What is Integration Testing?

Integration testing tests integration or interfaces between components, interactions to different parts of the system such as an operating system, file system and hardware or interfaces between systems.

  • Also after integrating two different components together we do the integration testing. As displayed in the image below when two different modules ‘Module A’ and ‘Module B’ are integrated then the integration testing is done.

  • Integration testing is done by a specific integration tester or test team.
  • Integration testing follows two approach known as ‘Top Down’ approach and ‘Bottom Up’ approach as shown in the image below:

Below are the integration testing techniques:

1. Big Bang integration testing:

In Big Bang integration testing all components or modules are integrated simultaneously, after which everything is tested as a whole. As per the below image all the modules from ‘Module 1’ to ‘Module 6’ are integrated simultaneously then the testing is carried out. What is big bang integration testing

Advantage: Big Bang testing has the advantage that everything is finished before integration testing starts.

 Disadvantage: The major disadvantage is that in general it is time consuming and difficult to trace the cause of failures because of this late integration.

2. Top-down integration testing: Testing takes place from top to bottom, following the control flow or architectural structure (e.g. starting from the GUI or main menu). Components or systems are substituted by stubs. Below is the diagram of  ‘Top down Approach’:

What is top down integration testing

Advantages of Top-Down approach:

  • The tested product is very consistent because the integration testing is basically performed in an environment that almost similar to that of reality
  • Stubs can be written with lesser time because when compared to the drivers then Stubs are simpler to author.

Disadvantages of Top-Down approach:

  • Basic functionality is tested at the end of cycle

3. Bottom-up integration testing: Testing takes place from the bottom of the control flow upwards. Components or systems are substituted by drivers. Below is the image of ‘Bottom up approach’:

what is bottom up integrati

Advantage of Bottom-Up approach:

  • In this approach development and testing can be done together so that the product or application will be efficient and as per the customer specifications.

Disadvantages of Bottom-Up approach:

  • We can catch the Key interface defects at the end of cycle
  • It is required to create the test drivers for modules at all levels except the top control

Incremental testing:

  • Another extreme is that all programmers are integrated one by one, and a test is carried out after each step.
  • The incremental approach has the advantage that the defects are found early in a smaller assembly when it is relatively easy to detect the cause.
  • A disadvantage is that it can be time-consuming since stubs and drivers have to be developed and used in the test.
  • Within incremental integration testing  a range of possibilities exist, partly depending on the system architecture.

Functional incremental: Integration and testing takes place on the basis of the functions and functionalities, as documented in the functional specification.

Advertisements

What is Black Box Testing?

Black box testing treats the system as a “black-box”, so it doesn’t explicitly use Knowledge of the internal structure or code. Or in other words the Test engineer need not know the internal working of the “Black box” or application.

Main focus in black box testing is on functionality of the system as a whole. The term ‘behavioral testing’ is also used for black box testing and white box testing is also sometimes called ‘structural testing’. Behavioral test design is slightly different from black-box test design because the use of internal knowledge isn’t strictly forbidden, but it’s still discouraged.

Each testing method has its own advantages and disadvantages. There are some bugs that cannot be found using only black box or only white box. Majority of the applications are tested by black box testing method. We need to cover majority of test cases so that most of the bugs will get discovered by blackbox testing.

Black box testing occurs throughout the software development and Testing life cycle i.e in Unit, Integration, System, Acceptance and regression testing stages.

Tools used for Black Box testing:
Black box testing tools are mainly record and playback tools. These tools are used for regression testing that to check whether new build has created any bug in previous working application functionality. These record and playback tools records test cases in the form of some scripts like TSL, VB script, Java script, Perl.

Advantages of Black Box Testing
– Tester can be non-technical.
– Used to verify contradictions in actual system and the specifications.
– Test cases can be designed as soon as the functional specifications are complete

Disadvantages of Black Box Testing
– The test inputs needs to be from large sample space.
– It is difficult to identify all possible inputs in limited testing time. So writing test cases is slow and difficult
– Chances of having unidentified paths during this testing

Methods of Black box Testing :

Graph Based Testing Methods:
Each and every application is build up of some objects. All such objects are identified and graph is prepared. From this object graph each object relationship is identified and test cases written accordingly to discover the errors.

Error Guessing:
This is purely based on previous experience and judgment of tester. Error Guessing is the art of guessing where errors can be hidden. For this technique there are no specific tools, writing the test cases that cover all the application paths.

Boundary Value Analysis:
Many systems have tendency to fail on boundary. So testing boundry values of application is important. Boundary Value Analysis (BVA) is a test Functional Testing technique where the extreme boundary values are chosen. Boundary values include maximum, minimum, just inside/outside boundaries, typical values, and error values.

Extends equivalence partitioning
Test both sides of each boundary
Look at output boundaries for test cases too
Test min, min-1, max, max+1, typical values

BVA techniques:
1. Number of variables
For n variables: BVA yields 4n + 1 test cases.
2. Kinds of ranges
Generalizing ranges depends on the nature or type of variables
Advantages of Boundary Value Analysis
1. Robustness Testing – Boundary Value Analysis plus values that go beyond the limits
2. Min – 1, Min, Min +1, Nom, Max -1, Max, Max +1
3. Forces attention to exception handling

Limitations of Boundary Value Analysis
Boundary value testing is efficient only for variables of fixed values i.e boundary.

Equivalence Partitioning:
Equivalence partitioning is a black box testing method that divides the input domain of a program into classes of data from which test cases can be derived.

How is this partitioning performed while testing:
1. If an input condition specifies a range, one valid and one two invalid classes are defined.
2. If an input condition requires a specific value, one valid and two invalid equivalence classes are defined.
3. If an input condition specifies a member of a set, one valid and one invalid equivalence class is defined.
4. If an input condition is Boolean, one valid and one invalid class is defined.

What is Regression Testing?

Regression testing is type of testing carried out to ensure that changes made in the fixes or any enhancement changes are not impacting the previously working functionality. It is executed after enhancement or defect fixes in the software or its environment. It can be difficult to determine how much re-testing is needed, especially near the end of the development cycle.

This testing is done to make sure that new code changes should not have side effects on the existing functionalities. It ensures that old code still works once the new code changes are done.

Need of Regression Testing

Regression Testing is required when there is a

  • Change in requirements and code is modified according to the requirement
  • New feature is added to the software
  • Defect fixing
  • Performance issue fix

Regression Testing Techniques

Software maintenance is an activity which includes enhancements, error corrections, optimization and deletion of existing features. These modifications may cause the system to work incorrectly. Therefore, Regression Testing becomes necessary. Regression Testing can be carried out using following techniques:

Retest All

  • This is one of the methods for regression testing in which all the tests in the existing test bucket or suite should be re-executed. This is very expensive as it requires huge time and resources.

Regression Test Selection

  • Instead of re-executing the entire test suite, it is better to select part of test suite to be run
  • Test cases selected can be categorized as 1) Reusable Test Cases 2) Obsolete Test Cases.
  • Re-usable Test cases can be used in succeeding regression cycles.
  • Obsolete Test Cases can’t be used in succeeding cycles.

Prioritization of Test Cases

  • Prioritize the test cases depending on business impact, critical & frequently used functionalities. Selection of test cases based on priority will greatly reduce the regression test suite.

Selecting test cases for regression testing

It was found from industry data that good number of the defects reported by customers were due to last minute bug fixes creating side effects and hence selecting the test case for regression testing is an art and not that easy.  Effective Regression Tests can be done by selecting following test cases –

  • Test cases which have frequent defects
  • Functionalities which are more visible to the users
  • Test cases which verify core features of the product
  • Test cases of Functionalities which has undergone more and recent changes
  • All Integration Test Cases
  • All Complex Test Cases
  • Boundary value test cases
  • Sample of Successful test cases
  • Sample of Failure test cases

This type of testing typically carried out by testing specialist peoples. The automated testing methods are the best and safe option to carry out the Regression testing. In the market there are many free as well as free (open source) software testing tools are available. In the larger software development project the Regression testing is commonly used.

Some of the most commonly used regression testing tools list:

It is a verification method & mainly done in system testing but it applicable to all levels of software testing (Unit testing, Integration testing, System testing and Acceptance testing.)

Regression Testing and Configuration Management

Configuration Management during Regression Testing becomes imperative in Agile Environments where code is being continuously modified. To ensure effective regression tests, observe the following :

  • Code being regression tested should be under a configuration management tool
  • No changes must be allowed to code, during the regression test phase.  Regression test code must be kept immune to developer changes.
  • The database used for regression testing must be isolated. No database changes must be allowed

Difference between Re-Testing and Regression Testing:

Retesting means testing the functionality or bug again to ensure the code is fixed. If it is not fixed, defect needs to be re-opened. If fixed, defect is closed.

Regression testing means testing your software application when it undergoes a code change to ensure that the new code has not affected other parts of the software.

How to define the scope of regression testing?

The scope of testing is based on the how large fixes or enhancements of features. If this is impacting in the larger area of the application then scope will increase proportionally & it requires the deep testing of application including relevant test cases execution. But to analyze this tester should get the correct inputs from developer to decide the scope.

But if you are testing GUI applications the doing regression testing is very difficult. If the changes made in the GUI then all test cases written will not reused. In case if the project is big then it is very hectic work to update the GUI test cases. You have to write it again different test case for updated GUI once again.

What is Functional Testing?

Functional testing is a type of testing which verifies that each function of the software application operates in conformance with the requirement specification. This testing mainly involves black box testing and it is not concerned about the source code of the application.

Each and every functionality of the system is tested by providing appropriate input, verifying the output and comparing the actual results with the expected results. This testing involves checking of User Interface, APIs, Database, security, client/ server applications and functionality of the Application Under Test. The testing can be done either manually or using automation

What do you test in Functional Testing?

The prime objective of Functional testing is   checking the functionalities of the software system. It mainly concentrates on –

  • Mainline functions:  Testing the main functions of an application
  • Basic Usability: It involves basic usability testing of the system. It checks whether an user can freely navigate through the screens without any difficulties.
  • Accessibility:  Checks the accessibility of the system for the user
  • Error Conditions: Usage of testing techniques to check for error conditions.  It checks whether suitable error messages are displayed.

Functional Testing Process:

In order to functionally test an application, following steps must be observed.

  1. Understand the Requirements
  2. Identify test input (test data)
  3. Compute the expected outcomes with the selected test input values
  4. Execute test cases
  5. Comparison of actual and computed expected result

Functional Vs Non-Functional Testing:

Functional Testing Non-Functional Testing
Functional testing is performed using the functional specification provided by the client and verifies the system against the functional requirements. Non-Functional testing  checks the  Performance, reliability, scalability and other non-functional aspects of the software system.
Functional testing is executed first Non functional testing should be performed after functional testing
Manual testing or automation tools can be used for functional testing Using tools will be effective for this testing
Business requirements are the inputs to functional testing Performance parameters like speed , scalability are inputs to non-functional testing.
Functional testing describes what the product does Nonfunctional testing describes how good the product works
Easy to do manual testing Tough to do manual testing
Types of Functional testing are

  • Unit Testing
  • Smoke Testing
  • Sanity Testing
  • Integration Testing
  • White box testing
  • Black Box testing
  • User Acceptance testing
  • Regression Testing
Types of Non functional testing are

  • Performance Testing
  • Load Testing
  • Volume Testing
  • Stress Testing
  • Security Testing
  • Installation Testing
  • Penetration Testing
  • Compatibility Testing
  • Migration Testing

Functional testing tools:

There are several tools available in the marker to perform functional testing. They are explained as follows:

  • Selenium – Popular Open Source Functional Testing Tool
  • QTP – Very user-friendly Functional Test tool by HP
  • JUnit– Used mainly for Java applications and this can be used in Unit and system testing
  • soapUI – This is an open source functional testing tool, mainly used for Web service testing. It supports multiple protocols such HTTP, SOAP and JDBC.
  • Watir – This is functional testing tool for web applications. It supports tests executed at the web browser and uses ruby scripting language

What is Unit Testing?

Unit testing of software applications is done during the development (coding) of an application.

The objective of unit testing is to isolate a section of code and verify its correctness. In procedural programming a unit may be an individual function or procedure

The goal of unit testing is to isolate each part of the program and show that the individual parts are correct. Unit testing is usually performed by the developer.

Why do Unit Testing? Why it is important?

Sometimes software developers attempt to save time by doing minimal unit testing. This is a myth because skimping on unit testing leads to higher defect fixing costs during system testing, integration testing and even beta testing after the application is completed. Proper unit testing done during the development stage saves both time and money in the end.

How to Create Unit Test Cases

Unit testing is commonly automated, but may still be performed manually. The IEEE does not favor one over the other. A manual approach to unit testing may employ a step-by-step instructional document.

Under the automated approach-

  • A developer could write another section of code in the application just to test the function. They would later comment out and finally remove the test code when the application is done.
  • They could also isolate the function to test it more rigorously. This is a more thorough unit testing practice that involves copy and pasting the function to its own testing environment to other than its natural environment. Isolating the code helps in revealing unnecessary dependencies between the code being tested and other units or data spaces in the product. These dependencies can then be eliminated.

A coder may use a UnitTest Framework to develop automated test cases. Using an automation framework, the developer codes criteria into the test to verify the correctness of the unit. During execution of the test cases, the framework logs those that fail any criterion. Many frameworks will also automatically flag and report in a summary these failed test cases. Depending upon the severity of a failure, the framework may halt subsequent testing.

Mock Objects

Unit testing relies on mock objects being created to test sections of code that are not yet part of a complete application. Mock objects fill in for the missing parts of the program. For example, you might have a function that needs variables or objects that are not created yet. In unit testing, those will be accounted for in the form of mock objects created solely for the purpose of the unit testing done on that section of code.

Unit Testing Tools

There are several automated tools available to assist with unit testing. We will provide a few examples below:

  • Rational Software – Rational Software by IBM has a unittest feature known as “Rational Test Realtime”. The software contains a complete range of testing tools for much more than just unit testing. It is used for Ada, Java, C and C++. It creates unit tests by reverse engineering the software. Operating systems it supports include Windows, Linux, Solaris, HP-UX and AIX. Go to http://www-01.ibm.com/software/rational/ to learn more.
  • JavaScript Assertion Unit- Also known as jsAsserUnit, this Freeware JavaScript unit testing tool can be used on any platform that supports JavaScript. It is available at http://jsassertunit.sourceforge.net/docs/index.html
  • CUT – CUT is a Freeware unittest tool for C, C++ and Objective C. It is great for embedded software testing frameworks and desktop applications on Linux and Windows operating systems. Learn more at sourceforge.net by going to http://sourceforge.net/projects/cut/.
  • Dotunit – Dotunit is a .net framework Freeware unit testing tool. Part of Junit on the Microsoft .net framework, Dotunit is used for automating unit testing on windows systems. This is another tool from sourceforge.net, so look for it at: http://dotunit.sourceforge.net/

Those are just a few of the available unit testing tools. There are lots more, especially for C languages and Java, but you are sure to find a unit testing tool for your programming needs regardless of the language you use.

Extreme Programming & Unit Testing

Unit testing in Extreme Programming involves the extensive use of testing frameworks. A unit test framework is used in order to create automated unit tests. Unit testing frameworks are not unique to extreme programming, but they are essential to it. Below we look at some of what extreme programming brings to the world of unit testing:

  • Tests are written before the code
  • Rely heavily on testing frameworks
  • All classes in the applications are tested
  • Quick and easy integration is made possible

Unit Testing Myth

Myth: It requires time and I am always overscheduled

My code is rock solid! I do not need unit tests.

Myths by their very nature are false assumptions. These assumptions lead to a vicious cycle as follows –

Truth is Unit testing increase the speed of development.

Programmers think that integration testing will catch all errors and do not unit test. Once units are integrated, very simple errors which could have very easily found and fixed in unit tested take very long time to be traced and fixed.

Unit Testing Benefits and Advantage

  • Developers looking to learn what functionality is provided by a unit and how to use it can look at the unit tests to gain a basic understanding of the unit API.
  • Unit testing allows the programmer to refactor code at a later date, and make sure the module still works correctly (i.e. Regression testing). The procedure is to write test cases for all functions and methods so that whenever a change causes a fault, it can be quickly identified and fixed.
  • Due to the modular nature of the unit testing, we can tests parts of project without waiting for others to be completed.

Unit Testing Limitations

  • Unit testing can’t be expected to catch every error in a program. It is not possible to evaluate all execution paths even in the most trivial programs
  • Unit testing by its very nature focuses on a unit of code. Hence it can’t catch integration errors or broad system level errors.

It’s recommended unit testing be used in conjunction with other testing activities.

Unit Testing Techniques

  • Structural Techniques
  • Functional Testing Techniques
  • Error Based Techniques

Unit Testing Best Practices

  • Unit Test cases should be independent. In case of any enhancements or change in requirements, unit test cases should not be affected.
  • Test only one code at a time.
  • Follow clear and consistent naming conventions for your unit tests
  • In case of change in code in any module, ensure there is a corresponding unit test case for the module and the module passes the tests before changing the implementation
  • Bugs identified during unit testing must be fixed before proceeding to the next phase in SDLC
  • Adopt a “test as your code” approach. The more code you write without testing the more paths you have to check for errors.
UNIT Testing Tutorial - Learn in 10 Minutes

Summary

As you can see, there can be a lot involved in unit testing. It can be complex or rather simple depending on the application being tested and the testing strategies, tools and philosophies used. Unit testing is always necessary on some level. That is a certainty.

Why Automated Testing?

Every software development group tests its products, yet delivered software always has defects. Test engineers strive to catch them before the product is released but they always creep in and they often reappear, even with the best manual testing processes. Test Automation software is the best way to increase the effectiveness, efficiency and coverage of your software testing.

Manual software testing is performed by a human sitting in front of a computer carefully going through application screens, trying various usage and input combinations, comparing the results to the expected behavior and recording their observations. Manual tests are repeated often during development cycles for source code changes and other situations like multiple operating environments and hardware configurations. An automated testing tool is able to playback pre-recorded and predefined actions, compare the results to the expected behavior and report the success or failure of these manual tests to a test engineer. Once automated tests are created they can easily be repeated and they can be extended to perform tasks impossible with manual testing. Because of this, savvy managers have found that automated software testing is an essential component of successful development projects.

Automated software testing has long been considered critical for big software development organizations but is often thought to be too expensive or difficult for smaller companies to implement. SmartBear’s Tools are affordable enough for single developer shops and yet powerful enough that our customer list includes some of the largest and most respected companies in the world.

Automated Software Testing Saves Time and Money

Software tests have to be repeated often during development cycles to ensure quality. Every time source code is modified software tests should be repeated. For each release of the software it may be tested on all supported operating systems and hardware configurations. Manually repeating these tests is costly and time consuming. Once created, automated tests can be run over and over again at no additional cost and they are much faster than manual tests. Automated software testing can reduce the time to run repetitive tests from days to hours. A time savings that translates directly into cost savings.

Vastly Increases Your Test Coverage

Automated software testing can increase the depth and scope of tests to help improve software quality. Lengthy tests that are often avoided during manual testing can be run unattended. They can even be run on multiple computers with different configurations. Automated software testing can look inside an application and see memory contents, data tables, file contents, and internal program states to determine if the product is behaving as expected. Test automation can easily execute thousands of different complex test cases during every test run providing coverage that is impossible with manual tests.

Testing Improves Accuracy

Even the most conscientious tester will make mistakes during monotonous manual testing. Automated tests perform the same steps precisely every time they are executed and never forget to record detailed results. Testers freed from repetitive manual tests have more time to create new automated software tests and deal with complex features

Automation Does What Manual Testing Cannot

Even the largest software and QA departments cannot perform a controlled web application test with thousands of users. Automated testing can simulate tens, hundreds or thousands of virtual users interacting with a network, software and web applications.

Automated QA Testing Helps Developers and Testers

Shared automated tests can be used by developers to catch problems quickly before sending to QA. Tests can run automatically whenever source code changes are checked in and notify the team or the developer if they fail. Features like these save developers time and increase their confidence.

QA and Dev Team Morale Improves

This is hard to measure but we’ve experienced it first hand. Executing repetitive tasks with automated software testing gives your team time to spend on more challenging and rewarding projects. Team members improve their skill sets and confidence and, in turn, pass those gains on to their organization.

There are several different types of software tests that can be automated.

Software Test Automation Tools

Automated software testing is becoming more and more important for many software projects in order to automatically verify key functionality, test for regressions and help teams run a large number of tests in a short period of time. Many teams (especially larger projects) still require a significant amount of manual functional testing in addition to automated testing, either because of the lack of sufficient resources or skills to automate all tests.

There are various tools that help software teams build and execute automated tests. Many teams are actively using unit tests as part of their development efforts to verify critical parts of their projects such as libraries, models and methods.

Please see below for a list of popular and useful tools, projects, books and resources to get started with automated software testing.

Automated Testing Tools for Web Applications –

Automatically testing your web application is a good way to ensure that new versions of your application don’t introduce bugs and regressions. Automation of your web application testing also allows your development team to make changes and refactor code with more confident, as they can quickly verify the functionality of the application after every change.

However, actually building automated tests for web applications can be challenging because the user interface of your application might change regularly, because of incompatibilities between browsers and because you usually need to support various server or client platforms. The following tools make it easier to build and execute automated tests for your web application.

Open Source

Selenium

Selenium is a popular automated web testing tool and helps you automate web browsers across different platforms. Selenium has the support of some of the largest browser vendors who have taken steps to make Selenium a native part of their browser.

Open Source

Watir

Watir is a set of Ruby libraries for automating web browsers and allows you to write tests that are easy to read and maintain. Watir drives browsers the same way people do (it clicks links, fills in forms, presses buttons etc.) and also checks results such as whether expected text appears on the page.

Open Source

Windmill

Windmill is a web testing tool designed to help testers automate and debug web applications. It comes with a cross-browser test recorder, JavaScript integration and an interactive shell to automate web browsers.

Commercial

Ranorex

Ranorex allows you to automate your web application testing (among other things) and both record user interactions and play them back to execute your tests. Ranorex is one of the more popular commercial tools to build and run automated web and GUI tests.

Open Source

SoapUI

SoapUI is a cross-platform functional testing tool. It has been specifically designed to help automatically test APIs such as SOAP and REST interfaces to ensure the interoperability of different applications.

Open Source

Sahi

Sahi is a tool for automation of web application testing. Sahi is available as a free open source edition as well as a commercial Pro edition.

Commercial

Tellurium

Tellurium is a web automation tool that allows you to design and write your automated tests using plain English without any scripting or programming experience.