What is Integration Testing?

Integration testing tests integration or interfaces between components, interactions to different parts of the system such as an operating system, file system and hardware or interfaces between systems.

  • Also after integrating two different components together we do the integration testing. As displayed in the image below when two different modules ‘Module A’ and ‘Module B’ are integrated then the integration testing is done.

  • Integration testing is done by a specific integration tester or test team.
  • Integration testing follows two approach known as ‘Top Down’ approach and ‘Bottom Up’ approach as shown in the image below:

Below are the integration testing techniques:

1. Big Bang integration testing:

In Big Bang integration testing all components or modules are integrated simultaneously, after which everything is tested as a whole. As per the below image all the modules from ‘Module 1’ to ‘Module 6’ are integrated simultaneously then the testing is carried out. What is big bang integration testing

Advantage: Big Bang testing has the advantage that everything is finished before integration testing starts.

 Disadvantage: The major disadvantage is that in general it is time consuming and difficult to trace the cause of failures because of this late integration.

2. Top-down integration testing: Testing takes place from top to bottom, following the control flow or architectural structure (e.g. starting from the GUI or main menu). Components or systems are substituted by stubs. Below is the diagram of  ‘Top down Approach’:

What is top down integration testing

Advantages of Top-Down approach:

  • The tested product is very consistent because the integration testing is basically performed in an environment that almost similar to that of reality
  • Stubs can be written with lesser time because when compared to the drivers then Stubs are simpler to author.

Disadvantages of Top-Down approach:

  • Basic functionality is tested at the end of cycle

3. Bottom-up integration testing: Testing takes place from the bottom of the control flow upwards. Components or systems are substituted by drivers. Below is the image of ‘Bottom up approach’:

what is bottom up integrati

Advantage of Bottom-Up approach:

  • In this approach development and testing can be done together so that the product or application will be efficient and as per the customer specifications.

Disadvantages of Bottom-Up approach:

  • We can catch the Key interface defects at the end of cycle
  • It is required to create the test drivers for modules at all levels except the top control

Incremental testing:

  • Another extreme is that all programmers are integrated one by one, and a test is carried out after each step.
  • The incremental approach has the advantage that the defects are found early in a smaller assembly when it is relatively easy to detect the cause.
  • A disadvantage is that it can be time-consuming since stubs and drivers have to be developed and used in the test.
  • Within incremental integration testing  a range of possibilities exist, partly depending on the system architecture.

Functional incremental: Integration and testing takes place on the basis of the functions and functionalities, as documented in the functional specification.

What is Black Box Testing?

Black box testing treats the system as a “black-box”, so it doesn’t explicitly use Knowledge of the internal structure or code. Or in other words the Test engineer need not know the internal working of the “Black box” or application.

Main focus in black box testing is on functionality of the system as a whole. The term ‘behavioral testing’ is also used for black box testing and white box testing is also sometimes called ‘structural testing’. Behavioral test design is slightly different from black-box test design because the use of internal knowledge isn’t strictly forbidden, but it’s still discouraged.

Each testing method has its own advantages and disadvantages. There are some bugs that cannot be found using only black box or only white box. Majority of the applications are tested by black box testing method. We need to cover majority of test cases so that most of the bugs will get discovered by blackbox testing.

Black box testing occurs throughout the software development and Testing life cycle i.e in Unit, Integration, System, Acceptance and regression testing stages.

Tools used for Black Box testing:
Black box testing tools are mainly record and playback tools. These tools are used for regression testing that to check whether new build has created any bug in previous working application functionality. These record and playback tools records test cases in the form of some scripts like TSL, VB script, Java script, Perl.

Advantages of Black Box Testing
– Tester can be non-technical.
– Used to verify contradictions in actual system and the specifications.
– Test cases can be designed as soon as the functional specifications are complete

Disadvantages of Black Box Testing
– The test inputs needs to be from large sample space.
– It is difficult to identify all possible inputs in limited testing time. So writing test cases is slow and difficult
– Chances of having unidentified paths during this testing

Methods of Black box Testing :

Graph Based Testing Methods:
Each and every application is build up of some objects. All such objects are identified and graph is prepared. From this object graph each object relationship is identified and test cases written accordingly to discover the errors.

Error Guessing:
This is purely based on previous experience and judgment of tester. Error Guessing is the art of guessing where errors can be hidden. For this technique there are no specific tools, writing the test cases that cover all the application paths.

Boundary Value Analysis:
Many systems have tendency to fail on boundary. So testing boundry values of application is important. Boundary Value Analysis (BVA) is a test Functional Testing technique where the extreme boundary values are chosen. Boundary values include maximum, minimum, just inside/outside boundaries, typical values, and error values.

Extends equivalence partitioning
Test both sides of each boundary
Look at output boundaries for test cases too
Test min, min-1, max, max+1, typical values

BVA techniques:
1. Number of variables
For n variables: BVA yields 4n + 1 test cases.
2. Kinds of ranges
Generalizing ranges depends on the nature or type of variables
Advantages of Boundary Value Analysis
1. Robustness Testing – Boundary Value Analysis plus values that go beyond the limits
2. Min – 1, Min, Min +1, Nom, Max -1, Max, Max +1
3. Forces attention to exception handling

Limitations of Boundary Value Analysis
Boundary value testing is efficient only for variables of fixed values i.e boundary.

Equivalence Partitioning:
Equivalence partitioning is a black box testing method that divides the input domain of a program into classes of data from which test cases can be derived.

How is this partitioning performed while testing:
1. If an input condition specifies a range, one valid and one two invalid classes are defined.
2. If an input condition requires a specific value, one valid and two invalid equivalence classes are defined.
3. If an input condition specifies a member of a set, one valid and one invalid equivalence class is defined.
4. If an input condition is Boolean, one valid and one invalid class is defined.

What is Regression Testing?

Regression testing is type of testing carried out to ensure that changes made in the fixes or any enhancement changes are not impacting the previously working functionality. It is executed after enhancement or defect fixes in the software or its environment. It can be difficult to determine how much re-testing is needed, especially near the end of the development cycle.

This testing is done to make sure that new code changes should not have side effects on the existing functionalities. It ensures that old code still works once the new code changes are done.

Need of Regression Testing

Regression Testing is required when there is a

  • Change in requirements and code is modified according to the requirement
  • New feature is added to the software
  • Defect fixing
  • Performance issue fix

Regression Testing Techniques

Software maintenance is an activity which includes enhancements, error corrections, optimization and deletion of existing features. These modifications may cause the system to work incorrectly. Therefore, Regression Testing becomes necessary. Regression Testing can be carried out using following techniques:

Retest All

  • This is one of the methods for regression testing in which all the tests in the existing test bucket or suite should be re-executed. This is very expensive as it requires huge time and resources.

Regression Test Selection

  • Instead of re-executing the entire test suite, it is better to select part of test suite to be run
  • Test cases selected can be categorized as 1) Reusable Test Cases 2) Obsolete Test Cases.
  • Re-usable Test cases can be used in succeeding regression cycles.
  • Obsolete Test Cases can’t be used in succeeding cycles.

Prioritization of Test Cases

  • Prioritize the test cases depending on business impact, critical & frequently used functionalities. Selection of test cases based on priority will greatly reduce the regression test suite.

Selecting test cases for regression testing

It was found from industry data that good number of the defects reported by customers were due to last minute bug fixes creating side effects and hence selecting the test case for regression testing is an art and not that easy.  Effective Regression Tests can be done by selecting following test cases –

  • Test cases which have frequent defects
  • Functionalities which are more visible to the users
  • Test cases which verify core features of the product
  • Test cases of Functionalities which has undergone more and recent changes
  • All Integration Test Cases
  • All Complex Test Cases
  • Boundary value test cases
  • Sample of Successful test cases
  • Sample of Failure test cases

This type of testing typically carried out by testing specialist peoples. The automated testing methods are the best and safe option to carry out the Regression testing. In the market there are many free as well as free (open source) software testing tools are available. In the larger software development project the Regression testing is commonly used.

Some of the most commonly used regression testing tools list:

It is a verification method & mainly done in system testing but it applicable to all levels of software testing (Unit testing, Integration testing, System testing and Acceptance testing.)

Regression Testing and Configuration Management

Configuration Management during Regression Testing becomes imperative in Agile Environments where code is being continuously modified. To ensure effective regression tests, observe the following :

  • Code being regression tested should be under a configuration management tool
  • No changes must be allowed to code, during the regression test phase.  Regression test code must be kept immune to developer changes.
  • The database used for regression testing must be isolated. No database changes must be allowed

Difference between Re-Testing and Regression Testing:

Retesting means testing the functionality or bug again to ensure the code is fixed. If it is not fixed, defect needs to be re-opened. If fixed, defect is closed.

Regression testing means testing your software application when it undergoes a code change to ensure that the new code has not affected other parts of the software.

How to define the scope of regression testing?

The scope of testing is based on the how large fixes or enhancements of features. If this is impacting in the larger area of the application then scope will increase proportionally & it requires the deep testing of application including relevant test cases execution. But to analyze this tester should get the correct inputs from developer to decide the scope.

But if you are testing GUI applications the doing regression testing is very difficult. If the changes made in the GUI then all test cases written will not reused. In case if the project is big then it is very hectic work to update the GUI test cases. You have to write it again different test case for updated GUI once again.

What is Functional Testing?

Functional testing is a type of testing which verifies that each function of the software application operates in conformance with the requirement specification. This testing mainly involves black box testing and it is not concerned about the source code of the application.

Each and every functionality of the system is tested by providing appropriate input, verifying the output and comparing the actual results with the expected results. This testing involves checking of User Interface, APIs, Database, security, client/ server applications and functionality of the Application Under Test. The testing can be done either manually or using automation

What do you test in Functional Testing?

The prime objective of Functional testing is   checking the functionalities of the software system. It mainly concentrates on –

  • Mainline functions:  Testing the main functions of an application
  • Basic Usability: It involves basic usability testing of the system. It checks whether an user can freely navigate through the screens without any difficulties.
  • Accessibility:  Checks the accessibility of the system for the user
  • Error Conditions: Usage of testing techniques to check for error conditions.  It checks whether suitable error messages are displayed.

Functional Testing Process:

In order to functionally test an application, following steps must be observed.

  1. Understand the Requirements
  2. Identify test input (test data)
  3. Compute the expected outcomes with the selected test input values
  4. Execute test cases
  5. Comparison of actual and computed expected result

Functional Vs Non-Functional Testing:

Functional Testing Non-Functional Testing
Functional testing is performed using the functional specification provided by the client and verifies the system against the functional requirements. Non-Functional testing  checks the  Performance, reliability, scalability and other non-functional aspects of the software system.
Functional testing is executed first Non functional testing should be performed after functional testing
Manual testing or automation tools can be used for functional testing Using tools will be effective for this testing
Business requirements are the inputs to functional testing Performance parameters like speed , scalability are inputs to non-functional testing.
Functional testing describes what the product does Nonfunctional testing describes how good the product works
Easy to do manual testing Tough to do manual testing
Types of Functional testing are

  • Unit Testing
  • Smoke Testing
  • Sanity Testing
  • Integration Testing
  • White box testing
  • Black Box testing
  • User Acceptance testing
  • Regression Testing
Types of Non functional testing are

  • Performance Testing
  • Load Testing
  • Volume Testing
  • Stress Testing
  • Security Testing
  • Installation Testing
  • Penetration Testing
  • Compatibility Testing
  • Migration Testing

Functional testing tools:

There are several tools available in the marker to perform functional testing. They are explained as follows:

  • Selenium – Popular Open Source Functional Testing Tool
  • QTP – Very user-friendly Functional Test tool by HP
  • JUnit– Used mainly for Java applications and this can be used in Unit and system testing
  • soapUI – This is an open source functional testing tool, mainly used for Web service testing. It supports multiple protocols such HTTP, SOAP and JDBC.
  • Watir – This is functional testing tool for web applications. It supports tests executed at the web browser and uses ruby scripting language

What is Unit Testing?

Unit testing of software applications is done during the development (coding) of an application.

The objective of unit testing is to isolate a section of code and verify its correctness. In procedural programming a unit may be an individual function or procedure

The goal of unit testing is to isolate each part of the program and show that the individual parts are correct. Unit testing is usually performed by the developer.

Why do Unit Testing? Why it is important?

Sometimes software developers attempt to save time by doing minimal unit testing. This is a myth because skimping on unit testing leads to higher defect fixing costs during system testing, integration testing and even beta testing after the application is completed. Proper unit testing done during the development stage saves both time and money in the end.

How to Create Unit Test Cases

Unit testing is commonly automated, but may still be performed manually. The IEEE does not favor one over the other. A manual approach to unit testing may employ a step-by-step instructional document.

Under the automated approach-

  • A developer could write another section of code in the application just to test the function. They would later comment out and finally remove the test code when the application is done.
  • They could also isolate the function to test it more rigorously. This is a more thorough unit testing practice that involves copy and pasting the function to its own testing environment to other than its natural environment. Isolating the code helps in revealing unnecessary dependencies between the code being tested and other units or data spaces in the product. These dependencies can then be eliminated.

A coder may use a UnitTest Framework to develop automated test cases. Using an automation framework, the developer codes criteria into the test to verify the correctness of the unit. During execution of the test cases, the framework logs those that fail any criterion. Many frameworks will also automatically flag and report in a summary these failed test cases. Depending upon the severity of a failure, the framework may halt subsequent testing.

Mock Objects

Unit testing relies on mock objects being created to test sections of code that are not yet part of a complete application. Mock objects fill in for the missing parts of the program. For example, you might have a function that needs variables or objects that are not created yet. In unit testing, those will be accounted for in the form of mock objects created solely for the purpose of the unit testing done on that section of code.

Unit Testing Tools

There are several automated tools available to assist with unit testing. We will provide a few examples below:

  • Rational Software – Rational Software by IBM has a unittest feature known as “Rational Test Realtime”. The software contains a complete range of testing tools for much more than just unit testing. It is used for Ada, Java, C and C++. It creates unit tests by reverse engineering the software. Operating systems it supports include Windows, Linux, Solaris, HP-UX and AIX. Go to http://www-01.ibm.com/software/rational/ to learn more.
  • JavaScript Assertion Unit- Also known as jsAsserUnit, this Freeware JavaScript unit testing tool can be used on any platform that supports JavaScript. It is available at http://jsassertunit.sourceforge.net/docs/index.html
  • CUT – CUT is a Freeware unittest tool for C, C++ and Objective C. It is great for embedded software testing frameworks and desktop applications on Linux and Windows operating systems. Learn more at sourceforge.net by going to http://sourceforge.net/projects/cut/.
  • Dotunit – Dotunit is a .net framework Freeware unit testing tool. Part of Junit on the Microsoft .net framework, Dotunit is used for automating unit testing on windows systems. This is another tool from sourceforge.net, so look for it at: http://dotunit.sourceforge.net/

Those are just a few of the available unit testing tools. There are lots more, especially for C languages and Java, but you are sure to find a unit testing tool for your programming needs regardless of the language you use.

Extreme Programming & Unit Testing

Unit testing in Extreme Programming involves the extensive use of testing frameworks. A unit test framework is used in order to create automated unit tests. Unit testing frameworks are not unique to extreme programming, but they are essential to it. Below we look at some of what extreme programming brings to the world of unit testing:

  • Tests are written before the code
  • Rely heavily on testing frameworks
  • All classes in the applications are tested
  • Quick and easy integration is made possible

Unit Testing Myth

Myth: It requires time and I am always overscheduled

My code is rock solid! I do not need unit tests.

Myths by their very nature are false assumptions. These assumptions lead to a vicious cycle as follows –

Truth is Unit testing increase the speed of development.

Programmers think that integration testing will catch all errors and do not unit test. Once units are integrated, very simple errors which could have very easily found and fixed in unit tested take very long time to be traced and fixed.

Unit Testing Benefits and Advantage

  • Developers looking to learn what functionality is provided by a unit and how to use it can look at the unit tests to gain a basic understanding of the unit API.
  • Unit testing allows the programmer to refactor code at a later date, and make sure the module still works correctly (i.e. Regression testing). The procedure is to write test cases for all functions and methods so that whenever a change causes a fault, it can be quickly identified and fixed.
  • Due to the modular nature of the unit testing, we can tests parts of project without waiting for others to be completed.

Unit Testing Limitations

  • Unit testing can’t be expected to catch every error in a program. It is not possible to evaluate all execution paths even in the most trivial programs
  • Unit testing by its very nature focuses on a unit of code. Hence it can’t catch integration errors or broad system level errors.

It’s recommended unit testing be used in conjunction with other testing activities.

Unit Testing Techniques

  • Structural Techniques
  • Functional Testing Techniques
  • Error Based Techniques

Unit Testing Best Practices

  • Unit Test cases should be independent. In case of any enhancements or change in requirements, unit test cases should not be affected.
  • Test only one code at a time.
  • Follow clear and consistent naming conventions for your unit tests
  • In case of change in code in any module, ensure there is a corresponding unit test case for the module and the module passes the tests before changing the implementation
  • Bugs identified during unit testing must be fixed before proceeding to the next phase in SDLC
  • Adopt a “test as your code” approach. The more code you write without testing the more paths you have to check for errors.
UNIT Testing Tutorial - Learn in 10 Minutes

Summary

As you can see, there can be a lot involved in unit testing. It can be complex or rather simple depending on the application being tested and the testing strategies, tools and philosophies used. Unit testing is always necessary on some level. That is a certainty.

What is a Defect Life Cycle or a Bug Life Cycle in Software Testing?

Defect life cycle is a cycle which a defect goes through during its lifetime. It starts when defect is found and ends when a defect is closed, after ensuring it’s not reproduced. Defect life cycle is related to the bug found during testing.

The bug has different states in the Life Cycle. The Life cycle of the bug can be shown diagrammatically as follows:

bug cycle Continue Reading