Types of Software Testing
Nowadays, there is no generally accepted definition of the “type of software testing”. It is not uncommon when methods, levels or even test design technics are defined as the type of testing. For example, sometimes, white box testing, integration testing or even boundary testing is considered as testing types.
It is necessary to refer to official sources to avoid ambiguous interpretations. International Software Testing Qualifications Board (ISTQB) is the respectable international organization that offers certifications in software testing, which are recognized in a whole world.
According to ISTQB definition, testing types are “means of clearly defining the objective of a certain level for a program or project”. The tester focuses on a particular test objective during test case execution. Depending on its objectives, there are four software testing types:
Functional testing
Non-functional testing
Structural testing
Change related testing
Of course, there are a lot of subtypes of testing. You can see them on the picture below:
So, let’s know more about all of the testing types.
Functional testing
Functional testing verifies that each function of the software application operates in conformance with the requirement specification. Functional testing shows “What the system does”. The goal of this testing is to check whether the system is functionally perfect.
Five steps need to be keeping in mind in the Functional testing:
Preparation of test data based on the specifications of functions
Business requirements are the inputs to functional testing
Find out of output of the functions based on functional specifications;
The execution of test cases
Observe the actual and expected outputs.
There are two perspectives in which testing functionality can be done from Requirement-based testing and
Business-process-based testing.
Requirements-based testing is performed in strict accordance with the defined requirements.
Business-process-based testing is performed in accordance with the knowledge based on the day-to-day business use of the system.
Functional testing advantages:
Functional testing simulates actual system usage.
It is executed in the conditions close to the customer’s ones.
No system structure assumptions are made while providing functional testing.
It is easy to do manual testing.
Functional testing limitations:
There is the high possibility of redundant testing.
Logical errors in software could be missed while providing functional testing.
It is easy to find and use tools for functional testing. The most well-known are: Selenium (both web and desktop application), Robotium(Android app), Linux Test Project, JUnit, Sprinter by Hewlett Packard Entreprise (manual testing), Browserstack (both automated and manual testing), Usersnap (manual testing).
Non-functional testing types
Non-functional testing types are concerned with the non-functional requirements. Non-functional testing helps to estimate the readiness of a system according to the different criteria which are not covered by functional testing. Unlike to functional testing, it shows “How good the system works”.
Let’s consider the variety of plenty subtypes of non-functional testing.
User Interface (UI) testing is aimed to ensure the graphic user interface of application meets the specifications. This helps to evaluate design elements such as layout, colors, fonts, font sizes, labels, text boxes, text formatting, captions, buttons, lists, icons, links, and content.
Approaches for UI testing: the manual based, record and reply, model-based.
The most popular special tools and frameworks for UI testing: FitNesse, iMacros, Coded UI, Jubula, LoadUI.
User Experience (UX) testing is aimed to verify customer interaction with company’s products and services. Actually, UX is much more than the visual interface of your product. It contains:
The impressions the customer takes away from the interaction with the product or service.
The process customer should go through to discover company’s product or service.
The sequence of actions customer takes as he/she interacts the interface.
The special tools and frameworks for UX testing: Usabilla, Omnigraffle, Visual Web Optimizer, UXPin, Crazy Egg.
Storage testing verifies the app under test, stores the relevant data in the correct directories and it has sufficient space to prevent unexpected termination due to insufficient disk space. It helps to determine whether or not the application uses more memory than estimated as filling the disk space can cause significant downtime.
The tools are often used for storage testing: HCIbench, Iometer, Diskspd Utility.
Operational testing is aimed to evaluate a system or component in its operational environment. Using it we can ensure system and component compliance in the application’s standard operating environment.
Operational testing is concerned to the system implementation behavior. This testing mainly focuses on operational readiness of the system, which is supposed to mimic the production environment.
Security testing is aimed to ensure the information system protects data and maintains functionality as intended. Penetration testing and Vulnerability testing are the species of Security testing types.
Penetration testing is the malicious source attack simulation, which allows evaluating the security of a computer system or network.
Vulnerability testing is aimed to evaluate the quantum of risks involved in the system in order to reduce the probability of the event. It helps to prevent problems which may affect the application integrity and stability.
The tools like Retina CS Community, OWASP Zed Attack Proxy, Veracode, Google Nogotofail, and SQL Map are often used for Security testing.
Configuration testing performed to check the system with each one of the supported software and hardware configurations:
OS Configuration – Win 7 32 bit/64 bit, Win 8 32 bit/64 bit, Win 7 32 bit/64 bit, Win 8 32 bit/64 bit.
Database Configuration – Oracle, DB2, MySql, MSSQL Server, Sybase.
Browser Configuration – IE 10, IE 11, Mozilla Firefox, Google Chrome.
There are different kinds of Configuration testing, such as Conversion testing, Cross-browser testing, Binary Portability testing, and Cross-Platform testing.
Conversion testing ensures the right data converting from existing systems for use in replacement systems.
Cross-browser testing performed to verify the correct work of application or system in different browser configurations: Mozilla Firefox, Google Chrome, Internet Explorer, and Opera etc.
Unlike to Cross-browser testing, Cross-platform testing is aimed to evaluate the work of application in different OS: Windows, iOS/Mac OS, Linux, Android, and BlackBerry etc.
Binary Portability testing helps to evaluate the portability of the software by executing the software on different platforms and environment. It is used for confirmation of an Application Binary Interface (ABI) specification.
The main tools which are often used for all kinds of Configuration testing: BrowserStack, CrossBrowserTesting by Smart Bear, Litmus, Browsera, Rational Clearcase by IBM, Ghostlab.
Localization testing performed to adapt a globalized application to a particular culture/locale. This process involves translating all native language strings to the target language and customizing the GUI so that it is appropriate for the target market. Globalization testing and Internationalization testing are ones of its kinds.
Globalization testing checks the proper functionality of the product with any of the culture/locale settings using every type of international input possible.
Internationalization testing checks the right content externalization in different languages and locations.
Localization is usually done using some combination of in-house resources, independent contractors and full-scope services of a localization company. Here are some tools for providing Localization testing – eggPlant, Babylon.NET by Redpin, and smartCAT.
Performance testing intends to determine how a system performs in terms of responsiveness and stability under a certain load. The kinds of Performance testing: Stress testing, Load testing, Stability testing, Volume testing, Concurrency testing, Scalability testing, Endurance testing etc.
Stress testing estimates the behavior of a system at or beyond the limits of its anticipated workload.
Load testing performed to evaluate the behavior of a system at increasing workload.
Stability testing is aimed to verify if the application can continuously perform well within or just above the acceptable period.
Volume testing allows analyzing the system performance by increasing the volume of data in the database. It verifies any values may become large over time (such as accumulated counts, logs, and data files), can be accommodated by the program and will not cause the program to stop working or degrade its operation.
Concurrency testing performed to identify the defects in an application when multiple users login to the application. Using it we have an ability to identify and measure the problems in system parameters such as response time, throughput, locks/deadlocks or any other issues associated with concurrency.
Scalability testing evaluates system’s ability to grow by increasing the different indicators, such as workload per user, or the number of concurrent users, or the size of a database.
Endurance testing defines problems that may occur with prolonged execution. It evaluates the behavior of a system when a significant workload is given continuously.
Ramp testing is the kind of Endurance testing, which consists of raising an input signal continuously until the system breaks down.
Apache JMeter, HP LoadRunner, Silk Performer from Micro Focus, WebLOAD, and Gatling are often used for performing different kinds of performance testing.
Recovery testing is aimed to evaluate the system ability for recovering from crashes, hardware failures, or other catastrophic problems. It is performed by the testing teams.
TestDisk, Recuva by Piriform, Wise Data Recovery by WiseCleaner and Restoration by Softonic are the main special tools for Recovery testing.
Compatibility testing checks the application’s compatibility within different environments: hardware, software, operating system, network environment. There are two kinds of this type of testing: Backward Compatibility, Forward Compatibility testing.
Backward Compatibility testing ensures new version of the product to continue to work with the older product.
Forward Compatibility testing provides the connection with the future version of the product.
Browsershots and MultiBrowser could be used for free for Compatibility testing.
Usability testing performed to evaluate a product or service by testing it with representative users. It helps to define user ability to learn to operate, prepare inputs for, and interpret outputs of a system or component.
Accessibility testing is the kind of Usability testing which determines the user-friendly level of a product to the people having disabilities (deaf, blind, mentally disabled).
The popular tools for Usability testing: User Zoom, Reflector, Loop11.
Structural testing
Structural testing checks the implementation of the program or code via testing of the structure of the software system or its components. The tester concentrates on the work of software during structural testing. It can be used at all levels of testing.
The main aims of the Structural testing:
Obvious inadequacies identification
Functional testing complementation
To understand if something is missing in our test suite
Structural testing techniques:
Statement Coverage verifies the each statement in a program is executed at least once during program testing.
Path Coverage is aimed to satisfy coverage criteria for each logical path through the program.
Branch Coverage verifies if each branch condition for the program has true or false values.
Condition Coverage is similar to the Branch Coverage. The main difference is Condition Coverage tests for conditional as well as not-conditional branches.
Advantages of Structural testing:
Dead code elimination
There is an ability to find out bugs at an early stage.
It ensures the more thorough software testing.
Structural Testing is not the time-consuming process.
Disadvantages of Structural testing:
Structural Testing is expensive.
It requires knowledge of the code.
It requires the strong knowledge of the tool used for testing.
The special tools for Structural testing: JBehave, Cucumber, JUnit, Cfix.
Change related testing
Change related testing is provided to ensure that previously eradicated bugs have been fixed and to catch bugs that may have been accidentally appeared into a new version. According to these goals, there are two subtypes of Change related testing: Confirmation testing (Re-testing) and Regression testing.
They often confuse the Confirmation testing with the Regression testing. As a rule, they need to be held one after another. Let’s clarify the difference between these subtypes of Change related testing.
Firstly you should perform Confirmation testing to ensure the bug has indeed been successfully removed. Put it simply, the test case that originally detected the bug is executed again and this time it should pass with no problems.
Regression testing consists not only of the detected bug test cases. It is performed not only for verifying bug fixing but also to ensure that new defects have not come up or discovered after the changes.
Hope, the picture below will help you to better understand Change related testing process.
The tools for Change related testing: Selenium, HP Quick Test Professional, TestComplete, TestDrive, SoapUI.
Now, you have got an idea about plenty of testing types. Some of them are used every day by a huge number of specialists, another are used very seldom. Anyway the all life is not enough to learn fundamentally each of testing types. But practice is the best way to do it. Hope that the information presented here will help you in your future activity in QA. It would be great to know your opinions about testing types in the comments.
Last updated
