As has been described previously, software testing is a key component of the software development process. Broadly speaking, it is an investigation that provides information about the quality of the software product. Software testing is part of quality assurance, a discipline that aims to purge errors and eliminate problems from the developed software.
For this two-part series of articles we have decided to conduct an overview of some of the most common software testing methodologies, providing different angles software testing can be examined from, showcasing various testing techniques, and describing the different levels on which software can be tested.
Verification and validation
Probably the first and most obvious way to look at software testing and quality assurance in general is to ask what it does and what it solves. In its simplest form, software testing consists of processes of verification and validation (often shortened to V&V). Verification and validation are best encapsulated by this aphorism:
Verification: are we building the product right?
Validation: are we building the right product?
Verification often involves so-called static tests and validation mostly deals with dynamic testing, which we will describe further in the text.
Testing methodologies
Once the purpose of software testing has been established, one can take a look at different testing methodologies. There are three commonly used methodologies that are very different, but quite intuitive:
Static/dynamic testing
Static testing refers to various methods of examining the software without actually running it. This commonly includes:
code reviews, walkthroughs, inspections and other group activities that facilitate peer code review. Pair programming, when two programmers cooperatively write code, while being a development technique, can be partly attributed to static software testing;
automated precompilation analysis - most integrated development environments and code editors validate the code for syntax and logic errors before running it.
While static software testing does not look like a sophisticated process, it can distinguish quite a lot of defects and system-level mistakes before they grow into bigger issues. Static testing is a verification technique.
Dynamic testing is a set of techniques that investigate the dynamic behavior of code. Software commonly deals with input that can change over time, so dynamic testing examines how software responds to such input. Unlike static testing, dynamic testing also involves validation.
The “box” method
The box method stems from a traditional testing metaphor. In it the box symbolizes the point of view that the tester takes when designing the test cases: white box (as in a transparent box or glass box) refers to testing the the software if it was a see-through box - the tester is testing the software internals, while black box symbolizes zero knowledge of the software structure - the tester acts as if they were a user oblivious to how the software product works.
One cannot go ahead without mentioning grey-box testing - a hybrid approach that gained prominence in mid-2000’s. In grey-box testing a tester designs test cases with the software structure in mind, but executes them at the user-level.
Exploratory testing
Exploratory testing is a term coined by developer and tester Cem Kaner in 1984 in reference to a method of simultaneously combining learning, test design and test execution. It can be better explained as a testing methodology that is opposed to scripted testing where all tests are predesigned. Exploratory testing is supposed to allow testers to obtain better knowledge of the product and hence be more effective. Replicated studies have shown that exploratory testing results in the same amount of defects found, but is more efficient than other methods in terms of time.
While the purpose of testing is clear, it is important to understand that there are a great variety of testing methods and approaches. Testing cannot be performed indefinitely, so it is crucial to establish key criteria for the product to be tested against - usually these criteria become apparent during the first two steps of the software development life cycle.
In the next article we will explore levels at which testing can be carried out as well as describe various testing techniques.