Learn Data-Based Test Techniques (CTAL-TA) with Interactive Flashcards

Master key concepts in Data-Based Test Techniques through our interactive flashcard system. Click on each card to reveal detailed explanations and enhance your understanding.

Domain Testing: Equivalence Partitions with Open and Closed Borders

In the ISTQB Advanced Test Analyst syllabus, domain testing combines equivalence partitioning and boundary value analysis. It is used where input or output domains are defined by numeric ranges, conditions or combinations of variables. A domain is split into equivalence partitions, sets of values the system is expected to treat the same way. Each partition is limited by borders, and defects tend to cluster at these borders. Domain testing therefore asks exactly where a border lies and which side of it the border value belongs to. A closed border includes the border value in the partition. It is usually expressed with inclusive operators such as <= or >=. For example, in the rule 18 <= age <= 65, both 18 and 65 belong to the valid partition. An open border excludes the border value from the partition. It is usually expressed with strict operators such as < or >. For example, in the rule 0 < weight < 100, the values 0 and 100 are not valid. Domain testing selects specific points for each border. The ON point is the value exactly on the border. The OFF point is the value closest to the border on the other side, measured by the smallest meaningful increment, such as 1 or 0.01. For a closed border, the ON point lies inside the partition and the OFF point lies outside it. For an open border it is the other way round: the ON point lies outside the partition and the OFF point lies inside it. IN points from the interior of the partition and OUT points from outside it are also tested. Testing both the ON and OFF points detects shifted borders and wrong operators, such as < coded instead of <=. Open borders should not be confused with open-ended partitions, which have no defined upper or lower limit. For those, the analyst must find a practical limit from specifications, data types or system constraints. Correctly classifying borders as open or closed is essential for deriving precise test conditions, expected results and coverage measures.

Domain Testing: ON and OFF Points

In the ISTQB Advanced Level Test Analyst syllabus, domain testing is a data-based technique that extends equivalence partitioning and boundary value analysis. It applies to input domains defined by one or more variables, often with linear boundary conditions such as x + y <= 100. The technique focuses on finding defects in how boundaries are implemented. Typical defects include a shifted boundary, a tilted boundary, a wrong closure (for example, < coded instead of <=), and missing or extra boundaries. To find these defects, the tester selects specific test points relative to each boundary. An ON point is a value that lies exactly on the boundary. An OFF point is a value as close as possible to the ON point, using the smallest meaningful increment (epsilon) for the data type, but on the opposite side of the boundary. Which side the OFF point falls on depends on the boundary closure. For a closed boundary (<= or >=), the ON point belongs to the domain, so the OFF point lies just outside it. For example, with x <= 10 for integers, the ON point is 10 and the OFF point is 11. For an open boundary (< or >), the ON point does not belong to the domain, so the OFF point lies just inside it. With x < 10, the ON point is 10 and the OFF point is 9. In both cases, the ON and OFF points always produce different outcomes. This is what lets them detect closure errors and boundary shifts. IN points, which lie inside the domain away from the boundaries, and OUT points, which lie outside, may be added as typical values. A common strategy is the 1x1 approach: one ON point and one OFF point per boundary. For n-dimensional linear boundaries, Beizer's N x 1 strategy uses n ON points and one OFF point to detect tilted boundaries. Test Analysts should define epsilon carefully, consider shared boundaries between adjacent domains, and avoid redundant points so that tests stay efficient yet thorough.

Domain Testing: IN and OUT Points

In the ISTQB Advanced Level Test Analyst syllabus, domain testing is a data-based technique that extends equivalence partitioning and boundary value analysis to domains defined by one or more variables. A domain is a region of the input space bounded by conditions, often linear inequalities such as x + y <= 100. Defects are often found at these borders, for example a wrong operator (< instead of <=) or a shifted or tilted boundary. Domain testing uses four kinds of test points: ON, OFF, IN and OUT. ON points lie exactly on a boundary. OFF points lie as close as possible to the boundary on the opposite side from the ON point. For a closed boundary (<=) the OFF point is outside the domain, and for an open boundary (<) it is inside. IN points are values that lie inside the domain and satisfy all of its boundary conditions, but are not on or next to any boundary. They represent typical, valid members of the domain. OUT points are values that lie outside the domain and violate at least one boundary condition, again away from the boundaries. They represent typical invalid inputs. IN and OUT points have several uses. First, when a test focuses on one boundary of a multidimensional domain, the variables not involved in that boundary are given IN-point values. This keeps those variables valid and ensures any failure is caused by the boundary under test. Second, IN and OUT points confirm that the domain is processed correctly as a whole: an IN point should be accepted and handled by the domain's logic, and an OUT point should be rejected or routed to another domain. Third, they help reveal major faults, such as a missing domain, extra processing, or the wrong domain being selected. IN and OUT points add little to precise boundary checking, which is the job of ON and OFF points. However, they complement them by giving cheap, representative coverage of the interior and exterior of each domain. Test Analysts select them to improve defect detection while keeping the number of test cases manageable.

Simplified Domain Coverage

In the ISTQB Advanced Level Test Analyst syllabus, domain testing is a data-based test technique for input domains that are defined by several variables and described by boundary conditions, often linear inequalities such as x + y <= 10. It extends boundary value analysis from one variable to many. A domain is a region in a multidimensional input space, and each boundary is a condition that separates inputs the system should treat one way from inputs it should treat another way. Simplified domain testing, based on work by Jeng and Weyuker, reduces the number of test points while still finding typical boundary defects. These defects include shifted boundaries, tilted boundaries, and closure errors, such as coding < where <= was intended. The technique uses ON points and OFF points. An ON point lies exactly on the boundary. An OFF point lies as close as possible to the ON point but on the other side of the boundary. For a closed boundary, which includes its edge, the ON point belongs to the domain and the OFF point lies outside it. For an open boundary, which excludes its edge, the ON point lies outside the domain and the OFF point lies inside it. Some approaches also use IN and OUT points, which lie clearly inside or outside the domain, to confirm general behavior. In the simplified approach, each boundary normally needs only one ON point and one OFF point. The test analyst identifies all boundaries, chooses these point pairs, determines the expected domain for each point, and derives test cases. Simplified domain coverage is the number of boundaries for which both the ON and OFF points have been tested, divided by the total number of boundaries, shown as a percentage. Full coverage gives good confidence in the boundaries with few tests. The main limitation is that it assumes simple, mostly linear boundaries, and nonlinear domains may need additional points.

Reliable Domain Coverage

In the ISTQB Advanced Level Test Analyst syllabus, data-based test techniques include equivalence partitioning, boundary value analysis, and domain analysis. Reliable Domain Coverage is best understood as a test design goal rather than a separately named technique: choose test data so that passing tests give justified confidence that the software behaves correctly across each whole input domain, not only at the values actually tested. The idea comes from testing theory, notably Goodenough and Gerhart's notion of a reliable test criterion. A domain is a set of input values, often defined by several variables and constrained by boundaries such as x + y <= 100. Domain testing assumes that all values inside a correctly identified subdomain are processed in the same way. If that assumption holds, one representative value can stand for the whole subdomain, and the coverage is reliable. If it is wrong, for example because hidden internal logic treats some values differently, the coverage is unreliable and defects may escape. In practice, test analysts pursue reliability with boundary-focused strategies. Domain analysis selects ON points, which lie on a boundary, and OFF points, which lie just beyond it, plus IN and OUT points where useful. A common approach is Beizer's 1x1 strategy, which uses one ON point and one OFF point per boundary. For linear boundaries, it can detect typical domain errors such as shifted, tilted, missing, or extra boundaries and wrong relational operators, for example < instead of <=. Coverage is usually measured as the percentage of identified partitions, boundary values, or domain test points that have been exercised. Reliability depends on correct domain identification, consistent processing within each partition, appropriate boundary precision, and attention to interacting variables. Analysts strengthen it by deriving domains carefully from specifications, involving domain experts, and adding tests where assumptions are uncertain. High coverage figures alone do not guarantee reliability. Confidence comes from well-founded partitions and boundaries combined with systematic test point selection.

Combinatorial Testing: Parameters, Values and Interaction Failures

In the ISTQB Advanced Level Test Analyst syllabus, combinatorial testing is a data-based test technique used when a test object has several input parameters, each with several possible values, and testing every combination is impractical. Instead of exhaustive testing, the Test Analyst selects a subset of combinations that still meets a defined coverage criterion.

Parameters are the factors that can affect the behavior of the test object, such as operating system, browser, user role, payment method or configuration setting. Values are the specific options each parameter can take, for example Windows, macOS or Linux. Values are usually derived with other techniques, such as equivalence partitioning or boundary value analysis, so that each value represents a meaningful class of inputs. The total number of possible combinations is the product of the number of values per parameter. Five parameters with four values each already give 1,024 combinations, which is why a reduction strategy is needed.

Interaction failures are failures that occur only when particular values of two or more parameters appear together. For example, a defect might appear only when a specific browser is used with a specific language setting. Empirical studies show that most such failures are triggered by interactions between a small number of parameters, typically two or three. This finding justifies pairwise testing (2-wise coverage), where every possible pair of values for any two parameters is covered by at least one test case. Higher strengths, such as 3-wise, give more thoroughness at the cost of more tests. Each-choice coverage is the weakest level and only requires each value to appear at least once.

The Test Analyst uses tools, orthogonal arrays or covering arrays to generate efficient test sets. The analyst must also handle constraints, meaning combinations that are invalid or impossible, and should combine combinatorial testing with risk analysis. Critical value combinations should be added explicitly, because the technique does not guarantee that every high-risk scenario is included.

Base Choice Coverage

Base Choice Coverage is a combinatorial test technique covered in the ISTQB Advanced Level Test Analyst syllabus as one of the data-based test techniques. It is useful when a test object has several input parameters, each with a set of possible values, and testing every combination would be impractical. The test analyst first picks a base choice for each parameter. This is a value that is typical, most frequently used, most important, or representative of normal operation. Combining the base choices of all parameters gives the base test, which represents the main expected scenario. Further tests are created by changing only one parameter at a time to each of its non-base values, while all other parameters keep their base values. Each additional test therefore differs from the base test in exactly one parameter. If parameter i has ki values, the number of tests required is 1 plus the sum of (ki - 1) across all parameters. For example, take Browser (Chrome, Firefox, Safari), OS (Windows, macOS) and Language (English, German, French), with Chrome, Windows and English as the base values. This yields the base test plus two browser variations, one OS variation and two language variations. That is six tests, compared with eighteen for all combinations. Coverage is calculated as the number of base choice combinations executed divided by the total number required, expressed as a percentage. The main advantage is that a failure in a variant test can be traced easily to the single changed value, which simplifies defect localization. The technique also concentrates effort on realistic usage. Its weaknesses are that it does not systematically test interactions between two non-base values, so it is weaker than pairwise coverage. Its effectiveness also depends on choosing meaningful base values, which requires domain knowledge and input from stakeholders. Base choice coverage subsumes each choice coverage, because every parameter value appears in at least one test. Test analysts may also apply constraints to exclude invalid or impossible combinations.

Pairwise Coverage

In the ISTQB Certified Tester Advanced Level Test Analyst (CTAL-TA) syllabus, pairwise testing belongs to combinatorial testing, one of the data-based test techniques. These techniques derive test cases from the input data space rather than from behaviour or structure. Pairwise testing addresses combinatorial explosion. When a test object has many parameters, each with several possible values, testing every combination quickly becomes impractical. For example, a system with 10 parameters of 3 values each has 3^10 = 59,049 combinations. Pairwise testing relies on the observation that most defects caused by parameter interactions are triggered by just two parameters together. Rather than covering every combination, it requires that every possible pair of values for every pair of parameters appears in at least one test case. Pairwise coverage measures how thoroughly this has been done. It is calculated as the number of distinct parameter-value pairs exercised by the tests, divided by the total number of possible pairs, expressed as a percentage. Achieving 100% pairwise coverage means every value of each parameter has been combined at least once with every value of every other parameter. A single test case can cover many pairs at once, so the number of tests needed grows far more slowly than exhaustive testing. In the example above, 100% pairwise coverage can typically be achieved with only about 15 to 20 test cases. Test analysts usually generate pairwise test sets with covering arrays, orthogonal arrays, or dedicated tools such as PICT. The analyst must still identify the relevant parameters and their values, often using equivalence partitioning or boundary value analysis first. Constraints, meaning invalid or impossible combinations, must be handled explicitly. The main limitation is that pairwise testing may miss defects that only appear when three or more parameters interact. Where risk justifies it, analysts can apply stronger N-wise coverage, such as 3-wise, at the cost of more test cases. The analyst should also define expected results for every generated test, which the technique itself does not provide.

Reducing Combinations with Equivalence Partitioning and Classification Trees

In the ISTQB Advanced Level Test Analyst syllabus, data-based test techniques help the Test Analyst manage the combinatorial explosion that occurs when a test object has many inputs, each with many possible values. Testing every combination is usually impossible, so the number of combinations must be reduced systematically without losing defect-detection power. Equivalence partitioning (EP) is the first step. Each input parameter's domain is divided into partitions whose values the system is expected to process in the same way. Partitions can be valid or invalid. Instead of testing every individual value, the analyst selects one representative from each partition, often adding boundary values for ordered partitions. This greatly reduces the number of candidate values per parameter. The Classification Tree Method (CTM) builds on EP by visualising and organising the test-relevant aspects of the test object. The analyst identifies classifications, which are parameters or conditions such as payment type, customer category or browser. Each classification is broken down into classes, which are its equivalence partitions, and classes may be refined further into sub-classifications to form a hierarchical tree. Below the tree, a combination table is created in which each row is a test case that selects one class from each relevant classification. The tree makes the input domain transparent, supports reviews with stakeholders and helps reveal missing or overlapping partitions. CTM lets the analyst choose a coverage level based on risk. Minimum coverage requires every class to appear in at least one test case, so the number of tests roughly equals the largest number of classes in any classification. Pairwise coverage requires every pair of classes from different classifications to be combined at least once. Maximum coverage, meaning all combinations, is used only for high-risk areas. Constraints, such as classes that cannot logically occur together, remove infeasible combinations. Tool support helps generate and maintain the combination table. Together, EP and classification trees deliver a small, traceable, risk-appropriate test set with measurable coverage.

Constraints and Infeasible Combinations in Combinatorial Testing

In the ISTQB Advanced Level Test Analyst syllabus, combinatorial testing (such as pairwise or each-choice testing) is a data-based technique for selecting a small but systematic set of test cases from many interacting parameters. In real systems, however, parameters are rarely fully independent. Constraints are rules that restrict which parameter values can appear together, and infeasible combinations are value combinations that cannot occur or cannot be executed. Examples include a mobile app where the operating system 'iOS' cannot be combined with the browser 'Samsung Internet', or an insurance form where 'No vehicle' excludes any 'Engine size' value.

Constraints typically take forms such as 'excludes' (value A never appears with value B), 'requires' (value A always needs value B), or conditional rules (if Country = US, then Currency = USD). The Test Analyst identifies them from requirements, business rules, domain knowledge and technical limitations, and documents them in the parameter model.

Ignoring constraints causes problems. Generated test cases may be impossible to execute, waste effort, or be wrongly reported as failures. Simply deleting infeasible test cases afterwards is risky, because each deleted test may also cover other valid pairs that are then lost, reducing the intended coverage.

Recommended approaches include using constraint-aware combinatorial tools that generate covering arrays respecting the rules, replacing an infeasible value in a test case with a feasible one and checking that the lost pairs are covered elsewhere, adding extra test cases to restore missing coverage, or splitting the model into sub-models (for example, separate models per operating system).

The Test Analyst should also distinguish infeasible combinations from invalid ones. Invalid combinations, such as entering an expired card date, are possible inputs that the system should reject. These should be tested separately as negative tests rather than mixed into valid combinations, to avoid input masking. Coverage is then measured against the set of feasible combinations only.

Random Testing: Probability Distributions, Guided and Unguided Selection

In the ISTQB Advanced Level Test Analyst syllabus (v4.0), random testing is a data-based test technique. Test inputs are chosen randomly from the input domain of the test object rather than derived systematically from partitions or boundaries. It is useful when the input space is too large for exhaustive testing, when systematic techniques may carry the tester's biases, or as a complement that finds unexpected failures. Because many tests are usually generated, random testing is almost always automated and needs an effective test oracle, such as assertions, crash or exception detection, comparison with a reference implementation, or metamorphic relations.

Probability distributions: Every random selection follows a probability distribution, either stated explicitly or assumed. A uniform distribution gives each value in the domain an equal chance of selection, which suits general exploration of robustness. A non-uniform distribution, such as a normal distribution or one based on an operational profile, weights values by how likely they are in real use. This makes the tests more representative of production and supports reliability estimation. Test analysts must choose the distribution deliberately, because it determines which areas of the domain receive the most testing. A poor choice may leave rare but risky values almost untested.

Unguided selection: Inputs are generated purely randomly according to the chosen distribution, with no feedback from earlier tests. It is simple, cheap, unbiased and easy to scale. However, it may produce clustered or redundant inputs and can miss narrow failure regions, such as specific boundary values.

Guided selection: Information from previous tests or from the test object steers the generation of new inputs. Examples include adaptive random testing, which spreads inputs evenly across the domain by choosing candidates far from those already executed, and coverage-guided or search-based approaches such as fuzzing, which favour inputs that reach new code or come closer to violating a condition. Guided selection usually detects failures more efficiently but costs more to implement and run.

The Test Analyst selects the distribution and approach based on risk, oracle availability and tool support.

Benefits and Limitations of Random Testing

In the ISTQB Advanced Level Test Analyst syllabus, random testing is a data-based test technique. Test inputs are selected at random from the input domain, either uniformly or according to an operational profile that reflects expected real-world usage. Because it relies on generating many inputs automatically, it is usually supported by tools.

Benefits:
1. Reduced tester bias: Random selection is not limited by the tester's assumptions, so it can reveal failures that systematic techniques such as equivalence partitioning or boundary value analysis miss.
2. High volume at low cost: Once a generator and an automated oracle exist, thousands or millions of test cases can be run cheaply. This suits regression testing, robustness testing and fuzzing.
3. Reliability estimation: When inputs follow an operational profile, the results support statistical estimates of reliability, such as mean time between failures, because the tests mirror real usage.
4. Simple to design: Test analysts mainly need to define the input domain and its distribution, rather than deriving detailed test conditions for each case.
5. Complementary value: It works well alongside other techniques and adds diversity to the test suite.

Limitations:
1. Oracle problem: Determining the expected result for large numbers of random inputs is difficult. Without an automated oracle, such as a reference implementation, invariants or crash detection, the technique becomes impractical.
2. Weak coverage of specific values: Random selection may rarely hit boundaries, special values or narrow failure regions. Defects that depend on precise inputs may therefore remain undetected.
3. No guaranteed coverage: It does not systematically achieve partition, boundary or combinatorial coverage, so coverage is hard to measure and justify.
4. Reproducibility: Failures can be hard to reproduce unless random seeds and generated inputs are logged.
5. Domain modelling effort: Complex, structured or interdependent inputs require sophisticated generators. An inaccurate operational profile can produce misleading reliability figures.
6. Inefficiency: Many tests may be redundant and exercise the same behaviour, which wastes execution and analysis effort.

Random testing is therefore best used as a complement to systematic techniques, not as a replacement for them.

More Data-Based Test Techniques questions
350 questions (total)
Practice questions
One session at a time, always new questions