Test Basis, Test Oracle Availability and the Choice of Test Technique (ISTQB CTAL-TA)
Introduction
In the ISTQB Certified Tester Advanced Level Test Analyst (CTAL-TA) syllabus, the chapter on selecting test techniques asks one practical question: given what I actually have in front of me, which technique will give the most effective and efficient tests? Two factors strongly shape the answer:
• The test basis: what you test against.
• The test oracle: how you decide whether a result is correct.
This guide explains what these concepts are, why they matter, how they influence technique selection, and how to handle exam questions on them.
Why It Is Important
A technique is only as good as the information that feeds it. Applying a sophisticated technique to the wrong kind of input wastes effort and gives false confidence. Some examples:
• State transition testing applied to a requirement that has no state model produces guesswork.
• Decision table testing applied without clearly defined business rules leaves the expected results undefined.
Getting this right matters because it lets the Test Analyst:
• Match techniques to available information, so tests are traceable and justified.
• Detect test basis defects early, such as ambiguity, gaps and contradictions. Trying to apply a formal technique often exposes these problems.
• Handle oracle problems realistically. If no one can say what the correct output is, even well-designed tests cannot reveal failures.
• Choose experience-based or black-box approaches appropriately when documentation is weak, and justify that choice to stakeholders.
• Support automation of test design. Model-based testing and automated generation of test cases and expected results depend on a formal, machine-readable test basis and an automatable oracle.
What It Is
1. Test Basis
The test basis is the body of knowledge used as the foundation for test analysis and design. Typical sources are:
• Requirements specifications and user stories with acceptance criteria.
• Business rules and use cases.
• Design documents and architecture.
• Models such as state diagrams, activity diagrams, data models and BPMN.
• Interface specifications and regulatory standards.
• Existing systems, and the knowledge of domain experts.
The test basis can be:
• Formal / structured: precise models, decision tables, state charts, grammars or interface schemas.
• Semi-formal: well-written requirements, user stories with clear acceptance criteria, or use cases.
• Informal / weak / absent: vague stories, outdated documents, tacit knowledge, or legacy systems without documentation.
2. Test Oracle
A test oracle is a source that determines the expected result of a test, against which the actual result is compared. Common oracles include:
• The specification itself.
• An existing or legacy system, used for comparison or back-to-back testing.
• A reference implementation or model.
• A user manual, or the knowledge of a domain expert.
• Heuristic oracles, such as consistency, plausibility and comparable products.
• Metamorphic relations, where the relation between related inputs and outputs is known even if exact outputs are not.
• Statistical or partial oracles.
The test oracle problem arises when expected results are hard, expensive or impossible to determine. Typical cases are:
• Complex calculations and scientific simulations.
• AI/ML outputs.
• Big data processing.
• Non-deterministic systems.
3. Choice of Test Technique
The syllabus groups techniques into three families:
• Black-box (specification-based): equivalence partitioning, boundary value analysis, decision table testing, cause-effect graphing, state transition testing, classification tree, pairwise/combinatorial testing, use case testing, and domain analysis.
• Experience-based: error guessing, checklist-based testing, exploratory testing and defect-based approaches.
• White-box: mainly the Technical Test Analyst's area, but relevant context.
The test basis and oracle availability are among the key factors in choosing between them, alongside the following:
• Risk level, and the required coverage or rigor.
• Regulatory requirements.
• Tester knowledge and skills.
• Time and budget.
• Type of system and SDLC model.
• Expected types of defects.
• Tool support.
How It Works
Step 1: Assess the test basis
Ask these questions:
• Is it complete, correct, consistent and unambiguous?
• Is it formal enough to derive tests systematically?
• What form does it take: rules, states, workflows, data ranges, or combinations of parameters?
Step 2: Map the form of the test basis to suitable techniques
• Input/output data ranges and value domains: equivalence partitioning and boundary value analysis (domain analysis for multiple interacting variables).
• Business rules with combinations of conditions and actions: decision table testing or cause-effect graphing.
• Behavior that depends on history or events, such as modes, statuses or protocols: state transition testing.
• Many parameters with many values where exhaustive combinations are infeasible: pairwise/combinatorial testing or classification tree.
• Actor-system interaction scenarios and workflows: use case testing (or scenario testing based on user stories).
• Formal models such as UML or BPMN: model-based testing, with possible automated test generation.
• Weak, missing or outdated documentation, tight time constraints, or exploring new features: experience-based techniques such as exploratory testing, error guessing and checklists. These are often combined with black-box techniques as the basis improves.
Step 3: Assess oracle availability
• Strong oracle (precise expected results are available): formal techniques give full value, and comparison can often be automated.
• Partial or weak oracle: use heuristic oracles, domain experts, comparison with a legacy or competitor system, back-to-back testing, metamorphic testing, sanity and plausibility checks, or reviews of results.
• No practical oracle: lean toward exploratory and experience-based testing, where tester judgment acts as the oracle. Raise the gap as a risk and seek clarification from stakeholders.
Step 4: Consider combination and context
Techniques are often combined. In particular:
• Equivalence partitioning and boundary value analysis inside each decision table rule.
• Exploratory testing to supplement scripted tests.
• Higher risk or regulatory demands push toward more formal techniques with measurable coverage.
• Even then, a formal technique still needs a sufficiently defined basis. If one does not exist, the analyst should first help create or clarify it, for example by building a decision table or state model with the business.
Step 5: Automating test design
Automated test design needs two things:
• A formal, machine-readable test basis, such as a model, grammar or schema.
• An oracle that can be encoded, such as expected outputs from the model, assertions or invariants.
Without both, automation can generate inputs (for example, random testing or fuzzing) but cannot reliably check outputs. Only crashes, exceptions or rule violations can serve as implicit oracles.
Worked Examples
• Example 1: An insurance premium depends on age band, driving history and vehicle type, and the rules are documented in a table. Use decision table testing combined with boundary value analysis on the age bands. The rules themselves act as a strong oracle.
• Example 2: A legacy payroll system is being re-platformed, but the documentation is outdated. The old system is a reliable oracle, so use back-to-back/comparison testing. Combine it with experience-based techniques to cover undocumented behavior.
• Example 3: A recommendation engine has no exact expected output. Use metamorphic testing and heuristic oracles, for example: adding a purchased item should not make similar items disappear entirely. Add exploratory testing.
• Example 4: An ATM card status flow (active, blocked, expired) is shown in a state diagram. Use state transition testing covering all states, all valid transitions and selected invalid transitions.
• Example 5: An agile team has thin user stories and little time. Use exploratory testing in time-boxed sessions guided by test charters and checklists. Give feedback to refine acceptance criteria.
Exam Tips: Answering Questions on Test Basis, Test Oracle Availability and the Choice of Test Technique
1. Read the scenario for clues about the form of the test basis. Keywords point to techniques:
• 'Rules', 'conditions', 'combinations of conditions': decision table.
• 'Status', 'mode', 'event', 'sequence': state transition.
• 'Range', 'valid/invalid values', 'limits': equivalence partitioning and boundary value analysis.
• 'Many parameters', 'configurations', 'browser/OS combinations': pairwise or classification tree.
• 'Actors', 'main and alternative flows': use case testing.
2. Watch for statements that the documentation is missing, vague, outdated or still evolving. These almost always point to experience-based techniques (exploratory testing, error guessing, checklists), often combined with something else. Be suspicious of answers proposing a rigorous formal technique when no basis exists to support it.
3. Identify oracle clues.
• 'Existing system', 'previous version' or 'competitor product' suggests a comparison oracle (back-to-back testing).
• 'Expected results unknown', 'complex calculation' or 'non-deterministic' suggests the oracle problem. Look for heuristic, partial, statistical or metamorphic approaches, or consulting domain experts.
4. Prefer the answer that fits all the stated constraints. ISTQB questions often include distractors that are good techniques in general but ignore one constraint: time, risk, skills, regulation or documentation quality. Eliminate options that conflict with any given fact.
5. Remember that combination is often the best answer. If an option combines a black-box technique with exploratory testing, or decision tables with boundary value analysis, and it suits the context, it is frequently correct. 'Only one technique should ever be used' is a red flag.
6. High risk or safety-critical contexts call for more formal techniques with measurable coverage. Even here, the correct answer may involve first improving or formalizing the test basis, for example by creating a decision table or state model with stakeholders.
7. For automation questions, check both prerequisites. Automated test design requires a formal or machine-readable basis and an automatable oracle. If the oracle cannot be automated, generated tests have limited value beyond detecting crashes.
8. Distinguish the test basis from the test oracle. The basis tells you what to test. The oracle tells you whether the outcome is right. The specification can be both, but a legacy system or an expert can be an oracle without being a good basis for design.
9. Defects in the test basis are a benefit. Applying techniques, especially formal ones such as decision tables, reveals gaps, contradictions and ambiguities. Options mentioning early defect detection in requirements are often valid.
10. Use K-level awareness. CTAL-TA questions here are frequently K3/K4 (apply/analyze). Expect scenario questions where you must choose the 'most appropriate' technique, so justify the choice mentally using the basis, the oracle, risk and constraints.
11. Beware of absolute words. Options containing 'always', 'never' or 'guarantees' are usually wrong. Technique selection is context-dependent.
Quick Checklist for the Exam
• What does the test basis look like: rules, states, ranges, flows, combinations, or nothing reliable?
• Is the oracle strong, partial, or missing?
• What are the risk level and regulatory demands?
• What time, skills and tools are available?
• Which option best fits all these factors, possibly by combining techniques?
Summary
Technique selection is not a matter of preference. It follows from what you know about the system (the test basis) and how you can judge correctness (the test oracle).
• Formal, structured bases and strong oracles enable rigorous black-box techniques and automated test design.
• Weak bases and oracles push toward experience-based techniques, heuristic oracles, comparison testing and clarifying the basis.
In the exam, decode the scenario's clues, eliminate options that ignore any constraint, and favor sensible, context-driven combinations.
