Metamorphic Relations in Rule-Based Test Techniques (ISTQB CTAL-TA)
Overview
Metamorphic testing (MT) is one of the rule-based test techniques in the ISTQB Advanced Level Test Analyst (CTAL-TA) syllabus. The other is decision table testing. At the heart of metamorphic testing is the metamorphic relation (MR).
An MR is a rule describing how the outputs of a program should change, or stay the same, when its inputs are changed in a defined way. This guide covers why MRs matter, what they are, how to apply them, and how to answer exam questions on them.
1. Why Metamorphic Relations Are Important
The test oracle problem
Every test needs an oracle, which is a way to decide whether an actual result is correct. For many systems, working out the exact expected output is very hard or impossible. Typical examples:
• Scientific and numerical simulations, such as weather models or physics engines.
• Machine learning and AI systems, such as image classifiers or recommendation engines.
• Search engines and large database queries, where you cannot know the full correct result set.
• Compilers, optimisers, route planners and bioinformatics tools.
• Complex mathematical functions with no independent reference implementation.
How MRs help
Metamorphic testing removes the need to know the exact expected output of a single execution. Instead, it checks whether the relationship between the outputs of several related executions holds. If the relation is violated, a defect has been found, even though nobody knew the correct individual answer.
Further benefits
• Automation-friendly: once an MR is defined, many follow-up tests can be generated and checked automatically.
• Scalable: one MR can drive hundreds or thousands of test cases, including randomly generated source inputs.
• Domain-driven: MRs capture real properties of the domain, so they often reveal subtle defects that example-based tests miss.
• Complementary: MT works alongside other techniques and can reuse existing test cases as source test cases.
2. What a Metamorphic Relation Is
Definition
A metamorphic relation is a necessary property of the test object. It relates:
• a source test case, which is the original input and its output, and
• one or more follow-up test cases, whose inputs are derived from the source input by a transformation.
It also states the expected relation between their outputs.
Key terms
• Source test case: the initial test case. Its output does not need to be verified against an exact expected value.
• Follow-up test case: a test case whose input is produced by applying the MR's input transformation to the source input.
• Input transformation: the rule for deriving follow-up inputs, for example 'multiply x by 2', 'reverse the list' or 'add a search filter'.
• Output relation: the condition the outputs must satisfy, for example 'outputs are equal', 'output doubles' or 'follow-up results are a subset of source results'.
Classic examples
• Sine function: sin(x) = sin(π − x). Compute sin(1.2) and sin(π − 1.2). If the results differ beyond rounding tolerance, there is a defect. You never needed to know the exact value of sin(1.2).
• Sorting: permuting the input list must not change the sorted output.
• Shortest path: the shortest distance from A to B equals the distance from B to A in an undirected graph. Adding a new edge can never make the shortest path longer.
• Search engine: adding a restrictive filter, such as 'only 2023 results', must return a subset of the unfiltered results.
• Online shop: doubling the quantity of every item (with no discounts) should double the total price.
• Image classifier: slightly changing brightness or rotating an image of a cat should not change the label 'cat'.
• Average function: adding a constant k to every value increases the average by exactly k.
Common categories of MRs
• Equality / invariance: the output stays the same, as with permutation or irrelevant changes.
• Additive / multiplicative: the output shifts or scales predictably.
• Inclusion / subset: the follow-up output is contained in, or contains, the source output.
• Inversion / negation: an inverse operation returns the original, for example encrypt then decrypt.
• Ordering / monotonicity: the output increases or decreases consistently with the input.
3. How Metamorphic Testing Works
Step 1: Analyse the test basis and domain. Study specifications, domain knowledge, mathematical properties and user expectations to find properties the system must always satisfy.
Step 2: Identify MRs. Express each MR as an input transformation plus an output relation. Each MR must be a necessary property: any correct implementation must satisfy it.
Step 3: Generate source test cases. Use existing tests, randomly generated data or inputs from other techniques, such as equivalence partitioning or boundary values.
Step 4: Derive follow-up test cases. Apply the input transformation of each MR to each source input.
Step 5: Execute. Run both the source and follow-up test cases.
Step 6: Check the output relation.
• If the relation is violated, there is a failure: at least one of the executions is wrong.
• If the relation holds, the test passes. This does not prove that either output is correct.
Step 7: Measure coverage. Coverage is usually expressed as the number of MRs exercised by tests divided by the total number of MRs identified, shown as a percentage.
Worked example
The system under test is a function that returns the total price of a basket. The MR is: 'If every item quantity is multiplied by n (with no quantity discounts), the total multiplies by n.'
• Source: basket with 2 pens and 1 notebook gives a total of 7.40.
• Follow-up (n = 3): 6 pens and 3 notebooks. The expected relation is total = 3 × 7.40 = 22.20.
• Actual follow-up total = 21.90, so the MR is violated and a defect exists. Perhaps an undocumented discount was applied, or there is a rounding error.
Limitations to remember
• Identifying good MRs requires domain knowledge and creativity.
• A satisfied MR does not guarantee correctness. A defect can affect source and follow-up outputs in a way that still satisfies the relation.
• MRs usually check only part of the expected behaviour, so several diverse MRs are needed.
• Floating-point and non-deterministic systems need tolerances in the output relation.
• An incorrectly defined MR (one that is not truly necessary) produces false positives.
4. Exam Tips: Answering Questions on Metamorphic Relations
Tip 1: Link MT to the oracle problem. If a question describes a system where expected results are hard or impossible to determine, metamorphic testing is very likely the intended answer. Typical clues are AI/ML, simulations, complex calculations and search. Decision tables suit clear combinations of conditions and actions instead.
Tip 2: Know the vocabulary precisely. Be able to distinguish the following:
• source test case versus follow-up test case
• input transformation versus output relation
Exam distractors often swap these terms.
Tip 3: Validate a candidate MR. When asked which statement is a valid MR, check that it:
• involves at least two executions,
• defines how the follow-up input is derived,
• states a relation between outputs, and
• is always true for a correct system.
Reject options that state a single expected value, as that is an ordinary oracle. Also reject options that are only 'usually' true.
Tip 4: Derive follow-up tests mechanically (K3 questions). Take the given source input, apply the stated transformation exactly, and compute the expected relation. Watch for arithmetic traps, units and rounding tolerances.
Tip 5: Interpret results correctly.
• A violated relation means a defect is present. You may not know which execution is wrong.
• A satisfied relation means no failure was detected, which is not proof of correctness.
Options claiming that a passing MR 'proves the output is correct' are false.
Tip 6: Coverage calculation. MR coverage = MRs exercised ÷ MRs identified × 100%. Do not confuse this with the number of test cases. One MR may be exercised by many source/follow-up pairs but still counts once.
Tip 7: Spot domain constraints. Scenario details matter. Discounts, caps, rounding rules or business exceptions can make a seemingly obvious MR invalid. For example, 'double the quantity doubles the price' fails if bulk discounts exist. Read the scenario carefully for such conditions.
Tip 8: Recognise benefits and limitations. Expect questions such as 'Which is an advantage of metamorphic testing?'
• Correct answers: no need for exact expected results, easy to automate, generates many tests from few MRs, useful for AI systems.
• Typical limitations: needs domain expertise, cannot prove correctness, quality depends on the MRs chosen.
Tip 9: Eliminate distractors systematically. Wrong options often:
• describe a single test with a fixed expected result,
• confuse MT with regression or back-to-back testing (back-to-back compares two implementations; MT compares several executions of the same implementation),
• claim MT replaces all other techniques.
Tip 10: Practise with familiar properties. Before the exam, write MRs for everyday functions: sorting, searching, totals, averages, maps and routes, and unit conversion. This makes scenario-based K3 questions much faster to solve.
Summary
A metamorphic relation is a necessary property linking the inputs and outputs of multiple executions of a test object. It lets testers detect failures without knowing exact expected results, which makes it a powerful answer to the test oracle problem. For the exam, remember five things:
• source versus follow-up test cases
• input transformation plus output relation
• violation means a defect, but satisfaction is not proof of correctness
• MR coverage is MRs exercised divided by MRs identified
• read scenario constraints carefully before accepting an MR as valid
