Scenario-Based Reviewing

Overview
Scenario-based reviewing is a review technique covered in the Software Defect Prevention chapter of the ISTQB Advanced Level Test Analyst (CTAL-TA) syllabus. It sits within the topic of supporting phase containment, which means finding defects in the same phase where they were introduced. In practice, this means catching defects in requirements, user stories or design documents before they reach code and test execution.

For the Test Analyst, the technique is applied to the test basis, such as requirements, use cases, user stories and business process descriptions. The goal is to find defects early, when they are cheapest to fix. The syllabus expects you to be able to apply a review technique such as scenario-based reviewing (a K3 learning objective). Questions are therefore usually practical and scenario-driven, not just definitions.

1. Why Scenario-Based Reviewing Is Important
Early defect detection: Defects in specifications spread into design, code and tests. A requirement defect found in a review may cost a fraction of what the same defect costs once it reaches production. Scenario-based reviewing is one of the most effective ways for a Test Analyst to contribute to defect prevention.

Better guidance than a plain checklist: Checklist items are often generic, for example 'Is the requirement testable?'. A scenario tells the reviewer exactly how to walk through the work product. This gives better direction for finding specific defect types, such as:

• missing alternative flows
• inconsistent state handling
• undefined error behaviour

Uses the tester mindset: Test Analysts naturally think in terms of usage, inputs, outputs, preconditions and expected results. The technique turns that mindset into a structured review activity. It works almost like 'executing' the specification on paper before any code exists.

Reveals behavioural and logical defects: Reading a document line by line often misses defects that only appear when you follow a realistic sequence of events. Examples include:

• a step that references data never captured earlier
• a flow with no exit
• a business rule that contradicts a later step

Prepares later test design: The scenarios used in the review can often be reused or refined as the basis for test scenarios, use case tests and acceptance tests. Review effort therefore also feeds test analysis and design.

Supports phase containment and process improvement: Defects found and classified during scenario-based reviews give data for root cause analysis. That data can help prevent similar defects in future specifications.

2. What Scenario-Based Reviewing Is
Definition: In scenario-based reviewing, reviewers are given structured guidelines on how to read through the work product. These guidelines are based on scenarios of expected usage. Reviewers perform 'dry runs' of the work product by following these scenarios step by step.

Key characteristics:

• It works best when the work product is in a suitable format, such as use cases, user stories with acceptance criteria, workflows or process models.
• Scenarios give reviewers better guidelines for finding specific defect types than simple checklist entries.
• Reviewers should not be constrained only to the documented scenarios. Otherwise they may miss other defect types, especially missing features or missing requirements that no scenario touches.
• It is an individual review technique. It is used during the individual review (preparation) activity of a review process, for example the process described in ISO/IEC 20246.
• It can be used in different review types: informal reviews, walkthroughs, technical reviews and inspections.

How it compares with other review techniques (frequently examined):

• Ad hoc reviewing: little or no guidance. It depends heavily on reviewer skill and leads to duplicate findings.
• Checklist-based reviewing: reviewers check items from a list of questions based on past defects. It is systematic, but checklists can become generic, and reviewers may ignore anything not on the list.
• Scenario-based reviewing: reviewers follow scenarios or dry runs based on expected usage. It gives deeper, behaviour-oriented guidance for specific defect types. There is a risk of missing defects outside the scenarios.
• Perspective-based reading (role-based): reviewers take the viewpoint of different stakeholders, such as end user, tester, developer or operations. It often involves creating a draft derived work product, such as test cases.

Memory hook:

• Checklist = questions to answer
• Scenario = a path to walk through
• Perspective = a role to play

3. How Scenario-Based Reviewing Works
Step 1: Planning and preparation.

• The review leader or Test Analyst identifies the work product and the review objectives, such as finding defects in flows, business rules or error handling.
• Scenarios are selected or created from expected usage. Sources include use cases, personas, operational profiles, business processes and high-risk transactions.
• Scenarios may be assigned to different reviewers so the work product gets broader coverage.

Step 2: Define the scenarios as structured guidance.
Each scenario should describe:

• the actor and goal
• the preconditions and starting state
• the sequence of actions and data
• the expected outcomes
• variations to explore, such as alternative paths, exceptions, boundary data and invalid inputs

A scenario can also come with questions to ask at each step, for example: 'Is the system response specified? Are the data needed at this step available? What happens on failure?'

Step 3: Perform the dry run (individual review).

• The reviewer 'executes' the scenario against the specification, step by step.
• At each step, the reviewer checks that the specification defines what happens and that the behaviour is complete, correct, consistent, unambiguous and testable.
• Typical defects found include:
  • missing steps
  • undefined system responses
  • missing alternative or exception flows
  • contradictions between business rules
  • missing or inconsistent data definitions
  • impossible state transitions
  • unclear postconditions
  • unverifiable acceptance criteria

Step 4: Look beyond the scenario.
Reviewers should also note anything suspicious outside the defined scenarios. This is especially important for missing functionality, because that is the main weakness of the technique.

Step 5: Communicate and analyse findings.

• Findings are logged as review comments or defects, ideally classified by type and severity.
• They are discussed in the review meeting, if one is held. Duplicates are removed and the defects are passed to the author.

Step 6: Fix, report and improve.

• The author corrects the work product, and the corrections are checked.
• Defect data is used to improve the scenarios and checklists for future reviews.
• It also supports root cause analysis and defect prevention.

Worked example:
A use case describes 'Customer withdraws cash from an ATM'. The Test Analyst defines a scenario: 'Customer with valid card, correct PIN, requests an amount exceeding the daily limit'. Walking through the use case reveals several defects:

• The main flow describes PIN validation and dispensing, but no step checks the daily limit.
• No alternative flow covers a request above the limit.
• The message shown to the customer is not specified.
• The postcondition does not say whether the card is returned.

A second scenario, 'Customer enters wrong PIN three times', reveals that card retention is mentioned in a business rule but missing from the use case flows. This is an inconsistency.

A generic checklist item such as 'Are error conditions described?' might have been ticked off, because some error conditions exist. Only the dry run exposed the specific gaps.

Benefits and limitations summary:

• Benefits: focused, behaviour-oriented detection; good for specific defect types; less dependent on reviewer experience than ad hoc reviewing; scenarios can be reused for test design; early detection lowers cost.
• Limitations: it depends on the quality and coverage of the scenarios; defects outside the scenarios (especially missing features) can be overlooked; it is less effective on work products that do not describe behaviour or usage; creating good scenarios takes effort.

4. Exam Tips: Answering Questions on Scenario-Based Reviewing
Tip 1: Recognise the keywords. If a question mentions 'dry run', 'expected usage', 'structured guidelines on how to read', 'walking through use cases' or 'following a sequence of user actions', the answer is almost certainly scenario-based reviewing.

Tip 2: Know the classic weakness. A common question asks for the main risk. The answer is that reviewers may miss defects not covered by the scenarios, especially missing features. The mitigation is that reviewers should not restrict themselves to the documented scenarios. Checklist-based reviewing has the same weakness in respect of the checklist.

Tip 3: Distinguish it from checklist-based and perspective-based reviewing.

• Checklist = a list of questions or items, often derived from past defects.
• Scenario = usage-based dry runs that give better guidance for specific defect types than checklist entries.
• Perspective-based = stakeholder viewpoints, often creating derived work products.

Distractors often swap these descriptions.

Tip 4: Match the technique to the work product. Scenario-based reviewing fits best when the test basis is in a suitable format, such as use cases, user stories, workflows or process flows. If the question describes a behaviour-rich specification and asks which technique best finds missing alternative flows or undefined responses, choose scenario-based reviewing.

Tip 5: For K3 'apply' questions, simulate the dry run. You may be given a short specification or use case and asked which defect a scenario-based review would find. To answer:

1. Pick the scenario given, or the obvious one.
2. Trace each step.
3. Ask: 'Is this step and its outcome specified?'
4. Look for missing exception flows, undefined data, contradictory rules and unreachable or unterminated paths.

The correct answer is usually a defect that becomes visible only when you follow the sequence.

Tip 6: Eliminate answers that are not review defects. Options describing runtime failures, performance measurements or code-level defects are usually wrong. Scenario-based reviewing is static testing of a work product. It finds defects in the documentation, not failures in execution.

Tip 7: Watch for absolute statements. Be suspicious of options such as:

• 'Scenario-based reviewing finds all defect types'
• 'Reviewers must only check the documented scenarios'
• 'It requires no preparation'

These contradict the syllabus.

Tip 8: Link it to defect prevention and phase containment. If a question asks why a Test Analyst should take part in specification reviews, focus on early detection, phase containment, lower cost of fixing, and improved testability of the test basis.

Tip 9: Remember the role of the Test Analyst. The Test Analyst brings a testing and usage viewpoint. They are well placed to create realistic scenarios, including negative and boundary scenarios, and to judge whether expected results can be derived from the specification.

Tip 10: Read the scenario in the question carefully. ISTQB Advanced questions are often long and contain irrelevant detail. Identify:

• the work product being reviewed
• the review objective
• the constraints (time, reviewer experience, document format)

Then choose the technique or finding that best fits those factors.

Quick revision summary:

• What: a review technique where reviewers follow structured, usage-based scenarios (dry runs) through a work product.
• Why: it gives better guidance for specific defect types than checklists, finds behavioural and logical gaps early, and supports phase containment.
• How: define scenarios from expected usage, walk through the work product step by step, record deviations and gaps, and look beyond the scenarios.
• Watch out: defects outside the scenarios (especially missing features) can be missed, so reviewers should not be constrained to documented scenarios.