Impact Analysis for Regression Test Selection

Impact Analysis for Regression Test Selection (ISTQB CTAL-TA: Test Analyst in Risk-Based Testing)

Introduction
Impact analysis for regression test selection is a core skill for the ISTQB Advanced Level Test Analyst (CTAL-TA). It sits at the meeting point of risk-based testing, maintenance testing and test analysis. When software changes, the Test Analyst must decide which existing tests to re-run, which to update, and which new tests are needed. Impact analysis supplies the evidence for that decision. In practice it is the bridge between what changed and what we must test.

Why It Is Important
Running every test after every change is rarely possible, especially in agile, CI/CD and maintenance settings. Running too few tests lets regressions escape into production. Impact analysis helps in the following ways:
- It reduces risk efficiently. Testing effort goes to the areas most likely to break, or most costly if they break.
- It optimises time and cost. Unaffected areas are not tested at full depth, so release cycles stay short.
- It supports informed decisions. Stakeholders get a clear rationale for what was and was not retested, which aids release decisions and test reporting.
- It underpins maintenance testing. The ISTQB syllabi stress that maintenance testing (after modifications, migrations or retirement) relies heavily on impact analysis to set its scope.
- It improves regression suite management. It shows which test cases have become obsolete, which need updating and which gaps exist.
- It feeds product risk analysis. Changed and affected areas raise the likelihood of failure, so risk levels and test priorities should be updated.

What It Is
Impact analysis is the identification of all work products affected by a change, including the estimated resources needed to accomplish the change. In the ISTQB Glossary it is the assessment of change to the layers of development documentation, test documentation and components, in order to implement a given change to specified requirements.

For regression test selection, impact analysis answers four questions:
1. What has changed? This covers code, requirements, configuration, data, environment and third-party components.
2. What could be affected directly or indirectly? This covers dependent modules, interfaces, data flows, shared components and business processes.
3. How risky is each affected area? This combines likelihood of failure with business impact.
4. Which tests cover those areas? The answer drives selection, prioritisation and identification of gaps.

The result is a targeted regression test set made up of:
- Confirmation tests (re-tests) that verify the defect fix or change works.
- Regression tests that verify unchanged but potentially affected areas still behave correctly.
- New or updated tests for new or modified functionality.

How It Works: Step by Step

Step 1: Identify the change.
Collect the change details from change requests, defect reports, user stories, release notes, code commits or version control diffs. Clarify the scope: is it a bug fix, an enhancement, a refactoring, a platform upgrade or a configuration change?

Step 2: Use traceability to find affected items.
Bidirectional traceability is the Test Analyst's main tool here. It links requirements to design to code to test conditions to test cases. If requirement R7 changes, traceability shows every test condition and test case linked to R7. Poor traceability makes impact analysis weak and slow, and this is a common exam point.

Step 3: Analyse dependencies and ripple effects.
Look beyond the directly changed item. Consider:
- Calling and called modules (call graphs).
- Shared data (databases, files, global variables).
- Interfaces and integrations, including APIs and external systems.
- Business process flows that pass through the changed component.
- Non-functional characteristics such as performance, security and usability, which can be affected even when functionality is not.

Developers and architects often supply code-level dependency information. The Test Analyst adds a business and domain perspective, for example: which user workflows depend on this?

Step 4: Assess risk for affected areas.
Apply risk-based testing principles:
- Likelihood rises with change complexity, the number of dependencies, defect history (defect clustering) and developer unfamiliarity with the code.
- Impact depends on business criticality, number of users, financial, legal or safety consequences, and visibility.
Higher risk means more tests and earlier execution.

Step 5: Select and prioritise regression tests.
Common strategies include:
- Retest-all. This is safest but most expensive. It is used when impact cannot be determined or the risk is very high, for example in safety-critical systems or after major changes.
- Selective (modification-based) regression. Choose tests that cover changed and affected areas.
- Risk-based prioritisation. Order tests by the risk level of what they cover.
- Coverage-based selection. Choose tests that exercise changed code or requirements, sometimes supported by code coverage tools.
- A core or smoke regression subset. Always run a small set of tests for critical business functions, whatever the change.

Step 6: Identify gaps and maintain the testware.
- Add new tests where the change introduces behaviour not yet covered.
- Update tests whose expected results changed.
- Retire obsolete tests.

Step 7: Execute, monitor and refine.
Run the selected tests and analyse the results. If unexpected failures appear in areas judged unaffected, the impact analysis was incomplete. Feed that lesson back into the risk analysis and the traceability records.

Factors That Make Impact Analysis Difficult
The syllabi commonly cite these:
- Specifications such as requirements and design documents are missing, outdated or of poor quality.
- Test cases are not documented, or are poorly documented.
- Bidirectional traceability between tests and the test basis is missing or poor.
- Tool support is weak or absent.
- The people involved lack domain or system knowledge.
- Maintainability was given too little attention during development.
- Architectures are highly coupled.

Worked Example
An online banking application changes its interest calculation algorithm for savings accounts.
- Directly changed: the interest calculation module.
- Affected via traceability: requirements REQ-INT-01 to REQ-INT-04 and test cases TC-120 to TC-138.
- Ripple effects: monthly statements, annual tax reports, account balance display, the batch overnight job, and the data export to the regulator.
- Risk assessment: regulatory reporting has high impact (legal penalties). Statements have high visibility. Balance display is medium risk.
- Selection: run confirmation tests on the new algorithm, plus regression tests on statements, tax reports and the regulatory export, which get the highest priority. Add boundary tests for new interest bands and a core smoke suite for login and transfers. Login UI cosmetics, which are unaffected, are deprioritised.

Relationship to Other CTAL-TA Topics
- Risk-based testing: impact analysis updates risk likelihood after changes.
- Test techniques: new tests may use equivalence partitioning, boundary value analysis or decision tables.
- Maintenance testing: impact analysis defines its scope.
- Test tools: test management tools hold traceability, and automation makes broad regression affordable.
- Agile: continuous impact analysis happens during refinement and sprint planning.

Exam Tips: Answering Questions on Impact Analysis for Regression Test Selection

1. Know the definition by heart.
Expect questions asking what impact analysis is or what its purpose is. The answer centres on identifying the work products affected by a change, which then determines the scope of regression testing.

2. Traceability is almost always the key enabler.
If a question asks what most supports effective impact analysis, look for bidirectional traceability between the test basis and test cases. Missing traceability is the classic obstacle.

3. Distinguish confirmation testing from regression testing.
- Confirmation testing (re-testing) checks that the fix works.
- Regression testing checks that nothing else broke.
Distractors often swap these two terms.

4. In scenario questions, look for ripple effects.
The best answer usually includes indirectly affected areas such as interfaces, shared data and downstream reports, not just the changed module. Answers that test only the changed component are usually incomplete.

5. Combine impact with risk.
When asked to prioritise regression tests, choose the option that considers both what is affected and how critical or likely to fail it is. Pure technical coverage without business risk is usually not the best TA answer.

6. Recognise when retest-all is justified.
Retest-all fits when impact analysis is not possible (no documentation, no traceability), when the change is very large, or when the system is safety-critical. Otherwise selective, risk-based regression is preferred.

7. Watch for 'always' and 'never'.
Options such as always run the full regression suite or only test changed code are typically wrong because they ignore context and risk.

8. Remember the Test Analyst's angle.
The CTAL-TA focuses on business and domain aspects: requirements, user workflows, data and functional and usability characteristics. The Technical Test Analyst covers code-level analysis such as call graphs and control-flow coverage. If the question asks about the Test Analyst's contribution, choose business- and requirements-oriented actions.

9. Remember that impact analysis also updates the test basis and testware.
Correct answers may mention updating, adding or retiring test cases, not only selecting them.

10. Read for the trigger.
Maintenance triggers include modification, migration and retirement. Data migration requires regression plus data conversion testing. Retirement may require testing data archiving and retrieval. Select tests that match the trigger.

11. Use elimination.
For K3 and K4 scenario questions, eliminate options that:
- ignore affected interfaces,
- ignore high-risk areas,
- waste effort on clearly unaffected low-risk areas, or
- skip confirmation testing of the actual change.

12. Expect multi-step reasoning.
A typical question gives a change, a traceability matrix and risk levels, then asks which test cases to run first. Work through the matrix systematically:
1. Mark the directly linked tests.
2. Add the tests linked to dependent requirements.
3. Sort the tests by risk level.
4. Check that the answer includes both confirmation and regression tests.

Summary
Impact analysis for regression test selection works through these steps:
1. Identify what changed.
2. Trace it to everything it might affect.
3. Weigh those areas by risk.
4. Select and prioritise the right confirmation, regression and new tests.
It depends heavily on good documentation and traceability, and it aligns closely with risk-based testing. For the exam, remember that traceability enables it, ripple effects and business risk must be considered, and selective risk-based regression is preferred unless impact cannot be determined or the risk demands retest-all.