CRUD Consistency Testing and CRUD Coverage (ISTQB CTAL-TA Behavior-Based Test Techniques)
Overview
CRUD testing is a behavior-based test technique in the ISTQB Advanced Level Test Analyst (CTAL-TA) syllabus. CRUD stands for Create, Read, Update and Delete, the four basic operations any system can perform on persistent data entities such as Customer, Order, Invoice or Account. The technique models which parts of the system (functions, processes, use cases, user stories, screens or user roles) perform which operations on which data entities. It then derives tests that check two things. First, that this behavior is complete and consistent. Second, that the data stays correct and coherent throughout its whole life cycle.
Why It Is Important
Many serious defects in business systems are not found by testing a single function in isolation. They appear when data created in one place is read, changed or deleted somewhere else. Typical problems include:
- An entity can be created but never deleted or archived. This is a data retention or GDPR risk.
- An entity is read or updated by a process, but no process ever creates it. This is a missing function or a broken data flow.
- A record is deleted, but other entities still refer to it. These are orphaned records and referential integrity failures.
- An update in one module is not visible, or is shown wrongly, in another module.
- A user role is allowed to delete data it should only be able to read. This is an access control defect.
- Inconsistent business rules apply when the same entity is updated through different channels.
CRUD testing gives the Test Analyst a systematic, measurable way to find these problems. It works on two levels:
- Statically, by analysing the CRUD matrix. This finds gaps and anomalies in requirements or design early, which is cheap shift-left defect prevention.
- Dynamically, by executing tests that walk entities through their life cycle across different functions.
The technique is especially valuable for data-intensive systems, such as ERP, banking, insurance, CRM and e-commerce systems. It is also useful for integration and system integration testing, and for migration testing.
What It Is: The CRUD Matrix
The core work product is the CRUD matrix, also called an entity/process matrix.
- Rows are usually functions, processes, use cases or user roles.
- Columns are data entities. Some teams swap rows and columns; either layout works.
- Each cell contains the letters of the operations that the function performs on that entity, for example C, R, U, D, or combinations like CR or RU. An empty cell means no interaction.
Example matrix (rows are functions, columns are entities):
Function | Customer | Order | Invoice
Register customer | C | - | -
Place order | R | C | -
Amend order | R | RU | -
Ship order | R | RU | C
Close account | D | R | R
How It Works
Step 1: Identify entities and functions. Use requirements, data models (ERDs), use cases, user stories, process models and interface specifications as the test basis.
Step 2: Build the CRUD matrix. For each pair of function and entity, record the operations performed. Involve business analysts and developers, because building the matrix often surfaces ambiguities.
Step 3: Completeness check (static analysis of the matrix). Examine each entity column for anomalies.
- Is there at least one C? If not, where does the data come from? Perhaps an external interface or migration, which should be documented.
- Is there at least one R? Data that is never read may be unnecessary.
- Is there a U where the business requires changes?
- Is there at least one D, or a logical delete or archive? If not, how is the end of the data life cycle handled?
- Are there multiple creators or deleters? This can signal possible conflicts that need special attention.
In the example above, Invoice is never updated or deleted, and Order is never deleted. These are questions to raise with stakeholders. Such anomalies are not automatically defects, but each must be justified. This is review-based testing and is a cheap way to find requirement defects.
Step 4: Consistency testing (dynamic). Design test cases that execute sequences of operations on an entity across different functions. Each test checks that the result of one operation is correctly reflected by the others. Typical checks are:
- Create then Read: data created by function A is displayed correctly by function B.
- Update then Read: changes made in one function are visible and correct everywhere the entity is read.
- Delete then Read, Update or Delete: deleted data can no longer be read or updated, and an appropriate error appears. Related entities are handled correctly through cascade, block or nullify rules.
- Operations without a prior Create: trying to read, update or delete a non-existent entity is handled correctly.
- Concurrent or competing operations: two functions update the same entity.
- Role or permission consistency: each role can perform only the operations allowed by the matrix, and is denied the others. This is often done as negative tests.
A full life cycle test, such as C, then R, then U, then R, then D, then R, is a classic consistency test. It validates the whole history of an entity end-to-end.
Step 5: Measure CRUD coverage. Coverage shows how thoroughly the matrix has been exercised. The two common views are:
- Operation (cell) coverage: the number of CRUD operations in the matrix exercised by tests, divided by the total number of CRUD operations in the matrix, times 100 percent. Each individual C, R, U or D entry counts as one coverage item.
- Entity life cycle or sequence coverage: the proportion of relevant operation sequences per entity that have been tested, for example create-read, update-read, delete-read and full life cycles.
For example, suppose the matrix contains 12 operation entries and the tests exercise 9 of them. Operation coverage is then 9 / 12 = 75 percent. Full coverage means every entry in the matrix is exercised by at least one test. Higher thoroughness also covers the life cycle sequences and the negative operations implied by empty cells.
Relationship to Other Techniques
- CRUD testing complements state transition testing, because an entity's life cycle is itself a state model (created, active, updated, deleted).
- It supports scenario-based and use case testing, because life cycle sequences often follow business scenarios.
- It is related to data flow thinking, where definitions and uses of data correspond to creates and reads.
- It supports security testing, through access rights per role.
Strengths and Limitations
Strengths:
- Systematic and measurable.
- Finds integration and data life cycle defects.
- Useful as a static review tool.
- Easy for business stakeholders to understand.
Limitations:
- The matrix can become very large in big systems.
- It does not test detailed business rules or value domains by itself. Combine it with equivalence partitioning, boundary value analysis or decision tables.
- Its quality depends on an accurate and up-to-date data model.
Exam Tips: Answering Questions on CRUD Consistency Testing and CRUD Coverage
1. Know the acronym and the structure. CRUD means Create, Read, Update, Delete. The matrix maps functions, processes or roles against data entities. Expect questions that show a matrix and ask what it reveals.
2. Spot anomalies quickly. For each entity column, ask: Is there a C? An R? A U? A D? A missing C or D is the most common anomaly tested. The correct answer is usually to raise it as a question or potential defect, not to assume an error automatically.
3. Distinguish completeness from consistency. Completeness is static analysis of the matrix: are all needed operations present? Consistency is dynamic testing: do sequences of operations across different functions give coherent results? Options that confuse the two are distractors.
4. Calculate coverage carefully. Count each individual operation letter in each cell as a coverage item. A cell with RU counts as two items. Divide the operations exercised by the total and express the result as a percentage. Watch for traps where an option counts cells instead of operations, or counts empty cells.
5. Choose the minimum set of tests. In questions asking for the fewest test cases to reach full coverage, remember that one life cycle test can cover several operations at once, for example creating in function A, reading in B and deleting in C. Look for tests that combine operations efficiently.
6. Remember the negative tests. Reading, updating or deleting after a Delete, and unauthorized operations implied by empty cells, are typical consistency tests. Options mentioning these are often correct when the question asks about thoroughness or robustness.
7. Link the technique to the right context. CRUD testing is most suitable for data-intensive systems, integration testing and verifying data life cycles. If a scenario describes data passing between modules, or deletion side effects, CRUD testing is likely the intended answer.
8. Know the test basis. Typical sources are data models, ERDs, use cases, process descriptions and access rights specifications.
9. Watch for absolute wording. Statements such as 'CRUD testing finds all integration defects' or 'every entity must be deleted physically' are usually wrong. Logical deletion or archiving can be valid.
10. Apply K3 thinking. At the Advanced level, you must apply the technique, not just recall it. Practise drawing a small matrix, identifying gaps, writing life cycle sequences and computing coverage under time pressure.
