Writing Checklist Items as Yes or No Questions (ISTQB CTAL-TA: Experience-Based Testing)
Overview
Checklist-based testing is one of the experience-based test techniques covered in the ISTQB Advanced Level Test Analyst (CTAL-TA) syllabus. A checklist captures what experienced testers know about where software tends to fail and what matters to users. How each checklist item is worded largely determines how useful the checklist is. The ISTQB guidance is to phrase checklist items as questions, ideally ones answered with a clear yes or no, and each item should be checkable separately and directly. This guide explains what that means, why it matters, how to apply it, and how to handle exam questions on the topic.
1. What It Is
A checklist item written as a yes or no question is a single, specific, verifiable statement framed so that the tester can give a binary answer after observing the test object.
Weak item: Error handling
Weak item: Check that the login page works well and looks good
Strong item: Does the system display an error message when an incorrect password is entered?
Strong item: Is the account locked after five consecutive failed login attempts?
The strong items name one observable condition, can be answered without guesswork, and produce a result that can be recorded and compared.
2. Why It Is Important
Unambiguous results: A yes or no answer removes the interpretation that vague items such as 'usability is acceptable' invite. A 'no' signals a potential defect or deviation.
Consistency between testers: Different testers using the same checklist should reach the same answer. This makes experience-based testing more repeatable.
Measurable coverage: Because every item is a discrete question, coverage can be expressed as the proportion of items checked, and results can be reported clearly.
Faster execution: Testers spend time testing, not working out what an item means.
Easier defect reporting: A 'no' answer points directly to the condition that failed, which supports clear defect reports.
Easier maintenance: Atomic questions can be added, removed or updated individually as the product and the defect history change.
Usable in reviews too: The same principle applies to review checklists (for example, 'Does each requirement have a unique identifier?'), which supports checklist-based reviewing.
3. How It Works: Principles for Writing Good Items
a) One condition per item (atomic): Avoid compound questions joined with 'and' or 'or'. 'Are mandatory fields marked and validated?' should become two items. If one part passes and the other fails, a compound item cannot be answered with yes or no.
b) Checkable separately and directly: The tester should be able to verify the item on its own, by observation or a simple action, without depending on the result of another item.
c) Specific and observable: Replace subjective words such as 'good', 'fast', 'user-friendly', 'appropriate' or 'correct' with observable criteria. Instead of 'Is the response fast?', ask 'Is the search result displayed within 2 seconds?' where a threshold exists.
d) Consistent polarity: Phrase items so that 'yes' always means the desired outcome and 'no' indicates a problem. Mixing positive and negative wording ('Is there no truncation?') causes mistakes.
e) Right level of detail: Items should be neither too general (they give no guidance) nor too detailed (the checklist becomes a set of scripted test cases and loses flexibility). High-level checklists leave room for tester variation and creativity, while low-level items give more repeatability.
f) Exclude items that do not belong: Items that can be checked automatically, items better suited as entry or exit criteria, and items too general to act on should not be included.
g) Base items on experience and risk: Sources include defect history, common failure types, standards, quality characteristics (for example usability or security), user priorities and knowledge of why software fails.
h) Keep it manageable and up to date: Long checklists are hard to use. Review them regularly, add items for newly found defect types, and remove items that no longer find problems. Developers often learn to avoid the mistakes a checklist targets, so it can become less effective over time.
4. Worked Transformation Example
Original item: Verify the registration form handles input properly and is user friendly.
Problems: compound, vague ('properly', 'user friendly'), not directly checkable.
Rewritten as yes or no questions:
1. Is an error message displayed when the email field contains no @ symbol?
2. Does the form prevent submission when a mandatory field is empty?
3. Is each mandatory field visually marked?
4. Does the cursor move to the first invalid field after a failed submission?
5. Is entered data kept in the form after a validation error?
5. Benefits and Limitations
Benefits: structure for experience-based testing, consistency, reportable coverage, quick to use, good for reviews and for testing without detailed specifications.
Limitations: a checklist only finds what it asks about. Defects outside the items can be missed, very rigid items reduce the exploratory value, and answers still depend on tester skill. Checklists are often combined with exploratory testing or used alongside more formal techniques.
6. Exam Tips: Answering Questions on Writing Checklist Items as Yes or No Questions
Tip 1: Look for the atomic item. When asked which checklist item is best written, prefer the option that tests exactly one condition. Reject answers containing 'and', 'or' or several conditions in one sentence.
Tip 2: Reject vague adjectives. Options with 'good', 'appropriate', 'adequate', 'user friendly', 'properly' or 'correctly' without criteria are usually distractors.
Tip 3: Prefer the question form with a binary answer. 'Does X happen when Y?' beats 'Check X' or 'Test the Y function'. If the expected answer could be 'it depends' or 'partly', the item is poorly written.
Tip 4: Check polarity. The best answer usually makes 'yes' mean compliance. Be careful with double negatives in options.
Tip 5: Recognize what should not be in a checklist. Items that can be automated, items that are really entry or exit criteria, or items too general are typical wrong answers in 'which item should be removed or improved' questions.
Tip 6: Know the detail trade-off. Too general means little guidance. Too detailed means it behaves like scripted test cases and loses tester variation. Expect questions on this balance and on how high-level checklists increase coverage variation between testers.
Tip 7: Scenario questions. If given a poor checklist and asked for the best improvement, choose the answer that splits compound items, adds measurable criteria, or rephrases statements as direct questions. Avoid answers that simply add more items without fixing wording.
Tip 8: Remember the purpose. If asked why items are written as yes or no questions, the key words are: unambiguous, consistent, directly verifiable, measurable coverage and easy reporting.
Tip 9: Read every option fully. Distractors often look correct at first but contain a hidden second condition or a subjective word near the end.
Tip 10: Link to maintenance. Questions may ask how to keep checklists effective: update based on defect analysis, remove obsolete items, and keep the list a manageable length.
7. Quick Summary
Write each checklist item as a single, specific, observable question that can be answered yes or no, where yes means the desired behavior. Keep items checkable separately and directly, at the right level of detail, based on experience and risk, and maintained over time. In the exam, choose the atomic, measurable, unambiguous option and reject compound, vague or automatable items.
