Learn Testing Quality Characteristics (CTAL-TA) with Interactive Flashcards

Master key concepts in Testing Quality Characteristics through our interactive flashcard system. Click on each card to reveal detailed explanations and enhance your understanding.

Functional Completeness Testing

In the ISTQB Certified Tester Advanced Level Test Analyst syllabus, functional completeness is one of three sub-characteristics of functional suitability, as defined by ISO/IEC 25010. The other two are functional correctness and functional appropriateness. Functional completeness is the degree to which the set of functions covers all the specified tasks and user objectives. Functional completeness testing therefore asks whether everything the system is supposed to do has actually been implemented. It does not primarily ask whether each function works correctly, which is the focus of correctness testing. The Test Analyst is responsible for this testing because it relies on business and domain knowledge. The main approach is to check implemented functionality against the test basis, such as requirements specifications, user stories, use cases, acceptance criteria and business process models. Bidirectional traceability is central. Each requirement or user objective is linked to test conditions and test cases, so the analyst can spot specified functions that are missing, only partly implemented, or never tested. A requirements traceability matrix or a test management tool often supports this analysis, and coverage metrics show how much of the specified functionality has been exercised. Black-box techniques such as use case testing and scenario-based testing are especially useful. They let the analyst confirm that complete end-to-end tasks can be carried out, rather than only checking isolated features. Reviews of the test basis can also find incompleteness early, before any code exists. Completeness testing can take place at every test level, but it is most prominent during system testing and acceptance testing. At those levels, stakeholders confirm that the delivered product lets users achieve their goals. Gaps found during this testing may point to missed requirements, misunderstood user needs or scope creep, and they are reported as defects or change requests. Effective completeness testing gives stakeholders confidence that the software delivers the full scope agreed. It also lowers the risk of releasing a product that cannot support essential business activities.

Functional Correctness Testing

In the ISTQB Certified Tester Advanced Level Test Analyst syllabus, Functional Correctness Testing is one of the main ways to evaluate the quality characteristic of functional suitability, as defined in ISO/IEC 25010. Functional suitability has three sub-characteristics: functional correctness, functional appropriateness, and functional completeness. Functional correctness is the degree to which a product or system gives correct results with the needed degree of precision. Functional correctness testing checks that the software behaves as specified or as reasonably expected. This means producing the right outputs, performing calculations accurately, and handling data properly for valid and invalid inputs. The Test Analyst is mainly responsible for this testing because it relies on business domain knowledge and on understanding what the system should do from the user's and stakeholders' points of view. The Test Analyst uses requirements, user stories, specifications, business rules, and other test basis items to find test conditions and design test cases. The expected results come from a test oracle. A test oracle can be a specification, an existing legacy system, a domain expert, or calculated reference values. Without a reliable oracle, it is hard to decide whether an actual result is correct. Black-box test techniques are commonly used to design correctness tests. These include equivalence partitioning, boundary value analysis, decision table testing, state transition testing, and use case testing. Experience-based techniques, such as exploratory testing and error guessing, add to these techniques and help reveal defects the specifications may miss. Precision is especially important when testing calculations, such as rounding in financial or scientific applications. Functional correctness testing can take place at all test levels, from component testing through system and acceptance testing, with a different focus at each level. Its goal is to give stakeholders confidence that the system delivers accurate, trustworthy results. It also helps find defects early, lowering the risk of incorrect business outcomes after release.

Functional Appropriateness Testing

In the ISTQB Advanced Level Test Analyst syllabus, functional appropriateness is one of three sub-characteristics of functional suitability, as defined in ISO/IEC 25010. The other two are functional correctness and functional completeness. Functional appropriateness is the degree to which the system's functions help users accomplish their specified tasks and objectives. Functional appropriateness testing evaluates and validates whether a set of functions is suitable for its intended tasks. The key question is not just whether a function works, but whether it helps users reach their goals in a sensible and efficient way.

The Test Analyst usually bases this testing on the functional design, such as use cases, user stories, business process models and workflow descriptions. These sources show what users are trying to achieve, so the analyst can design scenarios that represent realistic end-to-end tasks. Suitable techniques include use case testing, scenario-based testing, exploratory testing and checklist-based reviews. Knowledge of the business domain helps the analyst judge what users actually need.

This testing is normally conducted during system testing. It may also occur in the later stages of integration testing and in acceptance testing, where real users or their representatives take part.

Defects found here show that the system will not meet users' needs in an acceptable way, even if each function is technically correct. Typical examples include:
- unnecessary steps that make a task cumbersome;
- missing shortcuts or support for common workflows;
- functions that produce output in a form users cannot easily use;
- features that do not match how the business actually operates.

Functional appropriateness overlaps with usability. It also differs clearly from functional correctness: a function can calculate a result accurately and still be inappropriate if it does not support the user's real task.

Test Analysts should involve stakeholders early, for example through reviews of requirements and prototypes. This helps uncover appropriateness problems before implementation, when they are cheaper to fix. Overall, functional appropriateness testing helps ensure the delivered product provides real value to users.

Usability Aspects: Interaction Capability, User Experience and Accessibility

In the ISTQB Advanced Level Test Analyst syllabus (v4.0), usability is treated as a broad quality area with three related aspects. Interaction capability is the ISO/IEC 25010:2023 term that replaces the older characteristic called usability. It describes how well specified users can interact with a product to exchange information and achieve their goals. Its sub-characteristics include appropriateness recognizability (users can tell whether the product suits their needs), learnability, operability, user error protection, user engagement (formerly user interface aesthetics), inclusivity, user assistance and self-descriptiveness. The Test Analyst checks these through reviews of requirements and UI prototypes, checklist-based and heuristic evaluations (for example Nielsen's heuristics), and usability tests in which representative users perform realistic tasks. During these tests, effectiveness, efficiency and satisfaction are measured, often with think-aloud protocols, task completion rates and error counts. User experience (UX) is wider than interaction capability. ISO 9241-210 defines it as a person's perceptions and responses resulting from the use or anticipated use of a product. UX covers emotions, beliefs, preferences, comfort and behaviour before, during and after use, and it is shaped by brand image, presentation, functionality, performance and support. Because UX is subjective, the Test Analyst relies on interviews, observation, A/B testing, analytics and standardized questionnaires such as SUS, SUMI or WAMMI to collect evidence about user satisfaction. Accessibility is the degree to which a product can be used by people with the widest range of characteristics and capabilities, including visual, auditory, motor and cognitive impairments, as well as age-related limitations. It is often driven by legal and regulatory obligations, for example WCAG guidelines, the European Accessibility Act or US Section 508. Accessibility testing combines automated scanners, manual checks of keyboard navigation, colour contrast and alternative text, and tests with assistive technologies such as screen readers. For all three aspects, the Test Analyst defines user profiles and personas, selects suitable techniques, involves real users where possible, and reports findings so that the product meets stakeholder expectations.

Test Analyst Contribution to Usability Reviews and Usability Test Sessions

In the ISTQB Advanced Level Test Analyst syllabus, usability is the degree to which users can achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use. Usability experts often lead this work, but the Test Analyst makes important contributions to both usability reviews and usability test sessions.

Usability reviews: The Test Analyst takes part in reviews early in the lifecycle, when defects are cheapest to fix. Typical work products include requirements, user stories, use cases, wireframes, mock-ups, prototypes and user interface specifications. The Test Analyst checks that usability requirements are clear and testable, such as measurable task completion times or error rates. They also check that user profiles, personas and context of use are defined. A key technique is heuristic evaluation, where the interface is systematically inspected against recognized usability principles. Examples include consistency, visibility of system status, error prevention, clear feedback, recognition rather than recall, and accessibility guidelines. Checklists and the Test Analyst's knowledge of user behaviour and the business domain help find navigation problems, confusing terminology and inconsistent layouts before code is written.

Usability test sessions: The Test Analyst helps plan and run sessions with representative users. Their contributions include:
- Defining usability test conditions and realistic task-based test scenarios that reflect real user goals.
- Preparing test environments and test data.
- Helping select participants who match the target user groups.
- Supporting moderation and observation, including think-aloud protocols.
- Recording observations such as task success, time taken, errors, hesitations and user comments.
- Using post-session questionnaires and standardized surveys such as SUS, SUMI or WAMMI to measure user satisfaction.
- Analysing results, classifying usability defects by severity, and reporting findings with clear evidence and improvement recommendations.

Test Analysts may support both formative evaluation, which improves the design during development, and summative evaluation, which checks whether usability requirements have been met. Through these activities, the Test Analyst helps ensure the product is easy to learn, efficient to use, accessible, and satisfying for its intended users.

User Questionnaires and Accessibility Conformance Levels

In the ISTQB Advanced Level Test Analyst syllabus, user questionnaires and accessibility conformance levels belong to usability and accessibility testing. Both help the Test Analyst judge how well a product serves its real users. User questionnaires are a way to measure user satisfaction, which is one aspect of usability alongside effectiveness and efficiency. After users finish representative tasks, for example in a usability test session, they complete a questionnaire about their experience. Standardized, publicly available questionnaires are preferred because they have been validated and give consistent, comparable results. Well-known examples include SUMI (Software Usability Measurement Inventory), WAMMI (Website Analysis and MeasureMent Inventory) and SUS (System Usability Scale). SUMI and WAMMI let teams compare their results against a database of earlier measurements, so a product can be benchmarked against industry norms or against its own previous releases. Questionnaires mainly provide subjective data about attitudes, perceived ease of use and satisfaction. They work best alongside objective measures such as task completion rates, error counts and time on task. The Test Analyst may select suitable questionnaires, help administer them and interpret the results to identify usability defects and improvement priorities. Accessibility conformance levels describe how far a product meets accessibility standards for people with disabilities, such as visual, auditory, motor or cognitive impairments. The most widely referenced standard is the W3C Web Content Accessibility Guidelines (WCAG). WCAG is built on four principles: perceivable, operable, understandable and robust. It defines three conformance levels. Level A is the minimum and covers essential barriers, such as providing text alternatives for images. Level AA is the commonly required target and is often written into legislation, covering items such as sufficient color contrast. Level AAA is the highest and strictest level, and it is not always achievable for all content. Legal and regulatory requirements, such as Section 508 in the USA or EN 301 549 in Europe, often specify the required level. The Test Analyst uses these levels to define test conditions, choose checklists and tools, involve users with disabilities, and report conformance status.

Adaptability Testing

In the ISTQB Advanced Level Test Analyst syllabus, adaptability testing is part of portability testing, together with installability and replaceability testing. Following ISO/IEC 25010, adaptability is the degree to which a product or system can be effectively and efficiently adapted for different or evolving hardware, software, or other operational or usage environments. Adaptability testing checks whether an application functions correctly in all of its intended target environments.

The target environments are typically defined in requirements or a product compatibility matrix. They may vary by hardware platform, operating system and version, browser, database, middleware, screen resolution or device type, network configuration, language and locale settings, and cloud or virtualised infrastructure. The Test Analyst identifies these environments and their combinations, then derives tests that exercise representative functionality in each one. Because the number of possible combinations can be very large, techniques such as pairwise or other combinatorial testing, equivalence partitioning of environment types, and risk-based prioritisation are used to select a manageable but meaningful subset.

Adaptability can also mean the ability to move software to a specified environment by following a predefined procedure, such as a migration, configuration, or porting process. Tests may then evaluate that procedure for correctness, completeness, clarity of documentation, and the effort required.

Adaptability tests are often performed together with installability tests: the software is first installed on a target environment, then functional and non-functional tests confirm that it behaves as expected. Typical defects include platform-specific failures, layout and rendering problems, missing dependencies, incorrect handling of locale-specific data, and performance degradation on certain configurations.

The Test Analyst contributes by analysing stakeholder needs, defining environment coverage criteria, designing test conditions and test cases, and helping specify the test environments needed. Close cooperation with technical test analysts and infrastructure teams is important because building and maintaining many environments can be costly. Virtualisation, emulators, device farms, and containers can reduce this cost while maintaining adequate coverage.

Installability Testing

In the ISTQB Advanced Level syllabi, installability is a sub-characteristic of portability, following the ISO/IEC 25010 quality model. ISO/IEC 25010 defines it as the degree of effectiveness and efficiency with which a product or system can be successfully installed and/or uninstalled in a specified environment. In recent syllabus versions, portability testing is covered mainly in the Technical Test Analyst syllabus. Test Analysts still often contribute, especially by evaluating installation procedures from the user's perspective and verifying that functionality works after installation. Installability testing checks the software and the procedures used to install it on its target environments, such as operating systems, hardware, browsers, databases or cloud platforms. Typical objectives include: (1) confirming that the software installs successfully by following the installation guide, wizard or automated script, under default and custom options; (2) verifying correct behaviour when installation fails partway through, for example after a power loss, a full disk or missing permissions, so that the system rolls back cleanly or reports clear errors; (3) testing partial, typical and complete installations, upgrades, patches and downgrades; (4) checking that uninstallation removes all components without damaging other applications or leaving residual files, registry entries or services; (5) confirming that prerequisites, such as libraries, runtimes or configuration settings, are detected and handled; (6) measuring the time and number of steps installation takes against stated requirements; and (7) confirming that the installed system works correctly, usually through smoke or regression tests. Installation documentation and wizards should also be evaluated for clarity and correctness, which overlaps with usability testing. Useful techniques include checklists, configuration or combinatorial testing across supported environments, and scenario-based tests derived from installation requirements. Virtual machines and containers help testers create clean, repeatable environments. Common defects include incomplete or corrupt installations, unclear messages, overwritten shared components, failed upgrades that lose user data, and incomplete removal. Effective installability testing reduces support costs, protects user satisfaction and supports smooth deployment in production.

Interoperability Testing and Data Exchange

In the ISTQB Advanced Level Test Analyst syllabus, interoperability is treated as a sub-characteristic of compatibility in the ISO/IEC 25010 quality model. ISO 25010 defines it as the degree to which two or more systems, products or components can exchange information and use the information that has been exchanged. Interoperability testing therefore checks two things: that data is transferred correctly between systems, and that the receiving system interprets and processes that data as intended. Data exchange is the core of this testing. The Test Analyst first identifies all interfaces and communication points. These include APIs, file transfers, message queues, databases, middleware and external services. The Test Analyst then analyses the data formats, protocols and standards involved, such as XML, JSON, EDI, HL7, SWIFT or proprietary formats. From this analysis, test conditions are derived to verify several properties. Mapping: data structures and fields map correctly between systems. Encoding: character sets, units and date formats are handled consistently. Robustness: mandatory, optional and invalid data elements are processed appropriately. Transactions: transactional integrity is maintained. Error handling: problems such as rejected messages, timeouts or corrupt data are dealt with gracefully. Test techniques such as equivalence partitioning, boundary value analysis, decision tables and use case or end-to-end scenario testing help to cover data variations and cross-system workflows. A key challenge is the test environment. It should represent the target configurations as closely as possible, including different operating systems, versions, browsers, partner systems and network conditions. Where real systems are unavailable, simulators, stubs or service virtualization may be used, with their limitations understood. Interoperability testing is especially important for several kinds of systems: systems of systems, integrations involving commercial off-the-shelf software, cloud and microservice architectures, and any environment where components are frequently updated. The Test Analyst contributes by specifying interface-related requirements, reviewing interface specifications, designing data-driven tests and reporting risks. Typical risks include incompatible versions or misinterpreted standards, which could cause silent data corruption or failed business processes in production.

Defects and Test Techniques in Interoperability Testing

In the ISTQB Advanced Level Test Analyst syllabus, interoperability is treated as a sub-characteristic of compatibility under ISO/IEC 25010. It is the degree to which two or more systems or components can exchange information and use the information that has been exchanged. Interoperability testing checks that the software under test works correctly with the other systems, components, platforms and interfaces it must interact with in its intended environments.

Typical defects found by interoperability testing are those that prevent correct data exchange. Examples include incorrect data formats or structures, such as mismatched XML or JSON schemas, wrong field lengths or wrong data types. Encoding, character set, date, time and number-format inconsistencies are also common. Other defects arise from protocol or interface mismatches, such as wrong API versions, unsupported message types, or incorrect sequencing and handshaking. Data may be lost, truncated or misinterpreted when it is passed between systems. Problems can also come from deviations from industry standards or published interface specifications, or from incompatibilities with particular operating systems, browsers, databases, middleware or third-party products. Many of these defects stay hidden until components built by different teams or vendors are integrated.

For test techniques, the Test Analyst mainly applies black-box techniques with a focus on data exchange. Equivalence partitioning and boundary value analysis are applied to interface parameters and message contents. Decision tables and state transition testing model interaction rules and protocol states. Use case and scenario-based tests cover end-to-end flows that span several systems. Because the number of possible environment configurations can be very large, combinatorial techniques such as pairwise testing help select a manageable, representative set of configurations. Selection is based on risk, market usage and stakeholder priorities. Conformance to relevant standards and interface specifications should be verified. Tests should run in representative or production-like environments, which may use simulators, stubs or service virtualization where real partner systems are unavailable. Interoperability testing is typically performed during system integration testing and is supported by reviews of interface specifications early in the lifecycle.

More Testing Quality Characteristics questions
282 questions (total)
Practice questions
One session at a time, always new questions