Back to PMI Agile Certified Practitioner (PMI-ACP)

A/B Testing

Technique for comparing two versions of a feature or product.

A/B Testing is an essential Agile practice which involves comparing two versions of a feature or product, a version A and a version B, with live users to determine which one performs better. It is an efficient way of testing new ideas and improvements, making data-informed decisions and continually learning about user behaviors and preferences.
5 minutes 5 Questions

A/B Testing is a statistical method used in Agile environments to compare two versions of a product feature, webpage, or application to determine which performs better. It's a data-driven approach to decision-making that helps teams optimize user experience and business outcomes. In an A/B test, u…

Concepts covered: Hypothesis Setting, Multivariate Testing, Validity Threats, Variable Selection, Randomized Division of Audience, Control Groups, Statistical Significance, Follow-up Experiments, Interpretation and Implementation, Measurement

Test mode:
Start practice test
PMI-ACP - A/B Testing Example Questions

Test your knowledge of A/B Testing

Question 1

Your agile team has developed an application for a client. The beta testers from the client's side highlight that the app lacks certain features they expected. What validity threat is posed in this scenario?

Correct Answer: Construct validity threat due to miscommunication or misunderstanding of client's requirements

The correct answer is 'Construct validity threat due to miscommunication or misunderstanding of client's requirements'. This scenario represents a communication gap between the team and the client, leading to unmet expectations. This is a construct validity threat where the design and execution of the software don't align with the needs of the client. On the other hand, 'External validity threat due to tester's personal preferences', 'Internal validity threat due to testing tools used', and 'Information bias as feedback is not accepted' are incorrect. External validity threats are about generalizing the findings beyond the scope of the test, which doesn't apply here. Internal validity threats would suggest measurement errors due to tools or procedures, but here the issue is about misaligned requirements. Information bias does not apply as there is no hint in the question text that feedback is being rejected or ignored.

Question 2

An Agile team is preparing their product for the first release. They are trying to decide on a measurement that reflects the completeness and readiness of the product. Which measurement would be the best solution?

Correct Answer: Definition of Done

The best measure of the completeness and readiness of a product for release in Agile is the 'Definition of Done'. The 'Definition of Done' provides a checklist that items must meet to be considered done. It ensures that all team members have the same shared understanding of what it means for work to be complete. It could include various parameters like coding completed, testing done, documentation finished etc. The other answers are not the correct measures of a product's readiness. 'Story Points' measure the size of a user story but do not indicate completeness of the product. 'Product Backlog' is a list of features or technical tasks which the team maintains and which, in total, represent the work to be done. But having a backlog does not necessarily mean product is ready. 'Velocity' is a measure of the amount of work a Team can tackle during a single Sprint and is the key metric in Scrum, yet it does not directly determine the completeness or the readiness of the product for release.

Question 3

Your Agile team is about to finish a sprint, and some members are concerned the team's velocity is not accurately reflecting the work completed. The team member uses a few charts to represent the progress, but one is suggested as the primary measurement moving forward. Which one could it be?

Correct Answer: Burn-up Chart

In Agile methodologies, a Burn-up Chart is recognized as a better indicator of team's velocity as it measures the amount of work completed over time. This chart will effectively represent their progress, showing both scope changes and the completed work, providing a better visualization. The others like Histogram, Pie Chart and Scatter Plot may provide some insight, but they typically do not provide an ongoing, long-term view of progress and cannot represent the work completed along with the changes in scope, making them less effective in reflecting the team's velocity.

More A/B Testing questions
106 questions (total)
Start 100 question test