Back to A/B Testing

Hypothesis Setting

5 minutes 5 Questions

In A/B testing, the first step is to set a hypothesis. This involves identifying a problem or opportunity in a product or service and making an educated guess about the best solution. The hypothesis outlines the expected outcome of the change that your testing, serving as a big picture goal for the A/B test. For example, if a webpage has a high bounce rate, the hypothesis might be that changing the page design will lower that rate. After the test, the hypothesis proves to be either true or false, making it a crucial concept in the A/B testing process.

Guide: Understanding and Answering Questions on Hypothesis Setting in A/B Testing for PMI-ACP Exam

Hypothesis setting is an essential step in the A/B testing process, aiming to make informed decisions and improve outcomes based on reliable data. It involves formulating a clear, testable prediction about the anticipated impact of a specific change.
It operates on the notion of a null hypothesis and an alternative hypothesis. The null hypothesis states no effect, and the alternative anticipates some effect.
To answer exam questions on this topic:

1. Understand the Concepts: Know the definitions, concept of null and alternative hypothesis, and their roles in A/B testing.
2. Frame Hypothesis Correctly: The exam may ask you to identify or create correct hypothesis settings, so practice writing these and understanding how changes in a scenario might change your hypothesis.
3. Contextual Understanding: Understand how hypothesis setting fits within the broader context of Agile Project Management and how it supports informed decision-making.
4. Practical Application: Be able to analyse a given scenario and correctly apply your knowledge of A/B testing and hypothesis setting.
Always read the question carefully to ensure you're addressing exactly what's being asked.

Test mode:
Start practice test
PMI-ACP - A/B Testing Example Questions

Test your knowledge of Amazon Simple Storage Service (S3)

Question 1

You, as an Agile practitioner, hypothesize that providing a voice-assistant feature will simplify the use of your application and increase user adoption rates. What should your Null Hypothesis be in this scenario?

Correct Answer: The inclusion of a voice-assistant feature will not have an impact on the user adoption rates.

A Null Hypothesis in scientific and data-driven scenarios is a general statement or the default position that there is no relationship between two measured phenomena. In this case, it would imply that the inclusion of a voice-assistant feature will not affect the user adoption rates. This claim can then be tested using data. If the testing disproves the null hypothesis, then the original hypothesis (that the voice-assistant feature will simplify the use of the application and increase user adoption rates) gains validity. The other options are incorrect as they either assume a negative impact on adoption rates due to the inclusion of the feature, suggest the opposite effect, or mention irrelevant aspects such as the user interface.

Question 2

If you hypothesize that by improving your mobile app loading speed, it will improve user satisfaction rates. Following the A/B test, the test results were inconclusive. What should be the Null Hypothesis in this circumstance?

Correct Answer: Improving the mobile app loading speed has no effect on user satisfaction rates.

In hypothesis testing, we start with a null hypothesis that assumes no association between the variables. In this instance, the null hypothesis should be that there is no relationship between improving the mobile app loading speed and user satisfaction rates. Therefore, the statement 'Improving the mobile app loading speed has no effect on user satisfaction rates.' is the most appropriate null hypothesis. Other options, such as the improvement of mobile app loading speed decreasing user satisfaction, or it increasing the app uninstall rate, or not being related to user behavior at all, aren't proper null hypotheses because they assume specific interactions or impacts.

Question 3

You hypothesize that integrating a social sharing feature will increase app user retention. As an Agile practitioner, how should you interpret contradictory results from two sets of user testing groups?

Correct Answer: The hypothesis may have some variables unaccounted for and should be revised

As an Agile practitioner, contradictory results don't necessarily mean the hypothesis is incorrect entirely, therefore answer 'The hypothesis is completely incorrect' can be dismissed. Also, 'One set of users did not understand the feature', and 'The social sharing feature is flawed' are not conclusions you can clearly draw from the information provided. Without more context, attributing the contradictory results to user misunderstanding or feature flaw would be premature. The best interpretation is that 'The hypothesis may have some variables unaccounted for and should be revised'. This is consistent with the principles of Agile, which stresses learning and adaption from empirical feedback. These results could potentially highlight unforeseen variables that should be considered in the hypothesis or experiment design.

Go Premium

PMI Agile Certified Practitioner Preparation Package (2024)

  • 3432 Superior-grade PMI Agile Certified Practitioner practice questions.
  • Accelerated Mastery: Deep dive into critical topics to fast-track your mastery.
  • Unlock Effortless PMI-ACP preparation: 5 full exams.
  • 100% Satisfaction Guaranteed: Full refund with no questions if unsatisfied.
  • Bonus: If you upgrade now you get upgraded access to all courses
  • Risk-Free Decision: Start with a 7-day free trial - get premium features at no cost!
More Hypothesis Setting questions
8 questions (total)
Start 8 question test