Learn

An A/B Test is a scientific method of choosing between two options (Option A and Option B). Some examples of A/B tests include:

• What number of sale items on a website makes customers most likely to purchase something: 25 or 50?
• What color button are customers more likely to click on: blue or green?
• Do people spend more time on a website if the background is green or orange?

For A/B tests where the outcome of interest (eg., whether or not a customer makes a purchase) is categorical, an A/B test is conducted using a Chi-Square hypothesis test. In order to determine the sample size necessary for this kind of test, a sample size calculator requires three numbers:

• Baseline conversion rate
• Minimum detectable effect (also called the minimum desired lift)
• Statistical significance threshold

In this lesson, we will discuss each of these numbers and how a data scientist might choose them.

### Instructions

Take a look at the sample size calculator provided in the workspace and try plugging in some numbers. Can you guess what each input means?