Learn

A/B Testing: Sample Size Calculators

Introduction

An *A/B Test* is a scientific method of choosing between two options (Option A and Option B). Some examples of A/B tests include:

- What number of sale items on a website makes customers most likely to purchase something: 25 or 50?
- What color button are customers more likely to click on: blue or green?
- Do people spend more time on a website if the background is green or orange?

For A/B tests where the outcome of interest (eg., whether or not a customer makes a purchase) is categorical, an A/B test is conducted using a Chi-Square hypothesis test. In order to determine the sample size necessary for this kind of test, a sample size calculator requires three numbers:

*Baseline conversion rate**Minimum detectable effect*(also called the*minimum desired lift*)*Statistical significance threshold*

In this lesson, we will discuss each of these numbers and how a data scientist might choose them.

### Instructions

Take a look at the sample size calculator provided in the workspace and try plugging in some numbers. Can you guess what each input means?