Experimental Design
Learn to design an experiment to make a decision using a hypothesis test.
Start
Key Concepts
Review core concepts you need to learn to master this subject
A/B Tests with Categorical Outcomes
Baseline Conversion Rate
Minimum Detectable Effect
Significance Threshold
When to Stop an A/B Test
Chi-Square Test for A/B Testing
Simulating A/B Testing Data
Significance Threshold
A/B Tests with Categorical Outcomes
A/B Tests with Categorical Outcomes
For some A/B tests, the outcome of interest is categorical. For example, if we are testing whether a green background or an orange background leads to more customer purchases on a website, then our outcome (purchase or no purchase) is categorical. A/B tests with categorical results should be conducted using a Chi-Square hypothesis test.
- 1An A/B Test is a scientific method of choosing between two options (Option A and Option B). Some examples of A/B tests include: * What number of sale items on a website makes customers most like…
- 2A/B tests usually compare an option that we’re currently using to a new option that we suspect might be better. In order to compare the two options, we need a metric. Often, our metric will be the …
- 3Suppose we’re running an A/B Test to find out if a new website layout drives more subscriptions than the current one. If the new layout is only a tiny percent better, would we really care? In ord…
- 4When we run an A/B test, we usually want to use the results of the test to make a decision: use version A or B? In order to make that decision, many data scientists use a pre-determined significanc…
- 5Suppose that a Product Manager is running an A/B Test for a redesign of a landing page. Before starting the test, she used a sample size calculator to determine the sample size: 2,200 total website…
What you'll create
Portfolio projects that showcase your new skills
How you'll master it
Stress-test your knowledge with quizzes that help commit syntax to memory