Key Concepts

Review core concepts you need to learn to master this subject

A/B Tests with Categorical Outcomes

For some A/B tests, the outcome of interest is categorical. For example, if we are testing whether a green background or an orange background leads to more customer purchases on a website, then our outcome (purchase or no purchase) is categorical. A/B tests with categorical results should be conducted using a Chi-Square hypothesis test.

A/B Testing: Sample Size Calculators
Lesson 1 of 2
  1. 1
    An A/B Test is a scientific method of choosing between two options (Option A and Option B). Some examples of A/B tests include: * What number of sale items on a website makes customers most like…
  2. 2
    A/B tests usually compare an option that we’re currently using to a new option that we suspect might be better. In order to compare the two options, we need a metric. Often, our metric will be the …
  3. 3
    Suppose we’re running an A/B Test to find out if a new website layout drives more subscriptions than the current one. If the new layout is only a tiny percent better, would we really care? In ord…
  4. 4
    When we run an A/B test, we usually want to use the results of the test to make a decision: use version A or B? In order to make that decision, many data scientists use a pre-determined significanc…
  5. 5
    Suppose that a Product Manager is running an A/B Test for a redesign of a landing page. Before starting the test, she used a sample size calculator to determine the sample size: 2,200 total website…
  6. 6
    Congratulations! You now know how to use a sample size calculator to figure out an appropriate sample size for an A/B Test! As a reminder, you learned about the following inputs to a sample-size ca…

What you'll create

Portfolio projects that showcase your new skills

Pro Logo

How you'll master it

Stress-test your knowledge with quizzes that help commit syntax to memory

Pro Logo