A/B tests usually compare an option that we’re currently using to a new option that we suspect might be better. In order to compare the two options, we need a metric. Often, our metric will be the percent of users who take a certain action after interacting with one of our options. For instance:
- The percent of customers who buy a t-shirt after visiting one of two versions of a website
- The percent of users who click on one of two versions of an ad
In the t-shirt example above, the baseline conversion rate is our estimate for the percent of people who will buy a shirt under the current website design.
We can generally calculate a baseline by looking at historical data for the option that we’re currently using. For example, suppose that 2000 people visited a website over the past three months and 320 of those visitors purchased a shirt. We could estimate the baseline rate as follows:
This number may be written as a proportion (eg., 0.16) or a percent (eg., 16%).
Instructions
Suppose that an online media company wants to run an A/B test to see if a re-designed subscribe button will make site visitors more likely click “subscribe”. Some historical data is saved for you in script.py:
number_of_site_visitorsis the number of people who visited the site over the past monthnumber_of_converted_visitorsis the number who clicked “subscribe”.
Use these two variables to calculate the baseline conversion rate and save the result as a variable named baseline_rate, then print it out. Report the percentage, not the proportion.