In the previous exercise, we calculated a one-sided p-value. In this exercise, we’ll estimate a p-value for a 2-sided test, which is the default setting for many functions in Python (and other languages, like R!).
In our 10-coin-flip experiment, remember that we observed 2 heads, which is 3 less than the expected value of 5 (50% of 10) if the null hypothesis is true. The two sided test focuses on the number of heads being three different from expectation, rather than just less than. The hypothesis test now asks the following question:
Suppose that the true probability of heads is 50%. What is the probability of observing either two or fewer heads OR eight or more heads? (Note that two and eight are both three away from five). The calculation now estimates the proportion of the null histogram that is colored in red:
This proportion can be calculated in Python as follows. Note that the
| symbol is similar to
'or', but works for comparing multiple values at once.
import numpy as np outcomes = np.array(outcomes) p_value = np.sum((outcomes <= 2) | (outcomes >= 8))/len(outcomes) print(p_value) #output: 0.12
We end up with a p-value that is twice as large as the one-sided p-value.
The code you wrote to generate
null_outcomes is available to you in script.py. Use
null_outcomes to calculate the p-value for a two-sided test (alternative hypothesis is that the purchase probability was DIFFERENT FROM 10%). Remember that, if the purchase rate is 10%, we expect 50 of the 500 visitors to make a purchase.
In other words, calculate the proportion of values in
null_outcomes that are less than or equal to 41 (the number of purchases we observed in our sample, which is 9 fewer than 50) OR greater than or equal to 59 (which is 9 purchases more than 50). Save this number as a variable named
p_value and print it out.
Again, try pressing run a few times to observe a few different estimates of
p_value. What do you think the true p-value is for this test?