P-value calculations and interpretations depend on the alternative hypothesis of a test, a description of the difference from expectation that we are interested in.
For example, let’s return to the 10-coin-flip example from earlier. Suppose that we flipped a coin 10 times and observed only 2 heads. We might run a hypothesis test with the following null and alternative hypotheses:
- Null: the probability of heads is 0.5
- Alternative: the probability of heads is less than 0.5
This hypothesis test asks the question: IF the probability of heads is 0.5, what’s the probability of observing 2 or fewer heads among a single sample of 10 coin flips?
Earlier, we used a for-loop to repeatedly (10000 times!) flip a fair coin 10 times, and store the number of heads (for each set of 10 flips) in a list named
outcomes. The probability of observing 2 or fewer heads among 10 coin flips is approximately equal to the proportion of those 10000 experiments where we observed 0, 1, or 2 heads:
import numpy as np outcomes = np.array(outcomes) p_value = np.sum(outcomes <= 2)/len(outcomes) print(p_value) #output: 0.059
This calculation is equivalent to calculating the proportion of this histogram that is colored in red:
We estimated that the probability of observing 2 or fewer heads is about 0.059 (5.9%). This probability (0.059) is referred to as a one-sided p-value.
The code you wrote to generate
null_outcomes is available to you in script.py. Use
null_outcomes to estimate the p-value for a binomial hypothesis test with the following null and alternative hypotheses:
- Null: the probability of a purchase was 10%
- Alternative: the probability of a purchase rate was LESS THAN 10%
In other words, calculate the proportion of values in
null_outcomes that are less than or equal to 41 (the observed number of purchases that we calculated earlier). Save this number as a variable named
p_value and print it out.
Try pressing “Run” a few times; You should see slightly different values of
p_value each time. What do you think the true probability is?