Learn

Hypothesis Testing with R

Significance Level

While a hypothesis test will return a p-value indicating a level of confidence in the null hypothesis, it does not definitively claim whether you should reject the null hypothesis. To make this decision, you need to determine a threshold p-value for which all p-values below it will result in rejecting the null hypothesis. This threshold is known as the *significance level*.

A higher significance level is more likely to give a false positive, as it makes is “easier” to state that there is a difference in the populations of your data when such a difference might not actually exist. If you want to be very sure that the result is not due to sampling error, you should select a very small significance level.

It is important to choose the significance level before you perform a statistical hypothesis test. If you wait until after you receive a p-value from a test, you might pick a significance level such that you get the result you want to see. For instance, if someone is trying to publish the results of their scientific study in a journal, they might set a higher significance level that makes their results appear statistically significant. Choosing a significance level in advance helps keep everyone honest.

It is an industry standard to set a significance level of `0.05`

or less, meaning that there is a `5%`

or less chance that your result is due to sampling error.

Before you run a hypothesis test on a set of data, you set your significance level to `0.05`

. The hypothesis test then returns a p-value of `0.1`

. Can you reject the null hypothesis? Update the value of `reject_hypothesis`

to `TRUE`

or `FALSE`

depending on your answer.