Learn about various probability rules through interactive applets and Python programming.

Rules of Probability

Let's dive into some key concepts we will use throughout this lesson: *union*, *intersection*, and *complement*. 

#### Union

The *union* of two sets encompasses any element that exists in either one or both of them. We can represent this visually as a *Venn diagram*. 

![A Venn diagram that shows two overlapping circles, one that represents event A and one that represents event B. Both of these circles are shaded blue.](https://static-assets.codecademy.com/skillpaths/master-stats-ii/intro-probability/union-venndiagram.svg)

For example, let's say we have two sets, *A* and *B*. *A* represents rolling an odd number with a six-sided die (the set *{1, 3, 5}*). *B* represents rolling a number greater than two (the set *{3, 4, 5, 6})*. The union of these two sets would be everything in either set *A*, set *B*, or both: *{1, 3, 4, 5, 6}*. We can write the union of two events mathematically as *(A or B)*.

#### Intersection

The *intersection* of two sets encompasses any element that exists in both of the sets. Visually: 

![A Venn diagram that shows two overlapping circles, one that represents event A and one that represents event B. Only the overlap of the two circles is shaded.](https://static-assets.codecademy.com/skillpaths/master-stats-ii/intro-probability/intersection-venndiagram.svg)

The intersection of the above sets (*A* represents rolling an odd number on a six-sided die and *B* represents rolling a number greater than two) includes any value that appears in both sets: *{3, 5}*. We can write the intersection of two events mathematically as *(A and B)*.

#### Complement

Lastly, the *complement* of a set consists of all possible outcomes outside of the set. Visually:

![A Venn diagram that shows a circle representing event A. Everything outside of event A is shaded blue.](https://static-assets.codecademy.com/skillpaths/master-stats-ii/intro-probability/complement-venndiagram.svg)

Consider set *A* from the above example (rolling an odd number on a 6-sided die). The complement of this set would be rolling an even number: *{2, 4, 6}*. We can write the complement of set *A* as *A<sup>C</sup>*. One key feature of complements is that a set and its complement cover the entire sample space. In this die roll example, the set of even numbers and odd numbers would cover all possible rolls: *{1, 2, 3, 4, 5, 6}*.

Union, Intersection, and Complement

We have introduced conditional probability as a part of the multiplication rule for dependent events. However, let's go a bit more in-depth with it as it is a powerful probability tool that has real-world applications.

For this problem, we will follow along the tree diagram on the right.

Suppose that the following is true (this is shown in the first set of branches in the diagram):
* 20 percent of the population has strep throat.
* 80 percent of the population does not have strep throat.

Now suppose that we test a bunch of people for strep throat. The possible results of these tests are shown in the next set of branches:

* If a person has strep throat, there is an 85% chance their test will be positive and a 15% chance it will be negative. This is labeled as:
```tex
\begin{aligned}
P(+ \mid ST) = 0.85 \\
\text{and} \\
P(- \mid ST) = 0.15 \\
\end{aligned}
```
* If a person does not have strep throat, there is a 98% chance their test will be negative and a 2% chance it will be positive. This can be labeled as:
```tex
\begin{aligned}
P(- \mid \text{NO ST}) = 0.98 \\
\text{and} \\
P(+ \mid \text{NO ST}) = 0.02 \\
\end{aligned}
```

Finally, let's look at the four possible pairs of outcomes that form the terminal branches of our diagram:
```tex
\begin{aligned}
P(\text{ST and +}) = 0.17 \\
P(\text{ST and -}) = 0.03 \\
P(\text{NO ST and +}) = 0.016 \\
P(\text{NO ST and -}) = 0.784 \\
\end{aligned}
```

Together, these add up to one since they capture all potential outcomes after patients are tested. 

It's great that we have all this information. However, we are missing something. If someone gets a positive result, what is the probability that they have strep throat? Notationally, we can write this probability as:

```tex
P(ST \mid +)
```

In the next exercise, we'll explore how we can use our tree diagram to calculate this probability.

This diagram outlines the possible outcomes for the events mentioned in the narrative.

With the narrative, you should have all the information necessary to do the next exercise's calculations!

Conditional Probability Continued

Congratulations, we have finished our exploration into the rules of probability! To recap, we have covered:

* The union and intersection of two events
* The complement of an event
* Independent and dependent events
* Mutually exclusive events
* How to calculate the union of two events using the addition rule
* What conditional probability is and how to calculate it
* How to calculate simultaneous events using the multiplication rule
* How to use tree diagrams to map out possible outcomes
* What Bayes' theorem is and how to calculate it
* Application of conditional probability and Bayes' theorem

Marvel in all that we have covered! For more practice, feel free to use the applet to the right and practice questions below. 

Review

Imagine that we flip a fair coin 5 times and get 5 heads in a row. Does this affect the probability of getting heads on the next flip? Even though we may feel like it's time to see "tails", it is impossible for a past coin flip to impact a future one. The fact that previous coin flips do not affect future ones is called *independence*. Two events are *independent* if the occurrence of one event does not affect the probability of the other.  

Are there cases where previous events DO affect the outcome of the next event? Suppose we have a bag of five marbles: two marbles are blue and three marbles are red. If we take one marble out of the bag, what is the probability that the second marble we take out is blue?

![A diagram of the possible outcomes of pulling two marbles out of a bag when pulling them out without replacement.](https://static-assets.codecademy.com/skillpaths/master-stats-ii/intro-probability/marble-diagram-1.svg)

This describes two events that are *dependent*. The probability of grabbing a blue marble in the second event *depends* on whether we take out a red or a blue marble in the first event. 

What if we had put back the first marble? Is the probability that we pick a blue marble second independent or dependent on what we pick out first? In this case, the events would be independent. 

![A diagram of the possible outcomes of pulling two marbles out of a bag when pulling them out with replacement.](https://static-assets.codecademy.com/skillpaths/master-stats-ii/intro-probability/marble-diagram-2.svg)

Why do we care if events are independent or dependent? Knowing this helps us quantify the probability of events that depend on preexisting knowledge. This helps researchers understand and predict complex processes such as:

* Effectiveness of vaccines
* The weather on a particular day
* Betting odds for professional sports games

We will explore applications of this further in the lesson!

Independence and Dependence

If we want to calculate the probability that a pair of dependent events both occur, we need to define conditional probability. Using a bag of marbles as an example, let's remind ourselves of the definition of dependent events:

![A diagram of the possible outcomes of pulling two marbles out of a bag when pulling them out without replacement.](https://static-assets.codecademy.com/skillpaths/master-stats-ii/intro-probability/marble-diagram-1.svg)

If we pick two marbles from a bag of five marbles without replacement, the probability that the second marble is red depends on the color of the first marble. We have a special name for this: *conditional probability*. In short, conditional probability measures the probability of one event occurring, given that another one has already occurred.

Notationally, we denote the word "given" with a vertical line. For example, if we want to represent the probability that we choose a red marble given the first marble is blue, we can write:
```tex 
P(\text{Red Second} \mid \text{Blue First})
```

From the above diagram, we know that:
```tex 
P(\text{Red Second} \mid \text{Blue First}) = \frac{3}{4}
```


What if we picked out two marbles with replacement? What does the conditional probability look like? Well, let's think about this. Regardless of which marble we pick out first, it will be put back into the bag. Therefore, the probability of picking out a red marble or a blue marble second is unaffected by the first outcome. 


Therefore, for independent events, we can say the following:

```tex 
\begin{aligned}

P(A \mid B) = P(A) \\
\text{and} \\
P(B \mid A) = P(B) \\

\end{aligned}
```


Conditional Probability

We have looked at the addition rule, which describes the probability one event OR another event (or both) occurs. What if we want to calculate the probability that two events happen simultaneously? For two events, *A* and *B*, this is *P(A and B)* or the probability of the intersection of *A* and *B*.

##### General Formula

The general formula for the probability that two events occur simultaneously is:
```tex
P(A \text{ and } B) = P(A) \cdot P(B \mid A)
```

However, for independent events, we can simplify this formula slightly.

##### Dependent Events

Let's go back to our bag of marbles example. We have five marbles: two are blue, and three are red. We pick two marbles without replacement. What if we want to know the probability of choosing a blue marble first AND a blue marble second?

Taking conditional probability into account, the multiplication rule for these two dependent events is:
```tex
\begin{aligned}
P(\text{Blue 1st and Blue 2nd}) = P(\text{Blue 1st}) \cdot P(\text{Blue 2nd} \mid \text{Blue 1st}) \\
P(\text{Blue 1st and Blue 2nd}) = \frac{2}{5} \cdot \frac{1}{4} \\
P(\text{Blue 1st and Blue 2nd}) = \frac{1}{10}
\end{aligned}
```

This is one potential outcome when picking two marbles out of the bag. One way to visualize all possible outcomes of a pair of events is a *tree diagram*.

Tree diagrams have the following properties:
* Each branch represents a specific set of events.
* The probabilities the terminal branches (all possible sets of outcomes) sum to one.
* We multiply across branches (using the multiplication rule!) to calculate the probability that each branch (set of outcomes) will occur.

In the browser to the right, you will be able to play with one!

##### Independent Events

For two independent events, the multiplication rule becomes less complicated. The probability of two independent events occurring is:
 
```tex
P(A \text{ and } B) = P(A) \cdot P(B)
```
This is because the following is true for independent events:

```tex
P(B \mid A) = P(B)
```
Let's look at the simplest example: flipping a fair coin twice. Event *A* is that we get tails on the first flip, and event *B* is that we get tails on the second flip. *P(A) = P(B) = 0.5*, so according to our formula, the probability of getting tails on both flips would be:
```tex 
P(A \text{ and } B) = 0.5 \cdot 0.5 = 0.25
```

Visually on a tree diagram, we see:
![This visual shows a tree diagram that outlines all possible outcomes of flipping a fair coin two times. ](https://static-assets.codecademy.com/skillpaths/master-stats-ii/intro-probability/coin-tree-diagram.svg)

Open the diagram if in a new window [here](https://static-assets.codecademy.com/skillpaths/master-stats-ii/intro-probability/coin-tree-diagram.svg) if you would like to zoom in for a better view.

Multiplication Rule

Now, it's time to apply these concepts to calculate probabilities. 

Let's go back to one of our first examples: event *A* is rolling an odd number on a six-sided die and event *B* is rolling a number greater than two. What if we want to find the probability of one or both events occurring? This is the probability of the union of *A* and *B*:
```tex
P(A \text{ or } B)
```

We can visualize this calculation as follows: 

![This gif shows three sequential images of a Venn diagram that outline the formula for P(A or B). In the Venn Diagram, there are two overlapping circles: one that corresponds to event A and one that corresponds to event B. In the first image, the event A circle is shaded blue and P(A) is added to the formula. In the second image, the event B circle is shaded red and the formula now shows P(A) + P(B). In the final image, the overlap of event A and event B is shaded green and the formula now shows P(A) + P(B) - P(A and B).](https://static-assets.codecademy.com/skillpaths/master-stats-ii/intro-probability/addition-rule-independent-venndiagram.gif)

This animation gives a visual representation of the addition rule formula, which is:
```tex
P(A \text{ or } B) = P(A) + P(B) - P(A \text{ and } B)
```
We subtract the intersection of events *A* and *B* because it is included twice in the addition of *P(A)* and *P(B)*. 

What if the events are mutually exclusive? On a single die roll, if event *A* is that the roll is less than or equal to 2 and event *B* is that the roll is greater than or equal to 5, then events *A* and *B* cannot both happen.

![This gif shows two sequential images of a Venn diagram that outline the formula for P(A or B) for independent events. In the Venn Diagram, there are two non-overlapping circles: one that corresponds to event A and one that corresponds to event B. In the first image, the event A circle is shaded blue and P(A) is added to the formula. In the second image, the event B circle is shaded red and the final formula now shows P(A) + P(B).](https://static-assets.codecademy.com/skillpaths/master-stats-ii/intro-probability/addition-rule-dependent-venndiagram.gif)

For mutually exclusive events, the addition rule formula is:
```tex
P(A \text{ or } B) = P(A) + P(B)
```

This is because the intersection is empty, so we don't need to remove any overlap between the two events.

Addition Rule

Probability is a way to quantify uncertainty. When we flip a fair coin, we say that there is a 50 percent chance (probability = 0.5) of it coming up tails. This means that if we flip INFINITELY many fair coins, half of them will come up tails. Similarly, when we roll a six-sided die, we say there is a 1 in 6 chance of rolling a five. 

What if we flip a coin in one hand and roll a die in the other at the same time. What is the probability that the coin comes up tails AND the die comes up as a five? Is there a way to quantify the probability that these two different events BOTH occur? In this lesson, we will walk through different rules of probability that help us quantify the probability of multiple random events.


Introduction

Imagine that you are a patient who has recently tested positive for strep throat. You may want to know the probability that you HAVE strep throat, given that you tested positive:

```tex
P(ST \mid +)
``` 

To calculate this probability, we will use something called *Bayes Theorem*, which states the following:

```tex
P(B \mid A) = \frac{P(A \mid B) \cdot P(B)}{P(A)}
```


Using Bayes' theorem:
 
```tex
P(ST \mid +) = \frac{P(+ \mid ST) \cdot P(ST)}{P(+)}
``` 
We know:
```tex
P(+ \mid ST) = 0.85
```
We also know:
```tex
P(ST) = 0.20
```

What about *P(+)*? Is this something we know? Well, let's think about this. There are four possible outcomes:
* Having strep throat and testing positive
* Having strep throat and testing negative
* Not having strep throat and testing positive
* Not having strep throat and testing negative

We only care about the two outcomes where a patient tests positives for *P(+)*. Therefore, we can say:
```tex
\begin{aligned}
P(+) = P(\text{ST and +}) + P(\text{NO ST and +}) \\
P(+) = 0.17 + .016 \\
P(+) = 0.186 \\
\end{aligned}
```

Finally, if we plug all of these into the Bayes' theorem formula, we get:

```tex
P(ST \mid +) = \frac{0.85 \cdot 0.20}{0.186} = 0.914
```
There is a 91.4% chance that you actually have strep throat given you test positive. This is not obvious from the information outlined in our tree diagram, but with the power of Bayes theorem, we were able to calculate it!

Let's practice some more of this in the questions below.

This diagram outlines the possible outcomes for the events mentioned in the narrative.

With the narrative, you should have all the information necessary to do the exercise's calculations!

Bayes' Theorem

Two events are considered *mutually exclusive* if they cannot occur at the same time. For example, consider a single coin flip: the events "tails" and "heads" are mutually exclusive because we cannot get both tails and heads on a single flip. 

We can visualize two mutually exclusive events as a pair of non-overlapping circles. They do not overlap because there is no outcome for one event that is also in the sample space for the other:

![A Venn diagram that shows two non-overlapping circles, one that represents event A and one that represents event B. Nothing is shaded in the Venn diagram.](https://static-assets.codecademy.com/skillpaths/master-stats-ii/intro-probability/mutually-exclusive-venndiagram.svg)

What about events that are not mutually exclusive? If event *A* is rolling an odd number and event *B* is rolling a number greater than two, these events are not mutually exclusive. They have an intersection of *{3, 5}*. Any events that have a non-empty intersection are not mutually exclusive.