In this lesson, you will learn how to calculate and interpret the variance of a dataset.


Variance in R

Great work! In this lesson you've learned about variance and how to calculate it.

In the example used in this lesson, the importance of variance was highlighted by showing data from test scores in classes taught by two different teachers. What story does variance tell? What conclusions can we draw from this statistic?

<img src = "https://content.codecademy.com/courses/statistics/variance/teachers.png" alt = "The histogram of scores from two different teacher's classes">

In the class with low variance, it seems like the teacher strives to make sure all students have a firm understanding of the subject, but nobody is exemplary.

In the class with high variance, the teacher might focus more of their attention on certain students. This might enable some students to ace their tests, but other students get left behind.

If we only looked at statistics like mean, median, and mode, these nuances in the data wouldn't be represented.

Review

We now have five different values that describe how far away each point is from the mean. That seems to be a good start in describing the spread of the data. But the whole point of calculating variance was to get one number that describes the dataset. We don't want to report five values &mdash; we want to combine those into one descriptive statistic.

To do this, we'll take the average of those five numbers. By adding those numbers together and dividing by `5`, we'll end up with a single number that describes the average distance between our data points and the mean.

Note that we're not _quite_ done yet &mdash; our final answer is going to look a bit strange here. There's a small problem that we'll fix in the next exercise.


Average Distances

Now that you have learned the importance of describing the spread of a dataset, let's figure out how to mathematically compute this number.

How would you attempt to capture the spread of the data in a single number? 

Let's start with our intuition &mdash; we want the variance of a dataset to be a large number if the data is spread out, and a small number if the data is close together.

<img src="https://content.codecademy.com/courses/statistics/variance/two_histograms.svg" alt = "Two histograms. One with a large spread and one with a smaller spread.">

A lot of people may initially consider using the range of the data. But that only considers two points in your entire dataset. Instead, we can include every point in our calculation by finding the difference between every data point and the mean. 

<img src="https://content.codecademy.com/courses/statistics/variance/difference.svg" alt="The difference between the mean and four different points.">

If the data is close together, then each data point will tend to be close to the mean, and the difference will be small. If the data is spread out, the difference between every data point and the mean will be larger.

Mathematically, we can write this comparison as 

```tex
\text{difference} = X - \mu
```
Where `X` is a single data point and the Greek letter `mu` is the mean.


Distance From Mean

Finding the mean, median, and mode of a dataset is a good way to start getting an understanding of the general shape of your data

However, those three descriptive statistics only tell part of the story. Consider the two datasets below:

```R
dataset_one <- c(-4, -2, 0, 2, 4)
dataset_two <- c(-400, -200, 0, 200, 400)
```

These two datasets have the same mean and median &mdash; both of those values happen to be `0`. If we only reported these two statistics, we would not be communicating any meaninful difference between these two datasets.

This is where *variance* comes into play. Variance is a descriptive statistic that describes how spread out the points in a data set are.


Variance

We're almost there! We have one small problem with our equation. Consider this very small dataset:

```R
c(-5, 5)
```
The mean of this dataset is `0`, so when we find the difference between each point and the mean we get `-5 - 0 = -5` and `5 - 0 = 5`.

When we take the average of `-5` and `5` to get the variance, we get `0`:

```tex
\frac{-5 + 5}{2} = 0
``` 

Now think about what would happen if the dataset were `c(-200, 200)`. We'd get the same result! That can't possibly be right &mdash; the dataset with `200` is much more spread out than the dataset with `5`, so the variance should be much larger!

The problem here is with negative numbers. Because one of our data points was `5` units below the mean and the other was `5` units above the mean, they canceled each other out! 

When calculating variance, if a data point was above or below the mean &mdash; all we care about is how far away it was. To get rid of those pesky negative numbers, we'll square the difference between each data point and the mean.

Our equation for finding the difference between a data point and the mean now looks like this:

```tex
\text{difference} = (X - \mu)^2
```


Square the Differences

Well done! You've calculated the variance of a data set. The full equation for the variance is as follows:

```tex
\sigma^2 = \frac{\sum_{i=1}^{N}{(X_i -\mu)^2}}{N}
```
Let's dissect this equation a bit. 
* Variance is usually represented by the symbol sigma squared. 
* We start by taking every point in the dataset &mdash; from point number `1` to point number `N` &mdash; and finding the difference between that point and the mean. 
* Next, we square each difference to make all differences positive.
* Finally, we average those squared differences by adding them together and dividing by `N`, the total number of points in the dataset.

All of this work can be done quickly using a function we provided. The `variance()` function takes a list of numbers as a parameter and returns the variance of that dataset.

```R
dataset <- c(3, 5, -2, 49, 10)
var <- variance(dataset)
```
