Learn about useful descriptive statistics and their practical application.

Statistical Thinking

As we're moving through the numeric variables in our musician dataset, we come across some interesting details when we inspect the income variable.

![Plot showing the distribution of the income variable. The incomes range from about $10,000 to $90,000. The bars indicating frequencies are bell-shaped but asymmetrical with a long tail on the right.](https://static-assets.codecademy.com/Courses/data-literacy/stats/ex5-income-large.svg)

![Table showing the mean of $34,795 and the standard deviation of $11,971 for the income variable](https://static-assets.codecademy.com/Courses/data-literacy/stats/ex5-mean-table.svg)

1. We notice that the shape of the distribution is different than the shape of the age distribution. There are quite a few musicians with higher incomes that are creating a longer tail on the right side.
2. We also notice that the mean indicates that the typical income is $34,795. This value seems a little high since most of the incomes seem to be between $15,000 and $40,000.

What we have learned is that the income distribution is skewed. A **skewed** distribution is asymmetrical with a steep change in frequency on one side and a flatter, trailing change in frequency on the other. Specifically, the income distribution is right-skewed (also called positively-skewed) because the tail is on the right side.

So why does the mean seem wrong? Remember, the mean is the sum of all the values in the dataset divided by the total count. That sum is made very large by all the higher incomes in that right tail. This makes the mean a greater number than we would like it be. When the data are skewed, the mean may not be the best measure of a typical observation.

There are a number of ways to deal with this issue. We will handle the problem with the income data by taking some alternative measurements.

Skewed Distributions

To start our summary for the mayor, let's describe some of the categorical variables in the musician dataset &mdash; those variables that contain qualitative information on the city's musicians. First, let's look at information about the `title` variable, which tells us the job title each musician holds. 

The following table shows:

* **frequency**: the count of musicians for each job title
* **proportion**: the frequency divided by the total number of musicians
* **percentage**: the proportion converted from a decimal to a percentage

![A 6 by 4 table with columns labeled job title, frequency, proportion, and percentage. Row 1: performer, 333, 0.35, 35%. Row 2: manager 113, 0.12, 12%. Row 3: producer, 87, 0.09, 9%. Row 4: educator, 239, 0.25, 25%. Row 5: composer, 186, 0.19, 19%. Row 6: total, 958, 1.00, 100%.](https://static-assets.codecademy.com/Courses/data-literacy/stats/ex2-jobs-table-narr.svg)

From the table, we can learn about the different job titles of musicians in the city. There are 333 performers out of a total of 958 musicians. The proportion of performers is 333 &#247; 958 = 0.35. To make this even easier to understand, we can convert the proportion to a percentage by multiplying it by 100: 35% of musicians in the city are performers.

We can also compare one category to another by checking the ratio of their frequencies. For example, there are far fewer managers than performers. Their ratio is 333 performers to 113 managers, which can be simplified by dividing: 333 &#247; 113 = 2.95. This means there are almost 3 performers for every manager in the city.

Try finding some other ratios from the table:

<details>
<summary>What is the ratio of educators to composers? (Click to Toggle Correct Answer)</summary>
The ratio is 239 educators to 186 composers. Simplified, this ratio is 239 &#247; 186 = 1.28, which is little more than 1 educator for every composer. There are not many more educators than composers in the city.
</details>
 

<details>
<summary>What is the ratio of performers to producers? (Click to Toggle Correct Answer)</summary>
The ratio is 333 performers to 87 producers. Simplified, this ratio is 333 &#247; 87 = 3.83, which means there are about 4 performers for every producer.
</details>

Describing Categorical Variables

One measure that we haven't covered that is usually talked about alongside the mean and median is the mode. The mode is defined as the value with the highest frequency, but we can also think of the mode as the value where the peak of the distribution occurs. While not great for computations, the mode can help us identify interesting features in a variable.

For instance, there might be more than one mode, such as in our distribution of years of experience. In the following plot, we can see there's one peak near the 10-year mark and another near the 30-year mark. We would call this distribution _bimodal_ because it has two modes.

![Histogram plot showing frequencies for the experience variable. There are two peaks: one at around 10 years and one at around 30 years. The title is "Bimodal distribution".](https://static-assets.codecademy.com/Courses/data-literacy/stats/ex8-bimodal.svg)

Sometimes bimodal distributions occur when there are differences across categories of another variable. Given that the city seems to have a lot of young people in bands, let's see if this pattern is reflected when we find the mean of each category of the `band` variable.

![Table showing mean years of experience for those in a band and those not. Those in a band have a mean of 14.4 years of experience. Those not in a band have a mean of 26.2 years of experience.](https://static-assets.codecademy.com/Courses/data-literacy/stats/ex8-table.svg)

These means are very different and very close to the locations of the modes in our plot. This indicates that there may be some differences in experience level between these two groups that are showing up in our distribution plots as two peaks.

By making this separation and then summarizing with the mean, we have **aggregated** our data. In this case, we have aggregated by summarizing a numeric variable (`experience`) across each value of a categorical variable (`band`).

We have aggregated some other data in tables in the learning environment. Do you see any interesting patterns?

An image of 3 tables titled "Aggregate Statistics." The first table shows median income by title: performer $40,387, manager $27,984, producer $36,897, educator $29,387, composer $35,027. The second table shows median and IQR of income by band membership: yes $31,683 $22,234, no $32,370 $15,984. The third table shows mean experience by instrument: voice 19 years, guitar 14 years, piano 34 years, drums 10 years, saxophone 28 years, violin 36 years.

Aggregate Data

Congratulations! We've finished our summary report for the mayor of Melody Metropolis, and you've just completed the lesson on statistical thinking! We covered a lot of information, including:

* Summary statistics give us measurements to describe our data.
* Categorical variables can be described with frequencies and proportions.
* Numeric variables can be described using measures of center and spread.
* Sometimes we need robust statistics like the median and IQR because our data is skewed or has outliers.
* We can aggregate data to get a look at relationships between pairs of variables.
* Scatter plots and correlation coefficients help us describe linear relationships between pairs of numeric variables.

And hopefully, the most important discovery you made is that learning statistics doesn't have to be a struggle. We gain valuable skills just by understanding some basic concepts. Enjoy using your new data vocabulary!

Image of a report with pie charts and percentages. The report is on a desk with other objects and papers.

Review

Let's return to our visualization of the distribution of musician ages.

![Bar plot showing the distribution of ages. Ages range from about 15 to 70. The frequencies determine the heights of the bars. The bars form a bell-shaped pattern.](https://static-assets.codecademy.com/Courses/data-literacy/stats/ex4-age-narr_corr.png)

* What would you say is the typical age of a musician in Melody Metropolis? 
* Are most musicians about this age, or are there lots of musicians of many different ages?

To answer these questions more specifically, we should take some measurements of our variable.

* The **mean**, also called the average, describes the center of a numeric distribution by adding all values and dividing by the count.
* The **standard deviation** describes the spread of values in a numeric distribution relative to the mean. It is calculated by finding the average squared distance from each data point to the mean and square-rooting the result.

The mean age of musicians is 40.6 years and the standard deviation is about 9.3 years. We might interpret this standard deviation as moderate variability in age. Had the standard deviation been 1 year, we might say there's hardly any variability in age. The plot of this narrow distribution might look something like the following:

![Bar plot showing the distribution of ages when the standard deviation is 1. Ages range from about 37 to 43 years. The frequencies determine the heights of the bars. The bars form a very narrow bell-shaped pattern.](https://static-assets.codecademy.com/Courses/data-literacy/stats/ex4-age-lowsd.svg)

The mean and standard deviation are common choices, especially for normal distributions. Their mathematical formulas have special properties that make them easy to use in other contexts, such as statistical testing. However, the mean and standard deviation are not always the best measurements to describe a distribution.

Mean and Standard Deviation

Let's find an alternative measure to the mean. We want to find a value that represents the typical musician income, but we don't want to use the actual values in the computation because the data are skewed.

One method would be to find the middle value when all values are arranged from smallest to largest. This value is called the **median**, but it's also referred to as the 50th percentile or the second quartile (Q2).

Let's look at some simple data in the learning environment where the median (Q2) is 13. Half the data points are less than 13, and half are greater than 13. 

These data span 22 values, ranging from 6 to 28. We could use this as our measure of spread, but what if the highest number wasn't 28 but 280? The median would still be 13, but now the range is 274 (280-6), which doesn't tell us a lot about the bulk of the data. 

A better measurement might be the **interquartile range (IQR)**. A quartile is simply a marker for a quarter (25%) of the data.
* The first quartile marks 25% (Q1 = 10).
* The second quartile marks 50% (Q2 = 13 &mdash; the median)
* The third quartile marks 75% (Q3 = 22)

The IQR is the difference between Q3 and Q1 (22-10 = 12), marking the range for just the middle 50% of the data.

Let's find out how the median and IQR work out for our income data.

![Table showing the median of income is $32,978 and the IQR is $17,150.](https://static-assets.codecademy.com/Courses/data-literacy/stats/ex6-income-stats.svg)

This looks better &mdash; the median of $32,978 is lower than the mean of $34,795 and seems more typical.

Median and IQR

Let's imagine we are working for the city government of the fictional city of Melody Metropolis. The mayor of Melody Metropolis wants to know more about the musicians who currently live in the city. The learning environment shows a dataset we have on musicians living in the city as of last year. How would you describe this dataset? See if you can answer any of the following questions:

* What does a typical musician's income look like? 
* Is there a wide range of musician ages?
* What proportion of the musicians in the dataset play guitar?

We can try to make generalizations by looking over the rows and columns, but it's difficult to answer these questions precisely. We need some kind of "data vocabulary" that can help us measure and describe the variables in the dataset. *Summary statistics* can be used for exactly this purpose! 

With a basic understanding of summary statistics, we can communicate and understand a lot more specific information about the musicians in the city. But learning statistics is often associated with a lot of negativity:

* Memorization of lots of math formulas
* Long calculations done by hand
* Confusing or meaningless interpretations

None of these struggles need to be part of learning to use statistics. In this lesson, we'll gain a conceptual understanding of how summary statistics can easily help us communicate and interpret our dataset.

Introduction

Now that we've learned about some of the categorical variables in our musician dataset, it's time to explore some numeric variables &mdash; those with quantitative data. There are a lot of ways we can describe the **distribution** of a numeric variable. A distribution is a function that shows all possible values of a variable and how frequently each value occurs. This may sound pretty technical, but visualizing the distribution can make it easy to understand.

In the learning environment, the distribution of musician ages is plotted with age on the x-axis and frequency on the y-axis. From this plot, we can see:

* Ages range from about 15 to 70.
* There are few musicians under 30 or over 50 years old.
* There are a lot of musicians between the ages of 30 and 50.

This distribution might be considered bell-shaped or hill-shaped and symmetrical. This is actually a very common pattern and is called a **normal distribution**.

![Plot of a normal distribution. The bars in the plots become larger and then smaller moving left to right, forming a bell shape.](https://static-assets.codecademy.com/Courses/data-literacy/stats/ex3-normal-graphic.svg)

Viewing a plot or knowing a variable is normally distributed gives us some general information, but still nothing specific. We need exact measurements to describe where the center of the distribution is and how wide the values are spread away from that center. There are several sets of statistics we may use for these measurements, and we will need to know when to use which combination.


A histogram plot of the musician age distribution with age in the x-axis and frequency on the y-axis. Age ranges from 0 to 70 years. Frequency ranges from 0 to 200. The frequency bars are in a bell shape. 

Describing Numeric Variables

Image of two distribution plots titled "Income Distribution." The top plot shows a right-skewed distribution with a median of $32,978 and a mean of $34,795. The bottom plot shows the same distribution with three additional incomes from Paul McCartney, BTS, and Beyonce. This plot has a median of $33,011 and a mean of $228,235.

For our income data, the difference between the mean ($34,795) and median ($32,978) was only about $2,000. You may be wondering: Is the difference ever larger?

Let's imagine some very famous celebrity musicians have all decided to move to Melody Metropolis. We know celebrities make much more money than the typical musician in our dataset. In the learning environment, we've added three new incomes to the distribution:

* **$48 million**: Paul McCartney, British musician of the Beatles 
* **$57 million**: BTS, South Korean K-pop band
* **$81 million**: Beyoncé, American singer-songwriter

The second plot shows that the median appears almost unaffected by the addition of these three gigantic incomes: the median moves from $32,978 to $33,011. However, the mean makes a drastic change from $34,795 to $228,235. The mean is now well beyond even the maximum in the original distribution. An income of $228,235 is definitely not a great measure of the center of our income distribution.

These celebrity incomes are examples of **outliers**, extreme values that are distant from the rest of the distribution. Just as with skewness, outliers tend to more heavily influence the mean than the median. This same pattern occurs with measures of spread: the standard deviation is more influenced by outliers and skewness than the interquartile range (IQR). 

Because the median and IQR are NOT heavily influenced by extreme values, we say they are **robust**. Robust statistics are often a better choice to measure the center and spread of a distribution that is skewed or has outliers.

Source:

[The World’s Top-Earning Musicians Of 2019](https://www.forbes.com/sites/zackomalleygreenburg/2019/12/06/the-worlds-top-earning-musicians-of-2019/?sh=87be291164e7)

Outliers and Robust Measures

Aggregating data is a way of exploring variable relationships. We specifically looked at relationships between a numeric variable and a categorical variable, but we should also examine relationships between two numeric variables.

For example, we might wonder: Does musician income vary with years of experience? To start, we can take a look at a **scatter plot** with experience on the x-axis and income on the y-axis. Each point in the plot represents a musician, and the coordinates of that point are the musician's experience (x) and income (y).

![Scatter plot showing income in USD by experience in years. The points form a pattern moving from the lower left to the upper right of the plot. They are not in a perfect line though. There is a range of incomes for every amount of experience.](https://static-assets.codecademy.com/Courses/data-literacy/stats/ex9-IEscatter.svg)

The cloud of points in the plot has a pattern. The points move from the lower left to the upper right part of the plot. In other words, lower levels of experience tend to be associated with lower incomes, and higher levels of experience tend to be associated with higher incomes. The points don't form a perfect line though &mdash; there is some variation.

We can describe this relationship more precisely by measuring the **correlation coefficient**. This number ranges from -1 to +1 and tells us two things about a linear relationship:

1. **Direction**: A positive coefficient means that higher values in one variable are associated with higher values in the other. A negative coefficient means higher values in one variable are associated with lower values of the other.
2. **Strength**: The farther the coefficient is from 0, the stronger the relationship and the more the points in a scatter plot look like a line.

The correlation coefficient for income and experience is 0.74 &mdash; the relationship is positive and moderately strong.