Learn the basics of aggregate functions in R and dplyr.

Aggregates in R

Sometimes, we want to group by more than one column.  We can do this by passing multiple column names as arguments to the `group_by` function.

Imagine that we run a chain of stores and have data about the number of sales at different locations on different days:

|location | date | day_of_week | total_sales|
|-|-|-|-|
|West Village | February 1 | W | 400 |
|West Village| February 2| Th| 450|
|Chelsea | February 1 | W | 375|
|Chelsea | February 2| Th | 390|
|...|...|...|...|

We suspect that sales are different at different locations on different days of the week.  In order to test this hypothesis, we could calculate the average sales for each store on each day of the week across multiple months.  The code would look like this:

```r
df %>%
  group_by(location,day_of_week) %>%
  summarize(mean_total_sales = mean(total_sales))
```
And the results might look something like this:

|location|day_of_week|mean_total_sales|
|-|-|-|
|Chelsea|M|402.50|
|Chelsea|Tu|422.75|
|Chelsea|W|452.00|
|...|...|...|...|
|West Village|M|390|
|West Village|Tu|400|
|...|...|...|...|

Calculating Aggregate Functions II

This lesson introduced you to aggregates in R using dplyr.  You learned:
* How to calculate summary statistics with `summarize()`
* How to perform *aggregate* statistics over individual rows with the same value or values using `group_by()`

Review

In this exercise, you will learn how to  *combine* all of the values from a column for a single calculation. This can be done with the help of the dplyr function `summarize()`, which returns a new data frame containing the desired calculation.

Some examples of this type of calculation include:
* The data frame `customers` contains the names and ages of all of your customers.  You want to find the median age:

```r
customers %>%
  select(age)
# c(23, 25, 31, 35, 35, 46, 62)
customers %>%
  summarize(median_age = median(age))
# 35
```
* The data frame `shipments` contains address information for all shipments that you've sent out in the past year.  You want to know how many different states you have shipped to.

```r
shipments %>%
  select(states)
# c('CA', 'CA', 'CA', 'CA', 'NY', 'NY', 'NJ', 'NJ', 'NJ', 'NJ', 'NJ', 'NJ', 'NJ')
shipments %>%
  summarize(n_distinct_states = n_distinct(states))
# 3
```
* The data frame `inventory` contains a list of types of t-shirts that your company makes. You want to know the standard deviation of the prices of your inventory.

```r
inventory %>%
  select(price)
# c(31, 23, 30, 27, 30, 22, 27, 22, 39, 27, 36)	
inventory %>% 
  summarize(sd_price = sd(price))
# 5.465595
```

The general syntax for these calculations is:
```r
df %>%
  summarize(var_name = command(column_name))
```
* `df` is the data frame you are working with
* `summarize` is a dplyr function that reduces multiple values to a single value
* `var_name` is the name you assign to the column that stores the results of the summary function in the returned data frame
* `command` is the summary function that is applied to the column by `summarize()`
* `column_name` is the name of the column of `df` that is being summarized

The following table includes common summary functions that can be given as an argument to `summarize()`:

|Command|Description|
|---------------|---------------|
|`mean()`| Average of all values in column|
|`median()`| Median value of column|
|`sd()`| Standard deviation of column|
|`var()`| Variance of column|
|`min()`| Minimum value in column|
|`max()`| Maximum value in column|
|`IQR()`| Interquartile range of column|
|`n_distinct()`| Number of unique values in column|
|`sum()`| Sum values of column|

Calculating Column Statistics

When we have a bunch of data, we often want to calculate aggregate statistics (mean, standard deviation, median, percentiles, etc.) over certain subsets of the data.

Suppose we have a grade book with columns `student`, `assignment_name`, and `grade`:

|student|assignment_name|grade|
|-|-|-|
|Amy|Assignment 1|96|
|Amy|Assignment 2|87|
|Bob|Assignment 1|91|
|Bob|Assignment 2|75|
|Chris|Assignment 1|83|
|Chris|Assignment 2|88|

We want to get an average grade for each student across all assignments.  We can do this using the helpful dplyr function `group_by()`.

For this example, we'd use the following piece of code:
```r
grades <- df %>%
  group_by(student) %>%
  summarize(mean_grade = mean(grade))
```

The output might look something like this:

|student|mean_grade|
|-|-|
|Amy|91.5|
|Bob|83|
|Chris|85.5|

In general, we use the following syntax to calculate aggregates:

```r
df %>%
  group_by(column_1) %>%
  summarize(aggregate_name = command(column_2))
```
* `column_1` (`student` in our example) is the column that we want to `group_by()`
* `column_2` (`grade` in our example) is the column that we want to apply `command()`, a summary function, to using `summarize()`
* `aggregate_name` is the name assigned to the calculated aggregate

In addition to the summary functions discussed in the last exercise (`mean()`, `median()`, `sd()`, `var()`, `min()`, `max()`, `IQR()` and `n_distinct()`), another helpful summary function, especially for grouped data, is `n()`. `n()` will return the count of the rows within a group, and does not require a column as an argument. To get the count of the rows in each group of students from our example:

```r
grades <- df %>%
  group_by(student) %>%
  summarize(count = n())
```

Calculating Aggregate Functions I

In this lesson you will learn about aggregates in R using dplyr. An aggregate statistic is a way of creating a single number that describes a group of numbers. Common aggregate statistics include mean, median, and standard deviation.

Additionally, you will learn how you can group data into different subsets based on column values. This can help narrow the focus of a summary statistic to a subset of a dataset. R you ready to get started?

Introduction

`group_by()` can also be used with the dplyr function `mutate()` to add columns to a data frame that involve per-group metrics.

Consider the same educational technology company's `enrollments` table from the previous exercise:

|user_id|course|quiz_score|
|-|-|-|
|1234|learn_r|80|
|1234|learn_python|95|
|4567|learn_r|90|
|4567|learn_python|55|

You want to add a new column to the data frame that stores the difference between a row's `quiz_score` and the average `quiz_score` for that row's `course`. To add the column:

```r
enrollments %>% 
  group_by(course) %>% 
  mutate(diff_from_course_mean = quiz_score - mean(quiz_score))
```
* `group_by()` groups the data frame by course into two groups: `learn-r` and `learn-python`
* `mutate()` will add a new column `diff_from_course_mean` which is calculated as the difference between a row's individual `quiz_score` and the `mean(quiz_score)` for that row's group (course)

The resulting data frame would look like this:

|user_id|course|quiz_score|diff_from_course_mean|
|-|-|-|-|
|1234|learn_r|80|-5|
|1234|learn_python|95|20|
|4567|learn_r|90|5|
|4567|learn_python|55|-20|

* The average `quiz_score` for the `learn-r` course is `85`, so `diff_from_course_mean` is calculated as `quiz_score - 85` for all the rows of `enrollments` with a value of `learn-r` in the `course` column.
* The average `quiz_score` for the `learn-python` course is `75`, so `diff_from_course_mean` is calculated as `quiz_score - 75` for all the rows of `enrollments` with a value of `learn-python` in the `course` column.

Combining Grouping with Mutate

While `group_by()` is most often used with `summarize()` to calculate summary statistics, it can also be used with the dplyr function `filter()` to filter rows of a data frame based on per-group metrics.

Suppose you work at an educational technology company that offers online courses and collects user data in an `enrollments` data frame:

|user_id|course|quiz_score|
|-|-|-|
|1234|learn_r|80|
|1234|learn_python|95|
|4567|learn_r|90|
|4567|learn_python|55|

You want to identify all the enrollments in difficult courses, which you define as courses with an average `quiz_score` less than `80`. To filter the data frame to just these rows:

```r
enrollments %>%
  group_by(course) %>%
  filter(mean(quiz_score) < 80)
```
* `group_by()` groups the data frame by `course` into two groups: `learn-r` and `learn-python`
* `filter()` will keep all the rows of the data frame whose per-group (per-course) average `quiz_score` is less than `80`

Rather than filtering rows by the individual column values, the rows will be filtered by their group value since a summary function is used! The resulting data frame would look like this:

|user_id|course|quiz_score|
|-|-|-|
|1234|learn_python|95|
|4567|learn_python|55|

* The average `quiz_score` for the `learn-r` course is `85`, so all the rows of `enrollments` with a value of `learn-r` in the `course` column are filtered out.
* The average `quiz_score` for the `learn-python` course is `75`, so all the rows of `enrollments` with a value of `learn-python` in the `course` column remain.