Learn how to analyze different statistical distributions using NumPy.

Statistical Distributions with NumPy

We can generate our own normally distributed datasets using NumPy. Using these datasets can help us better understand the properties and behavior of different distributions. We can also use them to model results, which we can then use as a comparison to real data. 

In order to create these datasets, we need to use a *random number generator*. The NumPy library has several functions for generating random numbers, including one specifically built to generate a set of numbers that fit a normal distribution: `np.random.normal`. This function takes the following keyword arguments:
-  **loc:** the mean for the normal distribution
- **scale:** the standard deviation of the distribution
- **size:** the number of random numbers to generate


```py
a = np.random.normal(0, 1, size=100000)
```
If we were to plot this set of random numbers as a histogram, it would look like this:
![histogram](https://content.codecademy.com/courses/numpy/symmetric.svg)


Normal Distribution, Part II

There are some complicated formulas for determining these types of probabilities. Luckily for us, we can use NumPy - specifically, its ability to generate random numbers. We can use these random numbers to model a distribution of numerical data that matches the real-life scenario we're interested in understanding. 

For instance, suppose we want to know the different probabilities of our basketball player making 1, 2, 3, etc. out of 10 shots. 

NumPy has a function for generating binomial distributions: `np.random.binomial`, which we can use to determine the probability of different outcomes.

The function will return the number of successes for each "experiment".  

It takes the following arguments:
- **N**: The number of samples or trials
- **P**: The probability of success 
- **size**: The number of experiments

Let's generate a bunch of "experiments" of our basketball player making 10 shots.  We choose a big **N** to be sure that our probabilities converge on the correct answer.
```py
# Let's generate 10,000 "experiments"
# N = 10 shots
# P = 0.30 (30% he'll get a free throw)

a = np.random.binomial(10, 0.30, size=10000)
```
Now we have a record of 10,000 experiments.  We can use Matplotlib to plot the results of all of these experiments:
```py
plt.hist(a, range=(0, 10), bins=10, normed=True)
plt.xlabel('Number of "Free Throws"')
plt.ylabel('Frequency')
plt.show()
```
![binomiall](https://content.codecademy.com/courses/numpy/free_throws.svg)

Binomial Distributions, Part II 

Most of the datasets that we'll be dealing with will be unimodal (one peak). We can further classify unimodal distributions by describing where most of the numbers are relative to the peak.

A *symmetric* dataset has equal amounts of data on both sides of the peak. Both sides should look about the same.
![histogram](https://content.codecademy.com/courses/learn-pandas/distribution-types-ii-symmetric-noline.svg)

A *skew-right* dataset has a long tail on the right of the peak, but most of the data is on the left.
![histogram](https://content.codecademy.com/courses/learn-pandas/distribution-types-ii-skew-right-noline.svg)

A *skew-left* dataset has a long tail on the left of the peak, but most of the data is on the right.
![histogram](https://content.codecademy.com/courses/learn-pandas/distribution-types-ii-skew-left-noline.svg)

The type of distribution affects the position of the mean and median. In heavily skewed distributions, the mean becomes a less useful measurement.

SYMMETRIC
![histogram](https://content.codecademy.com/courses/learn-pandas/distribution-types-ii-symmetric.svg)

SKEW-RIGHT
![histogram](https://content.codecademy.com/courses/learn-pandas/distribution-types-ii-skew-right.svg)

SKEW-LEFT
![histogram](https://content.codecademy.com/courses/learn-pandas/distribution-types-ii-skew-left.svg)


There is an image of three histograms stacked vertically. The top histogram is labeled "Graph A", the middle histogram is labeled "Graph B", and the bottom histogram is labeled "Graph C". Graph A ranges from 0.0 to 0.8 on the x-axis and 0 to 2500 on the y-axis. Graph A starts at (0.0, 0) and slowly curves upward till it peaks at about (0.25, 2500). Then it slowly curves downward (at a slightly less steep rate than it curved upward) until it ends at about (0.7, 0). Graph B ranges from 0.5 to 1.0 on the x-axis and 0 to 4000 on the y-axis. Graph B starts at (0.65, 0) and slowly curves upward till it peaks at about (0.93, 4000). Then it curves downward rapidly until it ends at about (1.0, 0). Graph C ranges from -4 to 4 on the x-axis and 0 to 3500 on the y-axis. Graph C starts around (-1, 0) and slowly curves upward till it peaks at about (0, 3300). Then it curves downward at the same rate it curved upward until it ends at about (3, 0).

Different Types of Distribution, Part II

We know that the standard deviation affects the "shape" of our normal distribution.  The last exercise helps to give us a more quantitative understanding of this.

Suppose that we have a normal distribution with a mean of 50 and a standard deviation of 10.  When we say "within one standard deviation of the mean", this is what we are saying mathematically:
```
lower_bound = mean - std
            = 50 - 10
            = 40

upper_bound = mean + std
            = 50 + 10
            = 60
```
It turns out that we can expect about 68% of our dataset to be between 40 and 60, for this distribution.

As we saw in the previous exercise, no matter what mean and standard deviation we choose, 68% of our samples will fall between +/- 1 standard deviation of the mean!

In fact, here are a few more helpful rules for normal distributions:
- **68%** of our samples will fall between +/- 1 standard deviation of the mean
- **95%** of our samples will fall between +/- 2 standard deviations of the mean
- **99.7%** of our samples will fall between +/- 3 standard deviations of the mean

Standard Deviations and Normal Distribution, Part II

Histograms and their datasets can be classified based on the shape of the graphed values. In the next two exercises, we'll look at two different ways of describing histograms. 

One way to classify a dataset is by counting the number of distinct *peaks* present in the graph. Peaks represent concentrations of data. Let's look at the following examples:

A *unimodal* dataset has only one distinct peak.
![histogram](https://content.codecademy.com/courses/numpy/distribution_type_i/unimodal_new.svg)

A *bimodal* dataset has two distinct peaks. This often happens when the data contains two different populations.
![histogram](https://content.codecademy.com/courses/numpy/distribution_type_i/bimodal_new.svg)

A *multimodal* dataset has more than two peaks.
![histogram](https://content.codecademy.com/courses/numpy/distribution_type_i/multimodal_new.svg)

A *uniform* dataset doesn't have any distinct peaks.
![histogram](https://content.codecademy.com/courses/numpy/distribution_type_i/uniform_new.svg)



Image of three histograms. The top histogram, labeled (a), has three distinct peaks. The middle histogram, labeled (b), has one centralized peak. The bottom histogram, labeled (c), has two peaks, uniformly distributed and connected.

Different Types of Distributions, Part I

When we first look at a dataset, we want to be able to quickly understand certain things about it:

- Do some values occur more often than others?
- What is the range of the dataset (i.e., the min and the max values)?
- Are there a lot of outliers?

We can visualize this information using a chart called a *histogram*.

For instance, suppose that we have the following dataset:

```py
d = [1, 1, 1, 2, 2, 2, 2, 2, 3, 3, 4, 4, 4, 4, 5]
```

A simple histogram might show us how many 1's, 2's, 3's, etc. we have in this dataset.

|Value|Number of Samples|
|:-:|:-:|
|1| 3|
|2| 5|
|3| 2|
|4| 4|
|5| 1|

When graphed, our histogram would look like this:
![histogram](https://content.codecademy.com/courses/numpy/histogram-i-narrative-1.png)

There is a histogram titled, "Histogram". The x-axis is labeled "Values" and ranges from 2 to 9. The y-axis is labeled "Number of Samples" and ranges from 0 to 5. The bars in the chart have the following numbers: value 2 has 2 samples, value 3 has 3 samples, value 4 has 4 samples, value 5 has 5 samples, value 6 has 2 samples, value 7 has 1 sample, value 8 has 2 samples, and value 9 has 2 samples.

Histograms, Part I

A university wants to keep track of the popularity of different programs over time, to ensure that programs are allocated enough space and resources. You work in the admissions office and are asked to put together a set of visuals that show these trends to interested applicants. How can we calculate these distributions? Would we be able to see trends and predict the popularity of certain programs in the future? How would we show this information?

In this lesson, we are going to learn how to use NumPy to analyze different distributions, generate random numbers to produce datasets, and use Matplotlib to visualize our findings. 

This lesson will cover:
- How to generate and graph histograms
- How to identify different distributions by their shape
- Normal distributions
- How standard deviations relate to normal distributions
- Binomial distributions


Introduction

We can graph histograms using a Python module known as *Matplotlib*.  We're not going to go into detail about Matplotlib’s plotting functions, but if you're interested in learning more, take our course <a href="https://www.codecademy.com/learn/data-visualization-python" target="_blank" rel="noopener noreferrer">Learn Data Visualization with Python</a>. 

For now, familiarize yourself with the following syntax to draw a histogram:

```py
# This imports the plotting package.  We only need to do this once.
from matplotlib import pyplot as plt 

# This plots a histogram
plt.hist(data)

# This displays the histogram
plt.show()
```

When we enter `plt.hist` with no keyword arguments, matplotlib will automatically make a histogram with 10 bins of equal width that span the entire range of our data.

If you want a different number of bins, use the keyword `bins`. For instance, the following code would give us 5 bins, instead of 10:

```py
plt.hist(data, bins=5)
```

If you want a different range, you can pass in the minimum and maximum values that you want to histogram using the keyword `range`.  We pass in a tuple of two numbers.  The first number is the minimum value that we want to plot and the second value is the number that we want to plot up to, but not including.

For instance, if our dataset contained values between 0 and 100, but we only wanted to histogram numbers between 20 and 50, we could use this command:
```py
# We pass 51 so that our range includes 50
plt.hist(data, range=(20, 51))
```

Here’s a complete example:

```py
from matplotlib import pyplot as plt

d = np.array([1, 1, 1, 2, 2, 2, 2, 2, 3, 3, 4, 4, 4, 4, 5])

plt.hist(d, bins=5, range=(1, 6))

plt.show()
```

![histogram](https://content.codecademy.com/courses/numpy/test.svg)

Histograms, Part III

Suppose we had a larger dataset with values ranging from 0 to 50. We might not want to know exactly how many 0's, 1's, 2's, etc. we have.

Instead, we might want to know how many values fall between 0 and 5, 6 and 10, 11 and 15, etc.

These groupings are called *bins*.  All bins in a histogram are always the same size.  The *width* of each bin is the distance between the minimum and maximum values of each bin.  In our example, the width of each bin would be 5.

For a dataset like this, our histogram table would look like this:

|Bin | Number of Values|
|-|-|
|(0, 5) | 2 |
|(6, 10) | 10 |
| (11, 15) | 11 |
| ... | ...|
|(46, 50) | 3|

And if we were to graph this histogram, it would look like this:
![histogram](https://content.codecademy.com/courses/numpy/histogram-ii-narrative-1.png)

There is a histogram titled, "Histogram". The x-axis is labeled "Values" and ranges from 5 to 40. The y-axis is labeled "Number of Samples" and ranges from 0 to 15. The x-axis is labeled by counts of 5 (5, 10, 15, etc.), and the bars are lined up between the numbers instead of being centered with each number. The bars in the chart have the following numbers: values 5-10 have 15 samples, values 10-15 have 2 samples, values 15-20 have 0 samples, values 20-25 have 3 samples, values 25-30 have 11 samples, values 30-35 have 12 samples, and values 35-40 have 3 samples.

Histograms, Part II

In a normal distribution, we know that the mean and the standard deviation determine certain characteristics of the shape of our data, but how exactly? 

Let’s do some exploration to find out!

Standard Deviations and Normal Distribution, Part I

Let's review! In this lesson, you learned how to use NumPy to analyze different distributions and generate random numbers to produce datasets. Here's what we covered:

- What is a histogram and how to map one using Matplotlib
- How to identify different dataset shapes, depending on **peaks** or distribution of data 
- The definition of a **normal distribution** and how to use NumPy to generate one using NumPy's random number functions
- The relationships between normal distributions and **standard deviations**
- The definition of a **binomial distribution**

Now you can use NumPy to analyze and graph your own datasets! You should practice building your intuition about not only what the data says, but what conclusions can be drawn from your observations.

Review

Let's return to our original question:

Our basketball player has a 30% chance of making any individual basket.  He took 10 shots and made 4 of them, even though we only expected him to make 3.  What percent chance did he have of making those 4 shots?

We can calculate a different probability by counting the percent of experiments with the same outcome, using the `np.mean` function.

*Remember that taking the mean of a logical statement will give us the percent of values that satisfy our logical statement.* 

Let's calculate the probability that he makes 4 baskets:

```py
a = np.random.binomial(10, 0.30, size=10000)
np.mean(a == 4)
```
When we run this code, we might get:
```py
>> 0.1973
```
*Remember, because we're using a random number generator, we'll get a slightly different result each time.  With the large *`size`* we chose, the calculated probability should be accurate to about 2 decimal places.*

So, our basketball player has a roughly 20% chance of making 4 baskets.

This suggests that what we observed wasn't that unlikely.  It's quite possible that he hasn't got any better; he just got lucky. 

Binomial Distributions and Probability

It's known that a certain basketball player makes 30% of his free throws.  On Friday night’s game, he had the chance to shoot 10 free throws. How many free throws might you expect him to make?  We would expect 0.30 * 10 = 3.

However, he actually made 4 free throws out of 10 or 40%.  Is this surprising?  Does this mean that he’s actually better than we thought?

The *binomial distribution* can help us.  It tells us how likely it is for a certain number of “successes” to happen, given a probability of success and a number of trials. 

 In this example:
- The *probability of success* was 30% (he makes 30% of free throws)
- The number of *trials* was 10 (he took 10 shots)
- The number of *successes* was 4 (he made 4 shots)

The binomial distribution is important because it allows us to know how likely a certain outcome is, even when it's not the expected one. From this graph, we can see that it's not that unlikely an outcome for our basketball player to get 4 free throws out of 10. However, it would be pretty unlikely for him to get all 10. 

![binomiall](https://content.codecademy.com/courses/numpy/free_throws.svg)




Binomial Distribution, Part I

The most common distribution in statistics is known as the *normal distribution*, which is a symmetric, unimodal distribution.

Lots of things follow a normal distribution:
- The heights of a large group of people
- Blood pressure measurements for a group of healthy people
- Errors in measurements

Normal distributions are defined by their mean and standard deviation.   The mean sets the "middle" of the distribution, and the standard deviation sets the "width" of the distribution.  A larger standard deviation leads to a wider distribution.  A smaller standard deviation leads to a skinnier distribution.

Here are a few examples of normal distributions with different means and standard deviations:

![normal_distribution](https://content.codecademy.com/courses/numpy/normal_distribution.svg)

As we can see, each set of data has the same "shape", but with slight differences depending on their mean and standard deviation.  