ENDS 12/4: Get 50% off your Pro annual membership using code [CYBER23](https://www.codecademy.com/checkout?periods=year&plan_id=proGoldAnnualV2&discountCode=CYBER23)


This lesson teaches learners how to perform Multiple Linear Regression using scikit-learn. It also teaches how to assess the model's accuracy using Residual Analysis. Thank you to the StreetEasy Research team for this dataset.

Multiple Linear Regression

Now that we have implemented Multiple Linear Regression, we will learn how to tune and evaluate the model. Before we do that, however, it's essential to learn the equation behind it.

**Equation 6.1** The equation for multiple linear regression that uses two independent variables is this: 

```tex
y = b + m_{1}x_{1} + m_{2}x_{2}
```

**Equation 6.2** The equation for multiple linear regression that uses three independent variables is this: 

```tex
y = b + m_{1}x_{1} + m_{2}x_{2} + m_{3}x_{3}
```

**Equation 6.3** As a result, since multiple linear regression can use any number of independent variables, its general equation becomes: 

```tex
y = b + m_{1}x_{1} + m_{2}x_{2} + ... + m_{n}x_{n}
```

Here, *m1*, *m2*, *m3*, ... *mn* refer to the **coefficients**, and *b* refers to the **intercept** that you want to find. You can plug these values back into the equation to compute the predicted *y* values. 

Remember, with `sklearn`'s `LinearRegression()` method, we can get these values with ease.

The `.fit()` method gives the model two variables that are useful to us:

- `.coef_`, which contains the coefficients
- `.intercept_`, which contains the intercept

After performing multiple linear regression, you can print the coefficients using `.coef_`. 

Coefficients are most helpful in determining which independent variable carries more weight. For example, a coefficient of -1.345 will impact the rent more than a coefficient of 0.238, with the former impacting prices negatively and latter positively.

Multiple Linear Regression Equation

Linear regression is useful when we want to predict the values of a variable from its relationship with other variables. There are two different types of linear regression models (<a href="https://www.codecademy.com/paths/data-science/tracks/dspath-supervised/modules/dspath-linear-regression/lessons/linear-regression" target="_blank">simple linear regression</a> and multiple linear regression). 

In predicting the price of a home, one factor to consider is the size of the home. The relationship between those two variables, price and size, is important, but there are other variables that factor in to pricing a home: location, air quality, demographics, parking, and more. When making predictions for price, our _dependent variable_, we'll want to use multiple _independent variables_. To do this, we'll use Multiple Linear Regression. 

**Multiple Linear Regression** uses two or more independent variables to predict the values of the dependent variable. It is based on the following equation that we'll explore later on:
```tex
y = b + m_{1}x_{1} + m_{2}x_{2} + ... + m_{n}x_{n} 
```

**StreetEasy Dataset**

You'll learn multiple linear regression by performing it on this dataset. It contains information about apartments in New York.

Introduction to Multiple Linear Regression

![StreetEasy Logo](https://content.codecademy.com/programs/machine-learning/multiple-linear-regression/streeteasy.jpg)

**<a href="https://www.streeteasy.com" target="_blank">StreetEasy</a>** is New York City's leading real estate marketplace — from studios to high-rises, Brooklyn Heights to Harlem.

In this lesson, you will be working with a dataset that contains a sample of 5,000 rentals listings in `Manhattan`, `Brooklyn`, and `Queens`, active on StreetEasy in June 2016.

It has the following columns:

- `rental_id`: rental ID
- `rent`: price of rent in dollars
- `bedrooms`: number of bedrooms
- `bathrooms`: number of bathrooms
- `size_sqft`: size in square feet
- `min_to_subway`: distance from subway station in minutes
- `floor`: floor number
- `building_age_yrs`: building's age in years
- `no_fee`: does it have a broker fee? (0 for fee, 1 for no fee)
- `has_roofdeck`: does it have a roof deck? (0 for no, 1 for yes)
- `has_washer_dryer`: does it have washer/dryer in unit? (0/1)
- `has_doorman`: does it have a doorman? (0/1)
- `has_elevator`: does it have an elevator? (0/1)
- `has_dishwasher`: does it have a dishwasher (0/1)
- `has_patio`: does it have a patio? (0/1)
- `has_gym`: does the building have a gym? (0/1)
- `neighborhood`: (ex: Greenpoint)
- `borough`: (ex: Brooklyn)

More information about this dataset can be found in the <a href="https://www.codecademy.com/content-items/d19f2f770877c419fdbfa64ddcc16edc" target="_blank">StreetEasy Dataset</a> article.

Let's start by doing exploratory data analysis to understand the dataset better. We have broken the dataset for you into:

- <a href="https://raw.githubusercontent.com/Codecademy/datasets/master/streeteasy/manhattan.csv" target="_blank">**manhattan.csv**</a>
- <a href="https://raw.githubusercontent.com/Codecademy/datasets/master/streeteasy/brooklyn.csv" target="_blank">**brooklyn.csv**</a>
- <a href="https://raw.githubusercontent.com/Codecademy/datasets/master/streeteasy/queens.csv" target="_blank">**queens.csv**</a>

StreetEasy Dataset

Great work! Let’s review the concepts before you move on:

- **Multiple Linear Regression** uses two or more variables to make predictions about another variable:

```tex
y = b + m_{1}x_{1} + m_{2}x_{2} + ... + m_{n}x_{n} 
```
- Multiple linear regression uses a set of independent variables and a dependent variable. It uses these variables to learn how to find optimal parameters. It takes a labeled dataset and learns from it. Once we confirm that it's learned correctly, we can then use it to make predictions by plugging in new `x` values.
- We can use scikit-learn's `LinearRegression()` to perform multiple linear regression.
- **Residual Analysis** is used to evaluate the regression model's accuracy. In other words, it's used to see if the model has learned the coefficients correctly.
- Scikit-learn's `linear_model.LinearRegression` comes with a `.score()` method that returns the coefficient of determination R&#178; of the prediction. The best score is 1.0.

Review

You've performed Multiple Linear Regression, and you also have the predictions in `y_predict`. However, we don't have insight into the data, yet. In this exercise, you'll create a 2D scatterplot to see how the independent variables impact prices. 

**How do you create 2D graphs?**

Graphs can be created using Matplotlib's `pyplot` module. Here is the code with inline comments explaining how to plot using Matplotlib's `.scatter()`:

```py
# Create a scatter plot
plt.scatter(x, y, alpha=0.4)

# Create x-axis label and y-axis label
plt.xlabel("the x-axis label")
plt.ylabel("the y-axis label")

# Create a title
plt.title("title!")

# Show the plot
plt.show()
```

We want to create a scatterplot like this:

<img src="https://content.codecademy.com/courses/matplotlib/visualization.png" title="Visualization"/>

Visualizing Results with Matplotlib

Now we have the training set and the test set, let's use scikit-learn to build the linear regression model!

The steps for multiple linear regression in scikit-learn are identical to the steps for simple linear regression. Just like simple linear regression, we need to import `LinearRegression` from the `linear_model` module:

 ```py
from sklearn.linear_model import LinearRegression
```

Then, create a `LinearRegression` model, and then fit it to your `x_train` and `y_train` data:

```py
mlr = LinearRegression()

mlr.fit(x_train, y_train) 
# finds the coefficients and the intercept value
```

We can also use the `.predict()` function to pass in x-values. It returns the y-values that this plane would predict:

```py
y_predicted = mlr.predict(x_test)
# takes values calculated by `.fit()` and the `x` values, plugs them into the multiple linear regression equation, and calculates the predicted y values. 
```

We will start by using two of these columns to teach you how to predict the values of the dependent variable, prices.


Multiple Linear Regression: Scikit-Learn

When trying to evaluate the accuracy of our multiple linear regression model, one technique we can use is **Residual Analysis**.

The difference between the actual value *y*, and the predicted value *ŷ* is the **residual _e_**. The equation is:

```tex
e = y - \hat{y}
```

In the StreetEasy dataset, *y* is the actual rent and the *ŷ* is the predicted rent. The real *y* values should be pretty close to these predicted *y* values.

`sklearn`'s `linear_model.LinearRegression` comes with a `.score()` method that returns the coefficient of determination R&#178; of the prediction.

The coefficient R&#178; is defined as:

```tex
1 - \frac{u}{v}
```
where *u* is the residual sum of squares:

```py
((y - y_predict) ** 2).sum()
``` 
and *v* is the total sum of squares (TSS):

```py
((y - y.mean()) ** 2).sum()
```

The TSS tells you how much variation there is in the y variable.

R&#178; is the percentage variation in y explained by all the x variables together.

For example, say we are trying to predict `rent` based on the `size_sqft` and the `bedrooms` in the apartment and the R&#178; for our model is 0.72 — that means that all the x variables (square feet and number of bedrooms) together explain 72% variation in y (`rent`).

Now let's say we add another x variable, building's age, to our model. By adding this third relevant x variable, the R&#178; is expected to go up. Let say the new R&#178; is 0.95. This means that square feet, number of bedrooms and age of the building *together* explain 95% of the variation in the rent.

The best possible R&#178; is 1.00 (and it can be negative because the model can be arbitrarily worse). Usually, a R&#178; of 0.70 is considered good.

Evaluating the Model's Accuracy

In our Manhattan model, we used 14 variables, so there are 14 coefficients:

```
[ -302.73009383  1199.3859951  4.79976742 
 -24.28993151  24.19824177 -7.58272473  
-140.90664773  48.85017415 191.4257324  
-151.11453388  89.408889  -57.89714551  
-19.31948556  -38.92369828 ]
```
- `bedrooms` - number of bedrooms
- `bathrooms` - number of bathrooms
- `size_sqft` - size in square feet
- `min_to_subway` - distance from subway station in minutes
- `floor` - floor number
- `building_age_yrs` - building's age in years
- `no_fee` - has no broker fee (0 for fee, 1 for no fee)
- `has_roofdeck` - has roof deck (0 for no, 1 for yes)
- `has_washer_dryer` - has in-unit washer/dryer (0/1)
- `has_doorman` - has doorman (0/1)
- `has_elevator` - has elevator (0/1)
- `has_dishwasher` - has dishwasher (0/1)
- `has_patio` - has patio (0/1)
- `has_gym` - has gym (0/1)

To see if there are any features that don't affect price linearly, let's graph the different features against `rent`.

**Interpreting graphs**

In regression, the independent variables will either have a positive linear relationship to the dependent variable, a negative linear relationship, or no relationship. A negative linear relationship means that as X values _increase_, Y values will _decrease_. Similarly, a positive linear relationship means that as X values _increase_, Y values will also _increase_. 

Graphically, when you see a downward trend, it means a negative linear relationship exists. When you find an upward trend, it indicates a positive linear relationship. Here are two graphs indicating positive and negative linear relationships:

![Positive and Negative Linear Relationships](https://content.codecademy.com/programs/machine-learning/multiple-linear-regression/correlations.png "Positive and Negative Linear Relationships")

Correlations

As with most machine learning algorithms, we have to split our dataset into:

- **Training set**: the data used to fit the model
- **Test set**: the data partitioned away at the very start of the experiment (to provide an unbiased evaluation of the model)
 
![Training Set vs. Testing Set](https://content.codecademy.com/programs/machine-learning/multiple-linear-regression/split.svg)

In general, putting 80% of your data in the training set and 20% of your data in the test set is a good place to start.

Suppose you have some values in `x` and some values in `y`:

```py
from sklearn.model_selection import train_test_split

x_train, x_test, y_train, y_test = train_test_split(x, y, train_size=0.8, test_size=0.2)
```

Here are the parameters:

- `train_size`: the proportion of the dataset to include in the train split (between 0.0 and 1.0)
- `test_size`: the proportion of the dataset to include in the test split (between 0.0 and 1.0)
- `random_state`: the seed used by the random number generator [optional]

To learn more, here is a <a href="https://www.codecademy.com/articles/training-set-vs-validation-set-vs-test-set" target="_blank">Training Set vs Validation Set vs Test Set article</a>.

Training Set vs. Test Set

Now let's rebuild the model using the new features as well as evaluate the new model to see if we improved!

For `Manhattan`, the scores returned:
```
Train score: 0.772546055982
Test score:  0.805037197536
```

For `Brooklyn`, the scores returned:
```
Train score: 0.613221453798
Test score:  0.584349923873
```

For `Queens`, the scores returned:
```
Train score: 0.665836031009
Test score:  0.665170319781
```

For whichever borough you used, let's see if we can improve these scores!
