# Linear Regression

Given a set of points, find a line that fits the data best! Even this simple form of regression allows us to predict future points.

Start## Key Concepts

Review core concepts you need to learn to master this subject

Gradient Descent in Regression

Gradient Descent in Regression

`Gradient Descent`

is an iterative algorithm used to tune the parameters in regression models for minimum loss.

*Gradient descent* step

*Gradient descent* step

The size of the step that gradient descent takes is called the *learning rate*. Finding an adequate value for the *learning rate* is key to achieve convergence. If this value is too large the algorithm will never reach the optimus, but if is too small it will take too much time to achieve the desired value.

- 1The purpose of machine learning is often to create a model that explains some real-world data, so that we can predict what may happen next, with different inputs. The simplest model that we can fi…
- 2In the last exercise, you were probably able to make a rough estimate about the next data point for Sandra’s lemonade stand without thinking too hard about it. For our program to make the same leve…
- 3When we think about how we can assign a slope and intercept to fit a set of points, we have to define what the
*best fit*is. For each data point, we calculate**loss**, a number that measures how… - 4The goal of a linear regression model is to find the slope and intercept pair that minimizes loss on average across all of the data. The interactive visualization in the browser lets you try to fi…
- 5As we try to minimize loss, we take each parameter we are changing, and move it as long as we are decreasing loss. It’s like we are moving down a hill, and stop once we reach the bottom: The pr…
- 6We have a function to find the gradient of b at every point. To find the m gradient, or the way the loss changes as the slope of our line changes, we can use this formula: \frac{2}{N}\sum_{i=1}^{N…
- 7Now that we know how to calculate the gradient, we want to take a “step” in that direction. However, it’s important to think about whether that step is too big or too small. We don’t want to oversh…
- 8How do we know when we should stop changing the parameters m and b? How will we know when our program has learned enough? To answer this, we have to define convergence.
**Convergence**is when the… - 9We want our program to be able to iteratively
*learn*what the best m and b values are. So for each m and b pair that we guess, we want to move them in the direction of the gradients we’ve calculat… - 10At each step, we know how to calculate the gradient and move in that direction with a step size proportional to our learning rate. Now, we want to make these steps until we reach convergence.
- 11We have constructed a way to find the “best” b and m values using gradient descent! Let’s try this on the set of baseball players’ heights and weights that we saw at the beginning of the lesson.
- 12Congratulations! You’ve now built a linear regression algorithm from scratch. Luckily, we don’t have to do this every time we want to use linear regression. We can use Python’s scikit-learn librar…
- 13We have seen how to implement a linear regression algorithm in Python, and how to use the linear regression model from scikit-learn. We learned: - We can measure how well a line fits by measuring …

## What you'll create

Portfolio projects that showcase your new skills

## How you'll master it

Stress-test your knowledge with quizzes that help commit syntax to memory