Linear Regression

Given a set of points, find a line that fits the data best! Even this simple form of regression allows us to predict future points.

Start

Key Concepts

Review core concepts you need to learn to master this subject

Gradient Descent in Regression

Gradient descent step

Gradient Descent in Regression

Gradient Descent is an iterative algorithm used to tune the parameters in regression models for minimum loss.

Gradient descent step

The size of the step that gradient descent takes is called the learning rate. Finding an adequate value for the learning rate is key to achieve convergence. If this value is too large the algorithm will never reach the optimus, but if is too small it will take too much time to achieve the desired value.

Linear Regression

Lesson 1 of 1

1
The purpose of machine learning is often to create a model that explains some real-world data, so that we can predict what may happen next, with different inputs. The simplest model that we can fi…
2
In the last exercise, you were probably able to make a rough estimate about the next data point for Sandra’s lemonade stand without thinking too hard about it. For our program to make the same leve…
3
When we think about how we can assign a slope and intercept to fit a set of points, we have to define what the best fit is. For each data point, we calculate loss, a number that measures how…
4
The goal of a linear regression model is to find the slope and intercept pair that minimizes loss on average across all of the data. The interactive visualization in the browser lets you try to fi…
5
As we try to minimize loss, we take each parameter we are changing, and move it as long as we are decreasing loss. It’s like we are moving down a hill, and stop once we reach the bottom: The pr…
6
We have a function to find the gradient of b at every point. To find the m gradient, or the way the loss changes as the slope of our line changes, we can use this formula: \frac{2}{N}\sum_{i=1}^{N…
7
Now that we know how to calculate the gradient, we want to take a “step” in that direction. However, it’s important to think about whether that step is too big or too small. We don’t want to oversh…
8
How do we know when we should stop changing the parameters m and b? How will we know when our program has learned enough? To answer this, we have to define convergence. Convergence is when the…
9
We want our program to be able to iteratively learn what the best m and b values are. So for each m and b pair that we guess, we want to move them in the direction of the gradients we’ve calculat…
10
At each step, we know how to calculate the gradient and move in that direction with a step size proportional to our learning rate. Now, we want to make these steps until we reach convergence.
11
We have constructed a way to find the “best” b and m values using gradient descent! Let’s try this on the set of baseball players’ heights and weights that we saw at the beginning of the lesson.
12
Congratulations! You’ve now built a linear regression algorithm from scratch. Luckily, we don’t have to do this every time we want to use linear regression. We can use Python’s scikit-learn librar…
13
We have seen how to implement a linear regression algorithm in Python, and how to use the linear regression model from scikit-learn. We learned: - We can measure how well a line fits by measuring …