Choosing a Linear Regression Model

Learn how to choose the best linear regression model for a particular research question.

Start

Key Concepts

Review core concepts you need to learn to master this subject

Choosing a Linear Model

R-Squared

Nested Models

Adjusted R-Squared

F-test

Log Likelihood

AIC and BIC

Training and Test Sets

Choosing a Linear Model

For multivariate datasets, there are many different linear models that could be used to predict the same outcome variable. Therefore, we need methods for comparing models and choosing the “best” one for the task at hand.

Choosing a Linear Regression Model

Lesson 1 of 1

1
Introduction
In this lesson, we’ll discuss some of the ways we can compare and choose linear regression models using a variety of different methods. For example, suppose that we work at a bike rental company a…
Start
2
R-Squared
R-squared is one of the most common metrics to evaluate linear regression models. We can interpret R-squared as the proportion of variation in an outcome variable that is explained by a linear re…
Start
3
R-Squared and the Dangers of Overfitting
Let’s again suppose that we want to use the StreetEasy data to predict rental prices in NYC. We have the following two models that we want to compare: # Fit model 1 model1 = sm.OLS.from_formula(‘r…
Start
4
Adjusted R-Squared
While R-squared is useful for comparing models with different sets of predictors, we saw that it could lead to overfitting when choosing between nested models. To address this issue, we can inste…
Start
5
F-Tests
In the previous exercises, we compared nested models based on adjusted R-squared. Another way to compare nested models is by using a hypothesis test called an F-test. Suppose we want to compare th…
Start
6
Log-Likelihood
So far, we’ve used R-squared, adjusted R-squared, and an F-test to compare models. These criteria are most useful for finding a model that best fits an observed set of data. They are often used whe…
Start
7
AIC and BIC
Similarly to R-squared, log-likelihood only increases as we add more predictors to a model. In the same way that adjusted R-squared penalizes R-squared for more predictors, there are criteria that …
Start
8
Training and Test Sets
Another way of choosing a model to make predictions for new data (also called out-of-sample prediction) is by using training and test datasets. The idea is that we only use PART of our data to …
Start
9
Review
Congratulations! In this lesson, you’ve learned a number of different methods for model comparison: * For choosing a model that best represents the data we have: * R-squared * Adjusted R-squar…
Start