We often have data with missing elements, as a result of a problem with the data collection process or errors in the way the data was stored. The missing elements normally show up as NaN (or Not a Number) values:

The num_guests value for the 3rd row is missing, and the bill value for the 5th row is missing. Some calculations we do will just skip the NaN values, but some calculations or visualizations we try to perform will break when a NaN is encountered.

Most of the time, we use one of two methods to deal with missing values.

Method 1: drop all of the rows with a missing value

We can use .dropna() to do this:

bill_df = bill_df.dropna()

This command will result in the DataFrame without the incomplete rows:

If we wanted to remove every row with a NaN value in the num_guests column only, we could specify a subset:

bill_df = bill_df.dropna(subset=['num_guests'])

Method 2: fill the missing values with the mean of the column, or with some other aggregate value.

We can use .fillna() to do this:

bill_df = bill_df.fillna(value={"bill":bill_df.bill.mean(), "num_guests":bill_df.num_guests.mean()})

This command will result in the DataFrame with the respective mean of the column in the place of the original NaNs: