In this exercise, we will see how we can process the data that is stored in a CSV file.

When working with a file that contains a large amount of data, generating every possible record using tuple() is inefficient. Recall from the previous exercise that map() returns an iterator that we can use to bring in data. We can apply the higher-order functions to this iterator to process the data and only bring in the relevant set of records. For example, let’s say we only want to retain tuples for houses built after 1985. We can do this by applying filter() with the lambda lambda q: q[2] > 1985 like so:

h = filter(lambda q: q[2] > 1985, map(lambda x: house(int(x[0]), int(x[1]), int(x[2]), int(x[3])), reader))

houses = tuple(h)
print(houses) # Print out tuples of house built after 1985

Now let’s say we want the most expensive house built after 1985. We can use reduce() to do this for us like so:

h = filter(lambda q: q[2] > 1985, map(lambda x: house(int(x[0]), int(x[1]), int(x[2]), int(x[3])), reader))
houses = tuple(h)

from functools import reduce

most_expensive = reduce(lambda x, y: x if x.list_price > y.list_price else y, houses)
print(most_expensive) # Print out tuples of house built after 1985

When working with CSV files, it is important to ensure they are properly formatted; some entries might have leading or trailing whitespaces that we must remove, for example.