Collections

Since strings are also considered containers, the `collections` module also provides a container wrapper for the string class. This contains all of the functionality of a regular string, but it includes the string's data inside of a property called `data`. Inheriting from this class allows us to create our own version of a string! Here is an example:

```python
from collections import UserString

# Create a class which inherits from the UserString class
class IntenseString(UserString):

    # A new method to capitalize and add exclamation points to our string
    def exclaim(self):
        self.data = self.data.upper() + '!!!'
        return self.data


    # Overwrite the count method to only count a certain letter
    def count(self, sub=None, start=0, end=0):
        num = 0
        for let in self.data:
            if let == 'P':
                num+=1
        return num


intense_string = IntenseString("python rules")

print(intense_string.exclaim())
print(intense_string.count())
```

This shows how we can add additional methods to the original container’s class or even overwrite existing methods. This is the same as inheriting from regular classes in Python.


Now let's create our own string class!

UserString

In Python, lists are one of the most common containers we use to work with data. Unfortunately, there are certain situations where they perform poorly. 

Let’s imagine a situation where we are processing a large document containing bug reports for an application. In order to prioritize the most important bugs, we want any normal bug reports to be appended to the end of the list and higher priority bugs to be at the front of the list (kind of like a priority list). As we fix the bugs, they can be removed from the front of the list. 

The program below is an example of what our implementation might look like using lists. Take some time to understand what the code is doing.    

```python
bug_data = []

loaded_bug_reports = get_all_bug_reports()

for bug in loaded_bug_reports:
    if bug['priority'] == 'high':
        # A list uses the insert method to append to the front
        bug_data.insert(0, bug)
    else:
        bug_data.append(bug)

# A list must provide an index to pop
next_bug_to_fix = bug_data.pop(0)
```

The problem with this implementation is that lists are not optimized for appending and popping large amounts of data, although they are great at accessing data at any index which you provide. 

To solve this problem, we can use `deque` containers. These are similar to lists, but they are optimized for appending and popping to the front and back, rather than having optimized accessing. Because of this, they are great for working with data where you don’t need to access elements in the middle very often or at all.  

Let's observe our same program but implemented with a `deque`: 
```python
from collections import deque

bug_data = deque()

loaded_bug_reports = get_all_bug_reports()

for bug in loaded_bug_reports:
    if bug['priority'] == 'high':
        # With a deque, we can append to the front directly
        bug_data.appendleft(bug)
    else:
        bug_data.append(bug)

# With a deque, we can pop from the front directly
next_bug_to_fix = bug_data.popleft()
```

More information about the `deque` container can be found in the [Python Documentation](https://docs.python.org/3/library/collections.html#collections.deque)

Now let's try working with a larger amount of data where a `deque` is more beneficial! 

Deque

Now that we've had a refresher on some of the built-in containers Python provides, let's dive into the `collections` module. 

The classes from the `collections` module are very similar to the built-in containers we've been already using, but they contain new methods and utilities. Each of these specialized containers focuses on a certain improvement to its built-in counterpart such as optimizing performance, better organization, fewer steps for performing tasks, and more!

In order to use classes from the `collections` module, we will first need to import the module into our code. This is different than the previous containers we’ve seen because they were built-in and did not require an import.

Here are some of the various ways importing will look like:

```python
# To import a single class or multiple classes
from collections import name_of_class, name_of_another_class

# To import all classes in the collections module
from collections import *

# Another way to import all classes in a module
import collections
```

For a more specific example, here is what importing the `OrderedDict` (one of the specialized containers) would look like. We will dive deeper into the details of this particular container later on in this lesson but for now just observe the syntax: 

```python
from collections import OrderedDict

orders = OrderedDict({'order_4829': {'type': 't-shirt', 'size': 'large', 'price': 9.99},
          'order_6184': {'type': 'pants', 'size': 'medium', 'price': 14.99},
          'order_2905': {'type': 'shoes', 'size': 12, 'price': 22.50}})

orders.move_to_end('order_4829')
orders.popitem()
```

You might have noticed that this is a similar example to the dictionary review in the last exercise, but there are some new methods that provide even more functionality to the traditional dictionary.

Here is a list of all of the advanced containers we will be looking at in this lesson:

**Advanced Containers**
 * `deque`
 * `namedtuple`
 * `Counter`
 * `defaultdict`
 * `OrderedDict`
 * `ChainMap`

**Container Wrappers**
 * `UserDict`
 * `UserList`
 * `UserString`

Introduction to Specialized Containers

One of the most common tasks we might have to do in a program is to count instances of an element in a collection. Below, we will examine a Python list and look at one particular way we count elements, and then see how the `collections` module allows us to improve upon our implementation using the `Counter` collection. 

First, lets define a list of items. Since we have been working on a application for our clothing store, lets stick with clothing items: 

```python
clothes_list = ['skirt', 'hoodie', 'dress', 'blouse', 'jeans', 'shoes', 'skirt', 'skirt', 'jeans', 'hoodie', 'boots', 'jeans', 'jacket', 't-shirt', 'skirt', 'skirt', 'dress', 'shoes', 'blouse', 'hoodie', 'skirt', 'boots', 'shoes', 'boots', 'jeans', 'hoodie', 'blouse', 'hoodie', 'shoes', 'shoes', 'blouse', 'boots', 'blouse', 'hoodie', 't-shirt', 'jeans', 'dress', 'skirt', 'jacket', 'boots', 'skirt', 'dress', 'jeans', 'jeans', 'jacket', 'jeans', 'shoes', 'dress', 'hoodie', 'blouse']
```

If we wanted to create a representation of how many of each item exist in our collection, we could use a loop and a dictionary. Here is what it might look like: 

```python
counted_items = {}
for item in clothes_list:
   if item not in counted_items:
       counted_items[item] = 1
   else:
       counted_items[item] += 1

print(counted_items)
```

This would output (in no particular order): 
```bash
{'skirt': 8, 'hoodie': 7, 'dress': 5, 'blouse': 6, 'jeans': 8, 'shoes': 6, 'boots': 5, 'jacket': 3, 't-shirt': 2}
```

While this is a perfectly sound solution to our counting problem, we can actually accomplish this goal much quicker using the `Counter` container!

The `Counter` container instantly counts elements for any hashable object. It stores the data as a dictionary where the keys are the elements and the values are the number of occurrences. Here is what the same problem looks like, but with the `Counter` container:

```python
from collections import Counter

counted_items = Counter(clothes_list)
print(counted_items)
```

Would Output:
```bash
Counter({'skirt': 8, 'jeans': 8, 'hoodie': 7, 'blouse': 6, 'shoes': 6, 'dress': 5, 'boots': 5, 'jacket': 3, 't-shirt': 2})
```

This allows us to create a much more elegant solution without many lines of code. Additionally, the `Counter` class has methods that provide further utility when working with our data. These methods include mathematical operations for subtracting one count dictionary from another, finding the most common elements, and even generating a new list of elements based on the number of occurrences. 

For more information about the `Counter` class, take a look at the [Python documentation](https://docs.python.org/3/library/collections.html#collections.Counter).

Let's now practice using the `Counter` class!


Counter

Dictionaries are another popular type of collection we use in our programs. Although they are great for a lot of situations, applications that rely heavily on them always run into a common issue. This issue deals with how to handle missing keys! 

When we try to access a key-value pair in a dictionary, but the key does not exist, a dictionary will normally throw a `KeyError`. Take a look at this example of accessing an invalid key from a normal dictionary:

```python
prices = {'jeans': 19.99, 'shoes': 24.99, 't-shirt': 9.99, 'blouse': 19.99}

print(prices['jacket'])
```

Would output: 
```error
KeyError: 'jacket'
```

Dealing with frequent `KeyError` exceptions can be quite cumbersome and in certain cases, it might be better to avoid throwing an error. One of the ways Python offers to deal with this issue is by having a default missing value in the dictionary, and this is exactly what the `defaultdict` collection does. Let's explore this new collection together! 

First, we import the class and set the default value:
```python
from collections import defaultdict

validate_prices = defaultdict(lambda: 'No Price Assigned')
```

Next, we can set the keys and values like a regular `dict`:
```python
validate_prices['jeans'] = 19.99
validate_prices['shoes'] = 24.99
validate_prices['t-shirt'] = 9.99
validate_prices['blouse'] = 19.99
```

Finally, we access an invalid key to observe the result:
```python
print(validate_prices['jacket'])
```

Would output:
```bash
No Price Assigned
```

Notice the following: 

- We set the default value using a lambda expression.
- Any time we try to access a key that does not exist, it automatically updates our `defaultdict` object by 
  creating the new key-value pair using the missing key and the default value. 


To read more about the `defaultdict` container, take a look at the [Python Documentation](https://docs.python.org/3/library/collections.html#collections.defaultdics)

Now let's try using a `defaultdict` to validate new content on our clothing store website!

DefaultDict

There is another way to store dictionaries or other mappings in Python. We have looked at the `defaultdict` and `OrderedDict` so far and they handle a lot of situations so what else could we possibly need?

Well, the `ChainMap` container allows us to store many mappings in an ordered group, but lookups (accessing the value using a key) are repeated for every mapping inside of the `ChainMap` until something is found or the end is reached.  If we try to modify the data in any way, then only the first mapping in the `ChainMap` will receive the changes. When accessing data, one way to think of the `ChainMap` is that it treats all of the stored dictionaries as one large dictionary, where if there are repeated keys, then the first found result is returned. Let’s see what this looks like with an example using a customer’s clothing dimensions!

First, we import the `ChainMap` container and set up our data.
```python
from collections import ChainMap

customer_info = {'name': 'Dmitri Buyer', 'age': '31', 'address': '123 Python Lane', 'phone_number': '5552930183'}

shirt_dimensions = {'shoulder': 20, 'chest': 42, 'torso_length': 29}

pants_dimensions = {'waist': 36, 'leg_length': 42.5, 'hip': 21.5, 'thigh': 25, 'bottom': 18}
```

Next, we initialize a `ChainMap` with the mappings which we want to use. In this case, the mappings are the dimensions dictionaries.
```python
customer_data = ChainMap(customer_info, shirt_dimensions, pants_dimensions)
```

Now we can access values from any of the stored mappings.
```python
customer_leg_length = customer_data['leg_length']
```

The parents property skips the first mapping and returns everything else (all of the parents of the first mapping).
```python
customer_size_data = customer_data.parents
```

We can directly modify the data only in the first dictionary.
```python
customer_data['address'] = '456 ChainMap Drive'
```

Note: In order to modify data from dictionaries which are deeper in the `ChainMap`, we will need to iterate through the dictionaries which are stored inside of it.

As we can see in this example, we create a new `ChainMap` using three different dictionaries. This allows us to access any of the `key:value` pairs stored inside.

Another interesting concept that the `ChainMap` uses is the concept of a parent mappings. If we use the `.parents` property, all mappings except the first one will be returned. This is because those mappings are considered to be the parent mappings to the first one. You can add a new "child" mapping to the front of the list of mappings using the `.new_child()` method. 

To find out more about this container, check out the [Python Documentation](https://docs.python.org/3/library/collections.html#collections.ChainMap).

Now let's use a `ChainMap` to keep track of our clothes business profits for the last 12 months!

ChainMap

Tuples, another common built-in container, are very useful for grouping together data that does not need to be modified in the future. Tuples do however run into an issue when they host various data and even nested data. Let’s examine a tuple containing actor data:
```python
actor_data_tuple = ('Leonardo DiCaprio', 1974, 'Titanic', 1997)
```

In this example, we are storing details about an actor that is unlikely to change (we can assume for now the actor's name will not change). While the tuple does a great job of creating a container that can keep ordered immutable data, it can become quite confusing to represent properties using numerical indices. For example: 

```py
actor_data_tuple[3]
```

Unless we explicitly define a variable name that describes what the third index represents, it's very hard to tell what data we are talking about. We would also need to make separate variables for each property! Thanks to the `collections` module, we have a solution to this problem.

The `namedtuple` collection allows us to have an immutable tuple object, but every element becomes self-documented. Let's examine our actor example but now refactored to use a `namedtuple`:

```py
from collections import namedtuple

# General Structure: namedtuple(typename, field_names, *, rename=False, defaults=None, module=None)

ActorData = namedtuple('ActorData', ['name', 'birth_year', 'movie', 'movie_release_date'])
```

In this example, we are defining an instance of the `namedtuple` collection with a _typename_ called `'ActorData'` and a sequence of strings called _field_names_ that represent the labels for the data we want to store. 

We are saying we want our `namedtuple` to be called `'ActorData'` and for it to have `name`, `birth_year`, `movie`, and `movie_release_date` properties. It's like creating a label system for the type of data inside of the tuple!

We can then define an instance of our `ActorData`:
```py
actor_data = ActorData('Leonardo DiCaprio', 1974, 'Titanic', 1997)
```

This then allows us to access the mapped property value to its associated name from before using the `.` notation: 
```py
print(actor_data.name)
```

Would return: 
```bash
Leonardo DiCaprio
```

Some things to note about `namedtuples`: 
- You may have noticed we use a CapWords convention when defining our `namedtuple`. This is because `namedtuple` actually returns a _subclass_ and thus falls under the conventions we use for classes. 
- The `field_names` argument can alternatively be a single string with each fieldname separated by whitespace and/or commas, for example, `'x y'` or `'x, y'`.
- At first glance, `namedtuples` might seem like it is trying to replicate a dictionary. While the key idea of labeling properties is the same in both structures, `namedtuples` have some key advantages over a regular dictionary: 
  - They are immutable and maintain their order, while a dictionary does not. 
  - They are more lightweight than dictionaries and take no more memory than a regular tuple.

There are other useful methods that a `namedtuple` uses such as converting from a `namedtuple` to a `dict`, replacing elements and field names, and even setting default values for attributes. More information about `namedtuple` containers can be found in the [Python Documentation](https://docs.python.org/3/library/collections.html#collections.namedtuple). 

Let's now practice using the `namedtuple` container!


Named Tuple

Nice work! We have learned about all sorts of advanced containers which can help make programming easier, more organized, and more optimized! We even learned how to make our own advanced containers using container wrappers. Let’s review the use case for each of the advanced containers from the `collections` class:

* `deque`
 * An advanced container which is optimized for appending and popping items from the front and back. For accessing many elements positioned elsewhere, it is better to use a `list`. 
* `namedtuple`
 * The `namedtuple` lets us create an immutable data structure similar to a `tuple`, but we don’t have to access the stored data using indices. Instead, we can create instances of our `namedtuple` with named attributes. We can then use the `.` operator to retrieve data by the attribute names.
* `Counter`
 * This advanced container automatically counts the data within a hashable object which we pass into it’s constructor. It stores it as a dictionary where the keys are the elements and the values are the number of occurrences. 
* `defaultdict`
 * An advanced container which behaves like a regular dictionary, except that it does not throw an error when trying to access a key which does not exist. Instead, it creates a new `key:value` pair where the value defaults to what we provide in the constructor for the `defaultdict`.
* `OrderedDict`
 * The `OrderedDict` combines the functionality of a `list` and a `dict` by preserving the order of elements, but also allowing us to access values using keys without having to provide an index for the position of stored dictionaries. 
* `ChainMap`
 * This interesting container combines multiple mappings into a single container. When accessing a value using a key, it will search through every mapping contained within until a match is found or the end is reached. It also provides some useful methods for grouping parent and child mappings. 
* `UserDict`
 * This is a container wrapper which lets us create our own version of a dictionary
* `UserList`
 * This is a container wrapper which lets us create our own version of a list
* `UserString`
 * This is a container wrapper which lets us create our own version of a string

Let's try combining some of these concepts by adding one last addition to our clothes store app!

Review of Specialized Containers

In Python, wrappers are modifications to functions or classes which change the behavior in some way. They are called wrappers because they "wrap" around the existing code to modify it. This is most commonly used with function wrapping, but we can also wrap classes. Let's take a look at an example of a class wrapper: 

First, we need a class to wrap around.
```python
class Customer:

  def __init__(self, name, age, address, phone_number):
    self.name = name
    self.age = age
    self.address = address
    self.phone_number = phone_number
```

Next, we create a wrapper class which stores an object of the class we are wrapping around. It also includes some additional functionality.
```python
class CustomerWrap(Customer):

  def __init__(self, name, age, address, phone_number):
    self.customer = Customer(name, age, address, phone_number)

  def display_customer_info(self):
    print('Name: ' + self.customer.name)
    print('Age: ' + str(self.customer.age))
    print('Address: ' + self.customer.address)
    print('Phone Number: ' + self.customer.phone_number)
```

Finally, we can create an object from the wrapper class to access the new functionality and the wrapped class contained inside.
```python
customer = CustomerWrap('Dmitri Buyer', 38, '123 Python Avenue', '5557098603')
customer.display_customer_info()

# Output
# Name: Dmitri Buyer
# Age: 38
# Address: 123 Python Avenue
# Phone Number: 5557098603
```

Wrapper classes allow us to create different variations of classes with different purposes while avoiding duplicate code. Since we use an instance of the wrapped class inside of it, it preserves all of the attributes and methods from the wrapped class and keeps us from having to re-type all of the code.

In the case of containers, the `collections` class has three different wrapper classes set up for us to modify! Because of this, we can refer to them as wrapper containers. The advanced containers which we have already been looking at are variations of the standard built-in containers, so using wrapper containers allows us to create our own versions as well.

The three wrapper containers we will be looking at are:

* `UserDict`
* `UserList`
* `UserString`


Container Wrappers

When keeping track of many different dictionaries with the built-in Python containers, we could try storing dictionaries in a list, or even a dictionary of dictionaries. This may work in some cases, but there are a few problems which might come up. 

When storing dictionaries in a list, the order is preserved, but we have to access the elements by their index before we can access the dictionary:

```python
first_order = {'order_2905': {'type': 'shoes', 'size': 12, 'price': 22.50}}
second_order = {'order_6184': {'type': 'pants', 'size': 'medium', 'price': 14.99}}
third_order = {'order_4829': {'type': 't-shirt', 'size': 'large', 'price': 9.99}}

list_of_dicts = [first_order, second_order, third_order]
```

In order to get the price of a specific order, we must know the index of it already before we can access the dictionary data stored inside:

```python
print(list_of_dicts[1]['order_6184']['price'])

# Output
# 14.99
```

On the other hand, depending on the Python version, the `dict` container can preserve the order, but it is difficult to move elements around:

```python
dict_of_dicts = {}
dict_of_dicts.update(first_order)
dict_of_dicts.update(second_order)
dict_of_dicts.update(third_order)

print(dict_of_dicts['order_6184']['price'])

# Output
# 14.99
```
Note: The `dict` class is unordered in earlier versions of python, so implementing it this way must have version 3.6 or greater.

To solve these issues, we can use an `OrderedDict`!

The `OrderedDict` container allows us to access values using keys, but it also preserves the order of the elements inside of it. Let’s take a closer look at the example of processing customer orders from earlier in the lesson: 

Import and create an `OrderedDict`.
```python
from collections import OrderedDict

orders = OrderedDict()
```

The order of the data is preserved when adding it to the `OrderedDict`:
```python
orders.update({'order_2905': {'type': 'shoes', 'size': 12, 'price': 22.50}})
orders.update({'order_6184': {'type': 'pants', 'size': 'medium', 'price': 14.99}})
orders.update({'order_4829': {'type': 't-shirt', 'size': 'large', 'price': 9.99}})
```

Data can be accessed using keys like a normal dictionary:
```python
# Get a specific order
find_order = orders['order_2905']
```

The order can be retrieved by converting it to a list then accessing by index:
```python
# Get the data in a list format
orders_list = list(orders.items())
third_order = orders_list[2]
```

When using an `OrderedDict`, we are able to use its methods for moving the data around. We can move an element to the back or front and pop the data from the back or front of the `OrderedDict`:
```python
# Move an item to the end of the OrderedDict
orders.move_to_end('order_4829')

# Pop the last item in the dictionary
last_order = orders.popitem()
```
Note: These two methods also accept boolean arguments which determine if the element is moved / popped from the front or back of the `OrderedDict`.

For more information about the `OrderedDict` container, take a look at the [Python Documentation](https://docs.python.org/3/library/collections.html#collections.OrderedDict).

Now let's try using an `OrderedDict` in our clothes app!


OrderedDict

In Python, there are many ways to store and organize data. So far, we have experienced adding elements to a list, writing key-value pairs to a dictionary, or even accessing data within tuples. 

Any object which stores data is called a _container_. If you have written code in Python, you have likely been using containers this whole time! 

We are familiar with Python's built-in containers (such as lists or dictionaries), but there are many other containers that exist in Python. These containers each specialize in a specific job and can be imported into your code from other modules or even be custom-made! In this lesson, we will be looking at these specialized containers from the Python `collections` module. 

We will start to dive deeper in the next exercises, but for now, let's take some time to review some of the common built-in containers we are most familiar with: 

**Lists**

Lists are an ordered group of elements. Elements can be added, removed, accessed, and modified.

```python
products = ['t-shirt', 'pants', 'shoes', 'dress', 'blouse']

products.append('jacket')
products.sort()
products.remove('shoes')
```

**Tuples**

Tuples are immutable objects which group multiple elements together. They are similar to lists, except that they cannot be modified once created. 

```python
searched_terms = ('clothes', 'phone', 'app', 'purchase', 'clothes', 'store', 'app', 'clothes')

term = searched_terms[2]
num_of_occurrences = searched_terms.count('clothes')
```

**Dictionaries**

Dictionaries are unordered groups of key-value pairs. 

```python
orders = {'order_4829': {'type': 't-shirt', 'size': 'large', 'price': 9.99}, 
          'order_6184': {'type': 'pants', 'size': 'medium', 'price': 14.99}
         }

order_4829_price = orders['order_4829']['price']
order_6184_size = orders['order_6184']['size']
orders['order_4829']['size'] = 'x-large'
num_of_orders = len(orders)
```

**Sets**

Sets are unordered groups of elements that cannot contain duplicates, elements cannot be modified. 

```python
old_products_set = {'t-shirt', 'pants', 'shoes'}
new_products_set = {'t-shirt', 'pants', 'blouse', 'dress'}
updated_products = new_products_set | old_products_set
removed_products = old_products_set - new_products_set
```

You can learn more about these built-in containers in [earlier lessons](https://www.codecademy.com/learn/learn-python-3) or in the [Python Documentation](https://docs.python.org/3/tutorial/datastructures.html).

Now that we have reviewed the most common containers in Python, let's practice using them, and then move on to exploring the specialized containers we mentioned earlier! 

Recap: Python Containers

In this lesson, we have seen advanced containers which modify the functionality of a dictionary such as the `defaultdict` and `OrderedDict`. The `UserDict` container wrapper lets us create our own version of a dictionary. This class contains all of the functionality of a normal `dict`, except that we can access the dictionary data through the `data` property. Here’s an example of creating a modified dictionary:

```python
from collections import UserDict

# Create a class which inherits from the UserDict class
class DisplayDict(UserDict):
    # A new method to increase the dictionary's functionality
    def display_info(self):
        print("Number of Keys: " + str(len(self.keys())))
        print("Keys: " + str(list(self.keys())))
        print("Number of Values: " + str(len(self.values())))
        print("Values: " + str(list(self.values())))

    # We can also overwrite a method from the dictionary class
    def clear(self):
        print("Deleting all items from the dictionary!")
        super().clear()

disp_dict = DisplayDict({'user': 'Mark', 'device': 'desktop', 'num_visits': 37})

disp_dict.display_info()

disp_dict.clear()
```

As shown in this code example, we can add additional methods and overwrite methods from the `UserDict `class. This is the same as inheriting from regular classes in Python.

Now let's create our own `dict` class!


UserDict

Not only can we create our own version of a dictionary, the `UserList` wrapper container lets us create our own `list` as well! This class contains all of the functionality of a regular `list`, but it also has a property called `data` which allows us to access the list contents directly. Here is an example of a modified `list` using the container wrapper:

```python
from collections import UserList

# Create a class which inherits from the UserList class
class CondenseList(UserList):

    # A new method to remove duplicate items from the list
    def condense(self):
        self.data = list(set(self.data))
        print(self.data)


    # We can also overwrite a method from the list class
    def clear(self):
        print("Deleting all items from the list!")
        super().clear()

condense_list = CondenseList(['t-shirt', 'jeans', 'jeans', 't-shirt', 'shoes'])

condense_list.condense()

condense_list.clear()
```

As shown in this code example, we can add additional methods and overwrite methods from the `UserList ` class. This is the same as inheriting from regular classes in Python.

Let's try creating our own `list` class!
