Learn about resource management using context managers!

Context Managers

Remember this? 

```py
def __exit__(self, *exc):
```

It's time to address the big mystery. What in the world is the `*exc` parameter in the `__exit__` method we have been writing so far?

Well, context managers play an important role in handling exceptions. Recall exceptions are errors that happen within the runtime of a code, terminating it before its completion. Within a context manager, the `__exit__` method is responsible for dealing with any exceptions. It can implement how to close the file and any other operations we want to perform if an exception occurs. 

So far, we have been using `*exc` to fill in the argument requirements for our context managers `__exit__` method. If we went back and wrote this instead: 
```py
def __exit__(self):
```

We would have been met with a puzzling error: 
```error
__exit__() takes 1 positional argument but 4 were given.
```

This is because the `__exit__` method needs four total arguments! In the past exercises, we ignored this requirement by using the `*` operator to tell the method we will pass a variable number of arguments even though we never did. It was a good way to put the above error on hold, but now let's dive into what these required arguments are and how to use them so that we can master the `__exit__` method.

The `__exit__` method has  three required arguments (in addition to `self`): 

1. An exception type: which indicates the class of exception (i.e. `AttributeError` class, or `NameError` class)
2. An exception value: the actual value of the error
3. A traceback: a report detailing the sequence of steps that caused the error and all the details needed to fix the error.

Let’s take a look at an example context manager that deals with exceptions in its `__exit__` method:

```Py
class OpenFile:
 
 def __init__(self, file, mode):
   self.file = file
   self.mode = mode

 def __enter__(self):
   self.opened_file = open(self.file, self.mode)
   return self.opened_file
 
 def __exit__(self, exc_type, exc_val, traceback):
   print(exc_type)
   print(exc_val)
   print(traceback)
   self.opened_file.close()
```
 
In this `__exit__` method, we are dealing with exceptions by adding a script that prints the exception values to the console. We can see the outcome of our simple exception handling when we run our `with` statement with an intentional failure:
 
```Py
with OpenFile("file.txt", "r") as file:
  # .see() is not a real method
  print(file.see())
```

Would output: 
```bash
<class 'AttributeError'>
'_io.TextIOWrapper' object has no attribute 'see'
<traceback object at 0x7f08dcfb5040>
```
```error
Traceback (most recent call last):
  File "script.py", line 14, in <module>
    print(file.see())
AttributeError: '_io.TextIOWrapper' object has no attribute 'see'
```

Once the `with` statement is run, we get the above error message that tells us that we have an `AttributeError`, that our object has no attribute `'see'`, and provides a traceback object. When an error occurs, the code stops, and resources (i.e., file in our earlier example) are still closed. The values of these three arguments are then thrown or suppressed. 

In contrast, if no error occurs in the `with` statement above, the `__exit__` method would have printed: 
```bash
None
None
None
```

Note that `exc_type`, `exc_value`, and `traceback` are completely arbitrary names. We can use any name we want for these parameters as long as it does not hinder the readability of our code. In general, it's best practice to be as descriptive as possible!

Now, let's experience exceptions in context managers for ourselves. 


Handling Exceptions I

In the previous exercise, we explored how to create a context manager using the `contextlib` module. However, we did not go over how to deal with errors just as we did with the class-based approach. Like any other pattern, you may run into errors when invoking your context manager using the `@contextmanager` decorator. 

For the class-based context manager, the `__exit__` method dealt with exceptions. For the decorator method, errors are most commonly dealt with within an `except` block. We will build on top of our `try`/`finally` block by incorporating an `except`. There are two main ways to deal with errors: 

- To throw an error and stop the execution of our entire program, we can:
  - Simply do nothing by excluding an `except` block

- To catch errors and continue the execution of our program, we can:
  - Handle the exception via an `except` block.

Let’s look at an example of what a decorator based context manager that catches errors can look like:
```py
from contextlib import contextmanager

@contextmanager
def open_file_contextlib(file, mode):
  open_file = open(file, mode)
 
try:
   yield open_file
 
 # Exception Handling
 except Exception as exception:
   print('We hit an error: ' + str(exception))
 
 finally:
   open_file.close()
 
with open_file_contextlib('file.txt', 'w') as opened_file:
 opened_file.sign('We just made a context manager using contexlib')
```

Notice:
- The inclusion of the `except` clause
- The `except` attempts to catch a generic `Exception` and, if it is hit, saves it to a variable `exception`.
  - Note: we can use any exception object, not just a generic one, if we know the specific exception we are trying to catch.
- The handler then prints out the error

When this context manager is called in the `with` statement above, it will hit the exception block because `.sign()` is not a file method. The output would look like this:

```bash
We hit an error: '_io.TextIOWrapper' object has no attribute 'sign'
```

This tells us what our error is, so we know what to fix. Now, let’s practice upgrading our `poem_files` context manager to catch exceptions. 

Contextlib Error Handling

Printing exceptions isn't the only way we can handle them in the `__exit__` method. An exception that occurs in a context manager can be handled in two ways: 

- If we want to throw an error when an error occurs, we can either:
  - Return `False` after the `.close()` method 
  - Do nothing

- If we want to suppress the error, we can:
  - Return `True` after the `.close()` method

Using a script similar to our earlier example, we can examine how this works:

```py
class OpenFile:
 
  def __init__(self, file, mode):
    self.file = file
    self.mode = mode
 
  def __enter__(self):
    self.opened_file = open(self.file, self.mode)
    return self.opened_file
 
  def __exit__(self, exc_type, exc_val, traceback):
    print(exc_type, exc_val, traceback)
    print("The exception has been handled")
    self.file.close()
    return True
```
 
Notice above that nothing changed except for the adding of `return True` to implement the suppression of an error. To see this in action, we'll call two `with` statements using this context manager; One that will throw an exception and another that will not. Let's observe the behavior: 
 
```py
with OpenFile("file.txt", "r") as file:
  # .see() is not a real method
  print(file.see())

with OpenFile("file.txt", "r") as file:
  print(file.read())
```
 
When we run this code, our output is as follows:
```
<class 'AttributeError'> '_io.TextIOWrapper' object has no attribute 'see' <traceback object at 0x7fedf822d180>

The exception has been handled

None None None
```

Here we see that:
- The error message we manually coded is printed but there is no automatic error message thrown by the program.
- Both `with` statements ran.

If we did not return `True`, the second (and all proceeding) `with` statements would not have run since an exception would be hit. 

Additionally, we can choose to handle a specific exception, while also suppressing it! This is useful if we want our context manager to not block the execution of other code, but also customize the output if a certain exception occurs. Here is an example of working with a `TypeError`: 

```py
class OpenFile:
 
  def __init__(self, file, mode):
    self.file = file
    self.mode = mode
 
  def __enter__(self):
    self.opened_file = open(self.file, self.mode)
    return self.opened_file
 
  def __exit__(self, exc_type, exc_val, traceback):
    self.file.close()
    if isinstance(exc_val, TypeError):
      # Handle TypeError here...
      print("The exception has been handled")
      return True
```

Notice the `if` statement that compares `exc_val` to a specific exception we are trying to catch. Anything we want to happen for this specific exception can occur in the conditional code block. Lastly, we return `True` to make sure we suppress the exception from arising and stopping the rest of our code from running.

**NOTE**: Remember that the `__exit__()` method's primary responsibility is to close the resource the context manager is working with. In this example, we close the file first, before handling the error, so that it will always execute.

Let's return to our poem context manager from earlier and implement some exception handling!

Handling Exceptions II

Congratulations! We have reached the end of the context managers lesson. Making it this far means that we have explored many of the core concepts behind context managers. Let's recap: 

**Context Managers:**
- Context managers are a form of resource management in python invoked by the `with` statement.
- They ensure that resources are closed/released after usage regardless of whether or not an error occurs. 
- They can be created from scratch using either the class-based method or the `contextlib` decorator-based method.
- Behind every context manager, there’s an  `__enter__` and `__exit__` method taking place.
- Context managers can be nested together to work with resources simultaneously.

**Class-Based Context Managers**
- They can be created from scratch with the manual implementation of the `__enter__` and `__exit__` method.
- The `__exit__` method takes three arguments: An exception type, exception value, and a traceback. The method can then handle exceptions.

**Decorator Based Context Managers**
- They can be created from scratch using the `contextlib` contextmanager decorator on a generator function
- In the `contextlib` method, the `except` block handles exception’s code block


Review

So far, we’ve only been using context managers within the context (Ha! Get it?) of one file. In most programs, there might be a need to use context managers for a couple of different scenarios that include working with multiple files! For example, we might want to:   
- Work with information from multiple files.
- Copy the same information to multiple files.
- Copy information from one file to another.

To accomplish this goal of working with multiple resources at once, context managers can be nested together in a `with` statement to manage multiple resources simultaneously.

Let’s imagine we have two files: a `teacher.txt` file and a `student.txt`. We want to copy all the information on the student file to the teachers. Our code might look like this: 

```py
with open('teacher.txt', 'w') as teacher, open('student.txt', 'r') as student:
 teacher.write(student.read())
```

Notice:
- The `with` statement is being called once but invoking two context managers. This is a single-line nested `with` statement.
- Each context manager is separated by a comma and has its own target variable.
- Our `teacher.txt` file is being opened in write mode because it will be written into and our `student.txt` is opened in read mode because we are attempting to copy the text into the teacher's file
- The resulting `teacher.txt` file will now include everything that was in the `student.txt` file.
- Here we have chosen to use the `open()` built-in function rather than a custom context manager. It is entirely possible to use our own in place of the `open()` function. 

We can also write the above `nested` context managers in a slightly different way:
``` py
with open("teacher.txt", "w") as teacher:
   with open("student.txt", "r") as student:
     teacher.write(student.read())
```

Notice that this syntax is almost similar to the first method. However, here are some differences to note:
- The `with` statement is being called twice
- The `with` statement statement to open `student.txt` in read mode is nested in the code block of the `with` statement that opens `teacher.txt` in write mode.
- This method, though slightly longer gives a clearer visual of `nesting` and is preferable when working with more than two context managers.

Let's practice nesting context manager with our `poem_files` decorator-based context manager from earlier!

 Nested Context Managers

When we use a computer, we tend to interact with its various resources. Similar to how we need to manage resources in our daily lives like our time and energy, resources on a  computer also need to be managed. 

For computers, the resources we manage usually come in the form of memory, storage, or power.  Since all resources are limited, if they are not managed well, it can lead to the computer running out of memory, space,  and even cause crashes. So how do we manage these resources? Well, one of the easiest ways (and one we already started to explore) is through the use of _context managers_. 

A context manager is an object that takes care of the assigning and releasing of resources (files, database connections, etc). Learning to properly use context managers will give our software benefits such as:

- Preventing resource leaks
- Preventing crashes
- Decreasing the vulnerability of our data
- Preventing program slow-down. 

Before we dive into context managers, let's examine what happens when we don't manage our resources properly.


Introduction to Resource Management

Now that we have an understanding of why we need context managers and the power of the `with` statement, it is essential for us to know what’s happening under the hood to gain a much deeper understanding of the concept. The best way to see the internal workings of a context manager (such as the `with` statement) is by creating our own!

One of the two approaches of creating context managers is referred to as the _class-based approach_. The class-based approach of writing context managers requires explicitly defining and implementing the following two methods inside of a class:

- An `__enter__` method
  - The `__enter__` method allows for the setup of context managers. This method commonly takes care of opening resources (like files). This method also begins what is known as the _runtime context_ - the period of time in which a script runs. In our previous examples, it was the time in which the code passed into the `with` statement code block was executed (basically everything under the `with` statement). 

- An `__exit__` method
  - The `__exit__` ensures the breakdown of the context manager. This method commonly takes care of closing open resources that are no longer in use.   

To visualize these methods and the approach, let’s take a look at a custom class-based context manager below:

```py
class ContextManager:
  def __init__(self):
    print('Initializing class...')
 
  def __enter__(self):
    print('Entering context...')
 
  def __exit__(self, *exc):
    print('Exiting context...')
```

Here, we defined a new class called `ContextManager` (to be extra explicit) and implemented the required methods. By defining these two methods, we are implementing the _context management protocol_ - a guideline for the required methods for a context manager. Don't get too caught up in the arguments passed to each method, we will talk through them in the next exercises, but they are required to not experience an error. 

Implementing the context management protocol allows us to immediately invoke the class using the `with` statement as shown below:

```py
with ContextManager() as cm:
  print('Code inside with statement')
```
Here we invoke the `ContextManager` class with a `with` statement. 
 
After running the code, our output of this context manager would be:
```
Initializing class...
Entering context...
Code inside with statement
Exiting context...
```

The above shows that our context manager class is executed in the following sequence:
1. `__init__` method
2. `__enter__` method
3. The code in the `with` statement block
4. `__exit__` method

Let's practice getting down the basics of writing a class-based context manager in addition to the execution flow before diving deeper into the `__enter__` and `__exit__` methods. 

Class Based Context Managers

We’ve learned that we can create our own context managers using the class-based method, but there’s an even simpler way of creating context managers. We can use a built-in Python module called `contextlib`!
 
The `contextlib` module allows for the creation of a context manager with the use of a generator function (a function that uses `yield` instead of `return`) and the contextlib decorator - `@contextmanager`. Instead of creating a class and definining `__enter__` and `__exit__` methods, we can use a simple function!

There are a few steps in the setup so let's take it slow and break down each step. First, we will need to import the built-in module into our script and grab the `@contextmanager` decorator: 
```py
from contextlib import contextmanager
```

Once we have successfully imported the module, we can automatically use the `@contextmanager` decorator to wrap a simple generator function: 

```py
from contextlib import contextmanager

@contextmanager
def open_file_contextlib(file, mode):
  opened_file = open(file, mode)
 try:
   yield opened_file
 finally:
   opened_file.close()
```

We are doing a few things here: 
1.  We have written a generator function called `open_file_contextlib` with the expectation that it will take in two arguments, a file and a mode.
2. We then use the built-in `open()` function to open the file (that we received as an argument) and save it to a variable called `opened_file`. 
3. The function then will attempt (via a `try` statement) to `yield` the opened file and complete whatever code we pass when we use it in conjunction with the `with` statement. More on this in a bit! 
4. Lastly the resource (file)  will be closed once all the code is done being executed.

If we think about this structure in sections relative to the class-based approach, it essentially breaks down into this: 
```py
@contextmanager
def generator_function(<parameters>):
    <setup section - equivalent to __enter__ >
    try:
        yield <value>
    finally:
        <cleanup section - equivalent to __exit__ >
```
 
Once we have created this function and denoted it as a context manager using the `@contextmanager` decorator, we can immediately use it like before in a `with` statement: 

```py
with open_file_contextlib('file.txt', 'w') as opened_file:
 opened_file.write('We just made a context manager using contexlib')
```

Following this pattern of creating context managers allows us to quickly convert generator functions to become context managers without the need to create any extra classes. Now, let's remake our poem context manager following this pattern!  

Introduction to Contextlib

One of the most common ways to manage resources in Python is to make sure the files we use in our scripts are properly closed after use. 

We already explored this concept when we used the `with` statement when operating on files. The `with` statement is the most common and pythonic way of invoking context managers in python. This means we have been using the concept of context managers all along!

Let’s recap how to use a `with` statement:
```py
with open("file_name.txt", "w") as file:
   file.write("How you gonna win when you ain't right within?")
```

Here is what is happening in our small script: 
1. The `with` statement calls the built-in `open()` function on `"file_name.txt"` with a mode of `"w"` which represents write mode. 
2. The `as` clause assigns the object opened (the file) to a target variable called `file`, which can be accessed inside of the context manager.
3. `file.write()` writes a sentence to `"file_name.txt"`

But, what exactly does this have to do with resource management? In order to answer this question, we need to take a peek behind the curtain and examine what our code looks like without a `with` statement. Here is what the same code would look like without the use of a context manager like `with`: 

```py
file = open("file_name.txt", "w")
try:
   file.write("How you gonna win when you ain't right within?")
finally:
   file.close()
```

The alternative to using `with` would require us to manually open (using `open()`) and close (using `close()`) the file we are working on. By using the `with` statement in the first example, it serves as a context manager where files are automatically closed after script completion and we don't ever have to worry about the possibility of forgetting to close a resource. Remember, leaving our resources open will hog up our finite computer resources. We are never guaranteed that Python will close the file for us if we happen to forget to do it! 

In the next exercise, we'll dive deeper into how context managers like the `with` statement are built. For now, let's start using it and seeing its power compared to the alternative `try`/`finally` clauses.  

A Familiar Face: The with Statement

Now that we know the structure of implementing our own class-based context manager. Let’s walk through a context manager that manages actual files as well as explore each of the methods we saw earlier. Here is what our context manager will look like: 

```py
class WorkWithFile:
  def __init__(self, file, mode):
    self.file = file
    self.mode = mode
 
  def __enter__(self):
    self.opened_file = open(self.file, self.mode)
    return self.opened_file
 
  def __exit__(self, *exc):
    self.opened_file.close()
```

We have written a class-based context manager called `WorkWithFile`! Let's break down each method and what happens inside of it. 

- The `__init__` method: 

  This method is standard across most classes, even ones that are not context managers themselves. In this case, we have three parameters: 

  - `self`: This is standard for any class we work with and allows us to work with methods and properties we assign to an instance of a class. 
  - `file`: Since we are working with files, we need to be able to take in a file argument when we call the class with a `with` statement. 
  - `mode`: Lastly, we need to provide the file a mode. This allows us to manage what our context manager will actually be doing, such as reading, writing, or both! 

  Both `file` and `mode` arguments allow us to accomplish the following syntax: 
  ```py
  with WorkWithFile('file.txt', 'r')
  ```

- The `__enter__` method: 

  This is where we deal with opening the file we want to work on. Since any new instance of our context manager will have a `file` and `mode` property, we can pass them into the `open()` function to open a specific file with a specific mode. Then, we save it as a variable called `self.opened_file,` and return it. 

  By returning `self.opened_file`, the file will be passed into the variable we define when we call it with the `with` statement. So for example: 
  ```py
  with WorkWithFile('file.txt', 'r') as file
  ```
  Will assign the open file `'file.txt'` to the variable called `file` that follows the `as` clause and 
thus allowing us to use it in the `with` statement code block (which we will look at shortly).

- The `__exit__` method: 

  Lastly, but one of the most important steps, we have to close the file we work on. Here we are still taking in a `*exc` argument, but we won't touch on that until the next exercise. For now, this method is solely responsible for closing the resource we opened in `__enter__`.

Now that we created our context manager, we can now use it in a `with` statement like so:
```py
with WorkWithFile("file.txt", "r") as file:
  print(file.read())
```

<details open>
<summary>Open this for a step by step breakdown of the above code!</summary>
<br>
Okay, here we go: 

1. The `with` statement executes and a context manager object is created from the class provided, with the values of the arguments required in the `__init__` method.
2. The `__enter__` method is run.
3. The `__enter__` method opens the file and returns a handle (reference) to the target variable `file` in the `with` statement. This is why in the method we `return self.opened_file`.
4. Then the code block in the `with` statement executes; printing the contents of the `file` to the console.
5. Once the code in the `with` statement is completed, The `__exit__` method is called.
6. The `__exit__` method closes the file.

</details>
<br>
Phew, these context managers are a handful! Take some time to let it all sink in, then let’s create our own context manager that works on poem files!