Learn to how perform linear algebra operations in Python using NumPy

Linear Algebra with Python

In addition to having built-in support for many linear algebra-related operations, Let's see how NumPy can create special matrices, such as the identity matrix. 

An identity matrix can be constructed using the `np.eye()` functions, which takes an integer argument that determines the *n x n* size of the square identity matrix.
```python
# 4x4 identity matrix
identity = np.eye(4)
```

In the output terminal, `identity` renders as:

```
[[1. 0. 0. 0.]
 [0. 1. 0. 0.]
 [0. 0. 1. 0.]
 [0. 0. 0. 1.]]
```

A matrix or vector of all zeros can be constructed using the `np.zeros()` function, which takes in a tuple argument for the shape of the NumPy array filled with zeros.
```python
# 5-element vector of zeros
zero_vector = np.zeros((5))
```
This will output in the terminal as:
```
[0. 0. 0. 0. 0.]
```
```python
# 3x2 matrix of zeros
zero_matrix = np.zeros((3,2))
```
This will output in the terminal as:
```
[[0. 0.]
 [0. 0.]
 [0. 0.]]
```

The transpose of a matrix can be accessed using the `.T` attribute of a NumPy array as shown below:
```python
A = np.array([[1,2],[3,4]])
A_transpose = A.T
```
If we print `A` and `A_transpose` out in the terminal, `A` outputs as:
```
[[1 2]
 [3 4]]
```
While `A_transpose` outputs as:
```
[[1 3]
 [2 4]]
```
Since `A_transpose` is the transpose of `A`, the rows and columns are swapped.

Special Matrices

In this lesson, we aim to implement the linear algebra concepts using Python. These Python implementations will include vectors, matrices, and some of their respective operations. Additionally, we will again look at problems like solving systems of linear equations using linear algebra.

To perform these linear algebra tasks in Python, we will work with NumPy, a popular numerical computing library. In addition to various built-in linear algebra capabilities, NumPy also works very well with other popular Python libraries used for data science applications such as pandas, SciPy (NumPy's parent library), and Matplotlib.

This image shows two intersecting planar graphs.

Introduction

The core of using NumPy effectively for linear algebra is using NumPy arrays. NumPy arrays are n-dimensional array data structures that can be used to represent both vectors (1-dimensional array) and matrices (2-dimensional arrays).

A NumPy array is initialized using the `np.array()` function, and including a Python list argument or Python nested list argument for arrays with more than one dimension.

For example, the following creates a NumPy array representation of a vector:
```python
v = np.array([1, 2, 3, 4, 5, 6])
``` 

We can also create a matrix, which is the equivalent of a two-dimensional NumPy array, using a nested Python list:

```python
A = np.array([[1,2],[3,4]])
``` 
If we print this NumPy array, it will look like the following:

```
[[1 2]
 [3 4]]
```

Matrices can also be created by combining existing vectors using the `np.column_stack()` function:

```python
v = np.array([-2,-2,-2,-2])
u = np.array([0,0,0,0])
w = np.array([3,3,3,3])

A = np.column_stack((v, u, w))
print(A)
```
Output:
```
[[-2  0  3]
 [-2  0  3]
 [-2  0  3]
 [-2  0  3]]
```

To access the shape of a matrix or vector once it's been created as a NumPy array, we call the `.shape` attribute of the array variable:

```python
A = np.array([[1,2],[3,4]])
print(A.shape)
``` 
Here, the output will be:

```
(2, 2)
```
since there are two rows and two columns.

To access individual elements in a NumPy array, we can index the array using square brackets. Unlike regular Python lists, we can index into all dimensions in a single square bracket, separating the dimension indices with commas. 

Thus in order to index the element equal to *2* in matrix *A*, we can do the following:
```python
A = np.array([[1,2],[3,4]])
print(A[0,1])
``` 
The first index accesses the specific row, while the second index accesses the specific column. Note that rows and columns are zero-indexed. In this example, the output will be the following:

```
2
```

We can also select a subset or entire dimension of a NumPy array using a colon. For example, if we want the entire second column of a matrix, we can index the second column and use an empty colon to select every row as such:
```python
A = np.array([[1,2],[3,4]])
print(A[:,1])
``` 
Output:
```
[2 4]
```

Note: `[2 4]` is a column vector, but it outputs in the terminal horizontally.

Using NumPy Arrays

Congratulations! You have completed your journey into linear algebra operations with NumPy. 

The Python library NumPy provides a powerful tool to programmatically perform linear algebra. A fundamental building block of NumPy, the NumPy array, allows us to represent vectors and matrices. These arrays can perform operations directly, such as addition and scalar multiplication using `+` and `*`, rerspectively.

NumPy also has built-in functions that allow us to perform more advanced operations like dot products, transposes, and matrix multiplication.

Finally, NumPy has a `numpy.linalg` sublibrary that provides more advanced functionality. This includes finding the norm (or magnitude/length) of a vector, finding the inverse of a square matrix, and solving a system of linear equations in *Ax=b* form.

All this functionality will allow us to use Python and NumPy to solve many linear algebra problems, including linear regression, in future lessons and articles.

Review

Now that we know how to use NumPy arrays to create vectors and matrices, we can learn how to perform various linear algebra operations using Python.

To multiply a vector or matrix by a scalar, we use inbuilt Python multiplication between the NumPy array and the scalar:
```python
A = np.array([[1,2],[3,4]])
4 * A
```
Written out mathematically, this is:

```tex
4 \begin{bmatrix}
1 & 2 \\
3 & 4 
\end{bmatrix}=
\begin{bmatrix}
4 & 8 \\
12 & 16 
\end{bmatrix}
```

Therefore, our output in Python is:
```
[[ 4  8]
 [12 16]]
```

To add equally sized vectors or matrices, we can again use inbuilt Python addition between the NumPy arrays.
```python
A = np.array([[1,2],[3,4]])
B = np.array([[-4,-3],[-2,-1]])
A + B
```

Written out mathematically, this is:

```tex
\begin{bmatrix}
1 & 2 \\
3 & 4 
\end{bmatrix}+
\begin{bmatrix}
-4 & -3 \\
-2 & -1 
\end{bmatrix}=
\begin{bmatrix}
-3 & -1 \\
1 & 3 
\end{bmatrix}
```
Therefore, our output in Python is:
```
[[ -3 -1]
 [1 3]]
```

Vector dot products can be computed using the `np.dot()` function:
```
v = np.array([-1,-2,-3])
u = np.array([2,2,2])
np.dot(v,u)
```
Written out mathematically, this is:

```tex
\begin{bmatrix}
-1 \\
-2\\ 
-3 \\
\end{bmatrix}\cdot
\begin{bmatrix}
2 \\
2 \\
2
\end{bmatrix} =
-12
```
Therefore, our output in Python is:
```
-12
```
Matrix multiplication is computed using either the `np.matmul()` function or using the `@` symbol as shorthand. It is important to note that using the typical Python multiplication symbol `*` will result in an elementwise multiplication instead.
```python
A = np.array([[1,2],[3,4]])
B = np.array([[-4,-3],[-2,-1]])

# one way to matrix multiply
np.matmul(A,B)
# another way to matrix multiply
A@B
```
Written out mathematically, this is:

```tex
\begin{bmatrix}
1 & 2 \\
3 & 4 
\end{bmatrix}\cdot
\begin{bmatrix}
-4 & -3 \\
-2 & -1 
\end{bmatrix}=
\begin{bmatrix}
-8 & -5 \\
-20 & -13 
\end{bmatrix}
```
Therefore, our Python output is:
```
[[ -8  -5]
 [-20 -13]]
```

Using NumPy for Linear Algebra Operations

Moving past special matrices, there are some more advanced linear algebra operations we can perform using NumPy. To start, we will be using the `numpy.linalg` sublibrary to perform the following operations:

The "norm" (or length/magnitude) of a vector can be found using `np.linalg.norm()`:
```python
v = np.array([2,-4,1])
v_norm = np.linalg.norm(v)
```
This outputs `v_norm` as:
```
4.58257569495584
```

The inverse of a square matrix, if one exists, can be found using `np.linalg.inv()`:
```python
A = np.array([[1,2],[3,4]])
print(np.linalg.inv(A))
```
This outputs:
```
[[-2.   1. ]
 [ 1.5 -0.5]]
```

Finally, we can actually solve for unknown variables in a system on linear equations in *Ax=b* form using `np.linalg.solve()`, which takes in the *A* and *b* parameters. Given:
```
 x + 4y -  z = -1
-x - 3y + 2z =  2
2x -  y - 2z = -2
```
We convert to *Ax=b* form and solve.
```python
# each array in A is an equation from the above system of equations
A = np.array([[1,4,-1],[-1,-3,2],[2,-1,-2]])
# the solution to each equation
b = np.array([-1,2,-2])
# solve for x, y, and z
x,y,z = np.linalg.solve(A,b)
```
If we `print((x, y, z))`, we see the following output:
```
(0, 0, 1.0)
```
showing that the solution to the system of equations is:
```
x = 0
y = 0
z = 1
```
