Introduction to Differential Calculus

We have talked at length about derivatives abstractly. But how do we actually find derivative functions? We will outline this in this exercise and the next one.

The answer is through a series of "rules" that we’ll introduce in this exercise and the next.  These rules are building blocks, and by combining rules, we can find and plot the derivatives of many common functions.  

To start, we can think that derivatives are *linear operators*, fancy language that means:

```tex
\frac{d}{dx} c f(x) = c f'(x)
```
This means that we can pull constants out when calculating a derivative. For example, say we have the following function:
```tex
f(x) = 4x^{2}
``` 
We can define the derivative as the following:
```tex
\frac{d}{dx}4x^{2} = 4\frac{d}{dx}x^{2}
```  

We can also say that
```tex
\frac{d}{dx}(f(x) + g(x)) = \frac{d}{dx}f(x) + \frac{d}{dx}g(x)
``` 
This means that the derivative of a sum is the sum of the derivatives. For example, say we have the following function:
```tex
f(x) = x^{2} + x^{3}
```
We can define the derivative as the following:
```tex
\frac{d}{dx}(x^{2}+x^{3}) =  \frac{d}{dx}x^{2}+ \frac{d}{dx}x^{3}
```

When products get involved, derivatives can get a bit more complicated. For example, let's say we have a function in the following form of two separate "parts":
```tex
f(x) = u(x)v(x)
```
We define the *product rule* as:
```tex
f'(x) =u(x)v'(x) + v(x)u'(x) 
```

Mnemonically, we remember this as “first times derivative of second plus second times derivative of first.” For example, say we have the following equation that is the product of two :
```tex
f(x) = x^2log(x)
```
We would use the product rule to find the derivative of f(x):
```tex
f'(x) = x^2\frac{d}{dx}(log(x))+log(x)\frac{d}{dx}(x^2)
```

Finally, one last rule to take with us into the next exercise is that the derivative of a constant is equal to 0:

```tex
\frac{d}{dx}c = 0
```
This is because for a constant there is never any change in value, so the derivative will always equal zero. For example, we can say:
```tex
\frac{d}{dx}5 = 0
```

Calculating Derivatives

The first concept we will look at is something called a *limit*. Limits quantify what happens to the values of a function as we approach a given point. This can be defined notationally as:
```tex
\lim_{x \rightarrow 6} f(x) = L
```
We can read this in simple terms as “the limit as *x* goes to 6 of f(x) approaches some value L”.  To evaluate this limit, we take points increasingly closer and closer to *6*&mdash; as close to 6 as we can get, but not 6 itself! We then evaluate where the function is headed at those points.  

If we look at the limit of a function as x approaches a value from one direction, this is called a *one-sided limit*. For example, we might look at the values of *f(5)*, *f(5.9)*, *f(5.999)* and see if they are trending towards the value of *f(6)*. This is represented as:
```tex
\lim_{x \rightarrow 6^{-}} f(x) = L
```
We read this as "the limit as x approaches 6 from the left side approaches some value L."

Whereas if we looked at the values of *f(6.1)*, *f(6.01)*, *f(6.0005)* and see if they are trending towards *f(6)*, we would represent this as:

```tex
\lim_{x \rightarrow 6^{+}} f(x) = L
```
We read this as "the limit as *x* goes to 6 from the right side approaches some value L."

This takes us to the final key concept about limits. The limit as *x* approaches 6 exists only if the limit as *x* approaches 6 from the left side is equal to the limit as *x* approaches 6 from the right side. This is written out as:

if
```tex

\lim_{x \rightarrow 6^{-}} f(x) = \lim_{x \rightarrow 6^{+}} f(x) = L

```
then

```tex

\lim_{x \rightarrow 6} f(x) = L

```
if
```tex

\lim_{x \rightarrow 6^{-}} f(x) \neq \lim_{x \rightarrow 6^{+}} f(x)

```
then

```tex

\lim_{x \rightarrow 6} f(x)\ does\ not\ exist

```

Limits

Derivative functions contain important information. Let's look at a couple of examples. Going back to our running example, *f(t)* describes a runner’s position as a function of time; therefore, the derivative function *f'(t)* describes the runner’s speed at any time *t*. Let's say we have a function *s(t)* that describes the real-time sales of a particular product; the corresponding derivative function *s'(t)* describes how those sales are changing in real-time. 

Let's look at the general information we can get as well. If *f'(x)* > 0, the corresponding function *f(x)* is increasing. Keep in mind that *f(x)* itself may be negative even though the derivative *f'(x)* is positive. The derivative says nothing about the value of the original function, only about how it is changing. Similarly, if *f'(x)* < 0, then the original function is decreasing. 

If *f'(x)* = 0, then the function is not changing. This can mean one of a few things. 
* It may mean that the function has reached a *local maximum* (or maxima). A local maximum is a value of x where *f'(x)* changes from positive to negative and thus hits 0 along the way. In *f(x)*, the local maximum is higher than all the points around it. 
* It may also mean that the function has reached what is called a *local minimum*. A local minimum is lower than the points around it. When *f'(x)* goes from negative values to 0 to positive values, a local minimum forms. 
* It may be an *inflection point*. This is a point where a function has a change in the direction of curvature. For example, the curve of the function goes from "facing down" to "facing up." Finding inflection points involves a second derivative test, which we will not get to in this lesson.

*Global maxima* and *global minima* are the largest or smallest over the entire range of a function.

In machine learning models, maxima and minima are key concepts as they’re usually what we are trying to find when we optimize!

Properties of the Derivative Function

Suppose we wanted to measure a runner’s instantaneous speed using a stopwatch. By instantaneous speed, we mean their speed at an exact moment in time.

Let’s define *f(t)* as the runner's distance from the start time at time *t*. How could we find the runner's instantaneous speed at 1 second after the race? We could record their positions at 1 second and some time after 1 second and then do some calculations. Let's say we measure their distance from the start at *f(1)* and *f(3)*.  The change in position is *f(3) - f(1)*, and the time between 1 second and 3 seconds is 2 seconds, so we can calculate the runner’s average speed to be the following:

```tex
\text{average speed} = \frac{f(3)-f(1)}{3-1}
```

However, this gives us the average speed, not the instantaneous speed. We don't know the runner's speed at *t=1*; we only know their speed on average over the 2-second interval.

If we repeated the process but instead took our second measurement at 1.1 seconds, we could be more accurate since we would find the average speed between 1 and 1.1 seconds. If we took our second measurement at a tiny increment, such as *t=1.0000001*, we would be **approaching** instantaneous speed and get a very accurate measurement. The animation to the right demonstrates this concept.

We can generalize this using limits. Define *t = 1* to be the first measurement time, and let’s say we wait *h* seconds until taking the second measurement.  Then the second time is *x+h*, and the positions at the two times are *f(x)* and *f(x+h)*. Repeating the exact process as above, we define the runner's average speed as:

```tex
\text{average speed} = \frac{f(x+h)-f(x)}{(x+h)-x}
```
Simplifies to:

```tex
\text{average speed} = \frac{f(x+h)-f(x)}{h}
```

Using limits, we can make *h* very small. By taking the limit as *h* goes to 0, we can find the instantaneous rate of change!

```tex
\text{instantaneous rate of change} = \lim_{h \rightarrow 0} \frac{f(x+h)-f(x)}{h}
```

This is called the *derivative at a point*, which is the function's slope (rate of change) at a specific point. In the next few exercises, we will dive further into derivatives.

**Note**: We have shown examples where *h* is positive, but it can also be a negative value approaching 0.   



Limit Definition of a Derivative

We’ve discussed how to compute derivatives by hand. However, oftentimes when working on a computer, we have functions given to us in array form rather than an explicit equation. In this exercise, we will investigate how to calculate derivatives using Python.

For example, let's say we have an array that represents the following function:
```tex 
f(x) = x^2 + 3 
```

We can define this function in two sets of arrays using the following code.
```python
from math import pow
# dx is the "step" between each x value
dx = 0.05
def f(x):
 # to calculate the y values of the function
 return pow(x, 2) + 3
# x values
f_array_x = [x for x in np.arange(0,4,dx)]
# y values
f_array_y = [f(x) for x in np.arange(0,4,dx)]
``` 
The `pow()` function within the `f(x)` function we defined allows us to calculate the values of *x2*, and we use a list comprehension to create a list of the y-values in a variable called `f_array_y`.
	
To compute the derivative of `f_array`, we use a NumPy function called `gradient()`. 

```python
f_array_deriv = np.gradient(f_array_y, dx)
```

`gradient()` takes in an array (in this case, a one-dimensional array) as its first argument. We also need to specify the distance between the *x* values, which is `dx`. For a one-dimensional array, `gradient()` calculates the rate of change using the following formula:

```tex
\frac{\Delta y}{\Delta x} = \frac{Change\ in\ y}{Change\ in\ x} 
``` 
As we know from the limit definition of the derivative, we want "Change in x" to be as small as possible, so we get an accurate instantaneous rate of change for each point.

Calculating Derivatives in Python

Congrats! You have finished your exploration of differential calculus. This has been a jam-packed lesson, so pat yourself on the back for making it through the material. Let's review what we have learned throughout the exercises:

#### Important Concepts
* A *limit* is the value of a function approaches as we move to some *x* value.
* The *limit definition of a derivative* shows us how to measure the instantaneous rate of change of a function at a specific point.
* A *derivative* is the slope of a tangent line at a specific point. The derivative of a function *f(x)* is denoted as *f'(x)*.
* The derivative of a function allows us to determine when a function is increasing, decreasing, or is at a minimum or maximum value.

#### Derivative Rules
We learned the following general derivative rules:
```tex
\begin{aligned}
\frac{d}{dx} c f(x) = c f'(x) \\
\frac{d}{dx}(f(x) + g(x)) = \frac{d}{dx}f(x) + \frac{d}{dx}g(x) \\
\frac{d}{dx}c = 0 \\
f(x) = u(x)v(x) \rightarrow f'(x) =u(x)v'(x) + v(x)u'(x)
\end{aligned}
```
We also learned how to calculate derivatives for explicit functions:
```tex
\begin{aligned}
\frac{d}{dx}x^{n} = nx^{n-1}
\frac{d}{dx}ln(x) = \frac{1}{x} \\
\frac{d}{dx}e^x = e^x \\
\frac{d}{dx}sin(x) = cos(x) \\
\frac{d}{dx}cos(x) = -sin(x)
\end{aligned}
```  

#### Derivatives in Python
* `np.gradient()` allows us to calculate derivatives of functions represented by arrays.

Review

optimization comic about optimizing work time and free time during a week

Data scientists often are asked to make data-driven recommendations on the best way to solve a problem.  Along the way, they build and fit statistical models to support their claims, make predictions, and more.  While creating these models, we often need to find the best solution out of all possible solutions, which is where calculus comes in.

Calculus gives us ways to determine how a response variable (like profit, or revenue, or click-through rate) changes with related independent variables (like gas prices, or click-through rate, or time). Understanding how independent variables change with respect to a response variable is essential for choosing an ideal model because it allows us to answer the following question: “what is the most effective model for our given situation?”

In this lesson, we will learn some fundamentals of calculus and gain some tools that data scientists use in their everyday jobs.


Introduction

In the previous exercise, we discussed the idea of the derivative at a point as the instantaneous rate of change at that point.  The idea behind the definition is that by finding how the function changes over increasingly small intervals starting at a point, we can understand how a function changes exactly at that point.  The formal expression of this concept is written as:

```tex
\lim_{h \rightarrow 0} \frac{f(x+h) - f(x)}{h}
```

An equivalent interpretation is that the derivative is the slope of the tangent line at a point.  A *tangent line*  at a point is the line that touches the function at that point. Just as the slope of a line describes how a line changes, the slope of the tangent line describes how a "curvier" function changes at a specific point. The visual below shows what a tangent line looks like:
![A graph shaped like a sin function with a tangent line on the left part of the graph.](https://upload.wikimedia.org/wikipedia/commons/thumb/0/0f/Tangent_to_a_curve.svg/220px-Tangent_to_a_curve.svg.png)
In the applet to the right, you can play around with the tangent line and see how it changes as you move to different values of *x*.

Our discussion of taking increasingly smaller intervals in the last exercise gives us a way to calculate the derivative of a function at a point, but this would be a laborious process if we always had to find the derivative at a point this way.    
	
Fortunately, many common functions have corresponding derivative functions.   Derivative functions are often denoted *f’(x)* (read *f* prime of *x*) or *df/dx* (read derivative of *f* at *x*).  If f(x) represents the function value at a point *x*, the corresponding derivative function *f’(x)* represents how a function is changing at the same point *x*.  Essentially, it saves us from having to draw a tangent line or take a limit to compute the derivative at a point.  In later exercises, we will see a number of different rules at our disposal that allow us to calculate the derivatives of common functions. 

The Derivative Function

So we now have an idea on where to start differentiating (that is, take the derivative of) things like:
```tex
\begin{aligned}
4\log(x) \\
x^2 + \log(x) \\
x^2\log(x) \\ 
\end{aligned}
```
Our final step is to understand how to differentiate specific items themselves, like *log(x)* and *x2*.

Let's start off with how to differentiate *polynomials*. As a reminder, these are expressions that include the following operations:
* addition
* subtraction
* multiplication
* nonnegative exponents

For example:
```tex
3x^4-2x^2+4
``` 

To differentiate polynomials, we use the *power rule*. This states the following:
```tex
\frac{d}{dx}x^{n} = nx^{n-1}
```
To take the derivative of *3x4*, we do the following:
```tex
\frac{d}{dx}3x^{4} = 4*3x^{4-1}=12x^{3}
```

Let's try a challenging problem. Find the derivative of the following polynomial:

```tex
4x^5+2x
```
Hint: use a concept we learned in the previous exercise. Check your solution below.

<details>
 <summary>Solution</summary>
<img src="https://static-assets.codecademy.com/skillpaths/ds-math/calculus/derivative-solution-1.png" alt="The final answer is 20*x^3+2]">
</details>
 

There are rules even beyond the power rule. Many common functions have defined derivatives. Here are some common ones:

```tex
\begin{aligned}
\frac{d}{dx}ln(x) = \frac{1}{x} \\
\frac{d}{dx}e^x = e^x \\
\frac{d}{dx}sin(x) = cos(x) \\
\frac{d}{dx}cos(x) = -sin(x)
\end{aligned}
``` 

Let's use these rules to try out some practice problems!

Practice Problem 1:
```tex
2x+ln(x)+cos(x)
```
Click below to check your solution.
<details>
 <summary>Solution</summary>
<img src="https://static-assets.codecademy.com/skillpaths/ds-math/calculus/deriv-2.png" alt="The final answer is 2+1/x-sin(x)]">
</details>
 

Practice Problem 2:
```tex
2x*ln(x)
```
Click below to check your solution.
<details>
 <summary>Solution</summary>
<img src="https://static-assets.codecademy.com/skillpaths/ds-math/calculus/derivative-sol-3.png" alt="The final answer is 2ln(x)+2]">
</details>
 

Taking derivatives is something that takes plenty of practice to get the hang of. As a learner in this course, your biggest takeaway should be that we can differentiate many common functions by using some combination of these rules in tandem.