Optimization, as one of the fundamental processes in several machine learning algorithms, is dependent on the leveraging of derivatives in order to determine in which fashion to go about updating a model’s parameters values, to maximize or minimize an objective function.

This blog post by AICoreSpot will serve as a tutorial and continue exploring the different strategies through which we can identify the derivatives of functions. Specifically, we will be delving into the product, power and quotient rules, which we can leverage to arrive to the derivatives of functions quicker than if we had to identify each derivative from first principles. Therefore, for functions that are particularly challenging, keeping these rules at hand to identify their derivatives will become more and more critical.

In this tutorial, you will find out about the product, power and quotient rules to identify the derivative of functions.

After finishing this tutorial, you will be aware of:

- The power rule to adhere to when identifying the derivative of a variable base, raised to a fixed power.
- How the product rules facilitates us to identify the derivative of a function that is defined as the product of another two (or more functions)
- How to the quotient rule enables us to identify the derivative of a function that is ratio of two differentiable functions.

Let’s begin:

This tutorial consists of three parts:

- The Power Rule
- The Product Rule
- The Quotient Rule

__The Power Rule__

If we possess a variable base raised to a fixed power, the rule to adhere to in order to identify its derivative is to bring down the power in front of the variable base, and then subtract the power by 1.

For instance, if we possess the function f(x) = x squared, which we would desire to identify the derivative, we first bring down 2 in front of x and the subtract the power by 1.

f(x) = x squared

f (x) = 2x

In the pursuit of comprehending better where this rule has its origins, let’s take the longer path and identify the derivative of f(x) by beginning from the definition of a derivative.

Here, we replace for f(x) = x squared and then proceed to simplify the expression.

As h approaches a value of 0, then this limit approaches 2x, which tallies with the outcome that we have received prior leveraging the power rule.

If applied to f(x) = x, the power rule provides us a value of 1. This is due to the fact that when we bring a value of 1 in front of x, and then reduce the power by 1, what we are left with is a value of 0 in the exponent. Since x to the power of 0 is equal to 1, then f(x) = (1) (x to the power of 0) = 1.

The best way to comprehend this derivative is to know that f(x) = x is a line that fits the form y = mx +b owing to the fact that f(x) = x is the same as f(x) = 1x + 0 (or y = 1x + 0). The slope (m) of this line is 1, therefore, the derivative is equivalent to 1. Or you can just memorize that the derivative of x is 1. But if you don’t remember both of these concepts, you can always leverage the power rule.

The power rule can be used on any power, regardless of whether it is positive, negative, or a fraction. We can also apply it to radical functions by first expressing their exponent (or power) as a fraction:

f(x) = x= x to the power of ½

f(x) = (1/2) x to the power of -1/2

__The Product Rule__

Let’s assume that we now possess a function, f(x) of which we like to identify the derivative, which is the product of another two functions u(x) = 2x squared and v(x) = x cubed.

f(x) = u(x) v(x) = (2x squared) (x cubed)

In order to find out how to identify the derivative of f(x), let’s first begin with identifying the derivative of the product of u(x) and v(x) directly.

(u(x) v(x)) = (2x squared) (x cubed) = (4x) (3xsquared) = 12x cubed

It is obvious that the second outcome does not tally with the initial one, and that is owing to the fact that we have not leveraged the product rule.

The product rule informs us that the derivative of the product of two functions can be identified as:

f(x) = u(x) v(x) + u(x) v(x)

We can arrive at the product rule if we work our way through application of the attributes of limits, beginning again with the definition of a derivative.

We are aware that f(x) = u(x) v(x) and therefore, we can replace for f(x) f(x+h):

At this level, our objective is to factorize the numerator into various limits that can subsequently be assessed independently. For this reason, the subtraction of terms, u(x) v(x+h) – u(x) v(x+h), shall be introduced into the numerator. Its introduction does not alter the definition of f(x) that we just gotten, but it will assist us in factorising the numerator.

The outcome expression seems complex, but, if take a closer look we realize that we possess common terms that can be factored out:

The expression can be broken down even more through application of the limit laws that enable us separate the sums and products into independent limits.

The answer to our issue has now become obvious. We can observe that the starting and final terms in the simplified expression correlate to the definition of the derivative of u(x) and v(x), which we can denote through u(x) and v(x) respectively. The second term approaches the ongoing and differentiable function, v(x) as h approaches zero, whereas the third term is u(x).

Therefore, we arrive again at the product rule:

f(x) = u(x) v(x) + u(x) v(x)

With this new utility in tow, let’s reconsider finding f(x) when u(x) = 2x squared and v(x) = x cubed.

f(x) = u(x) v(x) + u(x) v(x)

f(x) = (4x) (x cubed) + (2x squared) (3x squared) = 4x to the power of 4 + 6x to the power of 4 = 10x to the power of 4

The outcome derivative now is correctly matching with the derivative of the product, (u(x) v(x)) that we have gotten prior.

This was a fairly simplistic instance that we could have computed directly to begin with. Although, we might have more complicated issues consisting of functions that cannot be multiplied directly, to which we can simply apply the product rule. For instance:

f(x) = x squared sin x

f(x) = (x squared) (sin x) + (x squared) (sin x) = 2x sin x + x squared cos x

We can even have extension of the product rule to more than just two functions. For instance, say f(x) is currently defined as the product of three functions, u(x), v(x) and w(x):

f(x) = u(x) v(x) w(x)

We can the enforce application of the product rule as follows:

f(x) = u(x) w(x) + u(x) v(x) w(x) + u(x) v(x) w(x)

__The Quotient Rule__

Likewise, the quotient rule informs us how to identify the derivative of a function f(x), which is the ratio of two differentiable functions, u(x) and v(x):

We can then enforce application of the product rule on u(x) to get:

u (x) = f(x) v(x) + f(x) v(x)

Solving back for f(x) provides us:

One last step replaces for f(x) to arrive to the quotient rule:

Application of the quotient rule and simplification of the outcome expression:

From the Pythagorean identity from trigonometry, we are aware that cos squared x + sin squared x =1, therefore:

Hence, leveraging the quotient rule, we have easily identified that the derivative of tangent is the squared secant function.

#### The Product Power and Quotient Rules

Optimization, as one of the fundamental processes in several machine learning algorithms, is dependent on the leveraging of derivatives in order to determine in which fashion to go about updating a model’s parameters values, to maximize or minimize an objective function

#### A new framework and dataset for long-range memory

This blog post by AICoreSpot, we put forth a new long-range memory framework, the Compressive Transformer, combined with a new benchmark for book-level language modelling, PG19. We furnish the conceptual utilities required to comprehend this new research in the context of latest advancements in the realm of memory models and language modelling.

#### Leveraging WaveNet to reunite speech-disabled users with their voices

As a teen, Tim Shaw gave everything he had in the pursuit of football, his aspiration was to be a part of the NFL. After getting selected and playing for Penn State in his college years, his aspirations finally became a reality.

#### AlphaFold – Leveraging Artificial Intelligence for Scientific Discovery

In a recent research piece put out in Nature, it was demonstrated how artificial intelligence research can spearhead and accelerate new scientific discoveries and inventions. A devoted, interdisciplinary team was assembled in the pursuit of leveraging artificial intelligence to

#### Dopamine and temporal difference learning – an enlightened relationship between AI and neuroscience

Motivation and learning are guided by external and internal rewards. Several of our everyday behaviours are directed by forecasting

#### IoT history and what’s to come

The internet-of-things (IoT) has just lately become a part and parcel of our daily lives. It is around us anywhere we go to, connected vehicles on city roadways, home automation devices situated at your homes, smart office sensors integrated into the workplace,

#### The history of Artificial Intelligence

In the first part of the 20th century, science fiction novels and movies put forth the concept of artificially intelligent robots to the world. It started with the “heartless” Tin man from the Wizard of Oz and went on with the humanoid robot that emulated Maria in Metropolis. Midway through that century, we had a league of scientists, mathematicians, and philosophers with the idea of artificial intelligence (or AI) culturally integrated in their minds.

#### The future of transport

We are all well aware that the COVID-19 pandemic has brought about a lot of changes in the way we go about our everyday lives – and transit is definitely at the top of the list of changes. Monthly transit rides recorded a low of 156.6 million rides in the month of April 2020, which is a dramatic 81.3% lower in comparison to the 835.2 million rides taken just a year back.

#### RPA – Tale of the Tape

The term automation’s coinage can be attributed to Delmar S. Harder in 1948. Delmar was the Vice President of the Ford Motor Company when he coined the term, and he summarized his organization’s objective to defeat the competitors with the phrase “what we need is more automation.

#### Self-driving vehicles: The future

As of 2019, the number of people who died in vehicle crashes was 36,096. Approximately 94% of fatal crashes are still because of human error. The NHTSA (National Traffic Safety Admin.) intends to alter this and autonomous vehicle form a major part of the solution.