Taylor series expansion is an amazing idea, not just in the world of mathematics, but additionally within optimization theory, function optimization and ML. It is broadly leveraged in numerical computations when estimates of a function’s values at differing points are needed.

In this guide, you will find out about Taylor series and how to go about approximating the values of a function around differing points leveraging its Taylor series expansion.

After going through this guide, you will be aware of:

• Taylor series expansion of a function
• How to go about approximating functions leveraging Taylor series expansion

Tutorial Summarization

This guide is subdivided into three portions, which are:

1] Power series and Taylor series

2] Taylor polynomials

3] Function approximation leveraging Taylor polynomials

What is a power series?

The following is a power series about the centre x=a and constant coefficients c_0

What is a Taylor Series?

It is an incredible fact that functions which are limitlessly differentiable can produce a power series referred to as the Taylor series. Let’s assume we possess a function f(x) and f(x) contains derivatives of all orders on a provided interval, then the Taylor series produced by f(x)at x=a is provided by:

The second line of the expression in the above image provides the value of the kth coefficient.

If we have a set a=0, then we have an expression referred to as the Maclaurin series expansion of f(x).

Examples of Taylor Series Expansion

Taylor series produced by f(x) = 1/x can be identified by initially differentiating the function and identifying a general expression for the kth derivative.

The Taylor series with regards to several points can now be identified. For instance:

Taylor Polynomial

A Taylor polynomial of order k, produced by f(x) at x=a is provided by:

For the instance of f(x) = 1/x, the Taylor polynomial of order 2 is provided by:

Approximation through Taylor Polynomials

We can go about approximating the value of a function at a point x=a leveraging Taylor polynomials. The higher the order of the polynomial, the additional terms there are in that polynomial and the nearer the approximation is to the actual value of the function at that point.

In the image below, the function 1/x is plotted around the point x=1(left) and x=3(right). The line in green is the actual function f(x) = 1/x. The pink line indicates the approximation through an order 2 polynomial.

More instances of Taylor Series

Let’s observe the function g(x) = e^x. Observing the fact that the kth order derivative of g(x) is also g(x), the expansion of g(x) about x=a, is provided by:

Therefore, around x=0, the series expansion of g(x) is provided through (gotten by setting a=0).

The polynomial of order k produced for the function e^x around the point x=0 is provided by:

The plot here demonstrate polynomials of differing orders that estimate the value of e^x around x=0. We can observe that as we shift away from zero, we require more terms to approximate e^x more precisely. The green line indicating the actual function is hiding behind the blue line of the approximating polynomial of order 7.

Taylor Series within machine learning

A widespread method within machine learning for identifying the optimal points of a function is the Newton’s strategy. Newton’s method leverages the second order polynomials to go about approximating a function’s value at a point. These methods that leverage second order derivatives are referred to as second order optimization algorithms.

Extensions

This portion of this blog article details some ideas for extension of the tutorial that you might desire to explore.

• Newton’s method
• Second order optimization algorithms

Further Reading

This section furnishes additional resources on the subject if you are seeking to delve deeper.

Resources

Jason Brownlee’s standout resource on Calculus Books for Machine Learning

Books

Pattern recognition and machine learning by Christopher M. Bishop

Deep learning by Ian Goodfellow, Joshua Begio, Aaron Courville

Thomas Calculus, 14^th Edition, 2017 (based on the original works of George B. Thomas, revised by Joel Hass, Christopher Heil, Maurice Weir)

Calculus, 3^rd edition, 2017 (Gilbert Strang)

Calculus, 8^th edition, 2015. (James Stewart)

Conclusion

In this guide, you found out what Taylor series expansion of a function about a point is.

Particularly, you learned:

• Power series and Taylor series
• Taylor polynomials
• How to approximate functions around a value leveraging Taylor polynomials