An intro to Taylor Series
Taylor series expansion is an amazing idea, not just in the world of mathematics, but additionally within optimization theory, function optimization and ML. It is broadly leveraged in numerical computations when estimates of a function’s values at differing points are needed.
In this guide, you will find out about Taylor series and how to go about approximating the values of a function around differing points leveraging its Taylor series expansion.
After going through this guide, you will be aware of:
- Taylor series expansion of a function
- How to go about approximating functions leveraging Taylor series expansion
Tutorial Summarization
This guide is subdivided into three portions, which are:
1] Power series and Taylor series
2] Taylor polynomials
3] Function approximation leveraging Taylor polynomials
What is a power series?
The following is a power series about the centre x=a and constant coefficients c_0
What is a Taylor Series?
It is an incredible fact that functions which are limitlessly differentiable can produce a power series referred to as the Taylor series. Let’s assume we possess a function f(x) and f(x) contains derivatives of all orders on a provided interval, then the Taylor series produced by f(x)at x=a is provided by:
The second line of the expression in the above image provides the value of the kth coefficient.
If we have a set a=0, then we have an expression referred to as the Maclaurin series expansion of f(x).
Examples of Taylor Series Expansion
Taylor series produced by f(x) = 1/x can be identified by initially differentiating the function and identifying a general expression for the kth derivative.
The Taylor series with regards to several points can now be identified. For instance:
Taylor Polynomial
A Taylor polynomial of order k, produced by f(x) at x=a is provided by:
For the instance of f(x) = 1/x, the Taylor polynomial of order 2 is provided by:
Approximation through Taylor Polynomials
We can go about approximating the value of a function at a point x=a leveraging Taylor polynomials. The higher the order of the polynomial, the additional terms there are in that polynomial and the nearer the approximation is to the actual value of the function at that point.
In the image below, the function 1/x is plotted around the point x=1(left) and x=3(right). The line in green is the actual function f(x) = 1/x. The pink line indicates the approximation through an order 2 polynomial.
More instances of Taylor Series
Let’s observe the function g(x) = e^x. Observing the fact that the kth order derivative of g(x) is also g(x), the expansion of g(x) about x=a, is provided by:
Therefore, around x=0, the series expansion of g(x) is provided through (gotten by setting a=0).
The polynomial of order k produced for the function e^x around the point x=0 is provided by:
The plot here demonstrate polynomials of differing orders that estimate the value of e^x around x=0. We can observe that as we shift away from zero, we require more terms to approximate e^x more precisely. The green line indicating the actual function is hiding behind the blue line of the approximating polynomial of order 7.
Taylor Series within machine learning
A widespread method within machine learning for identifying the optimal points of a function is the Newton’s strategy. Newton’s method leverages the second order polynomials to go about approximating a function’s value at a point. These methods that leverage second order derivatives are referred to as second order optimization algorithms.
Extensions
This portion of this blog article details some ideas for extension of the tutorial that you might desire to explore.
- Newton’s method
- Second order optimization algorithms
Further Reading
This section furnishes additional resources on the subject if you are seeking to delve deeper.
Resources
Jason Brownlee’s standout resource on Calculus Books for Machine Learning
Books
Pattern recognition and machine learning by Christopher M. Bishop
Deep learning by Ian Goodfellow, Joshua Begio, Aaron Courville
Thomas Calculus, 14th Edition, 2017 (based on the original works of George B. Thomas, revised by Joel Hass, Christopher Heil, Maurice Weir)
Calculus, 3rd edition, 2017 (Gilbert Strang)
Calculus, 8th edition, 2015. (James Stewart)
Conclusion
In this guide, you found out what Taylor series expansion of a function about a point is.
Particularly, you learned:
- Power series and Taylor series
- Taylor polynomials
- How to approximate functions around a value leveraging Taylor polynomials