With regards to machine learning tasks such as classification or regression, approximation strategies have a critical role in learning from the data. Several machine learning methods approximate a function or a mapping between the outputs and the inputs through a learning algorithm.

In this guide, you will find out about approximation and its criticality in machine learning and pattern recognition.

After going through this guide, you will be aware of:

• What approximation is
• Criticality of approximation in machine learning

Tutorial Summarization

This guideline is divided into three portions, which are:

1] What is approximation?

2] Approximation when the form of function is unknown

3] Approximation when the form of function is known

What is Approximation?

Approximation is a commonly encountered. For instance, the irrational number π  can be approximated by the number 3.14. A more precise value is 3.141593, which stays an approximation. You can likewise approximate the values of all irrational numbers such as sqrt(3), sqrt(7), etc.

Approximation is leveraged whenever a numerical value, a model, a structure or a function is either not known or tough to compute. In this guide, we’ll concentrate on function approximation and detail its application to machine learning problems. There are two different cases:

1] The function is known but it is tough or numerically expensive to compute its precise value. In this scenario, approximation methods are leveraged to identify values, which are close to the function’s actual values.

2] The function itself is unknown and therefore a model or learning algorithm is leveraged to closely identify a function that can generate outputs close to the unknown function’s outputs.

Approximation when form of function is known

If the form of a function is known, a well known method in calculus and mathematics is approximation through Taylor series.

Another widespread method for approximation in calculus and mathematics is Newton’s method. It can be leveraged to approximate the roots of polynomials, therefore making it a useful strategy for approximation of the quantities like the square root of different values or the reciprocal of different numbers, etc.

Approximation When Form of Function is not known

In data science and machine learning, the assumption is that there is an underlying function that has the key to the relationship between the inputs and outputs. The form of this function is not known. Here, we speak about various machine learning problems that employ approximation.

Approximation in Regression

Regression consists of the forecast of an output variable when provided a grouping of inputs. Within regression, the function that truly maps the input variables to outputs is unknown. The assumption is that some linear or non-linear regression model can go about approximating the mapping of inputs to outputs.

For instance, we might have data connected to consumed calories each day and the corresponding blood sugar. To detail the relationship between the calorie input and blood sugar output, we can assume a straight line relationship/mapping function. The straight line is thus he approximation of the mapping of inputs to outputs. A learning method like the method of least squares is leveraged to identify this line.

Approximation in Classification

A typical instance of models that approximate functions in classification problems is that of neural networks. The assumption is that the neural network as a whole can go about approximating a true function that maps the inputs to the class labels. Gradient descent or some other learning algorithm is then leveraged to learn that function approximation through adjustment of the weights of the neural network.

Approximation in Unsupervised Learning

Below is a common instance of unsupervised learning. Here we possess points in 2D space and the label of none of these points is provided. A clustering algorithm generally assumes a model according to which a point can be allocated to a class or label. For instance, k-means learns the labels of data through the assumption that data clusters are circular, and therefore, allocates the same label or class to points lying in the same circle or an n-sphere in the scenario of multi-dimensional data. In the figure below we are approximating the relationship amongst points and their labels through circular function.

Extensions

This portion of the blog details some ideas for extensions of the tutorial that you may desire to explore:

• Maclaurin series
• Taylor’s series

Further Reading

This part of the blog furnishes additional resources on the subject if you are seeking to delve deeper.

Books

Patter recognition and machine learning by Christopher M. Bishop

Deep Learning by Ian Goodfellow, Joshua Begio, Aaron Courville

Thomas’ Calculus, 14^th edition, 2017 (based on the original works of George B. Thomas, revised by Joel Hass, Christopher Heil, Maurice Weir

Conclusions

In this guide, you found out about approximation. Particularly, you learned,

• Approximation
• Approximation when the form of a function is known
• Approximation when the form of a function is not known