Transduction or transductive learning are terms you might encounter within applied machine learning. The term is being leveraged with a few applications of recurrent neural networks on sequence prediction issues, like a few problems in the domain of natural language processing.

In this blog article, you will find out about transduction within the domain of machine learning.

After going through this post, you will be aware of:

• The definition of transduction generally and in some particular domains of study
• What transductive learning is within machine learning
• What transduction implies when speaking about sequence prediction problems.

Tutorial Summarization

This guide is subdivided into four portions, which are:

1] What is Transduction?

2] Transductive learning

3] Transduction in linguistics

4] Transduction in sequence prediction

What is Transduction?

Let’s begin with some fundamental dictionary definitions.

To transduce implies to convert anything into another form.

Transduce: To convert (something, like energy or a message) into another form or shape, basically sense organs transduce physical energy into a nervous signal.

It is a widespread term from the domain of electronics and signal processing, where a “transducer” is a general name for components or modules converting sounds to energy or the other way around.

All signal processing starts with an input transducer. The input transducer takes the input signal and converts it to an electrical signal. In signal-processing applications, the transducer can take several forms. A typical instance of an input transducer is a microphone.

Within biology, particularly genetics, transduction references to the process of a microorganism transferring genetic material to another microorganism.

Transduction: the action or procedure of transducing, particularly, the transfer of genetic material from one microorganism to another through a viral agent (like a bacteriophage)

So, typically, we can observe that transduction is about translating a signal into another form.

The signal processing description is the most important where sound waves are converted into electrical energy for some leveraging within a system. Every sound would be indicated by some electrical signature, at some selected level of sampling.

Transductive Learning

Transduction or transductive learning is leveraged in the domain of statistical learning theory to reference to forecasting particular instances provided particular instances from a domain.

It is compared with other variants of learning, like inductive learning and deductive learning.

Induction, obtaining the function from the provided data. Deduction, deriving the values of the provided function for points of interest. Transduction, obtaining the values of the unknown function for points of interest from the provided data.

It is a fascinating framing of supervised learning where the classical problem of “approximating a mapping function from data and leveraging it to make a prediction” is observed as more tough than is needed. Rather, particular forecasts are made directly from the real samples from the domain. No function approximation is needed.

The model of estimating the value of a function at a provided point of interest details a new idea of inference: shifting from the particular to the particular. We refer to this type of inference transductive inference. Observe that this idea of inference appears when one would like to obtain the ideal result from restricted amount of data.

A classical instance of a transductive algorithm is the k-Nearest Neighbours algorithm that doesn’t model the training data, but rather leverages it directly every time a forecast is needed.

Transduction is naturally connected to a grouping of algorithms referred to as instance-based, or case-based learning. Probably, the most widespread algorithm in this class is k-nearest neighbour algorithm.

Transduction in Linguistics

Conventionally, transduction has been leveraged when speaking about natural language, like within the domain of linguistics.

For instance, there is the notion of a “transduction grammar” that references to a grouping of rules for transforming instances of one language into another.

A transduction grammar details a structurally correlated pairing of languages. It produces sentence pairs, instead of sentences. The language-1 sentence is intended to be a translation of the language-2 sentence.

There is also the idea of a “finite-state-transducer” (FST) from the theory of computation that is invoked when speaking about translation activities for mapping one group of symbols to another. Critically, every input generates one output.

A finite state transducer is made up of a number of states. When transitioning between states an input symbol is consumed and an output symbol is emitted.

This leveraging of transduction when speaking about theory and classical machine translation color the leveraging of the term when speaking about advanced sequence prediction with recurrent neural networks on natural language processing activities.

Transduction in sequence prediction

In his guide/textbook on neural networks for language processing, Yoav Goldberg defines a transducer as a particular network model for NLP activities.

A transducer is narrowly defined as a model that outputs one time step for every input time step furnished. This maps to the linguistic usage, particularly with finite-state transducers.

Another option is to regard the RNN as a transducer, generating an output for every input it reads in.

He proposes this variant of model for sequence tagging in addition to language modelling. He goes on to indicate that conditioned-generation, like with the Encoder-Decoder architecture, may be viewed as a special case of the RNN transducer.

This last point is a surprise provided that the Decoder in the Encoder-Decoder model architecture permits a varied number of outputs for a provided input sequence, breaking “one output per input” in the definition.

More generally, transduction is leveraged within NLP sequence prediction activities, particularly translation. The definitions appear more relaxed than the strict one-output-per-input of Goldberg and the FST.

For instance, Ed Grefenstette, et al. detail transduction as mapping an input string to an output string.

Several natural language processing (NLP) activities can be perceived as transduction problems, that is learning to translate one string into another. Machine translation is a prototypical example of transduction, and latest outcomes signify that Deep RNNs have the capability to encode long source strings and generate coherent translations.

They go on to furnish a list of some particular NLP tasks that assist to make this wide definition and spelling correction, to inflectional morphology, and machine translation.

Alex Graves also leverages transduction as synonym for transformation and usefully also furnishes a list of instance NLP activities that meet the definition.

Several machine learning activities can be expressed as the transformation – or transduction – of input sequences into output sequences: speech recognition, machine translation, protein secondary structure prediction and text-to-speech to specify but a few:

In summary, we can restate a listing of transductive natural language processing activities as follows:

• Transliteration, generating words in a targeted form provided instances in a source form.
• Spelling correction, generating right word spelling provided incorrect word spelling
• Inflectional Morphology, generating new sequences provided source sequences and context.
• Machine translation, generating sequences of words in a targeted language provided instances in a source language.
• Speech Recognition, generating sequences of text provided sequences of audio.
• Protein secondary structure prediction, forecasting 3D structure provided input sequences of amino acids (not NLP)
• Text-to-speech, or speech synthesis, generating audio provided text sequences.

Lastly, in addition to the concept of transduction referring to wide classes of NLP problems and RNN sequence prediction models, a few new methods are overtly being named as such. Navdeep Jaitly et al. references to their new RNN sequence-to-sequence forecasting method as a “Neural Transducer” which technically RNNs for sequence-to-sequence forecasting method as a “Neural Transducer”, which technically RNNs for sequence-to-sequence forecast would also be.

We put forth a neural transducer, a more general class of sequence-to-sequence learning models. Neural transducer can generate chunks of outputs (potentially of zero length) as blocks of inputs arrive – therefore fulfilling the condition of being online. The model produces outputs for every block by leveraging a transducer RNN that implements a sequence-to-sequence model.

Further Reading

This part of the blog post furnishes additional resources on the subject if you are seeking to delve deeper.

Definitions

Merriam-Webster Dictionary definition of transduce

Digital Signal Processing Demystified, 1997

Transduction in genetics on Wikipedia

Learning Theory

The Nature of Statistical Learning Theory, 1995

Finite-state transducer on Wikipedia

Statistical machine translation, 2010

Lingustics

Handbook of Natural Language Processing, 2000

Finite-state transducer on Wikipedia

Statistical machine translation, 2010.

Sequence prediction

Neural network methods in natural language processing, 2017

Learning to transduce with unbounded memory, 2015

Sequence transduction with recurrent neural networks, 2012

A neural transducer, 2016

Conclusion

In this blog article, you found out about transduction in applied machine learning.

Particularly, you learned:

• The definition of transduction generally and in some particular domains of research.
• What transductive learning is within machine learning
• What transduction implies when talking about sequence prediction problems