Introduction to RNN

Introduction:

A recurrent neural network (RNN) is a form of artificial neural network that is designed to operate with time-series data or data that contains sequences. Ordinary feed-forward neural networks are only intended for data points that are unrelated to one another. However, if we have data in a sequence where one data point is dependent on the preceding data point, we must change the neural network to account for these dependencies. RNNs feature the idea of memory, which allows them to store the states or information of prior inputs in order to construct the sequence’s next output.

Why RNN?

RNNs were developed in response to many flaws in the feed-forward neural network:

1. Cannot deal with consecutive data
2. Only takes into account the current input
3. Cannot recall prior inputs

The RNN is the solution to these problems. An RNN can deal with sequential data, accepting both current and previously received inputs. Because of their internal memory, RNNs can remember past inputs.

Working:

Source: https://www.simplilearn.com/tutorials/deep-learning-tutorial/rnn

The information in Recurrent Neural Networks cycles in a loop to the middle-hidden layer.

The input layer ‘x’ receives the neural network’s input, analyses it, and sends it to the middle layer.

The middle layer ‘h’ can be made up of several hidden layers, each having its own activation functions, weights, and biases. A recurrent neural network can be used if the various parameters of different hidden layers are not affected by the preceding layer, i.e., the neural network does not have memory.

The Recurrent Neural Network will standardize the various activation functions, weights, and biases, resulting in identical values for each hidden layer. Then, rather than constructing numerous hidden layers, it will generate one and loop over it as many times as necessary.

Types of RNN:

There are 4 types of RNN namely:

1. One to One
2. One to Many
3. Many to One
4. Many to Many

1. One to One: This form of  neural network is called Vanilla Neural Network. It is applied to general machine learning issues with a single input and output.

Source: https://stats.stackexchange.com/questions/129411/how-can-recurrent-neural-networks-be-used-for-sequence-classification

2. One to Many: This neural network has a single input and several outputs. The image caption is an example of this.

Source: https://stats.stackexchange.com/questions/129411/how-can-recurrent-neural-networks-be-used-for-sequence-classification

3. Many to One: This RNN accepts a series of inputs and produces a single output. A good example of this type of network is sentiment analysis, in which a given sentence can be classified as having positive or negative thoughts.

Source: https://stats.stackexchange.com/questions/129411/how-can-recurrent-neural-networks-be-used-for-sequence-classification

4. Many to Many: This RNN takes a series of inputs and produces a series of outputs. One example is machine translation.

Source: https://stats.stackexchange.com/questions/129411/how-can-recurrent-neural-networks-be-used-for-sequence-classification

Problems solved using RNN:

1. Generating Text: We want to forecast the likelihood of each word given the previous words given a sequence of words.
2. Machine Translation: In that our input is a series of words in our source language, machine translation is analogous to language modeling (e.g. German). We wish to generate a string of words in our target language (e.g. English).
3. Speech Recognition: We can anticipate a sequence of phonetic segments together with their probabilities given an input sequence of acoustic signals from a sound wave.
4. Generating Image Descriptions: RNNs were utilized as a part of a model to generate descriptions for unlabelled photos, together with Convolutional Neural Networks.
5. Chatbots: Chatbots can respond to your questions. When a sequence of words is provided as input, a sequence of words is generated as output.