Reinforcement Learning for Speech Recognition using Recurrent Neural Networks

This work describes a voice recognition system that does not need an intermediate phonetic representation to convert audio input to text. The system is based on a mix of the the Connectionist Temporal Classification goal function and deep bidirectional LSTM recurrent neural network architecture . A new method is proposed in which the network is taught to reduce the likelihood of an arbitrary transcription loss function being encountered. without the aid of any lexicons or models, this allows for a direct optimization of WER. The system has a WER (word error rate) of 22 percent, 20 percent with simply a lexicon of authorized terms, 9 percent using a trigram language model. The error rate drops to 7 percent when the network is used in conjunction with a baseline system.