Recent advances in deep learning and automatic speech recognition have raised the accuracy of end-to-end speech recognition to a new level. However, recognizing personal content, such as contact names, remains a challenge. In this work, we present a personalization solution for an end-to-end system based on connectionist temporal classification. Our solution uses a class-based language model, in which a general language model provides context modeling for classes of named entities, and personal named entities are compiled into a separate finite-state transducer. Additionally, we introduce a phoneme-to-word model to map rare named entities to more frequent homophonic words, and also word pre-normalization for biases for rare word pieces, leading to another 48.9% relative improvement in the precision of entities with personal names in addition to an already customized model. base. This work allows our systems to match highly competitive custom hybrid systems in recognizing personally named entities.
