The neural transducer is an end-to-end model for automatic speech recognition (ASR). While the model is suitable for ASR transmission, the training process remains challenging. During training, memory requirements can quickly exceed the capacity of the latest generation GPUs, limiting batch sizes and sequence durations. In this paper, we analyze the time and space complexity of a typical transducer training setup. We propose a memory efficient training method that calculates transducer loss and gradients on a sample-by-sample basis. Introducing optimizations to increase the efficiency and parallelism of the sampling method. In a set of comprehensive benchmarks, we show that our sample method significantly reduces memory usage and performs at a competitive speed compared to the default batch calculation. As a highlight, we were able to calculate transducer loss and gradients for a batch size of 1024 and an audio duration of 40 seconds, using only 6 GB of memory.
