Speech processing systems often struggle to deliver clear audio in noisy environments. This challenge affects applications such as hearing aids, automatic speech recognition (ASR), and speaker verification. Conventional single-channel speech enhancement (SE) systems use neural network architectures such as LSTM, CNN, and GAN, but they are not without limitations. For example, attention-based models like Conformers, while powerful, require extensive computational resources and large data sets, which may be impractical for certain applications. These limitations highlight the need for scalable and efficient alternatives.
Introducing xLSTM-SENet
To address these challenges, researchers from Aalborg University and Oticon A/S developed xLSTM-SENet, the first xLSTM-based single-channel SE system. This system is based on the extended short-term memory (xLSTM) architecture, which refines traditional LSTM models by introducing exponential gate and matrix memory. These improvements address some of the limitations of standard LSTMs, such as restricted storage capacity and limited parallelization. By integrating xLSTM into the MP-SENet framework, the new system can effectively process magnitude and phase spectra, offering a simplified approach to speech enhancement.
Technical description and advantages
xLSTM-SENet is designed with a time-frequency (TF) domain encoder-decoder structure. At its core are the TF-xLSTM blocks, which use mLSTM layers to capture both temporal and frequency dependencies. Unlike traditional LSTMs, mLSTMs employ exponential gate for more precise storage control and a die-based memory design for higher capacity. The two-way architecture further improves the model's ability to use contextual information from past and future frames. Additionally, the system includes specialized decoders for magnitude and phase spectrums, which help improve speech quality and intelligibility. These innovations make xLSTM-SENet efficient and suitable for devices with limited computational resources.
Performance and findings
Evaluations using the VoiceBank+DEMAND dataset highlight the effectiveness of xLSTM-SENet. The system achieves comparable or better results than state-of-the-art models such as SEMamba and MP-SENet. For example, he recorded a Perceptual Evaluation of Speech Quality (PESQ) score of 3.48 and a Short-Term Objective Intelligibility (STOI) of 0.96. Additionally, composite metrics such as CSIG, CBAK, and COVL showed notable improvements. Ablation studies highlighted the importance of features such as exponential gate and bidirectionality in improving performance. While the system requires longer training times than some attention-based models, its overall performance demonstrates its value.
Conclusion
xLSTM-SENet offers a thoughtful answer to the challenges in single-channel speech enhancement. By leveraging the capabilities of the xLSTM architecture, the system balances scalability and efficiency with strong performance. This work not only advances the state of speech enhancement technology, but also opens the doors for its application in real-world scenarios, such as hearing aids and speech recognition systems. As these techniques continue to evolve, they promise to make high-quality speech processing more accessible and practical for a variety of needs.
Verify he Paper. All credit for this research goes to the researchers of this project. Also, don't forget to follow us on <a target="_blank" href="https://x.com/intent/follow?screen_name=marktechpost” target=”_blank” rel=”noreferrer noopener”>twitter and join our Telegram channel and LinkedIn Grabove. Don't forget to join our SubReddit over 65,000 ml.
Recommend open source platform: Parlant is a framework that transforms the way ai agents make decisions in customer-facing scenarios. (Promoted)
Nikhil is an internal consultant at Marktechpost. He is pursuing an integrated double degree in Materials at the Indian Institute of technology Kharagpur. Nikhil is an ai/ML enthusiast who is always researching applications in fields like biomaterials and biomedical science. With a strong background in materials science, he is exploring new advances and creating opportunities to contribute.
