Dominik Klement

Adapting Diarization-Conditioned Whisper for End-to-End Multi-Talker Speech Recognition

2026

We propose a speaker-attributed (SA) Whisper-based model for multi-talker speech recognition that combines target-speaker modeling with serialized output training (SOT). Our approach leverages a Diarization-Conditioned Whisper (DiCoW) encoder to extract target-speaker embeddings, which are concatenated …