Paden Tomasello

Hybrid Transducer and Attention based Encoder-Decoder Modeling for Speech-to-Text Tasks

2023 · arXiv (Cornell University)

Transducer and Attention based Encoder-Decoder (AED) are two widely used frameworks for speech-to-text tasks. They are designed for different purposes and each has its own benefits and drawbacks for speech-to-text tasks. In order to leverage …

SSR: Alignment-Aware Modality Connector for Speech Language Models

2025

Fusing speech into a pre-trained language model (SpeechLM) usually suffers from the inefficient encoding of long-form speech and catastrophic forgetting of pre-trained text modality.We propose SSR-CONNECTOR (Segmented Speech Representation Connector) for better modality fusion.Leveraging speech-text …