Yassir Fathullah

AudioChatLlama: Towards General-Purpose Speech Abilities for LLMs

2023 · arXiv (Cornell University)

In this work, we extend the instruction-tuned Llama-2 model with end-to-end general-purpose speech processing and reasoning abilities while maintaining the wide range of original LLM capabilities, without using any carefully curated paired data. The resulting …

Cross-Lingual Transfer Learning for Speech Translation

2024 · arXiv (Cornell University)

There has been increasing interest in building multilingual foundation models for NLP and speech research. This paper examines how to expand the speech translation capability of these models with restricted data. Whisper, a speech foundation …