Chao-Han Huck Yang

A Parameter-Efficient Learning Approach to Arabic Dialect Identification with Pre-Trained General-Purpose Speech Model

2023

In this work, we explore Parameter-Efficient-Learning (PEL) techniques to repurpose a General-Purpose-Speech (GSM) model for Arabic dialect identification (ADI). Specifically, we investigate different setups to incorporate trainable features into a multi-layer encoder-decoder GSM formulation under …

Whispering LLaMA: A Cross-Modal Generative Error Correction Framework for Speech Recognition

2023

Srijith Radhakrishnan, Chao-Han Yang, Sumeer Khan, Rohit Kumar, Narsis Kiani, David Gomez-Cabrero, Jesper Tegnér. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. 2023.

It's Never Too Late: Fusing Acoustic Information into Large Language Models for Automatic Speech Recognition

2024 · arXiv (Cornell University)

Recent studies have successfully shown that large language models (LLMs) can be successfully used for generative error correction (GER) on top of the automatic speech recognition (ASR) output. Specifically, an LLM is utilized to carry …

Bayesian Example Selection Improves In-Context Learning for Speech, Text, and Visual Modalities

2024 · arXiv (Cornell University)

Large language models (LLMs) can adapt to new tasks through in-context learning (ICL) based on a few examples presented in dialogue history without any model parameter update. Despite such convenience, the performance of ICL heavily …

Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks

2024 · arXiv (Cornell University)

Multimodal foundation models, such as Gemini and ChatGPT, have revolutionized human-machine interactions by seamlessly integrating various forms of data. Developing a universal spoken language model that comprehends a wide range of natural language instructions is …

Quantum Machine Learning: An Interplay Between Quantum Computing and Machine Learning

2024 · arXiv (Cornell University)

Quantum machine learning (QML) is a rapidly growing field that combines quantum computing principles with traditional machine learning. It seeks to revolutionize machine learning by harnessing the unique capabilities of quantum mechanics and employs machine …

Developing Instruction-Following Speech Language Model Without Speech Instruction-Tuning Data

2025

Recent end-to-end speech language models (SLMs) have expanded upon the capabilities of large language models (LLMs) by incorporating pre-trained speech models. However, these SLMs often undergo extensive speech instruction-tuning to bridge the gap between speech …