Yixiao Song

DEMETR: Diagnosing Evaluation Metrics for Translation

2022 · arXiv (Cornell University)

While machine translation evaluation metrics based on string overlap (e.g., BLEU) have their limitations, their computations are transparent: the BLEU score assigned to a particular candidate translation can be traced back to the presence or …

Does quantization affect models' performance on long-context tasks?

2025 · arXiv (Cornell University)

Large language models (LLMs) now support context windows exceeding 128K tokens, but this comes with significant memory requirements and high inference latency. Quantization can mitigate these costs, but may degrade performance. In this work, we …