Νικόλαος Παππάς

Modeling Context With Linear Attention for Scalable Document-Level Translation

2022

Document-level machine translation leverages inter-sentence dependencies to produce more coherent and consistent translations. However, these models, predominantly based on transformers, are difficult to scale to long documents as their attention layers have quadratic complexity in …

Semantic Text Transmission via Prediction with Small Language Models: Cost-Similarity Trade-off

2024

We consider the communication of natural language text from a source to a destination over noiseless and character-erasure channels. We exploit language's inherent correlations and predictability to constrain transmission costs by allowing the destination to …