PaperMetrixEvidence-aware discovery
EN
Interface language English EN العربية AR Türkçe TR
Create account
Researcher profile

Santiago Ontañón

2 papers in the PaperMetrix corpus

Publications

Papers by this author

  1. Big Bird: Transformers for Longer Sequences

    2020 · arXiv (Cornell University)

    Transformers-based models, such as BERT, have been one of the most successful deep learning models for NLP. Unfortunately, one of their core limitations is the quadratic dependency (mainly in terms of memory) on the sequence …

  2. LongT5: Efficient Text-To-Text Transformer for Long Sequences

    2022 · Findings of the Association for Computational Linguistics: NAACL 2022

    Recent work has shown that either (1) increasing the input length or (2) increasing model size can improve the performance of Transformer-based neural models. In this paper, we present LongT5, a new model that explores …