Koustuv Sinha

The RLLChatbot: a solution to the ConvAI challenge

2018 · arXiv (Cornell University)

Current conversational systems can follow simple commands and answer basic questions, but they have difficulty maintaining coherent and open-ended conversations about specific topics. Competitions like the Conversational Intelligence (ConvAI) challenge are being organized to push …

A Closer Look at Codistillation for Distributed Training

2020 · arXiv (Cornell University)

Codistillation has been proposed as a mechanism to share knowledge among concurrently trained models by encouraging them to represent the same function through an auxiliary loss. This contrasts with the more commonly used fully-synchronous data-parallel …

UnNatural Language Inference

2021

Koustuv Sinha, Prasanna Parthasarathi, Joelle Pineau, Adina Williams. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2021.

Masked Language Modeling and the Distributional Hypothesis: Order Word\n Matters Pre-training for Little

2021 · arXiv (Cornell University)

A possible explanation for the impressive performance of masked language\nmodel (MLM) pre-training is that such models have learned to represent the\nsyntactic structures prevalent in classical NLP pipelines. In this paper, we\npropose a different explanation: MLMs …

Masked Language Modeling and the Distributional Hypothesis: Order Word Matters Pre-training for Little

2021 · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing

A possible explanation for the impressive performance of masked language model (MLM) pre-training is that such models have learned to represent the syntactic structures prevalent in classical NLP pipelines. In this paper, we propose a …