Douwe Kiela

Gradient-based Adversarial Attacks against Text Transformers

2021 · arXiv (Cornell University)

We propose the first general-purpose gradient-based attack against transformer models. Instead of searching for a single adversarial example, we search for a distribution of adversarial examples parameterized by a continuous-valued matrix, hence enabling gradient-based optimization. …

Masked Language Modeling and the Distributional Hypothesis: Order Word\n Matters Pre-training for Little

2021 · arXiv (Cornell University)

A possible explanation for the impressive performance of masked language\nmodel (MLM) pre-training is that such models have learned to represent the\nsyntactic structures prevalent in classical NLP pipelines. In this paper, we\npropose a different explanation: MLMs …

I love your chain mail! Making knights smile in a fantasy game world:\n Open-domain goal-oriented dialogue agents

2020 · arXiv (Cornell University)

Dialogue research tends to distinguish between chit-chat and goal-oriented\ntasks. While the former is arguably more naturalistic and has a wider use of\nlanguage, the latter has clearer metrics and a straightforward learning signal.\nHumans effortlessly combine the …

Specializing Word Embeddings for Similarity or Relatedness

2015

We demonstrate the advantage of specializing semantic word embeddings for either similarity or relatedness. We compare two variants of retrofitting and a joint-learning approach, and find that all three yield specialized semantic spaces that capture …

Automatically Generating Rhythmic Verse with Neural Networks

2017

We propose two novel methodologies for the automatic generation of rhythmic poetry in a variety of forms. The first approach uses a neural language model trained on a phonetic encoding to learn an implicit representation …

Personalizing Dialogue Agents: I have a dog, do you have pets too?

2018 · arXiv (Cornell University)

Chit-chat models are known to have several problems: they lack specificity, do not display a consistent personality and are often not very captivating. In this work we present the task of making chit-chat more engaging …

SentEval: An Evaluation Toolkit for Universal Sentence Representations

2018 · arXiv (Cornell University)

We introduce SentEval, a toolkit for evaluating the quality of universal sentence representations. SentEval encompasses a variety of tasks, including binary and multi-class classification, natural language inference and sentence similarity. The set of tasks was …

Code-Switched Named Entity Recognition with Embedding Attention

2018

We describe our work for the CALCS 2018 shared task on named entity recognition on code-switched data. Our system ranked first place for MS Arabic-Egyptian named entity recognition and third place for English-Spanish.

What makes a good conversation? How controllable attributes affect human judgments

2019

Abigail See, Stephen Roller, Douwe Kiela, Jason Weston. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). 2019.

Dynamic Meta-Embeddings for Improved Sentence Representations

2018

While one of the first steps in many NLP systems is selecting what pre-trained word embeddings to use, we argue that such a step is better left for neural networks to figure out by themselves. …

Supervised Learning of Universal Sentence Representations from Natural\n Language Inference Data

2017 · arXiv (Cornell University)

Many modern NLP systems rely on word embeddings, previously trained in an\nunsupervised manner on large corpora, as base features. Efforts to obtain\nembeddings for larger chunks of text, such as sentences, have however not been\nso successful. …

Affordance-Compiled Intelligence: Observable-Only Cognitive Impedance Matching for No-Meta LLM-Integrated Systems

2026 · arXiv (Cornell University)

Affordance-Compiled Intelligence develops Cognitive Impedance Matching Theory (CIMT), an observable-only and no-meta protected compiler theory for LLM-integrated systems. The paper studies how a fixed model-policy can exhibit different operational capability when the surrounding world is …

Masked Language Modeling and the Distributional Hypothesis: Order Word Matters Pre-training for Little

2021 · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing

A possible explanation for the impressive performance of masked language model (MLM) pre-training is that such models have learned to represent the syntactic structures prevalent in classical NLP pipelines. In this paper, we propose a …

Retrieval Augmentation Reduces Hallucination in Conversation

2021

Despite showing increasingly human-like conversational abilities, state-of-the-art dialogue models often suffer from factual incorrectness and hallucination of knowledge (Roller et al., 2020). In this work we explore the use of neural-retrieval-in-the-loop architectures - recently shown …

True Few-Shot Learning with Language Models

2021 · arXiv (Cornell University)

Pretrained language models (LMs) perform well on many tasks even when learning from a few examples, but prior work uses many held-out examples to tune various aspects of learning, such as hyperparameters, training objectives, and …