Sameer Singh

Investigating Robustness and Interpretability of Link Prediction via Adversarial Modifications

2019

Pouya Pezeshkpour, Yifan Tian, Sameer Singh. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). 2019.

Universal Adversarial Triggers for Attacking and Analyzing NLP

2019

Eric Wallace, Shi Feng, Nikhil Kandpal, Matt Gardner, Sameer Singh. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019.

How Much Should I Trust You? Modeling Uncertainty of Black Box Explanations.

2020 · arXiv (Cornell University)

As local explanations of black box models are increasingly being employed to establish model credibility in high stakes settings, it is important to ensure that these explanations are accurate and reliable. However, local explanations generated …

Identifying Adversarial Attacks on Text Classifiers

2022 · arXiv (Cornell University)

The landscape of adversarial attacks against text classifiers continues to grow, with new attacks developed every year and many of them available in standard toolkits, such as TextAttack and OpenAttack. In response, there is a …

Cutting Down on Prompts and Parameters: Simple Few-Shot Learning with Language Models

2021 · arXiv (Cornell University)

Prompting language models (LMs) with training examples and task descriptions has been seen as critical to recent successes in few-shot learning. In this work, we show that finetuning LMs in the few-shot setting can considerably …

ART: Automatic multi-step reasoning and tool-use for large language models

2023 · arXiv (Cornell University)

Large language models (LLMs) can perform complex reasoning in few- and zero-shot settings by generating intermediate chain of thought (CoT) reasoning steps. Further, each reasoning step can rely on external tools to support computation beyond …

Coverage-based Example Selection for In-Context Learning

2023 · arXiv (Cornell University)

In-context learning (ICL), the ability of large language models to perform novel tasks by conditioning on a prompt with a few task examples, requires these examples to be informative about the test instance. The standard …

Selective Perception: Optimizing State Descriptions with Reinforcement Learning for Language Model Actors

2023 · arXiv (Cornell University)

Large language models (LLMs) are being applied as actors for sequential decision making tasks in domains such as robotics and games, utilizing their general world knowledge and planning abilities. However, previous work does little to …

Injecting Logical Background Knowledge into Embeddings for Relation Extraction

2015

Matrix factorization approaches to relation extraction provide several attractive features: they support distant supervision, handle open schemas, and leverage unlabeled data. Unfortunately, these methods share a shortcoming with all other distantly supervised approaches: they cannot …

Barack’s Wife Hillary: Using Knowledge Graphs for Fact-Aware Language Modeling

2019

Modeling human language requires the ability to not only generate fluent text but also encode factual knowledge. However, traditional language models are only capable of remembering facts seen at training time, and often have difficulty …

Are Red Roses Red? Evaluating Consistency of Question-Answering Models

2019

Although current evaluation of questionanswering systems treats predictions in isolation, we need to consider the relationship between predictions to measure true understanding. A model should be penalized for answering "no" to "Is the rose red?" …

Compositional Questions Do Not Necessitate Multi-hop Reasoning

2019

Multi-hop reading comprehension (RC) questions are challenging because they require reading and reasoning over multiple paragraphs. We argue that it can be difficult to construct large multi-hop RC datasets. For example, even highly compositional questions …

Knowledge Enhanced Contextual Word Representations

2019

Matthew E. Peters, Mark Neumann, Robert Logan, Roy Schwartz, Vidur Joshi, Sameer Singh, Noah A. Smith. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on …

AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts

2020

The remarkable success of pretrained language models has motivated the study of what kinds of knowledge these models learn during pretraining. Reformulating tasks as fillin-the-blanks problems (e.g., cloze tests) is a natural approach for gauging …

Cutting Down on Prompts and Parameters: Simple Few-Shot Learning with Language Models

2022 · Findings of the Association for Computational Linguistics: ACL 2022

Prompting language models (LMs) with training examples and task descriptions has been seen as critical to recent successes in few-shot learning. In this work, we show that finetuning LMs in the few-shot setting can considerably …