Tushar Khot

Exploring Markov Logic Networks for Question Answering

2015

Elementary-level science exams pose sig-nificant knowledge acquisition and rea-soning challenges for automatic question answering. We develop a system that rea-sons with knowledge derived from text-books, represented in a subset of first-order logic. Automatic extraction, while …

Combining Retrieval, Statistics, and Inference to Answer Elementary Science Questions

2016 · Proceedings of the AAAI Conference on Artificial Intelligence

What capabilities are required for an AI system to pass standard 4th Grade Science Tests? Previous work has examined the use of Markov Logic Networks (MLNs) to represent the requisite background knowledge and interpret test …

SciTaiL: A Textual Entailment Dataset from Science Question Answering

2018 · Proceedings of the AAAI Conference on Artificial Intelligence

We present a new dataset and model for textual entailment, derived from treating multiple-choice question-answering as an entailment problem. SciTail is the first entailment set that is created solely from natural sentences that already exist …

Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge

2018 · arXiv (Cornell University)

We present a new question set, text corpus, and baselines assembled to encourage AI research in advanced question answering. Together, these constitute the AI2 Reasoning Challenge (ARC), which requires far more powerful knowledge and reasoning …

Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answering

2018

We present a new kind of question answering dataset, OpenBookQA, modeled after open book exams for assessing human understanding of a subject. The open book that comes with our questions is a set of 1326 …

AdvEntuRe: Adversarial Training for Textual Entailment with Knowledge-Guided Examples

2018

We consider the problem of learning textual entailment models with limited supervision (5K-10K training examples), and present two complementary approaches for it. First, we propose knowledge-guided adversarial example generators for incorporating large lexical resources in …

QASC: A Dataset for Question Answering via Sentence Composition

2020

Composing knowledge from multiple pieces of texts is a key challenge in multi-hop question answering. We present a multi-hop reasoning dataset, Question Answering via Sentence Composition (QASC), that requires retrieving facts from a large corpus …

Complexity-Based Prompting for Multi-Step Reasoning

2022 · arXiv (Cornell University)

We study the task of prompting large-scale language models to perform multi-step reasoning. Existing work shows that when prompted with a chain of thoughts (CoT), sequences of short sentences describing intermediate reasoning steps towards a …

Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions

2023

Prompting-based large language models (LLMs) are surprisingly powerful at generating natural language reasoning steps or Chains-of-Thoughts (CoT) for multi-step question answering (QA). They struggle, however, when the necessary knowledge is either unavailable to the LLM …