Yelong Shen

Modeling Large-Scale Structured Relationships with Shared Memory for Knowledge Base Completion

2017

Recent studies on knowledge base completion, the task of recovering missing relationships based on recorded relations, demonstrate the importance of learning embeddings from multi-step relations. However, due to the size of knowledge bases, learning multi-step …

ReasoNet: Learning to Stop Reading in Machine Comprehension

2016 · arXiv (Cornell University)

Teaching a computer to read and answer general questions pertaining to a document is a challenging yet unsolved problem. In this paper, we describe a novel neural network architecture called the Reasoning Network (ReasoNet) for …

Controllable Natural Language Generation with Contrastive Prefixes

2022 · Findings of the Association for Computational Linguistics: ACL 2022

To guide the generation of large pretrained language models (LM), previous work has focused on directly fine-tuning the language model or utilizing an attribute discriminator. In this work, we propose a novel lightweight framework for …

CAMERO: Consistency Regularized Ensemble of Perturbed Language Models with Weight Sharing

2022 · arXiv (Cornell University)

Model ensemble is a popular approach to produce a low-variance and well-generalized model. However, it induces large memory and inference costs, which are often not affordable for real-world deployment. Existing work has resorted to sharing …

ToRA: A Tool-Integrated Reasoning Agent for Mathematical Problem Solving

2023 · arXiv (Cornell University)

Large language models have made significant progress in various language tasks, yet they still struggle with complex mathematics. In this paper, we propose ToRA a series of Tool-integrated Reasoning Agents designed to solve challenging mathematical …

MTL-LoRA: Low-Rank Adaptation for Multi-Task Learning

2024 · arXiv (Cornell University)

Parameter-efficient fine-tuning (PEFT) has been widely employed for domain adaptation, with LoRA being one of the most prominent methods due to its simplicity and effectiveness. However, in multi-task learning (MTL) scenarios, LoRA tends to obscure …

LoRC: Low-Rank Compression for LLMs KV Cache with a Progressive Compression Strategy

2024 · arXiv (Cornell University)

The Key-Value (KV) cache is a crucial component in serving transformer-based autoregressive large language models (LLMs), enabling faster inference by storing previously computed KV vectors. However, its memory consumption scales linearly with sequence length and …

Beyond Pass@1: Self-Play with Variational Problem Synthesis Sustains RLVR

2025 · arXiv (Cornell University)

Reinforcement Learning with Verifiable Rewards (RLVR) has recently emerged as a key paradigm for post-training Large Language Models (LLMs), particularly for complex reasoning tasks. However, vanilla RLVR training has been shown to improve Pass@1 performance …

Stochastic Answer Networks for Machine Reading Comprehension

2018

We propose a simple yet robust stochastic answer network (SAN) that simulates multi-step reasoning in machine reading comprehension. Compared to previous work such as ReasoNet which used reinforcement learning to determine the number of steps, …

LoRA Fine-Tuning of a 3B Code LLM for Algorithmic Efficiency

2021 · arXiv (Cornell University)

An important paradigm of natural language processing consists of large-scale pre-training on general domain data and adaptation to particular tasks or domains. As we pre-train larger models, full fine-tuning, which retrains all model parameters, becomes …