Tengyu Ma

Iterative Feature Matching: Toward Provable Domain Generalization with Logarithmic Environments

2021 · arXiv (Cornell University)

Domain generalization aims at performing well on unseen test environments with data from a limited number of training environments. Despite a proliferation of proposal algorithms for this task, assessing their performance both theoretically and empirically …

Symbol tuning improves in-context learning in language models

2023 · arXiv (Cornell University)

We present symbol tuning - finetuning language models on in-context input-label pairs where natural language labels (e.g., "positive/negative sentiment") are replaced with arbitrary symbols (e.g., "foo/bar"). Symbol tuning leverages the intuition that when a model …

A Latent Variable Model Approach to PMI-based Word Embeddings

2016 · Transactions of the Association for Computational Linguistics

Semantic word embeddings represent the meaning of a word via a vector, and are created by diverse methods. Many use nonlinear operations on co-occurrence statistics, and have hand-tuned hyperparameters and reweighting methods. This paper proposes …

A Simple but Tough-to-Beat Baseline for Sentence Embeddings

2017 · International Conference on Learning Representations

The success of neural network methods for computing word embeddings has motivated methods for generating semantic embeddings of longer pieces of text, such as sentences and paragraphs. Surprisingly, Wieting et al (ICLR'16) showed that such …

Document-Level Relation Extraction with Adaptive Thresholding and Localized Context Pooling

2021 · Proceedings of the AAAI Conference on Artificial Intelligence

Document-level relation extraction (RE) poses new challenges compared to its sentence-level counterpart. One document commonly contains multiple entity pairs, and one entity pair occurs multiple times in the document associated with multiple possible relations. In …