Ruoxi Jia

InfoBERT: Improving Robustness of Language Models from An Information\n Theoretic Perspective

2020 · arXiv (Cornell University)

Large-scale language models such as BERT have achieved state-of-the-art\nperformance across a wide range of NLP tasks. Recent studies, however, show\nthat such BERT-based models are vulnerable facing the threats of textual\nadversarial attacks. We aim to address …

LAVA: Data Valuation without Pre-Specified Learning Algorithms

2023 · arXiv (Cornell University)

Traditionally, data valuation (DV) is posed as a problem of equitably splitting the validation performance of a learning algorithm among the training data. As a result, the calculated data values depend on many design choices …

The Mirrored Influence Hypothesis: Efficient Data Influence Estimation by Harnessing Forward Passes

2024

Large-scale black-box models have become ubiquitous across numerous applications. Understanding the influence of individual training data sources on predictions made by these models is crucial for improving their trustworthiness. Current influence estimation techniques involve computing …