When Claims Evolve: Evaluating and Enhancing the Robustness of Embedding Models Against Misinformation Edits

Online misinformation remains a critical challenge, and fact-checkers increasingly rely on claim matching systems that use sentence embedding models to retrieve relevant fact-checks.However, as users interact with claims online, they often introduce edits, and it remains unclear whether current embedding models used in retrieval are robust to such edits.To investigate this, we introduce a perturbation framework that generates valid and natural claim variations, enabling us to assess the robustness of a wide-range of sentence embedding models in a multi-stage retrieval pipeline and evaluate the effectiveness of various mitigation approaches.Our evaluation reveals that standard embedding models exhibit notable performance drops on edited claims, while LLM-distilled embedding models offer improved robustness at a higher computational cost.Although a strong reranker helps to reduce the performance drop, it cannot fully compensate for first-stage retrieval gaps.To address these retrieval gaps, we evaluate train-and inference-time mitigation approaches, demonstrating that they can improve in-domain robustness by up to 17 percentage points and boost out-of-domain generalization by 10 percentage points.Overall, our findings provide practical improvements to claim-matching systems, enabling more reliable fact-checking of evolving misinformation.Mitigation Approaches Covid-19 no pass ordinary flu for how e dey kill people Covid-19 no pass ordinary flu for how e dey kill people Covid-19 is no more deadly than the ordinary flu.Teacher Model Student Model Covid-19 is only as deadly as the seasonal flu MSE-Loss MSE-Loss Knowledge Distillation Claim Normalization q q' q' q'' Input Claim (q): Covid-19 is only as deadly as the seasonal flu Fact Check: COVID-19 has a significantly higher mortality rate than seasonal flu, with greater severity and hospitalizations LLM As a Perturber LLM As a Verifier Perturbation Generation N candidate rewrites verified perturbations COVID-19 IS ONLY AS DEADLY AS THE SEASONAL FLU Covid-19 no pass ordinary flu for how e dey kill people COVID-19 is onli as dedli as the siznl flu BM25 +0.0 +0.0 -15.2 -15.0 +0.5 -2.6 -2.7 -12.4 -0.8 -0.2 -9.2 -6.7 -5.0 -1.6 all-distilroberta-v1 -0.4 -35.4 -15.8 -13.9 +1.8