Integrating Semantic Representations in a Cross-Modal Approach to Fact-Checking

We propose a cross-modal approach with deep fusion of a language model and graph structures based on Abstract Meaning Representations (AMRs) enriched with Wikidata to address the fact-checking problem.We collect and make available a large dataset of factchecked claim sentences, and systematically compare a transformerbased model with Graph Neural Networks (GNNs) based on AMR graphs and extended with external information.Furthermore, we evaluate the integration of language models and GNNs for the fact verification task.While GNN models on AMR-based graphs alone yield lower scores than transformer based language models on their own, the combined cross-modal approach-leveraging a multilayer and deep interaction between textual and structural information-demonstrates the best performance.Finally, we evaluate the generalization capability of this cross-modal approach integrating AMR-based graph structures on out-of-domain English and German claims.