Zhenzhou Ji

Pre-training Language Models with Deterministic Factual Knowledge

2022 · arXiv (Cornell University)

Previous works show that Pre-trained Language Models (PLMs) can capture factual knowledge. However, some analyses reveal that PLMs fail to perform it robustly, e.g., being sensitive to the changes of prompts when extracting factual knowledge. …