Keyur Faldu

KI-BERT: Infusing Knowledge Context for Better Language and Domain Understanding

2021 · arXiv (Cornell University)

Contextualized entity representations learned by state-of-the-art transformer-based language models (TLMs) like BERT, GPT, T5, etc., leverage the attention mechanism to learn the data context from training data corpus. However, these models do not use the …