Qingxiu Dong

Neural Knowledge Bank for Pretrained Transformers

2022 · arXiv (Cornell University)

The ability of pretrained Transformers to remember factual knowledge is essential but still limited for existing models. Inspired by existing work that regards Feed-Forward Networks (FFNs) in Transformers as key-value memories, we design a Neural …

A Survey on In-context Learning

2022 · arXiv (Cornell University)

With the increasing capabilities of large language models (LLMs), in-context learning (ICL) has emerged as a new paradigm for natural language processing (NLP), where LLMs make predictions based on contexts augmented with a few examples. …