Marius Mosbach

Some steps towards the generation of diachronic WordNets

2019 · DSpace repository (University of Tartu)

We apply hyperbolic embeddings to trace the dynamics of change of conceptualsemantic relationships in a large diachronic scientific corpus (200 years).Our focus is on emerging scientific fields and the increasingly specialized terminology establishing around them.Reproducing …

On the Interplay Between Fine-tuning and Sentence-level Probing for Linguistic Knowledge in Pre-trained Transformers

2020

Fine-tuning pre-trained contextualized embedding models has become an integral part of the NLP pipeline. At the same time, probing has emerged as a way to investigate the linguistic knowledge captured by pre-trained models. Very little …

Fusing Sentence Embeddings Into LSTM-based Autoregressive Language Models

2022 · arXiv (Cornell University)

Although masked language models are highly performant and widely adopted by NLP practitioners, they can not be easily used for autoregressive language modelling (next word prediction and sequence probability estimation). We present an LSTM-based autoregressive …