Marek Rei

Online Representation Learning in Recurrent Neural Language Models

2015

We investigate an extension of continuous online learning in recurrent neural network language models. The model keeps a separate vector representation of the current unit of text being processed and adaptively adjusts it after each …

Attending to Characters in Neural Sequence Labeling Models

2016 · arXiv (Cornell University)

Sequence labeling architectures use word embeddings for capturing similarity, but suffer when handling previously unseen or rare words. We investigate character-level extensions to such models and propose a novel architecture for combining alternative word representations. …

Logical Reasoning with Span-Level Predictions for Interpretable and Robust NLI Models

2022 · arXiv (Cornell University)

Current Natural Language Inference (NLI) models achieve impressive results, sometimes outperforming humans when evaluating on in-distribution test sets. However, as these models are known to learn from annotation artefacts and dataset biases, it is unclear …

An Extended Sequence Tagging Vocabulary for Grammatical Error Correction

2023 · arXiv (Cornell University)

We extend a current sequence-tagging approach to Grammatical Error Correction (GEC) by introducing specialised tags for spelling correction and morphological inflection using the SymSpell and LemmInflect algorithms. Our approach improves generalisation: the proposed new tagset …

StateAct: Enhancing LLM Base Agents via Self-prompting and State-tracking

2024 · arXiv (Cornell University)

Large language models (LLMs) are increasingly used as autonomous agents, tackling tasks from robotics to web navigation. Their performance depends on the underlying base agent. Existing methods, however, struggle with long-context reasoning and goal adherence. …

A Joint Model for Word Embedding and Word Morphology

2016

This paper presents a joint model for performing unsupervised morphological analysis on words, and learning a character-level composition function from morphemes to word embeddings. Our model splits individual words into segments, and weights each segment …

Compositional Sequence Labeling Models for Error Detection in Learner Writing

2016

In this paper, we present the first experiments using neural network models for the task of error detection in learner writing. We perform a systematic comparison of alternative compositional architectures and propose a framework for …

Automatic Text Scoring Using Neural Networks

2016 · Apollo (University of Cambridge)

Automated Text Scoring (ATS) provides a cost-effective and consistent alternative to human marking. However, in order to achieve good performance, the predictive features of the system need to be manually engineered by human experts. We …