Alex Vitvitskyi

Never Give Up: Learning Directed Exploration Strategies

2020 · arXiv (Cornell University)

We propose a reinforcement learning agent to solve hard exploration games by learning a range of directed exploratory policies. We construct an episodic memory-based intrinsic reward using k-nearest neighbors over the agent's recent experience to …

Round and Round We Go! What makes Rotary Positional Encodings useful?

2024 · arXiv (Cornell University)

Positional Encodings (PEs) are a critical component of Transformer-based Large Language Models (LLMs), providing the attention mechanism with important sequence-position information. One of the most popular types of encoding used today in LLMs are Rotary …