Yare 3-6-9 Rotary Positional Encoding: A Digital-Root Frequency Schedule for RoPE
At a glance
- الاستشهادات
- 0
- المراجع
- 0
- Comments
- 0
Abstract
Rotary Positional Encoding (RoPE; Su et al. 2021) uses a geometric frequency progression theta_i = 10000^(-2i/d) that, while effective, has no principled structure governing its period or extrapolation behavior. We propose Yare 3-6-9 RoPE, two frequency schedules tuned to digital-root structure: a vortex_369 schedule alternating between two poles, and a digital_root_doubling schedule walking the orbit of 1 under doubling modulo 9 — the cycle (1, 2, 4, 8, 7, 5). Both schedules produce closed-form period structure analyzable without empirical probing. The reference implementation is a non-destructive runtime patch over the HuggingFace RoPE inv_freq buffer, preserving the standard contract and fully revert-able. Unit tests verify norm preservation, idempotency, and determinism. We outline a long-context benchmark suite (LongBench, needle-in-a-haystack at 8k/32k/128k, passkey retrieval) comparing both Yare schedules against vanilla RoPE, NTK-aware scaling, YaRN, and LongRoPE. Empirical results are deferred to v1.1.
Publication details
- DOI
- 10.5281/zenodo.20150473
- OpenAlex
- W7161000049
- Document type
- preprint
- Language
- EN
- Source
- Open MIND
- Last metadata update
Comments
تسجيل الدخول للانضمام إلى النقاش.