{
  "id": 11175907,
  "title": "RoPE vs sinusoidal positional encoding, measured: 55 logits of drift against 0.0005",
  "url": "https://urgent.news/2026/10/01/rope-vs-sinusoidal-positional-encoding-measured-55-logits-of-drift",
  "topic": "ai",
  "section": "AI",
  "published": "2026-10-01T11:15:38.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/miraceti/rope-vs-sinusoidal-positional-encoding-measured-55-logits-of-drift-against-00005-pph"
  },
  "original_language": "en",
  "account": "RoPE and sinusoidal positional encoding were evaluated with 55 logits of drift against 0.0005. Two token embeddings were kept five positions apart and slid down a 2048-token sequence. Sinusoidal absolute position encoding produced a 55.5-logit spread that changed sign 157 times. Rotary embeddings kept the same pair scoring -0.6102 every time, with only a 5.4e-4 difference. The code, output, and details of the calculation were provided, showing no positional encoding at all produced a score of -5.8183. The sinusoidal encoding showed fluctuating logit values, while RoPE remained consistent at -0.6102. The gap-0 rows served as a sanity check, with RoPE matching the no-positional-encoding score at various positions. The worst same-distance spread across various gaps was 7.458e-04 absolute and 2.658e-03 relative.",
  "summary": "Take two token embeddings. Keep them exactly five positions apart and slide the pair down a 2048-token sequence. Under sinusoidal absolute position encoding the attention score for that one unchanging pair runs from -33.91 to +21.61 — a 55.5-logit spread that changes sign 157 times on the way. Under RoPE the same pair scores -0.6102 every single time , to within 5.4e-4. That is the whole argument…",
  "key_points": [
    "RoPE showed consistent -0.6102 logits, while sinusoidal encoding fluctuated",
    "Sinusoidal encoding had 55.5-logit spread changing sign 157 times",
    "Gap-0 rows confirmed RoPE matched no-positional-encoding score"
  ],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}