Urgent.News

What's breaking now, across thousands of outlets.

More in AI

Gemini 4 Argon must reverse Google’s AI inertia

Good morning and happy October. Today, I’m cringing at this gaggle of billionaires pretending to be human , vibing to the new Tinashe album, Popstar , and heartily dining on all the brutal ‘Digger’ reviews . As a reminder, Inbox is the tech newsletter that goes beyond the news cycle.

RoPE vs sinusoidal positional encoding, measured: 55 logits of drift against 0.0005

Take two token embeddings. Keep them exactly five positions apart and slide the pair down a 2048-token sequence. Under sinusoidal absolute position encoding the attention score for that one unchanging pair runs from -33.91 to +21.61 — a 55.5-logit spread that changes sign 157 times on the way.

  • RoPE showed consistent -0.6102 logits, while sinusoidal encoding fluctuated
  • Sinusoidal encoding had 55.5-logit spread changing sign 157 times
  • Gap-0 rows confirmed RoPE matched no-positional-encoding score

More from Thursday 1 October →