Urgent.News

What's breaking now, across thousands of outlets.

AI

CaliBench: Are the Stochastic Dynamics of Video World Models Physically Calibrated?

Video world models approximate the stochastic distribution of physical outcomes through generative sampling, but existing benchmarks score individual generations or compare distributions coarsely over a whole dataset, leaving the fine-grained aleatoric uncertainty of specific phenomena untested. We introduce CaliBench, which scores outcomes in a physically interpretable discrete space - a bin…

We haven't written up this one. arXiv cs.AI has the full story — the link below goes straight to it.

Read the original at arxiv.org →

More in AI

Shadow Traffic Is the Only Honest Free Model Evaluation

The demo replay played in the dim glow of a conference room, and everyone agreed the free model sounded good. The same four prompts produced crisp summaries, polite error handling, and no obvious…

  • Shadow traffic approach compares free model with primary model
  • Real production traffic used to test free model's performance
  • Metrics like error rates, latency, token consumption analyzed

More from Monday 17 August →