Urgent.News

What's breaking now, across thousands of outlets.

AI

VDiff-Bench: A Challenging Benchmark for Fine-Grained Image Difference Identification

Multimodal Large Language Models (MLLMs) perform strongly on general visual understanding tasks such as visual question answering, yet they often struggle with a basic comparative skill: identifying what has changed between two similar images. We introduce VDiff-Bench, a challenging multiple-choice benchmark for fine-grained Image Difference Identification. VDiff-Bench contains 1,756 four-way…

We haven't written up this one. arXiv cs.AI has the full story — the link below goes straight to it.

Read the original at arxiv.org →

More in AI

Demystifying LLM Context Windows: How AI Memory Works (and Why It Fails)

Imagine asking an AI coding assistant to help refactor a complex application. At first, it gives sharp, accurate responses.

  • Context window limits LLMs' active memory during interactions
  • Tokenization and embeddings create numerical representations of text
  • Context window complexity scales quadratically with length

SWE-Gate: Why Passing Tests Isn't Enough for Agent-Generated Code

Coding agents pass tests but fail code review. Repository-level benchmarks measure test passage but ignore review acceptance criteria. This is the blind spot in every benchmark from SWE-bench onward.

  • SWE-Gate introduces a second evaluation gate beyond test success to assess review compliance.
  • 34% of agent-generated patches failed review constraints despite passing functional tests.
  • SWE-Gate separates functional correctness evaluation from constraint compliance evaluation.

Anthropic Drops Claude Sonnet 4.5: Extended Thinking Hits 200K Context for Agentic Coding

Anthropic just shipped Claude Sonnet 4.5, and the headline number isn't the benchmark score, it's the 200K token context window combined with what they're calling "extended thinking mode." For…

  • Claude Sonnet 4.5 introduces 200K token context window for agentic coding.
  • Extended thinking mode enables interleaving reasoning with action in tasks.
  • Benchmark shows 30% reduction in hallucinations and 77.2% GitHub issue resolution.

More from Saturday 5 September →