Anthropic found a way to prove which AI model answered you — but there’s a costly catch
Developers using AI APIs have little way of knowing whether the model answering their requests is actually the one they’re The post Anthropic found a way to prove which AI model answered you — but there’s a costly catch appeared first on The New Stack .
Developers utilizing AI APIs currently have no reliable method of determining which model is responding to their requests. A provider may redirect queries to a more affordable model, or a malicious actor might interfere with the responses, all without customers having an obvious means of detection. Anthropic aims to rectify this oversight with FlashProve, an open-source cryptographic system unveiled by the company's Frontier Red Team.
Announced on Tuesday, FlashProve could empower developers to confirm the third-party AI services they employ and identify tampered inference systems. However, the system remains a research prototype, and its proof generation process is prohibitively expensive for regular API usage.
FlashProve employs zkSNARKs to ascertain that a response emanates from a particular set of model weights without exposing those weights. However, the proof still necessitates a trusted origin; otherwise, its validity is questionable. Anthropic has deposited FlashProve on GitHub, enabling developers to download and test the system independently. Yet, it's important to note that FlashProve is still a research prototype and not part of the Claude API.
To illustrate the efficiency of FlashProve, Anthropic's benchmarks indicate that the system processed approximately 110 prompt tokens per second while generating proofs on a single Nvidia H100 GPU. Each generated proof was around 18.78 MB and took about 21 seconds to verify on a single CPU thread. Generating the proof is considerably more resource-intensive than standard inference using vLLM, requiring 70 to 95 times more GPU time.
Moreover, FlashProve generates proofs in batches post-request completion, as opposed to token-by-token verification.
Despite these limitations, FlashProve aims to shrink proof sizes to roughly 19 MB and verification times to seconds. However, generating such proofs remains expensive and impractical for applications demanding immediate verification of every response. For developers already mindful of every token consumed by inference, the added GPU overhead may be challenging to justify for routine API traffic. That said, Anthropic asserts that FlashProve is considerably swifter than previous open-source systems.
An earlier open-source system, VerInf, required 14.3 hours to generate a proof covering 1,000 tokens, resulting in a 93.6 GB proof that took nearly a day to verify. Anthropic credits Claude with much of FlashProve's development, including optimizations that expedited its cryptographic proofs. The researchers utilized Lean to formally verify the protocol's mathematical attributes, including the acceptance of valid proofs, rejection of forged proofs with a high probability, and absence of model weight revelation beyond what the responses disclose.
FlashProve encompasses two verifiers: a swifter C++ implementation and a Lean version backed by formal proofs. The C++ verifier has passed Anthropic's comparison tests but has not undergone formal verification. Neither has the GPU prover. Anthropic claims that both the C++ verifier and GPU prover have been subjected to tests against the formally defined protocol, yet neither possesses the same mathematical assurances as the Lean implementation.
It's worth noting that FlashProve is limited to 1,024 tokens of context. It also utilizes integer-based inference and greedy decoding, while production models typically rely on floating-point calculations and sampling. There's no word on when, or whether, FlashProve will be integrated into the Claude API.
Written by urgent.news from The New Stack's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
