Early Anthropic hire, former METR COO have found a way to rein in rogue AI agents
Their startup, Artificial Intelligence Underwriting Company (AIUC) has raised $40 million in a Series A lound led by Ribbit Capital, with participation from First Harmonic.
After Anthropic researcher Jacob Coxon resigned over fears AI could wipe out humanity by decade's end, founders Rune Kvist and Rajiv Dattani met to discuss potential solutions. Kvist, an early Anthropic employee, explained that as AI becomes smarter, it becomes increasingly difficult to control and adopt. Dattani, METR's former COO, noted that enterprises are now rejecting AI deployments due to concerns over system limitations.
To tackle these issues, Kvist and Dattani launched Artificial Intelligence Underwriting Company (AIUC). The startup offers a third-party audit and certification service for AI agents, inspired by cybersecurity's SOC 2 standard. AIUC has developed AIUC-1, a certification standard and testing service, which runs over 5,000 tests on agents to assess their behavior in scenarios involving jailbreaks, hallucinations, and data leaks.
The results are compiled into a detailed report, and AIUC verifies the final audit with human oversight. Investors like Ribbit Capital and First Harmonic have backed AIUC's $40 million Series A round, which follows a $15 million seed round led by Nat Friedman. The company aims to provide independent assessments of AI agents' safety, helping enterprises make informed decisions about adopting AI technology.
Written by urgent.news from TechCrunch's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
