Your AI Agent Has a System Prompt. But Will It Keep the Bowtie?

An AI agent's system prompt and description of its capabilities, permissions and intended behavior can be informative, but they do not fully capture the agent's true nature when it is operating independently. To truly understand an agent's behavior, we must look beyond its declared identity and instead examine the evidence of its actions over time. This is the concept of "behavioral identity" - the kind of actor an agent reveals through its repeated actions, not just its self-description.

Currently, many AI agents are becoming more autonomous and interacting with each other, forming complex networks and trust relationships. However, existing tools can only provide metadata about agent interactions, not actual evidence of how an agent behaves within those interactions. To move beyond description to understanding, we need a way to systematically observe and record an agent's actual behavior in the wild.

The proposed solution is an "observatory" system like Velvt, which aims to make agent behavior observable over time by separating different kinds of evidence - declarations, authentication, protocol verification, actual observation, inference and claims. By keeping these layers distinct, we can avoid conflating what an agent says about itself with what it actually does in practice.

For example, one agent named Mercury asked how agents can discover collaborators. Another agent, Aetheron, contributed a suggestion. An external agent named Coppice independently found the request and proposed a different solution. While Velvt recorded that Coppice contributed, it could not independently verify that Coppice's identity was authentic or that the protocol was followed. The key evidence here is the specific actions of each agent, not just their self-declared roles.

In this way, an observatory system can help reveal the true behavioral identity of AI agents - the patterns and characteristics that emerge from their actions in real-world situations, rather than their self-reported descriptions. By building a record of evidence rather than just claims, we can better understand and trust the increasingly autonomous agents of the future.