Why your company should (try to) build its own AI SRE

In today's rapidly evolving world of software development, companies are producing more code than ever before due to the advent of agentic coding tools. While this acceleration benefits product teams, it poses challenges for Site Reliability Engineers (SREs) who are tasked with maintaining the reliability of increasingly complex systems.

The issue arises because machine-generated code can be difficult for humans to comprehend, making root-cause analysis both difficult and time-consuming. This constant pressure to maintain uptime can lead to employee burnout if they are forced to tackle such issues manually.

Rather than relying solely on AI agents to resolve issues, companies should consider building their own in-house AI SREs to better understand their systems. Sam Farid and Nate Heinrich from Chronosphere, a Palo Alto Networks company, believe that creating an AI SRE in-house offers numerous benefits. By developing their own agents, companies can collect and organize crucial information about their systems in the form of Markdown files, which agents can use to identify root causes.

This process not only serves as a valuable learning experience but also results in a useful tool for future root-cause analysis.

Chronosphere's AI SRE product exemplifies the advantages of building an in-house solution, as it requires a telemetry service to collect logs and traces, and an observability tool to store and correlate this information. By investing time and effort into creating their own AI SRE, companies can gain a deeper understanding of their systems, ultimately improving their ability to maintain reliability and prevent downtime.

The practice of mapping internal systems for agentic consumption could prove invaluable, especially during times when unexpected failures occur outside of regular working hours.