How I built an AI planning tribunal in a weekend by orchestrating AI agents

This weekend, I constructed a tool called Setback for Google's All Things Agentic Hackathon. The tool transforms a resident's simple objection to a nearby development application into a real, legally relevant council submission, while also correctly refusing the parts that don't apply. The majority of the actual coding was performed by AI coding agents that I directed, rather than by me line-by-line. The primary focus of this account is on the process of creating the tool and the lessons learned during its development.

NSW planning law only permits a council to consider a limited list of factors when reviewing a development application nearby: overshadowing, privacy, height and bulk, heritage, and a few others. A decline in property value is not on this list, nor is disliking the applicant. Despite this, residents frequently submit objections, believing they are the reasons behind the council's decisions.

Setback employs a panel of AI reviewers to process these objections, breaking them down into individual grounds, and verifying each against the actual statutory list before submitting anything to the council.

The creation of Setback was primarily an experimental endeavor rather than a straightforward coding exercise. I orchestrated a multi-agent workflow, consisting of several coding agents that operated concurrently within the same codebase. After each iteration, an integration pass was conducted to reconcile the connections between the agents.

This approach is akin to managing a small remote team for a weekend, rather than simply prompting a single chatbot. The critical decisions involved establishing clear boundaries and reviewing processes, rather than focusing on individual lines of code.

During the development process, I encountered two significant bugs that only became apparent during a deployed, end-to-end run. One bug stemmed from an agent framework falling back to the wrong API due to an unset environment variable, which was hidden four layers deep within another library. The other, more concerning, bug involved the production job discarding every resident's uploaded evidence for an entire build wave, because two separate backend processes disagreed on where uploads were actually stored.

These issues could not be detected through a code review that only examined the diff; they required actual execution on real infrastructure to be identified.

To prevent similar issues in the future, I implemented a standing discipline of running the tool against real infrastructure, monitoring its output, and addressing any failures that arise. I also learned the importance of honesty and transparency, even under tight deadlines. When a review pass revealed that my design documents described a component that was designed but never built, I opted not to quietly patch the document.

Instead, I left the correction visible, as an overclaimed document would lose credibility. This principle extends to presentations as well, where the honesty of gaps is more credible than polished claims that fail to withstand scrutiny.