I made my coding agent audit my prompt before it writes anything
The cost of a vague prompt is paid at the end, not the start. You learn what the model assumed only after it has finished building on the assumption. That is the problem worth solving. You write down your intent, the agent acts on what you wrote, and whatever you left out gets inferred. The gaps never announce themselves. The model reaches for the reading its training makes likeliest, which is a…
In the realm of coding and prompting, a cost becomes evident only after the completion of a project, not before. The issue lies in the model's assumption leading to its actions after the completion of the task. When writing down one's intent, the agent executes based on that intent, and any missing information gets inferred by the model.
These gaps are never explicitly pointed out. The model, using its training data, makes the most likely guess about the average requester, but this may not be the best guess for the individual user. To address this issue, a skill called 'clarify' was developed. This skill serves as a set of standing instructions that the agent reads first, and it can also be summoned using the /clarify command.
Before commencing any work, the 'clarify' skill audits the prompt for any assumptions it might make on its own. It surfaces these assumptions and only proceeds once the user answers. However, auditing every gap would be impractical, as a prompt typically contains dozens of assumptions, most of which do not matter. To address this, the skill categorizes each assumption into three bands: settled, defaulted, and open.
Settled assumptions have only one possible reading and proceed without further inquiry. Defaulted assumptions have a likely reading, which the skill states out loud and allows the user to veto. Open assumptions, on the other hand, can lead to two different readings, each potentially changing the deliverable. These open assumptions trigger questions.
The effectiveness of the 'clarify' skill lies in its ability to identify the open assumptions that need clarification, distinguishing between the clarifying steps that should be kept and those that can be switched off after a day. The skill file emphasizes that the enemy is not ambiguity, but rather silent inference. An expert's prompt is highly compressed, with the unstated intent being the most crucial part.
Recovering this unstated intent from the user, rather than reconstructing it from the user's priors, is essential. A compressed prompt is not inherently bad; treating compression as a sign of absence is what produces confidently wrong results. The 'clarify' skill, available for free without any signup, can be downloaded from the GitHub repository at https://github.com/bryanthood-wph/clarify.
To use it, simply drop it into the project and call the /clarify command as needed. Further reading on the importance of prompts capturing blind spots can be found at https://medium.com/@brnthood/account-for-what-your-prompt-doesnt-say-b443fa3af231?sk=504afa509c9d9df58dbff3290df992a3.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
