I measured whether my coding agent follows its rules
My agent made the same mistake four times in one conversation. It kept writing files through a shell heredoc, and the heredoc kept eating a backslash, so the file looked right and ran wrong. Three notes in its memory store described exactly this mistake, and all three had been retrieved and read. I went back through the transcript to make sure. Then it did it again. Work in this space gets filed…
During a session, my coding agent repeatedly wrote files using a shell heredoc, which caused issues as the heredoc swallowed a backslash. Three instances of this mistake were noted in the agent's memory, all of which were retrieved and read correctly. The focus of the investigation was on determining whether the agent's actions changed due to the rules implemented.
The results showed that the changes made had little impact compared to expectations. Various attempts at fixing the issue also failed, making it difficult to notice the problems. The implementation of the rules did not yield the desired results.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
