I wrote a test for prompt injection. It passed while the attack worked.
This is a submission for DEV's Summer Bug Smash: Smash Stories powered by Sentry . I maintain a small CLI called llm-council . It puts one question to several models, hides the authorship, and has them rank each other's answers. I use it as an adversarial reviewer on my own work — the whole point is to get disagreement from something that has no reason to be polite to me. On 26 July I pointed it…
This submission is part of DEV's Summer Bug Smash, and it addresses a prompt injection vulnerability. The author maintained a CLI called llm-council, which ranks model answers and discovered an issue while testing it on its own repository. The vulnerability lies in the fact that a hostile voter or a model that had read the repository during training could potentially write a malicious response containing the fence closing marker, thereby bypassing the mitigation and freely appending text that is no longer treated as quoted data.
The author created a test to prove that such an attack could not occur, but it didn't actually test the security property it claimed to verify. The fix involved using a per-run random nonce to ensure that the closing marker was unguessable. The test was then updated to assert the property instead of the arithmetic, proving that only the markers emitted by the system carried the real nonce.
Other gaps in the defense were also addressed, such as ensuring that rankings and responses were fenced appropriately. The changes were verified by mutation testing, with the old test turning red when reverted to a static nonce or unfenced rankings.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
