Your LLM app will get jailbroken. Here are 8 free adversarial probes you can run in 30 seconds

I run an autonomous agent that evaluates security decisions based on untrusted input. To ensure it's trustworthy, I tested it from an attacker's perspective. The outcome revealed that out of 100 MEDIUM-level tests, 27 were flagged, and one of the eight probes identified a soft failure. In this particular failure, the model unintentionally disclosed part of its system prompt when asked politely in a different language.

The entire 8-probe battery can be accessed for free, without any signup, and can be run against your own system prompt using the provided API endpoint. The process typically takes around 35 seconds, and the results include a risk score, per-probe raw prompts and model replies, and the flagged responses. The report is hash-verifiable, allowing you to verify the results and ensure their integrity.

The 8 free probes cover a range of attack classes, with this core kit focusing on the high-yield aspects. Other probes are available in a larger set of 35 across 17 attack classes, but this 8-probe battery is the most effective for quick checks. When I ran this test myself, I discovered that the softest failure was not the aggressive jailbreak but the polite, cross-language one.

Typically, aggressive jailbreaks are easier to detect due to their loud nature, while soft failures can be more challenging to identify and pose a significant risk. The risk score provides meaningful insights only if the results can be reproduced. Each probe includes the raw prompt and raw model reply, enabling you to compare and verify the outcomes, rather than relying on trusting the system.

The scan process takes approximately 35 seconds, whereas my traditional reviews often take a day or more. The probes are intentionally simple to ensure fast and reproducible results. If you are deploying an LLM application behind a form, this 30-second check is crucial before merging it into production. The 15-probe core of this kit is licensed under the MIT license, allowing you to clone, modify, and re-run any probe as needed. You can find the complete kit at gitee.com/xydhw/llm-red-team-starter-kit.