{
  "id": 978483,
  "title": "An AI Capture-the-Flag Tournament: What the Scoreboard Counted",
  "url": "https://urgent.news/2026/08/15/an-ai-capture-the-flag-tournament-what-the-scoreboard-counted",
  "topic": "ai",
  "section": "AI",
  "published": "2026-08-15T06:51:26.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/megapixel99/an-ai-capture-the-flag-tournament-what-the-scoreboard-counted-56fp"
  },
  "original_language": "en",
  "account": "In April, the author ran an AI capture-the-flag tournament with five games between small open-weight models ranging in size from 1.0B to 2.5B parameters. Qwen 3.5 at 2.3B captured 13 flags without losing any, while the other four models captured a total of two flags. The tournament was designed to test whether model size impacts security reasoning and if multi-step exploitation is feasible for smaller models.\n\nAfter running 327 more games with larger models, the first conclusion drawn could not be confirmed. The tournament included five hosted models and one local 3B fine-tune, totaling 327 games with a scoreboard. However, two things in the final table contradicted the original report. The 189 bonus flags were distributed among the models, with the encrypted vault being the most challenging bonus flag, captured 23 times. Despite having more parameters than all other models, RNJ-1 8B finished last, while a 3B fine-tune led on main flags, capturing two bonus flags.\n\nThe prompt used in the tournament did not specify attack methods, allowing models to use various techniques. RNJ-1, with 7,351 attacker commands, mostly focused on enumerating the machine it was on rather than exploiting targets. It did not address any opponents, resulting in no bonus flags being credited to it. The checker used for flag captures did not compare the output against the actual input, which allowed RNJ-1 to earn credits for turns where it wrote flags into its own command.",
  "summary": "Code: Megapixel99/capture-the-flag In April I ran five games of an AI capture-the-flag tournament between five small open-weight models (1.0B to 2.5B parameters). Each was given root on an identical Ubuntu container and told to steal /root/flag.txt from the others while defending its own. Qwen 3.5 at 2.3B captured 13 flags and lost none. The other four captured two flags between them. Nobody…",
  "key_points": [
    "Qwen 3.5 captured 13 flags without losing any in AI capture-the-flag tournament",
    "Larger models showed no confirmed impact on security reasoning or multi-step exploitation",
    "RNJ-1 8B finished last despite having more parameters than other models"
  ],
  "editors_take": "The AI tournament's expanded results contradict the initial conclusion that model size directly impacts security reasoning, as a smaller fine-tune model outperformed a larger one, and RNJ-1's size did not guarantee success.",
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}