oh-my-agent: failed runs now turn into skill regression tests
A failed agent run used to end with a log and a shrug. This week oh-my-agent closed that loop: a failure can be captured as an incident, promoted to a skill fixture, and fed to an optimizer that edits the skill under a dispatch budget. It took 131 commits and moved the CLI from 14.7.11 to 14.13.1. What's new Incident capture and promotion : oma harness incident scan lists failed, blocked, and…
Failed agent runs have transformed into skill regression tests with the introduction of a new feature in oh-my-agent. When an agent run fails, it is now captured as an incident, promoted to a skill fixture, and fed to an optimizer to edit the underlying skill within a dispatch budget. This update required 131 commits to progress the CLI from version 14.7.11 to 14.13.1.
The incident capture and promotion process lists failed, blocked, and partial runs that no incident references. The incident promoter derives a regression fixture for the skill the agent used, but only when the recorded failing output also fails the fixture. The feedback scan chains capture, promotion, and optimization, keeping the last 64 KiB of runner log for fixture validation.
Budgeted skill evolution ensures that an edit is only applied if neither the validation nor training split regresses, and at least one metric improves. The maximum dispatches per run is now enforced, and any attempt to exceed it will result in promotion being blocked. Every applied edit records lineage, and promotions and rollbacks can trace it back.
Meta-optimization treats the optimizer prompt itself as the candidate, requiring a seeded paired bootstrap 95% interval above zero and at least three pairs for promotion. The first adopted procedure improved on the current one by 0.27 on average, with an interval of [0.04, 0.54], and added three grounding rules for proposed skill edits.
Routing evaluation shows the model every installed skill description and records whether it selects the target, neighbor, or nothing. The body helps section is separated from the skill gets selected section. A PreToolUse hook denies native Grep, Glob, and recursive shell search commands while a provider is configured, with the Serena tool to use mentioned. OMA_CI_ALLOW_NATIVE can be set to an escape hatch, and providers.code_intelligence_guard: off disables it.
Session-start notices announce skill and procedure promotions once, with the edit and lifts displayed in a single line. OMA Doctor gained an Evolution note. Orca reporting allows spawned child agents to report their lifecycle to Orca, and Qwen gained native agent definitions.
Several issues have been fixed, including a weekly cross-post job failure after model_preset moved to auto, raw output unwrapping for vendors without a response_jq path, and fixes for OS jobs calling oma schedule:run id and stale registrations. The skill judge now parses PASS/FAIL from the raw Claude envelope before scoring, and outputs are unwrapped before scoring.
The new features have improved optimization time, allowing four candidates to run in parallel, reducing a forty-minute optimization epoch to a more manageable time. Concurrency controls have been added, with a default concurrency of 4, and meta-optimization now overlaps inner runs across skills. Fixtures test what the body states, and the new twelve fixtures have significantly improved scores compared to the first version.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
