Octomind 0.44.2: I Removed the Agent's Ability to Grade Its Own Homework

I was constructing a feature last month when the Octomind AI informed me it had finished. When I inspected, I discovered that two out of five required edits had been modified, while the remaining two remained unchanged. The model had declared the task complete, even though it had only altered part of the work. More alarmingly, I observed it self-verifying by reading back the edit it had just made and implicitly praising its own performance.

The AI was essentially grading its own homework, which is not genuine validation. It is the model arbitrarily rating its own assignments. I could no longer tolerate my tool deceiving me. Therefore, in version 0.44.2, I removed the capability entirely. This is not a trivial modification; it represents a fundamental shift in how an AI coding assistant ought to behave.

I am prioritizing transparency over the illusion of proficiency. An AI that declares, "I cannot perform this task," is more beneficial than one that states, "completed!" and leaves you to identify the remaining gaps. What Actually Changed: Three Core Principles

1. Per-Condition Verification

The verification checkpoint no longer permits a general "appears correct" conclusion. Instead, each task is evaluated against a set of individual evidence conditions. If any condition is not met, the entire verification fails, regardless of the holistic "PASS" status. This change was inspired by instances where the AI only validated the straightforward path and declared the task finished, neglecting crucial edge cases.

Now, every specified condition, or those derived by the planning system, undergoes its own verification process. The verifier cannot overlook discrepancies.

2. External Planning

The model's planning tool has been removed, and verification is now handled by an external plan manager running its own lightweight model. This separation prevents the primary model from using planning as a form of procrastination. The main AI generates a plan, claims completion, and treats the plan as the work accomplished. The external planner is a separate model dedicated solely to checklist creation, devoid of code generation or verification. This separation ensures the primary model cannot hide behind a plan and falsely claim progress.

3. Persisted Verification Policy

Verification policies persist throughout a session and remain intact even after system restarts. They are now integrated into the governance hash, which tracks the session's integrity. Previously, the verification rules were lost whenever the session was restarted, necessitating constant re-specification of standards. This change eliminates that inconvenience, as the policy carries over with the session, ensuring consistent verification protocols. What You'll Notice Day-to-Day

You will encounter fewer false positives. The AI will inform you, "I could not verify condition 3," rather than simply stating, "done!" when it fails to meet a specific condition. This might appear slower initially, but it is genuinely honest. You receive accurate status updates instead of premature completion. The verification process is now untainted by fallback models.

If verification fails, the gate remains closed, signaling that the verification process has not succeeded. This ensures you are aware of any verification issues and can address them promptly. You will also notice that the planning process is more efficient. This is due to the planner being a separate, less resource-intensive model focused solely on generating checklists, rather than attempting to write code simultaneously.

The separation of concerns is evident in the reduced latency experienced after the upgrade. Upgrade Notes

Please be aware that this update may require some adjustments. When you upgrade, your config file will automatically migrate to version 5, with a backup created in your config directory. If any issues arise during the upgrade, you can refer to the backup. The plan MCP tool has been discontinued, which may affect existing embeddings that relied on this tool.

If you are an embedder, please reach out for assistance in migrating to the new architecture. The tap and schedule functionalities have been relocated to a dedicated orchestration server, providing cleaner and more efficient timing and coordination. Installation Instructions

For macOS users, you can install Octomind 0.44.2 using the command `brew install muvon/tap/octomind`. During the first run, the config file will automatically migrate to version 5, and a backup will be generated. For other platforms, please pull the latest version from the repository. The release tags are up to date and readily available.