Urgent.News

What's breaking now, across thousands of outlets.

Tech

The Model You Requested Is Not Always the Model That Ran

I found a small but expensive ambiguity while building APX: the model selected in APX was easy to mistake for the model that an external coding runtime would actually run. That sounds obvious once stated. APX can talk to its own configured engine, then delegate coding work to Codex or Claude Code. Those CLIs have their own configuration, their own supported model flags, and their own defaults.…

When building APX, a small but significant confusion emerged: the model selected by APX might not be the same model that an external coding runtime would actually execute. This issue is apparent once it is stated. APX can communicate with its own configured engine and then delegate coding tasks to Codex or Claude Code. These CLIs have their own configuration, supported model flags, and default settings.

However, a UI label can be misleading. If it mentions one model, users may reasonably assume the next coding run will use that model as well. Yet, this expectation is not a contract. In one instance, a session requested a model that was not available in the external runtime. The CLI responded with a 400 error, and the question—"What exactly did APX ask it to run?"—remained unclear.

To address this, the boundary was changed. From now on, APX will record three distinct facts: the requested model, the model passed to the runtime, and the model that was actually executed. These facts should be recorded honestly. Previously, the APX documentation implied a promise that the external CLI would inherit APX's chosen model.

However, the default behavior is narrower: an unpinned runtime will use its own configured default model. APX provides the work but should not claim control it does not have. The default remains unchanged. APX now explicitly communicates the choice in three ways: the runtime can specify its native model and, when supported, its reasoning effort.

The runtime configuration can pin a model and effort for subsequent calls. Additionally, a runtime can opt-in to follow APX's selected model, but only if that provider is native to the runtime. The order of precedence is important: a direct call takes precedence over a runtime configuration, which in turn takes precedence over APX's inherited setting, and the CLI default wins if none of the above apply.

This ensures the origin of the value is clear, rather than being accidental. It is essential to never pass one model vocabulary into another program. The implementation that mattered most was refusing to treat model identifiers as interchangeable strings. APX's engine selection includes a provider, a model, and an effort. However, an external CLI might only accept a bare model ID, a separate effort flag, or neither.

APX now separates this information before passing it to a runtime adapter. The Codex adapter can receive a model flag and a reasoning-effort setting; the Claude Code adapter can receive a model but not the effort setting. Other runtimes may declare that they do not accept a model from APX. This declaration prevents two undesirable outcomes: APX should not hand a provider-prefixed identifier to a CLI that cannot understand it, and it should not silently ignore a requested flag and execute something else.

If a caller instructs Codex to run an Anthropic model or asks a runtime without an APX model option to take one, the call will fail before the process begins. This is a specific, cheap, and actionable error. A silent fallback might complete successfully while contradicting the reason the user selected a particular model in the first place.

"Effective" should never be synonymous with guessed. A runtime may accept a model pin, but APX does not receive a universal, trustworthy report from every CLI indicating which model ultimately executed the request. Therefore, the result now includes a small model record containing the requested model, the model passed to the runtime, the effort used, the source of the request, and the effective model.

When APX pins a model, the "effective" field reports that pin. If APX does not pin a model, the result indicates the runtime's default (not reported by the CLI). This last string is intentionally straightforward, indicating that APX did not observe what it did not observe. It informs the user about the remaining uncertainty, which lies within the runtime's own configuration and reporting.

APX also logs the requested model and source in the runtime session record. A session should not become a mystery, especially when it is resumed. The same resolution method applies to ordinary runtime calls and agent-to-agent runtime replies, ensuring that a configured pin is not lost when the conversation arrives through a different route.

The tests focused on the boundary between APX and external runtimes, not just the parser. The tests cover precedence rules, provider mismatch errors, unsupported effort errors, and the unpinned default behavior. Additionally, tests run a simulated Codex executable to inspect the arguments it receives for both new runs and resumed ones.

This distinction caught the real risk: a value can be resolved correctly but lost at the adapter boundary. The sandbox behavior remains separate. When a resumed Codex session inherits its sandbox from the initial turn, adding a model pin should not inadvertently alter this unrelated rule. In conclusion, keeping these four key pieces of information—what was requested, what was actually passed, why that value was chosen, and what the caller truly knows about the result—provides a clear and trustworthy framework.

This approach is less magical than relying on a single model label and is far easier to debug, audit, and trust.

Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.

Read the original at dev.to →

More in Tech

More from Wednesday 14 October →