Updating AI agent model without breaking things

Before implementing any modifications, it is crucial to safeguard the existing system. Automated daily backups are standard practice, and any significant alteration should have its own distinct backup. The majority of models autonomously generate backups at least once weekly. Prior to executing any major update, it is advisable to create a manual backup.

Individuals have the discretion to update their Claude Code agents manually when a substantial new model becomes available. In the past, this involved transitioning from Opus 5 to Opus 5.5, bypassing the intermediate versions. The decision on when to update agents rests with the user; there is no obligation to adopt every new model as soon as it becomes available.

When preparing for an update, it is necessary to save more than just the model settings. The agent's instructions, memory, code, data, and connection settings must all be included in the backup. This ensures that if a rollback becomes necessary, every essential component of the system can be restored to its previous state. Additionally, it is essential to verify the backup date and attempt to restore it in a separate environment.

These backups should be stored separately from the main system and access to them should be restricted to prevent any potential exposure of personal data or connection settings. CISA advises conducting regular tests on recovery procedures and maintaining backups in isolation.

To evaluate the performance of the new model, it is recommended to run it independently from the operational agent first. The agent should be tasked with a series of routine activities as well as more challenging scenarios where precision is paramount. For instance, if the agent is responsible for managing a calendar, it is vital to verify whether it comprehends dates, identifies available time slots, and seeks confirmation before scheduling a meeting.

During these tests, real sending and data alterations must be disabled, or safe test records should be utilized. The agent's responses and actions should then be compared to those of the previous model, with particular attention paid to whether it adheres to instructions, utilizes tools appropriately, and conducts its checks accurately.

The speed and cost implications of the stronger model should also be considered, as a superior-performing model on paper does not necessarily guarantee optimal results for a specific task.

It is crucial to follow the model provider's guidelines, as behavior and settings may differ from those of the previous version. For example, Anthropic's Opus 5.5 guide outlines the differences from Opus 5 and provides instructions on configuring the reasoning effort. After switching to the new model, it is advisable to migrate one agent at a time rather than the entire system simultaneously.

Each agent should be thoroughly tested with real tasks and errors before progressing to the next one. Should any issues arise, it is possible to restore the previous model and settings, along with any relevant data from the tested backup. Actions taken in external services may require separate corrections. Ultimately, updating models is an integral part of AI system support and development, and the focus should be on how well the model performs the assigned tasks and whether a working state can be restored, rather than solely on the version number.