Two hotfixes in one day

Flash released two hotfixes in a single day. The first hotfix, version 0.5.6, was shipped in the morning. By evening, version 0.5.7 followed suit. The issue Flash faced was its inability to remember what it said in the previous message. The model was fine, but the math was not.

The problem was that Flash reserved room for the model's reply within the context window. On the reporter's setup, this reservation took up the entire context window, leaving barely any space for the conversation. The fix implemented a change where the reply would only take up a quarter of the window, with the rest of the conversation remaining intact.

An additional change included the fact that the last three exchanges of the conversation would always stay word for word. Any previous tool output would shrink before anything else was dropped. Moreover, summarizing was now turned off by default.

The history budget on the reporter's machine increased significantly from 1,024 tokens to approximately 114,000 tokens. This allowed for a single line of math to be processed seamlessly.

As part of the hotfix, a rewritten system prompt was introduced. This prompt encouraged the model to re-read the conversation, check before guessing, and verify before stating that it was done. The system prompt update was rolled out with a browser update that maintains user sign-in.

Flash now has the capability to send HTML pages, which are sandboxed within the web UI. The lesson emphasized the importance of shipping small updates. When a bug is encountered, fixing it again in the same day is acceptable.