Serving Sage-A1.5 on an RTX 4090: one broken flag, one queue, and a self-check mode to measure
Run Sage-A1.5 as a research agent on your own card and the surprises are not in the model. They hide in a reasoning flag, a request limit, a token budget and one system prompt string. Sage-A1.5 comes from Shanghai AI Laboratory (the InternScience organisation on Hugging Face), with a paper by Wenjie Lou and 98 co-authors. It is retrained from the Qwen team's open 35B model and the lab's earlier…
Sage-A1.5, a model developed by Shanghai AI Laboratory, can be run on a single RTX 4090 card with surprising efficiency. The model's architecture consists of a mixture-of-experts design, where weights are split into 256 small specialist blocks and each token is processed by eight specialists plus one shared block. This structure allows for significant compression, with only about 3 billion parameters working per token, leading to a compressed 22 GB GGUF file.
The reasoning flag in the model card may cause issues when using llama.cpp, as newer versions lack support for the 'deepthink' format. Additionally, sub-agents employed in the model's agentic orchestration feature may require a server capable of handling multiple requests simultaneously. The model's token budget includes hidden tokens used for reasoning, which count against the max_tokens limit.
Enabling verification mode, which checks the model's work before providing an answer, can significantly improve performance, raising the accuracy on the BrowseComp benchmark from 72.5% to 82.9%. However, this testing is limited to a single model and card, and does not provide a comprehensive evaluation of the model's research capabilities at scale.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
