{
  "id": 18880606,
  "title": "Serving Sage-A1.5 on an RTX 4090: one broken flag, one queue, and a self-check mode to measure",
  "url": "https://urgent.news/2026/11/08/serving-sage-a1-5-on-an-rtx-4090-one-broken-flag-one-queue-and-a-self",
  "topic": "tech",
  "section": "Tech",
  "published": "2026-11-08T06:36:39.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/inferencekey/serving-sage-a15-on-an-rtx-4090-one-broken-flag-one-queue-and-a-self-check-mode-to-measure-4cdj"
  },
  "original_language": "en",
  "account": "Sage-A1.5, a model developed by Shanghai AI Laboratory, can be run on a single RTX 4090 card with surprising efficiency. The model's architecture consists of a mixture-of-experts design, where weights are split into 256 small specialist blocks and each token is processed by eight specialists plus one shared block. This structure allows for significant compression, with only about 3 billion parameters working per token, leading to a compressed 22 GB GGUF file. The reasoning flag in the model card may cause issues when using llama.cpp, as newer versions lack support for the 'deepthink' format. Additionally, sub-agents employed in the model's agentic orchestration feature may require a server capable of handling multiple requests simultaneously. The model's token budget includes hidden tokens used for reasoning, which count against the max_tokens limit. Enabling verification mode, which checks the model's work before providing an answer, can significantly improve performance, raising the accuracy on the BrowseComp benchmark from 72.5% to 82.9%. However, this testing is limited to a single model and card, and does not provide a comprehensive evaluation of the model's research capabilities at scale.",
  "summary": "Run Sage-A1.5 as a research agent on your own card and the surprises are not in the model. They hide in a reasoning flag, a request limit, a token budget and one system prompt string. Sage-A1.5 comes from Shanghai AI Laboratory (the InternScience organisation on Hugging Face), with a paper by Wenjie Lou and 98 co-authors. It is retrained from the Qwen team's open 35B model and the lab's earlier…",
  "key_points": [
    "Sage-A1.5 runs efficiently on a single RTX 4090 with compressed 22 GB GGUF file",
    "Reasoning flag and sub-agents may cause compatibility issues with llama.cpp",
    "Verification mode improves accuracy on BrowseComp benchmark from 72.5% to 82.9%"
  ],
  "editors_take": "The mixture-of-experts architecture enables high-performance reasoning on consumer hardware, though technical compatibility issues and single-card limitations restrict its practical deployment for large-scale agentic orchestration.",
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}