{
  "id": 18912685,
  "title": "How I built an opt-in Modal (DeepSeek) provider for my desktop AI companion — then benchmarked it and kept the old one",
  "url": "https://urgent.news/2026/11/08/how-i-built-an-opt-in-modal-deepseek-provider-for-my-desktop-ai",
  "topic": "ai",
  "section": "AI",
  "published": "2026-11-08T10:27:21.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/krish_verma_77e28d3fd63ca/how-i-built-an-opt-in-modal-deepseek-provider-for-my-desktop-ai-companion-then-benchmarked-it-1mbc"
  },
  "original_language": "en",
  "account": "Ankita AI is a desktop AI assistant created by Krish Verma, designed to assist users while they are away from their computers. Its voice replies are slower than expected, prompting Krish to create an opt-in provider using DeepSeek V4.1 Flash on Modal. The provider was built as a preset in the Ankita codebase, with the model, window size, floor, and token details stored in one map. The provider's auth credentials are passed through two proxy token parts, which are then combined into a single bearer token. The provider is opt-in and can be enabled through the voice settings on Ankita's desktop.\n\nTo benchmark the performance of the new provider, Krish first ran it against planned prompts, measuring the time to first token and the full reply. While Modal performed slightly better in terms of time, it also produced fewer output tokens than Baseten. Krish then ran the real voice loop, which includes the planner, agent prompt, tool schemas, and Sarvam TTS. The results showed that Modal was actually 32% slower than Baseten in the voice loop, despite the faster median time to first token in the planner prompts. This was due to the instability in Modal's response times, which varied greatly depending on whether its prefix was cached or not.\n\nThe instability in Modal's performance, rather than its median time, proved to be the main issue. The benchmarking process revealed that prefill and queue capacity on the deployment were the real bottlenecks, not the generation speed. Modal's decision to ship as an opt-in provider was made after verifying the benchmark results and confirming that the performance improvements were not just due to the bare planner prompts. The provider was kept as an optional addition to Ankita, with the decision to switch it to the default requiring further improvements in prefill capacity and caching.",
  "summary": "How I built an opt-in Modal (DeepSeek) provider for my desktop AI companion — then benchmarked it and kept the old one Ankita AI is a desktop AI assistant by Krish Verma — a little companion that lives on your desktop and does things while you are away. Its voice replies felt slow, and DeepSeek V4.1 Flash on Modal looked faster on paper: cheaper tokens, a 1M-token window, and a dedicated…",
  "key_points": [],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}