{
  "id": 1619673,
  "title": "Why I run speech-to-text locally instead of calling a cloud API",
  "url": "https://urgent.news/2026/08/18/why-i-run-speech-to-text-locally-instead-of-calling-a-cloud-api",
  "topic": "tech",
  "section": "Tech",
  "published": "2026-08-18T02:34:34.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/hannune/why-i-run-speech-to-text-locally-instead-of-calling-a-cloud-api-59j7"
  },
  "original_language": "en",
  "account": "Running speech-to-text locally instead of utilizing cloud APIs is a decision I've taken for my project. The issue with cloud STT is that transmitting audio to their servers raises privacy concerns, especially since the audio contains sensitive information like client names and project specifics. By running whisper locally, the audio remains on the machine, ensuring data privacy.\n\nThe setup comprises an RTX 3060 (12 GB VRAM) server, where audio files are uploaded from a phone via a progressive web app, stored on the same machine, and then processed. A transcription call is executed using the faster-whisper library with an int8 quantized model. The transcription process involves converting audio to segments, transcribing them, and then printing the results.\n\nThe VRAM situation is manageable on this setup as the 12 GB card is shared with other models. Whisper medium in int8 uses around 2.5-3 GB of VRAM when warm, which is sufficient as long as the models don't run simultaneously. However, when competing models are active, Whisper drops to CPU mode, slowing down the transcription process.\n\nWhile hosted Whisper endpoints are available and sometimes cheaper, they may not be the best choice for this project due to data residency issues. The tradeoff of a slower local run that keeps audio in-house is worth it when the content of calls needs to be protected. However, the accuracy of local Whisper deployment on shared GPU isn't yet fully optimized. Specific questions such as beam_size tuning and the impact on accuracy during inference time are yet to be addressed.",
  "summary": "Why I run speech-to-text locally instead of calling a cloud API Yesterday I wrote about deploying gemma, bge-m3, and whisper on a single server without enough VRAM for all three. This post is about why whisper is one of those three. The problem with cloud STT When you call a cloud speech-to-text API—OpenAI's Whisper endpoint, Google Speech-to-Text, Amazon Transcribe—the audio travels to their…",
  "key_points": [],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}