{
  "id": 14994887,
  "title": "Renting Out One T4 to Three Customers: Gemma 4, vLLM Multi-LoRA and a Rust Gateway",
  "url": "https://urgent.news/2026/10/16/renting-out-one-t4-to-three-customers-gemma-4-vllm-multi-lora-and-a",
  "topic": "tech",
  "section": "Tech",
  "published": "2026-10-16T16:48:42.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/xbill/renting-out-one-t4-to-three-customers-gemma-4-vllm-multi-lora-and-a-rust-gateway-3322"
  },
  "original_language": "en",
  "account": "This article presents a guide to renting out a single NVIDIA T4 NVIDIA graphics processing unit (GPU) on a Google Cloud virtual machine (VM) to multiple small businesses. The system featured employs Gemma 4 E2B, powered by vLLM, and a lightweight Rust gateway that manages access and billing for the shared GPU.\n\nThe demonstration reveals that a single T4 struggles when several customers simultaneously submit large prompts, causing latency to increase dramatically. However, vLLM’s priority scheduling capability effectively minimizes this bottleneck, bringing the response time back to approximately 6.52 seconds.\n\nA key aspect of this project is the isolation of each customer’s prompt and responses. This is achieved through keys, fine-tuned adapters, prefix caches, and a ledger that accurately matches vLLM’s token counters. In terms of billing, customers are charged based on GPU time, but a significant portion of this cost is offset by their token usage. For instance, a customer requesting lengthy prompts pays 27.2% of the VM’s cost for just 22.0% of the total tokens processed.",
  "summary": "This is a submission for the Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle This article provides a step by step guide to renting out one NVIDIA T4 on a Google Cloud VM to several small customers at once, with Gemma 4 E2B served by vLLM and a small Rust gateway in front. Every log, script and number is committed. https://github.com/xbill9/t4-timeshare One customer sending 4,000-token…",
  "key_points": [
    "Gemma 4 E2B runs on a single T4 GPU on Google Cloud VM",
    "vLLM’s priority scheduling reduces latency to ~6.52 seconds",
    "Customers billed based on GPU time and token usage"
  ],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}