{
  "id": 18777968,
  "title": "RTX 3090, RTX 5090, A100 and H100 on vLLM: the cheapest GPU flips with your answer time",
  "url": "https://urgent.news/2026/11/07/rtx-3090-rtx-5090-a100-and-h100-on-vllm-the-cheapest-gpu-flips-with",
  "topic": "tech",
  "section": "Tech",
  "published": "2026-11-07T19:50:19.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/qaisermehdi/rtx-3090-rtx-5090-a100-and-h100-on-vllm-the-cheapest-gpu-flips-with-your-answer-time-3pij"
  },
  "original_language": "en",
  "account": "The article compares the cost-effectiveness of four GPUs - RTX 3090, RTX 5090, A100 80GB PCIe, and H100 80GB SXM - when running the Qwen2.5-7B-Instruct model on vLLM. The cheapest option varies depending on the desired answer time.\n\nFor answers under 2 seconds, only the H100 can meet this requirement at $0.441 per 1 million output tokens. When allowing up to 3 seconds for answers, the H100 is $0.162, the RTX 5090 is $0.382, and the A100 is $0.803. In this scenario, the H100 is about 2.4 times cheaper than the RTX 5090.\n\nAs answer time increases to 4 seconds, the RTX 5090 becomes the cheapest at $0.120, followed by the H100 at $0.162, and the A100 at $0.243. At this point, the RTX 5090 is about 1.4 times cheaper than the H100.\n\nWith an answer time constraint of 8 seconds, the RTX 5090 is once again the most cost-effective at $0.050, more than double the price of the H100 at $0.099 and around 2 times cheaper than the A100 at $0.106.\n\nInterestingly, when running all four GPUs with the maximum number of requests (64) in flight, the H100 remains the most affordable at $0.162, even though the RTX 5090 is significantly slower at 5.4 seconds (p50). Conversely, when using the maximum number of requests (64) and a fixed answer time, the H100 is the most expensive at $0.162, while the RTX 5090 is $0.082 and the A100 is $0.106.\n\nThe article emphasizes that the choice of GPU depends on the balance between cost and answer time, as the most cost-effective option changes based on the specific requirements. Additionally, the cost per token can fluctuate depending on the cloud provider's pricing, making it essential to consider different pricing structures when evaluating GPU performance.",
  "summary": "I measured four GPUs serving the same 7B model on vLLM. Which one is cheapest per token has no single answer. It flips with how fast your answers must be. The setup Same model (Qwen2.5-7B-Instruct), same vLLM (0.27.1, default flags), same load on every card: prompts of about 512 tokens, 256-token answers, 1 to 256 requests at once, 60 seconds per level. Four rented cards on vast.ai: RTX 3090 at…",
  "key_points": [
    "H100 is cheapest for answers under 2 seconds at $0.441 per 1M tokens",
    "RTX 5090 becomes most cost-effective for answers up to 8 seconds at $0.050",
    "H100 remains most affordable with max requests (64) at $0.162"
  ],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}