{
  "id": 14106347,
  "title": "Training Transformers on 'Inference-Only' NPUs: How VKORE Reached 0.9999 Parity",
  "url": "https://urgent.news/2026/10/13/training-transformers-on-inference-only-npus-how-vkore-reached-0-9999",
  "topic": "ai",
  "section": "AI",
  "published": "2026-10-13T03:28:37.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/ai_openfree_b23025ef075cf/training-transformers-on-inference-only-npus-how-vkore-reached-09999-parity-m1"
  },
  "original_language": "en",
  "account": "VKORE, a software acceleration layer developed by VIDRAFT, enables inference-only NPUs to train transformer models while maintaining numerical parity with standard PyTorch. This innovation addresses the trend towards maximizing intelligence from existing silicon by allowing NPUs to handle both inference and training tasks, thereby reducing the need for separate GPU build-outs for fine-tuning. On a Korean NPU server with eight cards, VKORE increased single-user inference speed from 14.1 to 53.9 tokens per second (3.8x) and server throughput from 214 to 881 tokens per second (4.1x). Crucially, this performance boost was achieved without altering the underlying hardware, utilizing only the VKORE software layer. Surprisingly, VKORE demonstrated the capability to run real transformer LoRA training at 0.9999 numerical parity with PyTorch, achieving this at approximately 170 times the speed of the server's CPU. This breakthrough represents a significant shift in how data centers can optimize their resources, as NPUs, known for their power efficiency in inference tasks, can now also contribute to model training, thereby extending their utility and reducing overall operational costs.",
  "summary": "TL;DR: NPUs are sold as inference-only accelerators. VIDRAFT built VKORE, a software acceleration layer that leaves the chip untouched and (1) raises single-user inference 3.8x and server throughput 4.1x on the same NPU server, and (2) runs real transformer LoRA training on that NPU at 0.9999 numerical parity with standard PyTorch, about 170x faster than the same server's CPU. Measured on a…",
  "key_points": [
    "VKORE software accelerates inference-only NPUs for transformer model training.",
    "VKORE achieves 0.9999 numerical parity with PyTorch for LoRA training.",
    "VKORE boosts inference speed 3.8x and server throughput 4.1x on eight-card NPU server."
  ],
  "editors_take": "Repurposing inference-only NPUs for training tasks allows data centers to maximize existing silicon utility and reduce the necessity for expensive, separate GPU builds for model fine-tuning.",
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}