{
  "id": 20460722,
  "title": "CoreWeave connects custom training loops to model serving",
  "url": "https://urgent.news/2026/11/17/coreweave-connects-custom-training-loops-to-model-serving",
  "topic": "tech",
  "section": "Tech",
  "published": "2026-11-17T16:30:00.000Z",
  "source": {
    "name": "The New Stack",
    "slug": "the-new-stack",
    "url": "https://thenewstack.io/coreweave-forge-training-loops/"
  },
  "original_language": "en",
  "account": "Production traffic often reveals failures that cannot be fully understood through benchmarks or pre-deployment evaluations. This signal points to specific edge cases that users encounter, which must be addressed when refining a model. Determining whether the issue lies within the model itself or in areas like retrieval, context, prompts, tools, or serving is crucial. Changing the model weights alone won't resolve problems elsewhere in the system. However, when the model is the appropriate place for improvement, production evidence can enhance both quality and cost-effectiveness. A smaller model may replicate the behavior of a more expensive model; better examples can increase consistency; and an outcome-based reward aligns with the task's objectives. Refining an existing capable model may also prevent the need for a model migration to a new generation. Various techniques can be employed based on the failure mode. Model distillation is suitable when a powerful model already produces the desired behavior, and a smaller, cheaper model is needed to replicate it. Supervised fine-tuning (SFT) is appropriate when high-quality examples can teach consistent behavior. Reinforcement learning (RL) is necessary when outcomes can be reliably evaluated, but the ideal response cannot be specified in advance. These methods can be applied multiple times throughout a model's lifecycle, even if it requires additional setup. Production traces must be transformed into training data through production traces becoming training data. Post-training processes involve compute and orchestration, and the resulting model must undergo evaluation before being returned to inference. This process completes the loop. In reinforcement learning (RL), inference generates rollouts during training, and new weights must be returned to serving continuously throughout the process. CoreWeave has introduced several workflows that connect in CoreWeave Forge, including CoreWeave Model Distillation, CoreWeave RL Rollouts on CoreWeave Inference, and a programmable training API. The programmable API is currently in limited preview, with an official launch planned for later in 2026. To keep the data current, CoreWeave Model Distillation offers a managed workflow. CoreWeave Agent Lens captures inference traffic as the application runs, turning it into training data, relabeling weak outputs, testing base models and hyperparameters, deploying candidates to Serverless Inference, and comparing them with the production model. If the model already possesses the required capabilities for the task, fresh production data can enhance its fit without altering the serving profile already in place. This workflow was used to revisit a model previously deployed. In 2024, Method utilized OpenPipe to train Llama 3.1 8B to navigate bank interactive voice response (IVR) systems, utilizing GPT-4o outputs. In August 2026, fresh production data was incorporated, relabeled with GPT-5.6 Sol, and another version was trained. The candidate won 57.5% of head-to-head evaluations against the deployed version. The base model remained unchanged, but the fresh production data and Model Distillation in Forge enabled an earlier iteration. The 57.5% result represents the percentage of head-to-head evaluations the new version won.",
  "summary": "Production traffic shows failures that benchmarks or pre-deployment evaluations can’t fully reveal. It tells you which edge cases matter and The post CoreWeave connects custom training loops to model serving appeared first on The New Stack .",
  "key_points": [],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}