{
  "id": 18358848,
  "title": "Together AI let coding agents rewrite its GPU code. Then it kept humans on the deploy button.",
  "url": "https://urgent.news/2026/11/05/together-ai-let-coding-agents-rewrite-its-gpu-code-then-it-kept",
  "topic": "ai",
  "section": "AI",
  "published": "2026-11-05T18:56:57.000Z",
  "source": {
    "name": "The New Stack",
    "slug": "the-new-stack",
    "url": "https://thenewstack.io/together-compiler-gpu-optimization/"
  },
  "original_language": "en",
  "account": "Together AI is employing AI coding agents to optimize GPU code for its inference infrastructure, streamlining performance tuning that once required engineers to dedicate days or weeks of manual labor. The company unveiled Together Compiler on Wednesday, a system capable of generating, testing, and enhancing GPU kernels without direct guidance from engineers. Together AI asserts that this autonomous optimization loop can deliver gains overnight. However, they do not deploy changes on their own. The company explains that while the optimization loop is autonomous, the final evaluations and deployment process must be initiated by a human, who reviews the changes prior to production deployment. The rapid release of new AI models necessitates continuous optimization of inference infrastructure, as each model has distinct requirements, and code optimized for one workload may not perform optimally on another. Together AI reports that 80% of OpenRouter traffic now originates from models released within the past three months, leaving engineers constantly adjusting code to keep pace. To expedite testing, Together Compiler maintains the inference engine running and loads new code, drastically reducing initialization time from approximately 467 seconds to just nine seconds. With this approach, Together AI claims the system can execute 252 times more optimization experiments than existing agent-based systems. However, this figure represents the number of experiments Together Compiler can execute compared to Apple’s system, not the speed of AI model performance. The company also discloses inference throughput improvements of up to 2.6x compared to published benchmark configurations. These gains vary depending on the model and hardware, and have not undergone independent verification. The enhanced testing capabilities allow Together Compiler to uncover improvements overnight that could previously take engineers weeks of manual tuning. Faster inference does not automatically imply reduced costs. When asked if the performance gains would result in customer savings or primarily enhance the company’s internal efficiency, Together AI states that Together Compiler enhances Together AI’s infrastructure efficiency. For the same monetary investment, customers will experience improved performance. The claim that customers will get better performance for the same cost, rather than a price reduction. Together AI asserts that its compiler is currently handling trillions of tokens daily, highlighting the growing influence of coding agents in GPU performance optimization, with engineers overseeing work that agents can now accomplish overnight.",
  "summary": "Together AI is putting AI coding agents to work on the GPU code that runs its inference infrastructure, automating performance The post Together AI let coding agents rewrite its GPU code. Then it kept humans on the deploy button. appeared first on The New Stack .",
  "key_points": [],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}