Urgent.News

What's breaking now, across hundreds of outlets.

AI

AI Gateway Performance Comparison (2026): Python vs Go vs Rust vs Envoy, and Why the Benchmarks Disagree

Published charts that crown a fastest AI gateway usually claim a lead by a factor of forty. They disagree with each other because they do not measure the same interval. One number is time spent inside the process, with JSON and the network left out. Another is added latency at the client, with logging and spend tracking off. A third turns production features on, uses a slower mock, or runs a…

A comprehensive performance comparison of AI gateways, involving Python, Go, Rust, and Envoy, has been conducted. The results indicate that the choice of language significantly impacts the performance, with the benchmark results showing varying degrees of disparity. The methodology employed in the study involved measuring key aspects such as decoding, encoding, and replying, which are primarily CPU-intensive tasks.

Other factors like authentication, rate limiting, and logging were found to contribute to slower deployments due to database or Redis interactions. The architecture of the system also played a crucial role in determining the overall performance, with synchronous request logging being a notable factor that could adversely affect latency.

Memory usage per replica was another critical aspect, with idling processes in the tens of megabytes versus gigabytes leading to differences in fleet cost. The study also emphasized the implications of rollbacks, with a single static binary being more efficient than a stack with schema migrations, especially during incidents. A reproducible method was provided for readers to conduct their own performance tests, including a mock server, k6 script for streaming and concurrency variation, and a baseline and gateway test configuration. The publication of detailed configurations was encouraged to ensure the reliability of the results.

Rewritten by urgent.news; may contain errors — check the original.

Read the original at dev.to →

Report a problem with this story

More in AI

[國際] 亞馬遜抗衡輝達 傳砸2470億收購AI新銳Decart

亞馬遜(Amazon)據報將以約75億美元(約新台幣2470億)收購AI新創公司Decart,此舉有助亞馬遜雲端部門與輝達競爭。這筆交易最快下週初就可達成,但不排除破局可能。 金融時報披露,亞馬遜即將達成以75億美元收購新創公司Decart的交易,凸顯此一科技巨擘強化人工智慧(AI)能力的企圖心。…

Meta ads is now available in AWS Quick

AWS Quick now has a Meta ads AI connector. Meta advertisers can use natural language in Quick to create, manage, and analyze campaigns while staying connected to the Meta ad management network, and…

  • Meta ads integrated into AWS Quick with AI-powered connector
  • Enables natural language creation, management, and analysis of campaigns
  • Streamlines campaign planning to execution, reducing manual work

More from Sunday 22 November →