{
  "id": 20655213,
  "title": "How Salesforce is advancing global-scale model serving with cross-Region inference routing in Amazon SageMaker AI",
  "url": "https://urgent.news/2026/11/18/how-salesforce-is-advancing-global-scale-model-serving-with-cross",
  "topic": "ai",
  "section": "AI",
  "published": "2026-11-18T20:13:18.000Z",
  "source": {
    "name": "AWS Machine Learning",
    "slug": "aws-machine-learning",
    "url": "https://aws.amazon.com/blogs/machine-learning/how-salesforce-is-advancing-global-scale-model-serving-with-cross-region-inference-routing-in-amazon-sagemaker-ai/"
  },
  "original_language": "en",
  "account": "Salesforce, a global technology company, has developed a solution to efficiently serve models across 30+ global regions and hundreds of AWS accounts. This is achieved through cross-Region inference routing in Amazon SageMaker AI, a feature that lets applications route requests to the most suitable endpoint, regardless of its location.\n\nThe challenge Salesforce faced was managing capacity effectively, especially during overlapping peak periods for different customers. Hosting models in separate regions could lead to capacity fragmentation, where some regions have idle instances while others lack sufficient capacity. Before implementing cross-Region inference routing, Salesforce used an internal gateway to send traffic to remote regions when local endpoints couldn't handle the load. However, this setup had limitations, including the lack of capacity signals from remote endpoints and the need for traffic to traverse public AWS networking, increasing the risk of cascading failures.\n\nTo address these issues, Salesforce introduced the cross-Region inference router, a regional SageMaker AI resource within their account. Applications now invoke this router instead of individual endpoints, passing a routing target name along with the request. The router, which is a regional SageMaker AI resource, prioritizes local endpoints whenever possible. If a destination fails, it automatically retries the request on another available destination. This approach allows Salesforce to securely add capacity while maintaining strict data residency commitments.\n\nThe router operates based on recent request outcomes, adjusting traffic to healthy destinations. It collects signals such as scaling progress and instance launch failures to determine when to shift traffic. Model latency, on the other hand, doesn't influence traffic routing, as it often reflects long prompts or outputs rather than endpoint health. Salesforce integrated the router into its Internal Gateway, maintaining a per-request egress path where direct InvokeEndpoint calls were previously made. This integration ensures that the gateway's data residency check is the only routing decision, effectively routing requests to the local endpoint when necessary or to the router for cross-Region requests.",
  "summary": "Learn how Salesforce uses the cross-Region inference router in Amazon SageMaker AI to turn fragmented Regional capacity into reliable model serving. Salesforce adds capacity securely across Regions and accounts while keeping its data residency commitments, and sees per request where each call was served and why.",
  "key_points": [],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}