{
  "id": 3995008,
  "title": "Spreading the load: How Salesforce met Multi-AZ HA with SageMaker Inference Components",
  "url": "https://urgent.news/2026/08/28/spreading-the-load-how-salesforce-met-multi-az-ha-with-sagemaker",
  "topic": "ai",
  "section": "AI",
  "published": "2026-08-28T16:20:40.000Z",
  "source": {
    "name": "AWS Machine Learning",
    "slug": "aws-machine-learning",
    "url": "https://aws.amazon.com/blogs/machine-learning/spreading-the-load-how-salesforce-met-multi-az-ha-with-sagemaker-inference-components/"
  },
  "original_language": "en",
  "account": "When Salesforce sought to make its Agentforce AI foundation highly available (HA) across multiple Availability Zones (AZs), they encountered an issue with Amazon SageMaker AI Inference Components (ICs). While ICs reduced GPU costs by sharing them for multiple models, their default placement did not guarantee the Multi-AZ resilience Salesforce required. To address this, Salesforce leveraged the new IC Placement capability, surfaced through the SchedulingConfig parameter in the CreateInferenceComponent API.\n\nThe challenge was to prevent single points of failure in IC deployments. By default, SageMaker's placement algorithm optimizes deployments independently, potentially leading to uneven model distribution across AZs. This could result in instance-level or AZ-level failures, violating Salesforce's 2-AZ compliance bar for production models.\n\nAWS introduced the SchedulingConfig parameter to give customers control over IC copy placement across instances and AZs. Two key sub-parameters were AvailabilityZoneBalance and PlacementStrategy. AvailabilityZoneBalance ensured cross-AZ distribution, while PlacementStrategy controlled the distribution within each AZ. SPREAD strategy spread copies across as many instances as possible for fault isolation, while BINPACK packed copies onto fewer instances for utilization efficiency.\n\nA code example demonstrated deploying an IC with Multi-AZ HA placement. The CreateInferenceComponent call specified SPREAD placement strategy and PERMISSIVE AvailabilityZoneBalance, with CopyCount set to 4. This deployment placed 2 copies in each AZ, ensuring high availability.\n\nWhen scaling operations occurred, SageMaker maintained AZ balance through the configured SchedulingConfig parameters. New copies were placed to maintain even AZ distribution during scale-out, and copies were symmetrically removed during scale-in. It was crucial to avoid setting CopyCount to 1 for HA-critical models, as this would break 2-AZ compliance.\n\nTo further streamline maintenance, Salesforce configured their endpoint with a CONSOLIDATION scale-in policy. This periodic sweep consolidated IC copies, releasing idle instances while preserving AZ balance. The three pillars of the placement algorithm—balanced final distribution, availability-aware distribution, and endpoint update persistence—addressed Salesforce's HA requirements effectively.",
  "summary": "Learn how Salesforce used Amazon SageMaker AI Inference Component placement (the SchedulingConfig parameter) to distribute model copies across multiple Availability Zones, meeting their Multi-AZ high availability compliance requirements without sacrificing the cost efficiency of multi-model co-hosting.",
  "key_points": [],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}