Share GPU clusters across teams with isolation and fairness using Amazon SageMaker HyperPod
A reference architecture for securely sharing one Amazon SageMaker HyperPod EKS cluster across multiple teams, using AWS IAM Identity Center for authentication, per-team SageMaker Domains and Kubernetes namespaces for isolation, HyperPod Task Governance for fairness, and namespace-level cost allocation for chargeback.
Organizations with multiple teams working on generative AI tasks often require shared access to expensive GPU resources while ensuring isolation and fairness. A poorly designed multi-tenant architecture can lead to uncontrolled resource usage, poor isolation between teams, difficulty attributing costs, and hinder innovation. Amazon SageMaker HyperPod is an AI service designed to simplify the management of large compute clusters for generative AI workloads.
It orchestrates clusters using Amazon Elastic Kubernetes Service (Amazon EKS) or Slurm, enabling scalable distributed training, interactive development, and model inference. The post outlines a reference architecture for creating a multi-tenant environment on Amazon SageMaker HyperPod with EKS, utilizing AWS IAM Identity Center for centralized authentication, team-specific SageMaker AI domains, Kubernetes namespaces for workload isolation, HyperPod Task Governance for fair resource allocation, and namespace-level cost tracking for spending visibility and chargeback.
The architecture diagram illustrates how users from different teams authenticate through AWS IAM Identity Center and interact with their respective isolated namespaces on the HyperPod EKS cluster.
Rewritten by urgent.news; may contain errors — check the original.
Source: aws.amazon.com