Mixture-of-Experts kernel engineering on Amazon SageMaker AI
AWS engineers share kernel optimizations for fused Mixture-of-Experts (MoE) models on Amazon SageMaker AI: Swap-AB for WGMMA utilization, Triton backend routing for FP8 quantization, and eliminating Triton JIT recompilation. Together they deliver up to 10% higher throughput and up to 9% faster Time to First Token on H200 GPUs.
We haven't written up this one. AWS Machine Learning has the full story — the link below goes straight to it.
This story
This is one outlet's version. Read the fullest account.
- Deploy decision models with llama.cpp on Amazon SageMaker AI aws.amazon.com
