Cluster: Amazon EKS, Karpenter, Bottlerocket
GPUs: EC2 G and P instances (A10G, L4, L40S, RTX PRO 6000 Blackwell, H100, B200)
Capacity: Spot and On-Demand tiers, SOCI parallel image pull, S3 model loading
Networking and Observability: EFA, NCCL, Prometheus, Grafana, DCGM