A Reference Cloud Architecture for AI/ML Workloads

There is no single "best" cloud for AI/ML, but there is a best shape — small, isolated landing zones with explicit guardrails, an inference plane separated from training, and aggressive cost telemetry.

Separate planes Keep training (spiky, expensive) and inference (steady, latency-sensitive) on different account boundaries with different scaling rules.

Cost telemetry from day one You cannot optimize what you cannot see. Tag every workload, every model, every request.

The cheapest GPU is the one you turned off two minutes ago.