At this time, we’re unveiling the following Fairwater web site of Azure AI datacenters in Atlanta, Georgia. This purpose-built datacenter is linked to our first Fairwater web site in Wisconsin, prior generations of AI supercomputers and the broader Azure international datacenter footprint to create the world’s first planet-scale AI superfactory. By packing computing energy extra densely than ever earlier than, every Fairwater web site is constructed to effectively meet unprecedented demand for AI compute, push the frontiers of mannequin intelligence and empower each particular person and group on the planet to attain extra.
To fulfill this demand, we now have reinvented how we design AI datacenters and the techniques we run inside them. Fairwater is a departure from the standard cloud datacenter mannequin and makes use of a single flat community that may combine a whole lot of hundreds of the most recent NVIDIA GB200 and GB300 GPUs into an enormous supercomputer. These improvements are a product of many years of expertise designing datacenters and networks, in addition to learnings from supporting among the largest AI coaching jobs on the planet.
Whereas the Fairwater datacenter design is effectively suited to coaching the following technology of frontier fashions, it is usually constructed with fungibility in thoughts. Coaching has developed from a single monolithic job into a variety of workloads with totally different necessities (reminiscent of pre-training, fine-tuning, reinforcement studying and artificial knowledge technology). Microsoft has deployed a devoted AI WAN spine to combine every Fairwater web site right into a broader elastic system that allows dynamic allocation of numerous AI workloads and maximizes GPU utilization of the mixed system.
Under, we stroll by means of among the thrilling technical improvements that help Fairwater, from the best way we construct datacenters to the networking inside and throughout the websites.
Most density of compute
Trendy AI infrastructure is more and more constrained by the legal guidelines of physics. The pace of sunshine is now a key bottleneck in our means to tightly combine accelerators, compute and storage with performant latency. Fairwater is designed to maximise the density of compute to reduce latency inside and throughout racks and maximize system efficiency.
One of many key levers for driving density is enhancing cooling at scale. AI servers within the Fairwater datacenters are linked to a facility-wide cooling system designed for longevity, with a closed-loop strategy that reuses the liquid constantly after the preliminary fill with no evaporation. The water used within the preliminary fill is equal to what 20 houses devour in a yr and is simply changed if water chemistry signifies it’s wanted (it’s designed for 6-plus years), making it extraordinarily environment friendly and sustainable.
Liquid-based cooling additionally gives a lot greater warmth switch, enabling us to maximise rack and row-level energy (~140kW per rack, 1,360 kW per row) to pack compute as densely as attainable contained in the datacenter. State-of-the-art cooling additionally helps us maximize utilization of this dense compute in steady-state operations, enabling giant coaching jobs to run performantly at excessive scale. After biking by means of a system of chilly plate paths throughout the GPU fleet, warmth is dissipated by one of many largest chiller crops on the planet.
One other approach we’re driving compute density is with a two-story datacenter constructing design. Many AI workloads are very delicate to latency, which suggests cable run lengths can meaningfully impression cluster efficiency. Each GPU in Fairwater is linked to each different GPU, so the two-story datacenter constructing strategy permits for placement of racks in three dimensions to reduce cable lengths, which in flip improves latency, bandwidth, reliability and value.
Excessive-availability, low-cost energy
We’re pushing the envelope in serving this compute with cost-efficient, dependable energy. The Atlanta web site was chosen with resilient utility energy in thoughts and is able to attaining 4×9 availability at 3×9 value. By securing extremely accessible grid energy, we will additionally forgo conventional resiliency approaches for the GPU fleet (reminiscent of on-site technology, UPS techniques and dual-corded distribution), driving value financial savings for purchasers and sooner time-to-market for Microsoft.
We have now additionally labored with our business companions to codevelop power-management options to mitigate energy oscillations created by giant scale jobs, a rising problem in sustaining grid stability as AI demand scales. This features a software-driven answer that introduces supplementary workloads during times of decreased exercise, a hardware-driven answer the place the GPUs implement their very own energy thresholds and an on-site vitality storage answer to additional masks energy fluctuations with out using extra energy.
Slicing-edge accelerators and networking techniques
Fairwater’s world-class datacenter design is powered by purpose-built servers, cutting-edge AI accelerators and novel networking techniques. Every Fairwater datacenter runs a single, coherent cluster of interconnected NVIDIA Blackwell GPUs, with a sophisticated community structure that may scale reliably past conventional Clos community limits with current-gen switches (a whole lot of hundreds of GPUs on a single flat community). This required innovation throughout scale-up networking, scale-out networking and networking protocol.
By way of scale-up, every rack of AI accelerators homes as much as 72 NVIDIA Blackwell GPUs, linked through NVLink for ultra-low-latency communication throughout the rack. Blackwell accelerators present the best compute density accessible right now, with help for low-precision quantity codecs like FP4 to extend whole FLOPS and allow environment friendly reminiscence use. Every rack gives 1.8 TB of GPU-to-GPU bandwidth, with over 14 TB of pooled reminiscence accessible to every GPU.
These racks then use scale-out networking to create pods and clusters that allow all GPUs to perform as a single supercomputer with minimal hop counts. We obtain this with a two-tier, ethernet-based backend community that helps large cluster sizes with 800 Gbps GPU-to-GPU connectivity. Counting on a broad ethernet ecosystem and SONiC (Software program for Open Community within the Cloud – which is our personal working system for our community switches) additionally helps us keep away from vendor lock-in and handle value, as we will use commodity {hardware} as an alternative of proprietary options.
Enhancements throughout packet trimming, packet spray and high-frequency telemetry are core elements of our optimized AI community. We’re additionally working to allow deeper management and optimization of community routes. Collectively, these applied sciences ship superior congestion management, speedy detection and retransmission and agile load balancing, making certain ultra-reliable, low-latency efficiency for contemporary AI workloads.
Planet scale
Even with these improvements, compute calls for for big coaching jobs (now measured in trillions of parameters) are rapidly outpacing the ability and house constraints of a single facility. To serve these wants, we now have constructed a devoted AI WAN optical community to increase Fairwater’s scale-up and scale-out networks. Leveraging our scale and many years of hyperscale experience, we delivered over 120,000 new fiber miles throughout the US final yr — increasing AI community attain and reliability nationwide.
With this high-performance, high-resiliency spine, we will immediately join totally different generations of supercomputers into an AI superfactory that exceeds the capabilities of a single web site throughout geographically numerous places. This empowers AI builders to faucet our broader community of Azure AI datacenters, segmenting visitors primarily based on their wants throughout scale-up and scale-out networks inside a web site, in addition to throughout websites through the continent spanning AI WAN.
It is a significant departure from the previous, the place all visitors needed to experience the scale-out community whatever the necessities of the workload. Not solely does it present clients with fit-for-purpose networking at a extra granular stage, it additionally helps create fungibility to maximise the flexibleness and utilization of our infrastructure.
Placing all of it collectively
The brand new Fairwater web site in Atlanta represents the following leap within the Azure AI infrastructure and displays our expertise working the biggest AI coaching jobs on the planet. It combines breakthrough improvements in compute density, sustainability and networking techniques to effectively serve the large demand for computational energy we’re seeing. It additionally integrates deeply with different AI datacenters and the broader Azure platform to kind the world’s first AI superfactory. Collectively, these improvements present a versatile, fit-for-purpose infrastructure that may serve the total spectrum of contemporary AI workloads and empower each particular person and group on the planet to attain extra. For our clients, this implies simpler integration of AI into each workflow and the flexibility to create modern AI options that had been beforehand unattainable.
Discover out extra about how Microsoft Azure may also help you combine AI to streamline and strengthen improvement lifecycles right here.
Scott Guthrie is liable for hyperscale cloud computing options and companies together with Azure, Microsoft’s cloud computing platform, generative AI options, knowledge platforms and data and cybersecurity. These platforms and companies assist organizations worldwide clear up pressing challenges and drive long-term transformation.
Editor’s notice: An replace was made to extra clearly clarify how we optimize our community.
