Right this moment, we’re asserting the final availability of Amazon Elastic Compute Cloud (Amazon EC2) G7e cases that ship cost-effective efficiency for generative AI inference workloads and the best efficiency for graphics workloads.
G7e cases are accelerated by the NVIDIA RTX PRO 6000 Blackwell Server Version GPUs and are properly fitted to a broad vary of GPU-enabled workloads together with spatial computing and scientific computing workloads. G7e cases ship as much as 2.3 occasions inference efficiency in comparison with G6e cases.
Enhancements made in comparison with predecessors:
- NVIDIA RTX PRO 6000 Blackwell GPUs — NVIDIA RTX PRO 6000 Blackwell Server Version GPUs supply two occasions the GPU reminiscence and 1.85 occasions the GPU reminiscence bandwidth in comparison with G6e cases. Through the use of the upper GPU reminiscence supplied by G7e cases, you may run medium-sized fashions of as much as 70B parameters with FP8 precision on a single GPU.
- NVIDIA GPUDirect P2P — For fashions which might be too giant to suit into the reminiscence of a single GPU, you may break up the mannequin or computations throughout a number of GPUs. G7e cases scale back the latency of your multi-GPU workloads with help for NVIDIA GPUDirect P2P, which permits direct communication between GPUs over PCIe interconnect. These cases supply the bottom peer to look latency for GPUs on the identical PCIe swap. Moreover, G7e cases supply as much as 4 occasions the inter-GPU bandwidth in comparison with L40s GPUs featured in G6e cases, boosting the efficiency of multi-GPU workloads. These enhancements imply you may run inference for bigger fashions throughout a number of GPUs providing as much as 768 GB of GPU reminiscence in a single node.
- Networking — G7e cases supply 4 occasions the networking bandwidth in comparison with G6e cases, which implies you should utilize the occasion for small-scale multi-node workloads. Moreover, multi-GPU G7e cases help NVIDIA GPUDirect Distant Direct Reminiscence Entry (RDMA) with Elastic Material Adapter (EFA), which reduces the latency of distant GPU-to-GPU communication for multi-node workloads. These occasion sizes additionally help NVIDIA GPUDirectStorage with Amazon FSx for Lustre, which will increase throughput by as much as 1.2 Tbps to the cases in comparison with G6e cases, which implies you may shortly load your fashions.
EC2 G7e specs
G7e cases characteristic as much as 8 NVIDIA RTX PRO 6000 Blackwell Server Version GPUs with as much as 768 GB of whole GPU reminiscence (96 GB of reminiscence per GPU) and Intel Emerald Rapids processors. In addition they help as much as 192 vCPUs, as much as 1,600 Gbps of community bandwidth, as much as 2,048 GiB of system reminiscence, and as much as 15.2 TB of native NVMe SSD storage.
Listed below are the specs:
| Occasion title |
GPUs | GPU reminiscence (GB) | vCPUs | Reminiscence (GiB) | Storage (TB) | EBS bandwidth (Gbps) | Community bandwidth (Gbps) |
| g7e.2xlarge | 1 | 96 | 8 | 64 | 1.9 x 1 | As much as 5 | 50 |
| g7e.4xlarge | 1 | 96 | 16 | 128 | 1.9 x 1 | 8 | 50 |
| g7e.8xlarge | 1 | 96 | 32 | 256 | 1.9 x 1 | 16 | 100 |
| g7e.12xlarge | 2 | 192 | 48 | 512 | 3.8 x 1 | 25 | 400 |
| g7e.24xlarge | 4 | 384 | 96 | 1024 | 3.8 x 2 | 50 | 800 |
| g7e.48xlarge | 8 | 768 | 192 | 2048 | 3.8 x 4 | 100 | 1600 |
To get began with G7e cases, you should utilize the AWS Deep Studying AMIs (DLAMI) on your machine studying (ML) workloads. To run cases, you should utilize AWS Administration Console, AWS Command Line Interface (AWS CLI) or AWS SDKs. For a managed expertise, you should utilize G7e cases with Amazon Elastic Container Service (Amazon ECS), Amazon Elastic Kubernetes Service (Amazon EKS). Help for Amazon SageMaker AI can be coming quickly.
Now obtainable
Amazon EC2 G7e cases can be found at present within the US East (N. Virginia) and US East (Ohio) AWS Areas. For Regional availability and a future roadmap, search the occasion kind within the CloudFormation sources tab of AWS Capabilities by Area.
The cases may be bought as On-Demand Cases, Financial savings Plan, and Spot Cases. G7e cases are additionally obtainable in Devoted Cases and Devoted Hosts. To study extra, go to the Amazon EC2 Pricing web page.
Give G7e cases a strive within the Amazon EC2 console. To study extra, go to the Amazon EC2 G7e cases web page and ship suggestions to AWS re:Put up for EC2 or by means of your regular AWS Help contacts.
— Channy
