AMD has announced its brand-new Instinct MI350P PCIe GPU accelerator. This is their first PCIe design in years, and it’s built for demanding AI tasks.
With the Instinct MI350P PCIe GPU, AMD gives businesses a way to boost their AI computing power without needing to buy expensive new equipment. The MI350P’s PCIe design means it’s an easy-to-install solution that delivers a lot of performance in a standard dual-slot, server-focused design.
Here are some of its key features:
- It natively supports lower-precision MXFP6 and MXFP4, which allow for very high data processing.
- It speeds up calculations by supporting “sparsity” for most common 8-bit and 16-bit precisions.
- It offers an estimated 2,299 teraflops (TFLOPS) and up to 4,600 peak TFLOPS at MXFP4, making it the most powerful enterprise PCIe card available right now.
- It includes an estimated 144GB of high-bandwidth memory 3e (HBM3E), running at up to 4TB/s.
- It uses an open ecosystem with affordable development tools, which makes it easier to set up and helps reduce ongoing costs.
Looking at the technical details, the AMD Instinct MI350P uses the CDNA 4 architecture. It’s built using the same TSMC 3nm process technology with a 4 XCD setup, which is half the amount found in the MI350X. It also has a single I/O die, made with TSMC’s 6nm FinFET process. The chip contains 128 compute units, which translates to 8,192 Stream processors and 512 Matrix cores. These cores run at a peak clock speed of 2200 MHz. The entire chip has 73 billion transistors.
For memory, the Instinct MI350P includes 128 MB of LLC in the form of Infinity Cache within the GPU. It also has 144 GB of fast HBM3E memory, which works across a 4096-bit wide pathway, providing 4 TB/s of memory speed. For comparison, the MI350X has 288 GB of HBM3E memory over an 8192-bit interface. The PCIe card is 10.5 inches (267mm) long and has a passively cooled design, which is perfect for servers. AMD also uses a 16-pin connector to meet the card’s 600W power draw. It can also be set to use as little as 450W.
In terms of raw power, the AMD Instinct MI350P offers:
- 4.6 PFLOPs MXFP4
- 4.6 PFLOPs MXFP6
- 2.3 PFLOPs MXFP8
- 2.3 PFLOPs FP16 (Sparsity)
- 1.15 PFLOPs FP16
- 72 TFLOPs FP16
- 72 TFLOPs FP32
- 36 TFLOPs FP64
- 2.3 POPs INT8
- 4.6 POPs INT8 (Sparsity)
- 1.15 PFLOPs BFloat16
- 2.30 PFLOPs BFloat16 (Sparsity)
As you can see, the AMD Instinct MI350 series, including the MI350P, greatly speeds up various AI calculation formats used in businesses, such as MXFP6 and MXFP4.
The MI350P will go up against NVIDIA’s H200 NVL, which is NVIDIA’s last PCIe-based GPU accelerator. That card has 141 GB of HBM3E memory and uses the Hopper H200 GPU. NVIDIA has also released the RTX PRO 6000 Blackwell server edition, but that uses the standard GB202 chip instead of the more powerful GB200, which is the true server option. The RTX PRO 6000 Blackwell has 96 GB of GDDR7 memory. H200 NVL GPUs typically cost around $30,000 to $40,000 US.
