Skynet Smiles: AMD Unveils Monster Instinct MI300 Hybrid AI Accelerator

At AMD’s CES keynote this week, Chairman and CEO Dr. Lisa Su seemingly saved the best for last. At the very end, she held up an absolute monster of an APU, describing it as the first data center chip that combines a CPU, GPU, and memory on a single package. It’s called the AMD Instinct MI300, and it packs a whopping 146 billion transistors–46 billion more than found in Intel’s sophisticated data center chip, Ponte Vechio. AMD says the MI300 will be coming in the latter half of 2023, so for now it just wanted to show it off a bit.

The chip is absolutely massive; when she held it aloft, it was the size of her palm. The MI300 sports 24 Zen 4 cores along with its CDNA 3 GPU architecture and 128GB of high-bandwidth memory (HBM3). AMD didn’t mention how many CUs were being used on the GPU side. The CPU and GPU chiplets are 3D-stacked on top of a base die that connects them to the eight stacks of HBM. The design uses nine 5nm chiplets 3D-stacked onto four 6nm chiplets. It’s unclear how the CPU and GPU chips are divided amongst the chiplets and AMD hasn’t revealed it yet. The four 6nm base dies are active interposers that handle communications and I/O between the chiplets and the memory stacks.

AMD says this integrated approach allows for a significant boost in performance as well as efficiency. As an example, Dr. Su says it destroys the company’s previous generation product, the Instinct MI250X. Overall it offers 8X the AI performance along with a 5X uplift in performance-per-watt. In addition, she said the MI300 can reduce the training time for AI models from months to just weeks. This could theoretically save companies millions of dollars in electricity costs. The integration of the CPU and GPU also allows them to work on the same data simultaneously. Additionally, since the HBM3 is integrated, it might be able to function without separate DRAM modules.

Although the company’s 4th Gen Epyc “Genoa” chips are widely available and high-volume, that probably won’t be the case with the MI300. AMD didn’t divulge pricing because it’s not releasing it any time soon, but it will not be cheap. It’s currently slated to power the 2-exaflop El Capitan supercomputer at Lawrence National Laboratory. This computer will be powered on by the end of 2023 and will offers double the performance of the world’s current fastest supercomputer, Frontier, which is also AMD-powered. Frontier was the first supercomputer to break the exaflop barrier. Now it sounds like El Capitan could be the first to 2 exaflops.

Now read: