Samsung Electronics has introduced a next-generation architecture that vertically stacks memory on top of AI chips to push past the performance and power efficiency limits of artificial intelligence semiconductors.

Kim Kyoung-ryun, Vice President of Samsung Electronics' DRAM Development Office, unveiled the three-dimensional (3D) architecture "zHBM" during a keynote address at "FMS 2026," a memory industry event held at the Santa Clara Convention Center in California on August 4 (local time).

This is a next-generation 3D memory designed to reaffirm Samsung's drive to reclaim market leadership.

Unlike the conventional approach of connecting High Bandwidth Memory (HBM) alongside the AI accelerator (xPU), zHBM is an architecture that directly stacks HBM on top of the AI accelerator.

The letter "z" was added to signify utilizing the third dimension of height (the Z-axis), moving beyond the two dimensions of width and length (the X and Y axes).

Samsung explained that this approach minimizes the data movement distance between the accelerator and memory, resolving data bottlenecks while maximizing bandwidth and power efficiency.

Even when compared to HBM5, the upcoming generation of HBM, zHBM delivers up to an eightfold improvement in performance per Graphics Processing Unit (GPU) and up to a threefold increase in performance-per-watt.

In terms of thermal management, its thermal resistance drops to between 10% and 25% of that of HBM5, cutting thermal resistance by more than half.

Emphasizing the necessity of such an architectural shift, Vice President Kim stated, "With conventional packaging structures, a 'memory wall' exists as a physical limit to shortening the physical distance between the accelerator chip and memory."

He also introduced that HBM5, currently in the development stage, will implement Gate-All-Around (GAA) transistors for the first time in HBM history, adopting a 2-nanometer (nm) foundry ultra-fine process. He added that applying a heat dissipation structure technology to the sides of the core die has improved thermal control by more than 20% compared to the previous generation.

Lee Jin-yeop, Vice President of the Flash Development Office, who co-presented the keynote, showcased "zNAND-O," a next-generation NAND solution optimized for on-device AI environments.

The "z" stands for vertical stacking, while "O" was appended to signify optimization for "on-device."

Samsung explained that zNAND-O simultaneously resolves the capacity shortage of DRAM and the low bandwidth issues of conventional NAND, delivering high capacity, high bandwidth, and fast response times to effectively support data processing.

Vice President Lee shared that test-running a GPT model with 120 billion parameters showed zNAND-O achieving the same tokens-per-second rate as a conventional DRAM server while costing only one-sixth as much.

Holding up a physical ultra-compact enterprise SSD named "BM1773" during his keynote, he drew laughter from the audience by saying, "Hard to believe, but this single small storage chip is worth as much as a luxury sedan. If I were to drop this on the floor, I would have to start looking for a new job starting tomorrow."

Additionally, Samsung Electronics became the first in the industry to unveil its 10th-generation V-NAND, "V10 BV-NAND," which vertically stacks over 400 layers.

Samsung Electronics plans to leverage these 3D memories to address the growing demand for high-performance computing and AI infrastructure.

(Photo courtesy of Samsung Electronics, Yonhap News)