What is HBM Memory | Features & Types

Graphics cards make use of VRAM memory chips to store information. One of the types of memory that exist for graphics cards is High Bandwidth Memory (HBM) which has some advantages over other types of memory.

Let’s discuss HBM memory in further more details that what it and what is it for?

What is HBM Memory

It is one of the types of high-speed computer memory characterized by offering synchronous dynamic random access with 3D stacking.

HBM memory developed and mainly used for high-performance graphics accelerators, network devices, ASICs, and AI FPGAs for high-performance data centers, supercomputers, and some models of gaming graphics cards.

See Also: HBM2 vs. HBM3 (High-Bandwidth Memory)

The first HBM memory chip was manufactured by SK Hynix in 2013. The first serial products to integrate HBM memory were AMD Fuji GPUs released in 2015.

What HBM offers is high bandwidth with lower power consumption than DDR and GDDR standards.

This is accomplished by stacking up to eight DRAM arrays and an optional base array that can include a buffer and test logic circuitry. The stacked memories are vertically interconnected by silicon vias (TSVs) and micro-bumps.

Normally HBM memories are installed on the same substrate as the GPU and CPU.

AMD and NVIDIA use different variants of HBM memory integrated into the same package as professional GPUs. HBM memories can also be integrated into the same CPU package.

NVIDIA and AMD use special designs for these GPUs that integrate an interposer that allows HBM memories to be installed next to the GPU and thus obtain better performance and less consumption.

The problem is that the manufacture of HBM memories is expensive and the GPUs that integrate them are also expensive as they need additional elements.

History of High Bandwidth Memory

This type of memory began to be developed in 2008 and it was AMD before. Initially they were looking for solution that could create memories and correct the problem of consumption and the form factor.

The aim was to achieve a solution that would satisfy the growing needs of a computer’s amount of memory, but occupying less space and consumption.

AMD in the following years was working on the development of different procedures for stacking dies. The team in charge of developing these memories was Bryan Black, a senior member of AMD.

It should be noted that AMD sought the collaboration of industry partners with experience in the manufacture and development of memories.

One of the most prominent was SK Hynix, who already had previous experience in building 3D stacked memory.

It also included the Taiwanese UMC, specialized in interposers, and Amkor Technology and ASE, two companies specialized in the encapsulation process. HBM was finished developing in 2013, the year SK Hynix manufactured the first memory chip.

JEDEC adopted HBM memory as a standard in October 2013, as a result of a request made by AMD and SK Hynix in 2010. Mass manufacturing of HBM memory by SK Hynix started in 2015.

The first commercial product to use HBM memory was the AMD Fiji GPU, which was released in June 2015, with the AMD Radeon R9 Fury graphics card using it.

Samsung in January 2016 began early manufacturing of HBM2 memories. In the same month JEDEC accepted HBM2 memories as a standard.

The first commercial solution to use HBM2 memory was the NVIDIA Tesla P100 graphics card, which launched in April 2016.

Intel soon after, in June 2016, released Xeon Phi accelerators that feature 8 stacks of HCDRAM memory, a variant of HBM memories manufactured by Micron.

Main Features of HBM Memory

These memories stand out as an especially exceptional solution for memory-intensive computing tasks. The main characteristics of these memories are:

1. Eliminates the processing bottleneck

HBM memories have the characteristic of being able to be installed directly in the GPU/CPU package. This allows is to reduce the information processing time.

They connect with the GPU/CPU through high performance and efficient pathways. Although they are not physically integrated into the GPU/CPU, the transfer speed is incredible.

2. Energy efficiency

Opting for HBM memories allows you to improve the energy efficiency of the system. Few memory chips are required per GPU, which allows lower power consumption as compared to GDDR memories.

HBM memories are next to the GPU, in the same package, while the GDDR memories are around the GPU package, which also significantly reduces power consumption.

3. Small Form Factor

HBM memories are characterized by having a space-saving design. For the same amount of memory on a graphics card, HBM memory takes up to 94% less than GDDR5 memory.

Not HBM memories are integrated into the GPU package, the size of the PCB can be reduced compared to GDDR memories.

4. Bandwidth

HBM memories offer a much higher bandwidth than GDDR memories. This makes HBM memories optimal for tasks such as Artificial Intelligence, Deep Learning, supercomputing and other situations where there is a huge flow of data.

5. Manufacturing Costs

Not everything is bright with HBM memories, there is also a shadow and it is the manufacturing cost.

It is estimated that the manufacturing cost of HBM memory wafers is between 30-50% more than a DRAM (GDDR) memory wafer.

This is due to the characteristics and manufacturing characteristics. That’s why this type of memory is not used in gaming graphics cards as it is much more expensive.

Types of HBM Memory

We are now going to do a brief review of the different variants of HBM memories that have been appearing.

Some of the memories that appear here are not yet commercialized, since they are still in the development phase.

1. HBM Memories

HBM memories offer a larger memory bus than other DRAM standards (such as DDR4 and GDDR5).

An HBM stack makes use of four DRAM dies (4-Hi) that have two 128-bit channels per die for a total of 8 channels and a 1024-bit memory interface (128GB/s bandwidth).

Each of the memory stacks can have a maximum capacity of 4GB and up to 16GB. A GPU with four 4-Hi HBM stacks can offer a memory bandwidth of 4096 bits.

GDDR5 memories only offer 32 bits per channel and a maximum of 16 channels, thus offering a 512-bit memory interface.

2. HBM2 Memories

It is the evolution of HBM memories with significant improvements in performance and capacity.

HBM2 memories are characterized by offering a bandwidth of 2656GB/s per stack and a limit of 8GB of memory per stack.

They are also characterized by offering the possibility of integrating up to 8 memory stacks, offering a total capacity of 64GB HBM2.

3. HBM2e Memories

We could say that these memories are a “simple” update of the HBM2 memories. These new HBM2e memories are characterized by offering a bandwidth of up to 307GB/s per memory stack.

This update allows stacking of up to 12 memory layers (12-Hi) allowing a total of 24GB per stack.

HBM2e has received two variants

• Samsung: It has developed the HBM2 memories that have been called Flash-bolt HBM2e and that has eight matrices per memory stack. These memories offer a capacity of 16GB and a bandwidth of 460GB/s per memory stack
• SK Hynix: A variant of the HBM2e memory which you have not renamed. This revision offers a capacity of 16GB and a bandwidth of up to 460GB/s.

4. HBM3 Memories

By the end of 2020, Micron announced the HBM2e update, which was initially called HBM next memory. Later these memories have been renamed as HBM3 memories.

These new memories will offer a bandwidth per memory stack of 665GB/s. Each of the memory stacks will be able to support up to 64GB of capacity in a 16-matrix (16-Hi) format.

Additionally, these memories have licensed the DBI Ultra 2.5D/3D hybrid interconnect technology to Xperi Corp.

5. HBM-PIM Memories

Samsung in February 2021 announced the development of “special” HBM memories with memory processing. Such memories have been fully developed for in-memory AI computing.

This allows to increase the capacity of large-scale data processing.

It enables a DRAM-optimized AI engine to be installed within each memory stack that enables parallel processing and minimizes data movement.

See Also: How to choose a Good Graphics Card for Gaming

According to Samsung, the performance of the system will be doubled and in addition, the energy consumption will be reduced by 70%, without the need to change hardware or software.

Conclusion

High Bandwidth Memory memories has become essential for segments like artificial intelligence, deep learning and supercomputing, among others.

HBM is a memory standard that has been adopted for graphics cards intended for advanced computing. AMD was the one who started its development to improve energy efficiency and memory sizes could be scaled.

But another reason is that the GCN architecture had a problem which required large bandwidth or at least. The greater the memory bandwidth, the better performance will be.

This has been left behind with the RDNA architecture for gaming and the CDNA architecture for advanced computer graphics.

We see that HBM memories have evolved very well, but it is difficult for them to end up being used in the gaming market. As these memories are difficult and expensive to manufacture than GDDR DRAM memories.

We saw how the HBM2 memories meant that the AMD Radeon Vega 64 were excessively expensive with respect to the performance they offered.

Although many suggest that HBM memories will end up replacing GDDR memories, but at the moment this seems quite economically unfeasible.

What do you think about High Bandwidth Memory (HBM) memories?