HBM May Leave the GPU Neighborhood

HBM was created to solve a very specific problem. It packs multiple memory dies vertically, giving GPUs more bandwidth and more capacity without taking much board space. For AI accelerators, that once looked like the perfect match.

Now the appetite of modern GPUs is growing faster than HBM can comfortably follow. Memory makers are already discussing a bold idea with customers: move HBM away from the GPU package and place it on a separate board.

Optical Links Could Change the Rules

Today, HBM sits close to the GPU because distance matters. Move it too far away, and speed suffers. Engineers now want to use optical interfaces to keep data moving quickly between the GPU and a separate memory module.

This shift could let one GPU access several times more HBM than today. It would also reduce the pressure to keep stacking more layers inside each HBM chip. Current designs already reach 16 layers, and future versions may hit 20. Yet every extra layer makes production harder and more expensive.

Bigger Memory Could Reshape AI Hardware

We believe this idea could make future AI systems more flexible and powerful. Larger pools of HBM may help companies train bigger models and process heavier workloads without redesigning everything around one crowded GPU package.

You may eventually see faster AI services, better data processing, and more capable cloud tools. But this will not be simple. Chip packagers, memory makers, and accelerator vendors must solve tough design and cost challenges first.

If you found this topic useful, share it with your network and read more of our articles on next generation hardware and AI infrastructure.