Maybe I'm missing something in the initial comparison but at @2:08 you can see the HBv4 and HBv5 are NOT configured the same - The 5 has 384 compute nodes to the 4 only having 176 and "total memory" is double on the 5 - so the performance difference illustrated (2.6x faster) would be more than expected. How is that an apples to apples comparison?
Good observation. In this case we're measuring memory bandwidth, not total memory or total CPU. The increase in bandwidth for HBv5 is more a function of the increased number of channels between HBM and physical CPU cores as described at 8:50 up to 128 total channels - along with direct physical proximity to the also on-package CPU cores. This is compared to 12 memory channels using traditional 12 DIMMs, which are adjacent to, but not directly on-package with the physical CPU cores in HBv4.
Azure HBv5 VMs use AMD EPYC 9004 Processors with 3D V-Cache. More information on these can be found here techcommunity.microsoft.com/blog/azurehighperformancecomputingblog/announcing-azure-hbv5-virtual-machines-a-breakthrough-in-memory-bandwidth-for-hp/4303504
@ander1482the SoCs used in HBv5 have Epyc cores with HBM on the Soc package. These processors don't have the 3D V-cache like the Genoa-X processors do.
In the video they specifically mention 3D cache, but also the HBM memory and package config for the MI300 platform. So it’s a custom high cache high HBM MI300C with all CPU cores (likely 96, with some reserved for OS). Serious HPC! It wouldn’t surprise me to see AMD selling more of this type of compute to other supers.
@@tufttugger I see the reference for 3D V-Cache in the label on the SoC graphic at 7:40, but I think that's just a typo in the graphic. The HBv5 custom SoC doesn't have "Genoa-X" processor cores with the 3D V-Cache; it's built on standard Genoa cores. You are correct that each SoC has 96 cores and 128 GB of HBM, some of which will be reserved for the host OS/hypervisor services.
Maybe I'm missing something in the initial comparison but at @2:08 you can see the HBv4 and HBv5 are NOT configured the same - The 5 has 384 compute nodes to the 4 only having 176 and "total memory" is double on the 5 - so the performance difference illustrated (2.6x faster) would be more than expected. How is that an apples to apples comparison?
Good observation. In this case we're measuring memory bandwidth, not total memory or total CPU. The increase in bandwidth for HBv5 is more a function of the increased number of channels between HBM and physical CPU cores as described at 8:50 up to 128 total channels - along with direct physical proximity to the also on-package CPU cores. This is compared to 12 memory channels using traditional 12 DIMMs, which are adjacent to, but not directly on-package with the physical CPU cores in HBv4.
That was very insightful!
Impressive.
But i dont get it. Its custom silicon or Epyc with V-cache?
Azure HBv5 VMs use AMD EPYC 9004 Processors with 3D V-Cache. More information on these can be found here techcommunity.microsoft.com/blog/azurehighperformancecomputingblog/announcing-azure-hbv5-virtual-machines-a-breakthrough-in-memory-bandwidth-for-hp/4303504
@ as far as I know those processors don’t have HBM memory, only Intel Xeon MAX.
@ander1482the SoCs used in HBv5 have Epyc cores with HBM on the Soc package. These processors don't have the 3D V-cache like the Genoa-X processors do.
In the video they specifically mention 3D cache, but also the HBM memory and package config for the MI300 platform. So it’s a custom high cache high HBM MI300C with all CPU cores (likely 96, with some reserved for OS). Serious HPC! It wouldn’t surprise me to see AMD selling more of this type of compute to other supers.
@@tufttugger I see the reference for 3D V-Cache in the label on the SoC graphic at 7:40, but I think that's just a typo in the graphic. The HBv5 custom SoC doesn't have "Genoa-X" processor cores with the 3D V-Cache; it's built on standard Genoa cores.
You are correct that each SoC has 96 cores and 128 GB of HBM, some of which will be reserved for the host OS/hypervisor services.