Hot Chips 31 Live Blogs: Gen-Z Chipset for Exascale Fabric

06:08PM EDT - That's a wrap, time for a break.

06:07PM EDT - Using 16FF TSMC CoWoS

06:04PM EDT - Cadet Cluster up to 13 DL-385

06:03PM EDT - Cadet is an early access platform

06:03PM EDT - Prototype builds coming soon

06:01PM EDT - This shows the three chip types

06:01PM EDT - labs

06:01PM EDT - Test chips already in the laps

06:00PM EDT - Next gen will use 400g signalling, PCIe G5

06:00PM EDT - 5x messaging vs the competition

05:59PM EDT - One mask/one reticle does one switch chip, one bridge chip, and three e20 modules

05:59PM EDT - All manufactured on one set of masks

05:58PM EDT - Uses bridge die and switch die

05:58PM EDT - Also have electrical to optical module

05:58PM EDT - 80ns idle switch latency

05:57PM EDT - 1D all to all mesh

05:57PM EDT - 25.6mm x 47.5mm with 5 chips

05:57PM EDT - 14B transistors

05:57PM EDT - 2.5D packaging

05:56PM EDT - 6 fibers per dicetion = 1.27 Tbps per optical module

05:56PM EDT - 212 Gbps per fiber with 4 fibers

05:56PM EDT - 2 bits per symbol with PAM4

05:55PM EDT - Integrated optics into the chipset avoids the need for excess SerDes activations

05:54PM EDT - Optical simplifies it, and saves power

05:53PM EDT - Can be cost competitive with copper in the rack

05:52PM EDT - Motivation for optical interconnect

05:52PM EDT - packet deadlines eliminate any chance of duplicate packets in the fabric

05:51PM EDT - Support explicit congestion notification to the requesters

05:51PM EDT - Chipset manages fabric congestion

05:50PM EDT - low latency 2ns FEC is available if needed

05:50PM EDT - 3 different levels of CRC, 2 levels of retry

05:49PM EDT - Five of these switches are interconnected to build the 60-port switch

05:48PM EDT - age-based arbitration

05:48PM EDT - 16 virtual channels

05:48PM EDT - Fully buffered crossbar organization

05:48PM EDT - Can do it to any fabric attached memory

05:47PM EDT - Support direct load-store access

05:46PM EDT - Provide 12 optical Gen-Z links

05:46PM EDT - Gen-Z bridge connected to four x16 PCIe 4.0 links

05:46PM EDT - Built with Rome as it has enough PCIe

05:45PM EDT - Bridges are required as no CPU is natively Gen-Z today

05:44PM EDT - 60-port switch module has integrated optical Gen-Z links

05:44PM EDT - 3D HyperX scalable to 61440 nodes with 4 swich crossings

05:44PM EDT - Develop a topology to create system fabric based on Gen-Z and HyperX topology

05:43PM EDT - Supports direct attach and switched fabric

05:43PM EDT - Supports cluster based interconnects

05:43PM EDT - Gen-Z is memory semantic

05:42PM EDT - Customers are doing these workloads, and requiring infrastructure to do them

05:42PM EDT - Dataset sizes are creating complexity and compute requirements

05:41PM EDT - Big data analytics and modelling are what customers are running

05:41PM EDT - Funded in part by DoE

05:40PM EDT - One of the key competing interconnects of the future is Gen-Z, and Hewlett Packard Enterprise have a Gen-Z chipset to show at Hot Chips today.