Hot Chips 2026: d-Matrix stacks AI accelerator directly on custom DRAM for 100 TB/s per card — TSMC 4nm compute die bonded face-to-face at a 36-micron pitch on top of a custom-designed die

d-Matrix unveiled a new 3D DRAM accelerator called Raptor at the Hot Chips 2026 conference, showcasing a TSMC 4nm compute die bonded face-to-face with a custom-designed DRAM die at a 36-micron pitch. This innovative design delivers 100 TB/s of bandwidth per card using just 32GB of memory. Co-founder and CTO Sudeep Bhoja stated that the energy cost of this vertical interface is 0.37 pJ/bit, compared to around 2.4 pJ/bit for HBM4-based designs, resulting in a 4.7 times higher throughput per card.

The company has not disclosed who manufactures the DRAM die, but it has partnered with TSMC for the logic die and Alchip for the ASIC design and 2.5D/3D packaging. The custom DRAM die not only serves as the interposer but also carries PCIe and die-to-die signals. While the company claims a power density of 0.5W per square millimeter, the DRAM's high junction temperature of 105°C significantly reduces retention time from 32ms to 4ms.

Raptor's unique architecture includes 840 banks per chiplet, with 72 (around 9%) acting as spares wired into a two-level mux chain, allowing for bank switching without disrupting overall performance. The company also employs a Reed-Solomon code and CRC for error correction. Despite these advancements, the DRAM's lower density compared to HBM4 may impact scalability, as d-Matrix expects to deploy Raptor in rack-scale configurations of 72 cards per deployment, providing enough memory capacity to handle large-scale model inference.