What good is going fast if you can't get past the next rack?

According to Mehta this kind of connectivity could support 512 GPUs in as few as eight racks, acting as a single scale-up system.

That’s a biggin!