Distributed Layout — 8×8 Matrix on 2×2 GPU Mesh
Fully sharded across 4 GPUs | click cell to see placement
Layout
Fully Sharded (S0S1)
Shard + Replica (S0R)
Shard + Offset (S0+O)
GPU Mesh (2×2)