Everything You Wanted to Know About Graph Neural Network Partitioning (But Were Afraid to Ask)

Graph Neural Networks (GNNs) are the de facto models for deep learning on graph datasets, but training GNNs on large-scale datasets remains a challenge. Data partitioning plays a critical role in distributed mini-batch GNN training, as it directly impacts memory usage, training speed, and model accuracy. This survey comprehensively compares prevalent partitioning strategies in the Deep Graph Library (DGL) framework across standard benchmarks and models of varying depths. We systematically analyze aspects such as partition sizes, training times, memory overhead, and the accuracy associated with each partitioning method. Through this analysis, we uncover practical insights and the inherent trade-offs of these strategies. Our findings reveal surprising cases where simpler partitioning approaches outperform more sophisticated schemes. We conclude by offering practical guidelines for GNN partitioning.