“State-of-the-art” (SOTA) claims in materials graph neural networks are typically grounded in marginal improvements on standard benchmarks, yet such gains often reflect properties of the evaluation protocol rather than advances in generalization. This analysis demonstrates that widely used practices—particularly random train–test splits, inconsistent baselines, and selective reporting—systematically inflate reported performance. At the same time, critical constraints, including interpolation-dominated evaluation, computational cost, limited extrapolation, and dataset bias, remain largely unacknowledged. Drawing on the 2017–2023 literature, the study shows that these artifacts and omissions distort how progress is measured and interpreted. By reframing SOTA as a context-dependent rather than universal claim, it proposes a shift toward multi-split, multi-metric, and reproducible evaluation that more accurately reflects real-world materials discovery conditions.
Institute for Advanced Materials Research Press