Search

The pervasive reliance on predictive accuracy metrics such as mean absolute error, root mean square error, and R² in materials artificial intelligence has created a fundamental misconception: that low prediction error equates to a valid measurement of a material’s property. This paper argues that accuracy alone is insufficient because an AI-generated property value may align closely with held-out test data yet fail to support the specific scientific or engineering inferences for which it is intended. Drawing on foundational measurement validity theory from psychometrics and the social sciences, the manuscript adapts these concepts to the unique context of AI-generated materials properties. It proposes a novel five-component conceptual theory of measurement validity tailored to machine-learning predictions of physical quantities such as band gaps, formation energies, and mechanical moduli. Five distinct dimensions of validity—construct, criterion, generalizability, robustness, and consequential—are articulated and illustrated with materials-specific scenarios. Finally, the framework offers concrete implications for authors, reviewers, and the broader materials informatics community, shifting validation practices from narrow accuracy reporting toward comprehensive evidence-based arguments that link predictions to intended uses. By distinguishing accuracy from validity, this conceptual framework aims to elevate the epistemological rigor of AI-driven materials discovery and design.