Institute for Advanced Materials Research Press Institute for Advanced Materials Research Press

Search

Search results:
Beyond Test-Set Error: A Conceptual Framework for Evaluating ML Potential Transferability Under Distribution Shift
Machine learning interatomic potentials (MLIPs) are increasingly used to accelerate atomistic simulation in materials science, yet their evaluation remains dominated by test-set error measured on held-out data drawn from the same distribution as the training set. Although this practice often yields low mean absolute errors for energies and forces, it provides limited evidence that a model will remain reliable when applied to the distribution shifts that define real deployment settings, including new compositions, defect structures, elevated temperatures, and non-equilibrium trajectories. This article develops a conceptual framework that distinguishes transferability from in-distribution accuracy and defines it as the ability of an MLIP to preserve predictive fidelity within application-relevant tolerances under explicitly characterized shifts in the joint distribution of structures and quantum-mechanical labels. The framework identifies four canonical forms of shift in computational materials science—compositional, structural, thermodynamic, and dynamical—and shows why current benchmarking practices systematically obscure them. To address this limitation, the study proposes a shift-aware evaluation protocol built around five components: shift characterization, sensitivity analysis, extrapolation-distance estimation, robustness criteria, and standardized reporting. Within this framework, Maximum Mean Discrepancy is adapted as a quantitative diagnostic for pre-deployment assessment of train–target divergence, while transferability is evaluated through five complementary dimensions: accuracy under shift, graceful degradation, uncertainty alignment, physical consistency, and compositional extrapolation. By replacing the tacit IID assumption with an explicit framework for reasoning about distribution shift, this work offers a common basis for evaluating, reporting, and comparing MLIP transferability across benchmarks and deployment scenarios, thereby supporting more trustworthy AI-driven materials discovery.
Journal of Computational and Data-Driven Materials Engineering
Original Research | Open access | 18 January 2022 | Article: 3

Prediction without Transferability: Domain Shift in Cross-Material AI Inference
The advent of computational and data-driven materials engineering has revolutionized the discovery and design of advanced materials, leveraging machine learning to navigate vast chemical spaces and predict properties from multimodal datasets. However, a critical challenge persists in the form of domain shifts, where AI models trained on one material class exhibit diminished predictive accuracy when inferred across disparate materials, undermining transferability in cross-material inference scenarios. This conceptual manuscript addresses this gap by introducing a novel framework that dissects the epistemic and computational underpinnings of such shifts within materials informatics ecosystems. Drawing from representation learning, graph neural networks, and uncertainty quantification paradigms, the proposed Cross-Material Inference Cascade (CMIC) framework conceptualizes domain shifts as emergent from mismatched representational hierarchies and inference pipelines, rather than mere data scarcity. It outlines structural layers for mitigating these shifts through adaptive representation alignments and feedback-driven discovery logics, without relying on empirical transfer learning techniques. Implications extend to high-throughput computation, autonomous discovery systems, and inverse design, fostering more resilient AI infrastructures in materials science. By emphasizing computational workflow dynamics and epistemic risk structures, this work provides interpretive insights for steering future data-driven paradigms toward robust cross-material predictions, enhancing the interoperability of foundation models and simulation-experiment couplings in the field.
Journal of Computational and Data-Driven Materials Engineering
Original Research | Open access | 18 September 2022 | Article: 89
Filters
Clear All





Access type