A Framework for Assessing Compositional Generalizability in High-Entropy Alloy ML Potentials

High-entropy alloys (HEAs) and multi-principal element alloys (MPEAs) occupy an enormous compositional space that conventional computational approaches cannot fully explore. A five-element system with 10 % concentration steps already contains millions of distinct nominal compositions, each further multiplied by exponentially many atomic configurations arising from configurational disorder. Machine learning (ML) interatomic potentials have been proposed as a scalable solution to accelerate property prediction and materials design in this vast space. Yet the compositional generalizability of these potentials—their capacity to make reliable predictions for compositions and local environments lying outside the training distribution—remains largely unexamined in a systematic way. Most existing benchmarks focus on interpolation within narrow ranges of ordered or equiatomic compounds, leaving critical extrapolation scenarios untested. This conceptual framework article identifies four distinct but interrelated dimensions of compositional generalizability in HEA ML potentials: (i) element extrapolation, (ii) concentration interpolation, (iii) multi-element recombination, and (iv) local environment diversity. It proposes a four-component assessment framework—training-set characterization, test-set design, dimension-specific generalization metrics, and a structured validation protocol—that enables researchers to quantify generalization gaps without relying on performance numbers or simulation results. For each dimension, explicit validation strategies and success criteria are defined, grounded in the literature on special quasirandom structures, graph-network representations, and equivariant architectures. The framework is deliberately conceptual, emphasizing definitions, relationships among components, assessment criteria, and diagnostic reasoning rather than empirical data. By adopting this framework, the community can move beyond ad-hoc testing and develop ML potentials that truly generalize across the high-dimensional alloy landscape. The implications extend to more reliable high-throughput screening, accelerated discovery of novel HEAs, and clearer guidance for training-set design and model architecture choices. Ultimately, systematic assessment of compositional generalizability will help ensure that ML potentials fulfill their promise for complex concentrated alloys.

High-entropy alloys (HEAs) and multi-principal element alloys (MPEAs) have emerged as a new paradigm in materials design, offering exceptional mechanical, thermal, and functional properties that arise precisely from their compositional complexity [1-5]. Unlike conventional alloys built around one or two principal elements, HEAs contain three to six or more elements in near-equiatomic or non-equiatomic proportions. This multi-principal nature generates an enormous compositional space: even a modest five-element system sampled at 10 % concentration increments yields millions of possible nominal compositions. Each composition, moreover, corresponds to an astronomical number of distinct atomic arrangements because of configurational disorder.

Machine learning interatomic potentials have been widely recognized as a powerful route to navigate this space efficiently [1, 6-10]. By learning directly from density-functional-theory data, these potentials can predict energies, forces, and derived properties orders of magnitude faster than first-principles calculations while retaining near-ab-initio accuracy within their domain of applicability. Recent advances in graph-network architectures [8] and equivariant neural networks [7] have further improved data efficiency and physical fidelity, making it feasible to train potentials on the disordered structures that characterize HEAs.

Nevertheless, a fundamental limitation persists. The compositional generalizability of HEA ML potentials—the ability of a model to produce accurate predictions for compositions and local atomic environments that differ substantially from those seen during training—is rarely assessed in a structured or reproducible manner. Existing literature reviews highlight progress in model development and applications to specific HEAs yet acknowledge that generalization across composition remains an open challenge [1]. Performance assessments of ML potentials typically focus on ordered compounds or narrow interpolation regimes, which do not capture the extrapolation demands inherent to HEA design [6, 11, 12]. Benchmarks that do include HEAs often rely on special quasirandom structures (SQS) for a single nominal composition and do not systematically probe how well the model handles new elements, new concentrations, novel element-pair combinations, or truly random (non-SQS) configurations [13, 14].

This conceptual framework article therefore introduces a systematic approach for assessing compositional generalizability in HEA ML potentials. The framework rests on four core dimensions of generalization that together span the practical needs of alloy discovery. It defines four assessment components that any research group can apply without additional experimentation or simulation, only conceptual reorganization of existing training and test data. Validation strategies tailored to each dimension provide clear success criteria expressed in terms of relative error ratios and diagnostic patterns rather than absolute metrics. By separating generalization performance along these four axes, the framework reveals specific weaknesses—whether in element coverage, concentration sampling, interaction learning, or configurational representation—and guides targeted improvements in training-set design and model architecture.

Compositional generalization in HEA ML potentials is intrinsically more demanding than in conventional alloy systems for six interrelated reasons that stem from the fundamental physics and data realities of complex concentrated alloys.

First, the composition space is high-dimensional. With three to six principal elements, each concentration can vary continuously between zero and one (subject to the sum-to-one constraint). Even coarse 10 % sampling steps produce combinatorial explosion. No training set, however large, can densely cover all combinations, forcing every practical model to extrapolate in composition space [1, 11].

Second, configurational disorder multiplies the difficulty. For any fixed nominal composition, the number of possible atomic arrangements grows exponentially with system size. A model must therefore learn not only how properties depend on average composition but also how they depend on the specific local atomic environments that realize that composition. Training data typically capture only a tiny fraction of these environments, so generalization requires the model to infer unseen local neighborhoods from limited examples [13-15].

Third, local-environment coupling is pervasive. Changing the concentration of one element alters the probability of every possible nearest-neighbor and second-nearest-neighbor shell for all other elements. Consequently, a model trained on a particular set of element-pair statistics cannot simply be queried at a new concentration; the entire local-environment distribution shifts in a coupled, non-independent manner [12, 16, 17].

Fourth, property landscapes are strongly non-linear. Mechanical, thermal, and electronic properties of HEAs are not simple linear superpositions of elemental contributions. Lattice distortion, short-range ordering, and electronic hybridization create intricate, non-monotonic dependencies on both composition and local chemistry. Linear interpolation in composition space therefore provides no guarantee of accurate property prediction, and small concentration changes can produce disproportionately large property shifts [3, 18, 19].

Fifth, data scarcity is acute. Generating high-quality density-functional-theory reference data for disordered HEAs is computationally expensive, especially when many distinct configurations must be sampled to represent the same nominal composition. Most published training sets remain modest in size and are heavily biased toward equiatomic or near-equiatomic compositions and ordered or SQS supercells [6, 20, 21].

Sixth, the very tools used to generate training structures have built-in limitations. Special quasirandom structures [13] are excellent for approximating the average disordered state of a given nominal composition within a periodic supercell, yet they cannot capture the full spectrum of concentration fluctuations, long-range disorder, or rare local environments that exist in truly random solid solutions. Consequently, a potential trained exclusively on SQS data may fail when confronted with configurations drawn from a genuinely random ensemble [14, 22, 23].

These six factors interact. High dimensionality and data scarcity force sparse sampling; configurational disorder and local-environment coupling demand that the model learn subtle, many-body interactions from that sparse data; non-linear landscapes amplify any extrapolation error; and SQS approximations further restrict the diversity of local environments presented during training. The result is that standard interpolation tests—however successful—provide little insight into whether a potential will remain reliable when applied to the new compositions that alloy designers actually wish to explore. A dedicated conceptual framework for assessing generalization is therefore not merely useful but essential if ML potentials are to realize their transformative potential for HEA discovery.

Compositional generalizability in HEA ML potentials can be decomposed into four orthogonal yet interacting dimensions. Each dimension isolates a distinct type of extrapolation that arises in practical alloy-design workflows.

Figure 1 maps compositional generalizability into four diagnostically distinct dimensions—element extrapolation, concentration interpolation, multi-element recombination, and local-environment diversity—arranged as a directional assessment structure from training-domain definition to out-of-distribution evaluation.

Element extrapolation probes whether a model can infer properties for an HEA incorporating an element absent from the training distribution, as when a potential calibrated on Co–Cr–Fe–Ni is deployed on Co–Cr–Fe–Ni–Mn or Co–Cr–Fe–Ni–V. Under such conditions, all atomic environments involving the added species are both statistically unseen and chemically distinct, leaving the model without direct empirical grounding for those neighborhoods. A related challenge emerges when composition varies continuously rather than at discretely sampled points: concentration interpolation evaluates behavior at intermediate concentrations not represented during training, where non-monotonic response surfaces or abrupt transitions render simple compositional interpolation inadequate. This issue extends further when previously unobserved combinations of elements must be resolved, since training confined to binary subsystems does not guarantee that higher-order interactions in ternary or quaternary alloys are correctly encoded, thereby testing whether the learned representation captures genuine many-body effects rather than pairwise regularities. Beyond compositional considerations, robustness also depends on sensitivity to configurational disorder, as models trained on SQS supercells may not generalize to fully random arrangements of identical nominal composition, exposing limitations in representing long-range fluctuations that SQS constructions intentionally minimize [13]. Although these facets are analytically separable, their coupling in realistic settings implies that deficiencies rarely occur in isolation, complicating attribution while underscoring the need for diagnostic decomposition.

This decomposition allows researchers to pinpoint whether a model’s weakness lies in elemental coverage, sampling density, interaction learning, or configurational representation.

Table 1 consolidates the four dimensions into an analytical typology by distinguishing the boundary crossed in each case, the form of novelty introduced at test time, and the specific failure logic each dimension is designed to expose.

The proposed framework comprises four interlocking components that together transform the abstract concept of compositional generalizability into an actionable evaluation protocol.

Rigorous evaluation begins with precise delineation of the training distribution, as meaningful claims of generalization depend on knowing the compositional and configurational scope the model has encountered. This entails explicit reporting of the elemental set, concentration ranges and their discretization, the actual co-occurrence of element pairs or higher-order combinations within training structures, and the number and type of configurations sampled per composition, including the distinction between SQS and fully random realizations. Such characterization establishes the effective support of the learned representation and, in turn, defines where extrapolative demands arise; absent this information, the boundary between interpolation and genuine out-of-distribution prediction remains indeterminate.

Building on this foundation, evaluation must isolate distinct sources of distributional shift through deliberately constructed hold-out sets that probe unseen elements, intermediate concentrations, novel multi-element combinations, and alternative configurational statistics. The key requirement is independence from the training workflow, since any procedural overlap risks subtle leakage that would inflate apparent robustness. Interpreting performance across these regimes further necessitates dimension-specific metrics rather than aggregate scores: an error ratio benchmarking degradation relative to in-distribution interpolation, an assessment of compositional distance in element–concentration space, and a failure rate defined by exceedance of a property-dependent tolerance threshold [24, 25]. Disaggregating these quantities prevents the obscuring of systematic weaknesses that would otherwise be averaged out.

A consistent validation protocol operationalizes these principles by coupling controlled training on a well-defined base set with targeted evaluation across the constructed test domains, followed by comparison against an interpolation baseline drawn from within the training support. The resulting generalization gap, expressed as the degradation in accuracy under these shifts, provides a model-agnostic measure of robustness. Crucially, this procedure abstracts from specific architectures or target properties, enabling reproducible assessment and facilitating cumulative progress across studies.

When applied consistently, the four components convert anecdotal observations about “good performance on HEAs” into precise, diagnosable statements about where and how generalization succeeds or fails. The framework therefore supplies the conceptual scaffolding needed to move HEA ML potential development from case-by-case validation to systematic, reproducible assessment.

Each dimension is accompanied by a concrete validation strategy that operationalizes the framework while remaining purely conceptual.

Strategy for element extrapolation begins with a base quaternary system such as Co–Cr–Fe–Ni. The potential is trained on structures that exclude one target element (for example, no Mn). It is then tested on structures of the quinary Co–Cr–Fe–Ni–Mn at the same nominal concentrations used in training. The success criterion is that the error ratio remains below 2× relative to the interpolation baseline. This threshold reflects the expectation that introducing an entirely new element is the most severe test; any larger degradation signals insufficient transferability of learned chemical interactions.

Strategy for concentration interpolation trains the model exclusively at discrete concentration steps (0 %, 20 %, 40 %, 60 %, 80 %, 100 % for each element, with other elements adjusted to maintain the total). Testing occurs only at the exact midpoints (10 %, 30 %, 50 %, 70 %, 90 %). Success is defined by an error ratio below 1.5× the interpolation baseline and the absence of systematic bias (errors not consistently positive or negative). This criterion ensures the model has captured the non-linear shape of the property landscape rather than merely memorizing the trained points.

Strategy for multi-element recombination trains the potential only on binary subsystems (all possible pairs among the constituent elements). Testing proceeds on ternary and quaternary compositions formed by recombining those binaries. The success criterion is that the error on the recombined higher-order systems remains correlated with the binary error; no abrupt jump in error ratio should occur. This confirms that the model has learned transferable many-body interactions rather than isolated pairwise statistics.

Strategy for local environment diversity trains exclusively on SQS supercells and tests on random configurations of identical nominal compositions (or the reverse). Because true configurational disorder is expected to be the hardest aspect, the tolerance is relaxed to an error ratio below 3× the baseline. The criterion acknowledges that SQS and random ensembles differ in subtle long-range statistics; a modest degradation is acceptable provided the model still captures average properties reliably.

These four strategies are modular. A research group can apply any subset or all of them depending on the intended use case of the potential. Because each strategy maps directly onto one dimension and one component of the framework, results are immediately interpretable: a failure in element extrapolation points to insufficient elemental diversity in training, a failure in concentration interpolation signals inadequate sampling of the property landscape, and so on. Collectively the strategies transform the abstract framework into a practical checklist that any HEA ML potential developer can follow.

Existing materials-property benchmarks such as the Materials Project, OQMD, and AFLOW have been instrumental in advancing ML potentials for crystalline solids [6, 8, 11]. These databases predominantly contain ordered compounds with well-defined stoichiometries and minimal configurational disorder. While they provide excellent testbeds for interpolation within narrow composition ranges, they do not represent the high-dimensional, disordered nature of HEAs and MPEAs. Consequently, a potential that performs well on these ordered benchmarks may still fail when confronted with the combinatorial complexity of multi-principal element systems [1, 12].

Benchmarking studies of ML interatomic potentials have similarly focused on performance metrics for elemental metals, binary alloys, or simple ternary compounds [6, 20, 26, 27]. For instance, large-scale assessments compare energy and force errors across thousands of structures drawn from equilibrium lattices or defect-containing supercells, yet they rarely include the vast compositional spaces characteristic of HEAs [6, 11]. Even when HEAs appear in such evaluations, testing is typically limited to a handful of equiatomic compositions generated with SQS and evaluated only for interpolation within that narrow slice of composition space [14, 15]. No systematic probe of element extrapolation, concentration interpolation, multi-element recombination, or local-environment diversity is performed.

The gap is particularly evident in reviews of ML for HEAs. Progress is documented in model architectures and applications to specific alloy families, but compositional generalization is acknowledged as an unresolved challenge rather than a quantified property [1, 28]. Transfer-learning studies across alloy families exist [7, 8], yet they do not formalize the four dimensions introduced here. Similarly, work on SQS for HEAs emphasizes their utility for approximating average disorder within a fixed composition but does not address how well potentials generalize when the underlying configurational ensemble changes [13, 14, 22].

This framework directly fills the missing structure. It provides HEA-specific test-set designs that can be layered onto any existing benchmark suite without requiring new data generation. By insisting on separate reporting for each of the four dimensions, the framework prevents the common practice of declaring “good performance on HEAs” after testing only equiatomic SQS structures. It therefore complements rather than replaces current benchmarks, supplying the additional layer of scrutiny needed for complex concentrated alloys. Adoption of the framework will enable future benchmark curators to include standardized HEA generalization suites, transforming ad-hoc validation into community-wide, reproducible assessment [1, 6, 11].

Interpreting results from the framework requires attention to the pattern of generalization gaps across the four dimensions rather than any single scalar value. Good compositional generalization is indicated when error ratios remain close to 1.0 for all dimensions, failure rates stay below 10 %, and no systematic bias (consistent over- or under-prediction) appears. Such outcomes demonstrate that the model has learned transferable representations of local chemistry, composition-property relationships, and configurational statistics.

Poor generalization manifests differently in each dimension. An error ratio exceeding 5× for element extrapolation signals that the model cannot transfer learned interactions to unseen elements; the potential is effectively memorizing element-specific features rather than general chemical principles [7, 12]. Systematic over- or under-prediction in concentration interpolation reveals that the property landscape is strongly non-linear and undersampled; the model is essentially performing piecewise interpolation instead of capturing the true functional form [3, 18]. Abrupt jumps in error for multi-element recombination indicate insufficient learning of higher-order interactions; the potential treats elements as independent rather than coupled [15, 16]. Finally, a large gap in local-environment diversity (error ratio > 3×) shows that SQS-based training has not equipped the model to handle realistic concentration fluctuations and long-range disorder [13, 14].

Table 2 converts the framework from a reporting device into a diagnostic tool by mapping distinct error patterns to likely structural causes, model-development responses, and scope-of-use decisions.

These patterns carry immediate diagnostic value. Failure confined to element extrapolation points to the need for broader elemental diversity in the training set. A concentration-interpolation shortfall suggests denser sampling of the composition grid or inclusion of more intermediate structures. Multi-element recombination problems call for explicit training on higher-order combinations or architectures that better capture many-body effects [8, 28]. Local-environment diversity failures indicate that SQS alone is insufficient; random configurations must be added to the training workflow.

Because the framework reports metrics separately, researchers can prioritize fixes. A model that excels on three dimensions but collapses on the fourth can still be used confidently within its validated scope. Over time, consistent application of the framework will reveal which architectures—graph networks [8], equivariant models [7], or hybrid approaches—naturally exhibit superior generalization, guiding the community toward more robust HEA potentials.

While the framework supplies a clear conceptual scaffold, it is subject to five practical limitations that users should acknowledge.

Despite its conceptual economy, comprehensive assessment imposes a tangible computational burden, as constructing independent test domains often necessitates generating and labeling additional structures when such data are absent. In high-dimensional compositional spaces, this may translate into hundreds of DFT evaluations, placing practical constraints on resource-limited settings [6, 20]. A related consideration concerns how success is defined: the use of relative error ratios, rather than fixed thresholds, reflects the context-dependent nature of acceptable performance, which varies with material system, target property, and application-specific tolerances. Under these conditions, a model adequate for elastic constants may remain insufficient for vacancy formation energies within the same alloy family, requiring careful calibration of criteria [1, 3].

Beyond these practicalities, interpretability is complicated by intrinsic coupling among generalization regimes, since extrapolation to unseen elements simultaneously introduces novel local environments and interaction patterns. Although separate reporting attenuates this effect, interdependence persists and can obscure attribution of failure modes [12, 16]. This limitation is further compounded by the restriction to zero-temperature compositional and configurational variability, leaving thermally driven phenomena—such as vibrational contributions, phase stability, and temperature-dependent ordering—outside the current scope, and thus pointing to an unresolved extension of the framework [18, 29]. Moreover, generalization remains inherently property-specific: predictive fidelity may be retained for total energies yet degrade for forces or elastic responses, reflecting the distinct sensitivities of these observables to local structural features. Consequently, robust evaluation requires property-resolved analysis rather than implicit assumptions of transferability [6, 26].

Despite these limitations, the framework remains a substantial advance over the absence of any structured assessment. Its modular design allows incremental adoption—starting with one or two dimensions—and future extensions can address the gaps identified here.

The framework carries direct implications for three stakeholder groups: model developers, benchmark designers, and end users.

For model developers the primary recommendation is to report compositional generalization metrics alongside standard accuracy benchmarks. Rather than stating that a potential “works well for HEAs,” developers should publish error ratios, failure rates, and generalization gaps for each of the four dimensions. This transparency will accelerate progress by highlighting which architectures—equivariant graph networks [7], universal graph frameworks [8], or hybrid descriptors—naturally generalize better. Developers should also redesign training sets to maximize coverage of element pairs and random configurations rather than relying solely on equiatomic SQS structures [13, 14].

Benchmark designers can use the framework to create standardized HEA generalization suites. By publishing pre-defined training and test splits that embody the four dimensions, the community can establish common reference sets analogous to those already used for ordered compounds. Inclusion of both SQS and random configurations will become a default requirement, elevating the quality of future ML potential evaluations [6, 11, 28].

For practitioners selecting a potential for alloy design campaigns, the framework supplies a simple decision checklist. Before deploying a model on novel compositions, practitioners should verify that the relevant dimensions have been tested and that generalization gaps remain within acceptable bounds. A potential that has not been subjected to this assessment should be treated with caution, especially when the target composition lies far from the training distribution [1, 15].

Collectively these implications shift the culture of HEA ML potential development from case-by-case validation to systematic, reproducible assessment. Over time the framework will reduce wasted effort on models that appear accurate in narrow tests yet fail in real design scenarios, accelerate discovery of truly generalizable potentials, and increase confidence in ML-driven HEA screening.

High-entropy alloys and multi-principal element alloys present an enormous compositional space whose exploration demands ML potentials capable of reliable extrapolation. Yet current benchmarks and validation practices rarely assess the very generalizability that is most critical for practical alloy design. This conceptual framework addresses that gap by defining four distinct dimensions of compositional generalizability—element extrapolation, concentration interpolation, multi-element recombination, and local environment diversity—and by organizing assessment around four interlocking components: training-set characterization, test-set design, dimension-specific metrics, and a structured validation protocol.

Explicit validation strategies for each dimension translate the framework into an actionable checklist that any research group can adopt without new experimentation. By reporting generalization gaps separately rather than as a single performance number, the framework reveals precise model weaknesses and guides targeted improvements in training data and architecture. Its relation to existing benchmarks is complementary: it adds the HEA-specific layer that ordered-compound suites lack. Interpretation guidelines clarify what constitutes success or failure, while acknowledged limitations ensure realistic application.

The implications are immediate and far-reaching. Model developers gain diagnostic clarity, benchmark curators obtain a blueprint for HEA-specific suites, and practitioners receive a defensible basis for model selection. Widespread adoption of this framework will elevate compositional generalization from an afterthought to a core reporting requirement, ensuring that ML potentials fulfill their promise for the discovery and design of next-generation complex concentrated alloys. The community is therefore urged to integrate these assessment practices into standard workflows so that future HEA ML potentials are not only accurate but demonstrably generalizable across the vast alloy landscape.

None

None

None

None

Ali Hassan, Noor Siddiqui, Bilal Khan & Sana Malik contributed to this work.

Authors and affiliations

Department of Materials Data Science and Engineering, Faculty of Engineering, Aga Khan University, Karachi, Pakistan
Ali Hassan, Noor Siddiqui & Sana Malik

Department of Computational Materials Analytics, Faculty of Technology, Qatar University, Doha, Qatar
Bilal Khan

Corresponding author

Correspondence to Ali Hassan

Open Access The author(s) retain copyright. This article is licensed under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License. It may be shared and adapted for non-commercial purposes with appropriate attribution, an indication of changes, and distribution of adaptations under the same license. Third-party material may be subject to separate terms identified in its credit line. View the license at https://creativecommons.org/licenses/by-nc-sa/4.0/.