Search

Transfer learning has become a dominant paradigm for addressing the persistent small-data limitation in computational materials science, particularly through pre-training graph neural networks on large-scale repositories and subsequent fine-tuning on task-specific datasets. This strategy assumes that universal atomic-scale representations—such as local coordination environments, bonding characteristics, and elemental embeddings—transfer effectively across domains. However, accumulating empirical evidence reveals a systematic and reproducible failure: in small-data regimes, fine-tuned models frequently underperform both zero-shot pre-trained models and models trained from scratch. This work presents a comprehensive failure-mode analysis demonstrating that such collapse is not incidental but structurally inevitable under common conditions in materials applications. Four interdependent mechanisms are identified—catastrophic forgetting, negative transfer, representation distortion, and overfitting amplification—arising from irreducible source–target domain mismatch, feature entanglement in equivariant graph architectures, and the statistical insufficiency of target datasets typically containing fewer than 10³ samples. Under these constraints, the corrective signal provided by target data is inadequate to overcome entrenched source biases or to reconfigure internal representations. Drawing exclusively on consolidated literature, we show that transfer learning failure emerges when domain similarity falls below critical thresholds and when data scarcity prevents stable optimization. The study further formalizes operational diagnostics—including baseline benchmarking, source-validation tracking, and learning-rate sensitivity analysis—and introduces mitigation strategies such as progressive unfreezing, elastic weight consolidation, replay-based regularization, and similarity-gated transfer decisions. By reframing transfer learning as a high-risk, condition-dependent strategy rather than a default solution, this work provides a principled foundation for robust model deployment in materials informatics. It challenges prevailing assumptions of universal transferability and advocates for systematic failure-mode reporting to ensure reliability in machine-learned interatomic potentials and property prediction workflows.

Lifelong learning, also known as continual or sequential fine-tuning, promises machine-learning models for materials science that continuously improve as new data streams arrive. In theory, a model trained on binary-alloy formation energies should seamlessly incorporate new knowledge about ternary systems, experimental band gaps, or updated DFT databases without losing prior accuracy. Yet in practice, lifelong learning systematically fails for materials property prediction. Sequential fine-tuning triggers catastrophic interference: the model rapidly forgets earlier tasks while attempting to master new ones. This failure mode analysis identifies four core failure modes that are especially destructive in the materials domain. Catastrophic forgetting occurs when shared parameters encoding old task knowledge are overwritten during fine-tuning on new tasks. Negative interference arises when conflicting signals from different data sources force the model into an impossible compromise. Representation overwriting destroys generic feature detectors in early layers that were useful across multiple materials tasks. Task imbalance allows later, larger, or easier datasets to dominate the optimization process, marginalizing earlier knowledge. The root causes are intrinsic to materials science: massive domain shifts between compositional spaces, overlapping but non-identical feature distributions, and the optimizer’s inability to preserve multi-task optima in a shared parameter space. These problems are documented across recent studies of machine-learning interatomic potentials, continual learning in mechanical networks, and sequential task learning on evolving materials databases. This paper provides precise detection signatures for each failure mode and outlines mitigation principles that can be applied immediately by practitioners. By exposing why sequential fine-tuning collapses in materials property prediction, the analysis lays the foundation for genuinely lifelong materials models that accumulate knowledge rather than repeatedly erasing it.

Foundation models and large language models (LLMs) are rapidly entering materials science, offering new interfaces for property prediction, synthesis planning, and literature mining. This review synthesizes some peer-reviewed publications, focusing on foundation models pre-trained on crystals, molecules, and text, as well as LLM-based approaches for materials tasks. Three primary use cases emerge: (1) property prediction from compositional or structural descriptions, (2) synthesis recipe generation, and (3) extraction of structured data from scientific text. Methods span zero-shot prompting, few-shot prompting, chain-of-thought reasoning, retrieval-augmented generation, full fine-tuning, parameter-efficient fine-tuning, and embedding-based adaptation. What transfers effectively includes generic chemical knowledge, structure–text relationships, qualitative trends, similarity search, and literature extraction. In contrast, what does not transfer includes quantitative property prediction to experimental accuracy, extrapolation beyond pre-training distributions, crystal stability assessment, physics-based reasoning, and hallucination-free synthesis planning. Despite promising demonstrations in common materials, LLMs and foundation models still lag behind specialized graph neural networks in quantitative tasks and fail on compositional or structural novelty. This review provides a systematic taxonomy of applications, a critical analysis of prompting and fine-tuning strategies, and a clear delineation of transfer limitations. Gaps remain in uncertainty quantification, multimodal data scarcity, and rigorous benchmarking against non-LLM baselines. Recommendations for practitioners and developers emphasize realistic expectations and hybrid human–AI workflows to accelerate materials discovery without over-reliance on ungrounded predictions.