Multi-fidelity modeling has become an indispensable paradigm in artificial intelligence for materials science, offering a structured way to integrate data from simulations of varying computational expense and accuracy to accelerate the discovery and optimization of novel materials while mitigating the prohibitive costs associated with high-fidelity methods alone. This review systematically examines the conceptual foundations, underlying assumptions, and inherent trade-offs of multi-fidelity approaches through a targeted analysis of 30 peer-reviewed publications published between 2017 and 2023, identified via a rigorous literature search across databases such as Web of Science and Scopus that employed the exact search strings specified in the reference discovery protocol. The conceptual foundations rest on the hierarchical organization of fidelity levels, wherein low-fidelity models deliver rapid, broad-coverage approximations that serve as scaffolds for correction and refinement by higher-fidelity calculations through surrogate-based information transfer, thereby enabling efficient navigation of high-dimensional material design spaces. Key assumptions—such as the presence of meaningful correlation and smoothness between fidelity outputs, as well as linearity in the mapping between them—are scrutinized alongside the trade-offs they impose between computational cost, predictive accuracy, generalization capacity, and uncertainty handling. Methods ranging from Gaussian process co-kriging to neural network transfer learning are conceptually surveyed for their role in bridging fidelity gaps. At the same time, materials-specific applications in alloys, polymers, and interfaces illustrate both demonstrated successes and context-dependent limitations. Significant gaps persist in the literature, notably the infrequent validation of core assumptions and the absence of standardized benchmarks for multi-fidelity tasks, prompting recommendations for explicit assumption testing, quantitative trade-off reporting, and community-driven development of open benchmarks and reporting standards to elevate the rigor of multi-fidelity materials AI. Through this structured examination, the review underscores that while multi-fidelity frameworks hold transformative potential, their conceptual maturity requires sustained critical attention to assumptions and trade-offs if they are to support next-generation materials innovation reliably.