The integration of artificial intelligence (AI) and machine learning (ML) into materials science has accelerated the discovery and design of novel materials by enabling high-throughput prediction of properties from composition, structure, and processing parameters. However, the reliability of these predictions is frequently compromised by uncertainties stemming from limited datasets, model approximations, experimental noise, and intrinsic variability in materials systems. This narrative review synthesizes recent advances in understanding uncertainty and reliability in materials AI. It covers fundamental concepts such as aleatoric and epistemic uncertainty; methods for quantification, including Bayesian neural networks, ensembles, and Gaussian processes; inconsistencies in terminology and language across the literature; and the downstream consequences for decision-making in materials engineering, design, and deployment. Emphasis is placed on calibration of uncertainty estimates, domain-of-applicability assessment, and risk-aware applications in safety-critical contexts such as structural alloys and energy materials. By highlighting best practices and gaps, the review advocates for standardized frameworks to build trust and facilitate industrial translation of materials AI. Key challenges include data scarcity in high-performance materials and the need for physics-informed UQ to mitigate overconfidence in extrapolative predictions. This synthesis underscores the importance of robust uncertainty handling for responsible AI deployment in materials innovation.