In the rapidly evolving domain of artificial intelligence for materials science, exploitation-driven approaches that prioritize the optimization of predefined objective functions—such as targeted properties, minimal prediction error, or high accuracy—have come to dominate the field, systematically constraining exploration to regions of chemical space that are already anticipated to yield incremental gains while leaving vast swaths of potentially transformative materials undiscovered. Yet, the history of scientific progress, from the serendipitous isolation of novel compounds to paradigm-shifting insights into structure-property relationships, demonstrates that genuine discovery is propelled not solely by goal-directed pursuit but by a deeper, intrinsic drive toward the unknown, a principle that has begun to find formal expression in artificial intelligence through concepts of intrinsic motivation, novelty-seeking, and curiosity-driven algorithms. This paper proposes algorithmic curiosity as a foundational design principle for materials AI: an AI system architecture that actively seeks novelty, uncertainty, surprising patterns, and unexplored regions of materials space, irrespective of immediate utility or alignment with pre-specified rewards. It articulates five core components—novelty detection, uncertainty seeking, surprise maximization, coverage maximization, and prediction error seeking—alongside four operational principles that translate this philosophical shift into practical system behavior. Finally, the proposal delineates a phased implementation path for the broader materials AI community, offering a blueprint that promises to rebalance the exploration-exploitation trade-off and restore the spirit of open-ended scientific inquiry at the heart of materials discovery. By elevating curiosity from a peripheral heuristic to a central architectural imperative, this framework aims to unlock scientific breakthroughs that goal-optimized systems are structurally incapable of anticipating.