The integration of artificial intelligence (AI) into materials science has ushered in an era of semi-autonomous systems that accelerate discovery through predictive modeling, high-throughput screening, and adaptive experimentation. These systems offer substantial promise for addressing global challenges in energy, sustainability, and advanced manufacturing; however, their reliance on data-driven inference introduces risks related to bias propagation, epistemic uncertainty, and misalignment with scientific values. Conventional approaches treat human oversight primarily as an external corrective mechanism—post hoc monitoring or intervention in response to model outputs. This paper proposes a conceptual reframing wherein human oversight is repositioned as an intrinsic element of system design. Rather than viewing control as supervision layered atop an autonomous core, oversight is conceptualized as deliberate architectural choices that embed human judgment into the foundational structure of semi-autonomous materials AI. Drawing on literature from materials informatics, data bias mitigation, explainable AI, and human-AI collaboration, the proposed framework delineates three interdependent dimensions: epistemic boundary-setting, value-aligned modulation, and adaptive reflexivity. This reframing shifts the discourse from mitigating human absence to engineering human presence, fostering systems that are inherently more robust, interpretable, and aligned with the normative goals of scientific inquiry. By reconceptualizing oversight as design, the framework offers a pathway to responsible integration of AI in materials discovery without presupposing full autonomy or diminishing human agency.