Yale professor John Onofrey, in this session from the Introduction to Medical Software course, examines how artificial intelligence and machine learning complicate the standard medical software lifecycle. He contrasts AI/ML development with traditional software engineering, focusing on the outsized role of training data quality and reproducibility. Onofrey walks through risk factors specific to these systems, including distributional shift, adversarial attacks, and performance drift over time, and argues that AI/ML components should be treated as inherently unsafe elements requiring robust surrounding system design. He covers failure mitigation strategies such as human in the loop review and explains why deployed models need continuous post market monitoring rather than one time validation. The sixteen minute lecture is dense with practical engineering concerns rather than theory, aimed at students who need to build clinically deployable systems around imperfect models.