Section outline

  • Lesson Goal: Explore the counterintuitive phenomenon in modern ML where using massively more parameters than data (over-parameterization) can actually improve performance. Students will learn what over-parameterization means, why traditionally it was feared due to overfitting, and how deep learning defied expectations through phenomena like double descent, leading to new understanding of generalization.

    • ·         Micro-topic 23.1: Overfitting Refresher – Why Too Many Parameters Can Be Problematic (Goal: Make sure students understand the classical view: more parameters → higher risk of overfitting, and what overfitting entails)

    • L23 MT1 FREE Overfitting Refresher SCORM12 SCORM package
      • Micro-topic 23.2: Deep Learning Revolution – Big Models, Big Data (Goal: Explain how circa 2012-2015, deep learning started using extremely large networks on large datasets, challenging the conventional wisdom on overfitting)
    • L23 MT2 FREE Deep Learning Revolution SCORM12 SCORM package
      • Micro-topic 23.3: The “Double Descent” Phenomenon (Goal: Introduce “double descent” – the discovery that beyond the classical overfitting point, increasing model size can lead to improved test performance again, and discuss why this happens)
    • L23 MT3 FREE Double Descent SCORM12 SCORM package
      • Micro-topic 23.4: Why Over-Parameterization Can Work – Intuitions and Implications (Goal: Give intuitive reasons and summarize implications of over-parameterization success: e.g., implicit regularization, easier optimization, and how this shapes strategy in building ML solutions)
    • L23 MT4 FREE Why Overparameterization Works SCORM12 SCORM package
      • Micro-topic 23.5: Embracing Over-Parameterization – Your Black Belt Advantage (Goal: Tie the concept back to the course theme: how understanding when and why over-parameterization works allows our students to outperform or effectively utilize AI, rather than fear complex models)
    • L23 MT5 FREE Black Belt Advantage SCORM12 SCORM package