Lesson 5: Opening the Black Box of a Neural Network

Micro-Topic 5.1: The “Black Box” Problem

Goal: Acknowledge and explain why neural networks are often called “black boxes” – it’s hard to know what exactly is happening inside – and why that can be a problem (or a challenge to overcome).

Micro-Topic 5.2: Inside a Simple Neural Network

Goal: Walk through a very small neural network manually to show how data flows and gets processed. Use a simple example with, say, 2 inputs, 2 hidden neurons, and 1 output neuron (a tiny network) to illustrate calculations of a forward pass.

Micro-Topic 5.3: Implementing the Green Screen ML Solution

Goal: Return to the green screen example from Lesson 1, but now actually outline a simple neural network model to perform it. Show how we would set up inputs (pixel color values), outputs (background vs foreground), and how the network would figure out the task, thereby reinforcing understanding of a practical neural net.

Micro-Topic 5.4: Debugging a Neural Network (Finding the Bug)

Goal: Illustrate how one might go about diagnosing issues in a neural network’s performance. This ties into “opening the black box” by not just peeking at weights, but using systematic approaches to find where a network might be going wrong. Use a hypothetical scenario (perhaps a network isn’t learning as expected, or it’s making a silly mistake) and describe debugging steps.

Micro-Topic 5.5: Interpreting Neural Networks – Towards Explainable AI

Goal: Briefly discuss methods or approaches that help interpret neural networks (connecting to the idea of opening the black box). This might include mention of feature importance for networks, visualization of convolutional filters or attention maps, etc., at a high level. Emphasize that while hard, it’s an active area so students appreciate efforts to make AI more transparent.