Dropout randomly sets some neurons to zero during training, forcing the network to be robust and preventing over-reliance on specific neurons.
Key Benefits:
• Prevents neurons from co-adapting too much
• Forces the network to learn redundant representations
• Reduces overfitting dramatically
• Improves generalization to new data
🎭 Think of it like...
A sports team where random players are benched each game. The team learns to work well even when key players are missing, making them more resilient overall!
Training vs Inference
🎲 Training Mode
Randomly drop neurons
(e.g., 20-50% of them)
Forces learning redundancy
🧠 Inference Mode
Use ALL neurons
Scale outputs appropriately
Get full network prediction
Critical Point:
Dropout is ONLY active during training. During inference, all neurons participate in the final prediction.
Interactive Dropout Visualization
Input Layer
Hidden Layer 1
Hidden Layer 2
Output Layer
Dropout Rate:30%
Mode:
🎲 Training Mode: Neurons will be randomly dropped during forward pass