CS5720 - Week 8
Slide 157 of 160

Pruning Neural Networks

What is Network Pruning?

Network pruning is a model compression technique that removes redundant or less important weights, connections, or entire neurons from a trained neural network to reduce its size and computational requirements.
Key Concepts:

Weight pruning - Remove individual connections
Neuron pruning - Remove entire neurons/channels
Structured vs Unstructured - Pattern of removal
Magnitude-based - Remove smallest weights
🌳 Think of it like...
Pruning a neural network is like trimming a tree - you remove the branches (weights) that don't contribute much to the tree's health, making it more efficient while maintaining its core function.

Benefits of Pruning

  • 📦
    Model Size Reduction
    Significantly smaller file sizes for storage and transfer
  • Faster Inference
    Fewer computations mean faster predictions
  • 🧠
    Memory Efficiency
    Lower memory footprint for deployment
  • 🔋
    Energy Savings
    Reduced power consumption on mobile devices

Interactive Pruning Demonstration

Original Network
Pruned Network
Pruning Ratio
30% weights pruned
70% Parameters Remaining
1.4x Speed Improvement
-2.1% Accuracy Change
Prepared by Dr. Gorkem Kar