CS5720 - Week 3
Slide 42 of 60

Universal Approximation Theorem (Intuitive)

The Theorem

Universal Approximation Theorem
A neural network with a single hidden layer containing a finite number of neurons can approximate any continuous function to arbitrary accuracy, given appropriate activation functions and sufficient neurons.
What does this mean in plain English?

Neural networks are like universal function approximators - they can learn to mimic ANY continuous function, just like how a skilled artist can draw any shape with enough pencil strokes!

The Power

🎨 The Artist Analogy
Think of neurons as brushstrokes. With enough brushstrokes of different sizes and colors, an artist can paint any scene. Similarly, with enough neurons, a network can "paint" any function!
This theorem tells us that neural networks are incredibly flexible. They're not limited to learning specific types of patterns - they can learn ANY pattern that exists in your data.

The Catch

⚠️ But There's a Problem...
Just because you CAN approximate any function with a single hidden layer doesn't mean you SHOULD. It might require millions of neurons! Deep networks are often more practical.
The theorem doesn't tell us:
• How many neurons we need
• How to find the right weights
• Whether it's computationally feasible

Interactive Function Approximation

5
🧠
More Neurons = Better Fit
As you add neurons, the network can capture more complex patterns and approximate the target function more accurately.
📈
Width vs Depth
While a wide network CAN work, deep networks are often more efficient, learning hierarchical representations.
⚡
Practical Implications
This theorem gives us confidence that neural networks can solve our problems - if we design them right!
🎯
The Learning Challenge
The real challenge isn't whether a network CAN learn something, but HOW to train it efficiently.
← Previous Next →
Prepared by Dr. Gorkem Kar