CS5720 - Week 8
Slide 141 of 160

Autoencoders: Unsupervised Learning

What are Autoencoders?

An autoencoder is a type of neural network that learns to compress data into a smaller representation and then reconstruct it. It learns to copy its input to its output while discovering useful patterns in the data.
Key Characteristics:

Unsupervised learning - no labels needed
Self-supervised - the input is also the target
Dimensionality reduction - compresses information
Feature learning - discovers meaningful representations
🎨 Think of it like...
An autoencoder is like an artist who must paint a portrait using only a few brush strokes. They learn to capture the essential features while discarding unnecessary details.

Key Components

  • 🔽
    Encoder
    Compresses input into a compact representation
  • 🔒
    Bottleneck/Latent Space
    The compressed representation of the data
  • 🔼
    Decoder
    Reconstructs the original input from compression
Remember:
The magic happens in the bottleneck - it forces the network to learn the most important features!

Autoencoder Architecture Visualization

Input
(784 dims)
x₁
x₂
...
xₙ
Encoder
(128 dims)
h₁
h₂
...
Bottleneck
(32 dims)
z₁
z₂
Decoder
(128 dims)
d₁
d₂
...
Output
(784 dims)
x̂₁
x̂₂
...
x̂ₙ
Goal: Minimize reconstruction error: ||x - x̂||²
Prepared by Dr. Gorkem Kar