Dense Connectivity
DenseNet takes connectivity to the extreme: each layer receives inputs from all preceding layers and feeds into all subsequent layers.
The Core Idea:
• If skip connections help, why not connect everything?
• Layer L receives feature maps from layers 0, 1, 2, ..., L-1
• Maximum information flow and feature reuse
• Dramatic parameter reduction while maintaining performance
🕸️ Dense Connectivity Pattern
Each layer concatenates feature maps from all previous layers as input, creating a densely connected network structure.
🌱 Growth Rate Concept
Each layer adds a fixed number of feature maps (growth rate k), leading to linear growth in channel dimensions.
Key Benefits
-
⚡ Parameter Efficiency
Fewer parameters than ResNet while achieving better performance
-
🌊 Strong Gradient Flow
Direct connections to all previous layers eliminate vanishing gradients
-
♻️ Feature Reuse
All layers have access to all previous features, maximizing information usage
-
📡 Implicit Deep Supervision
Each layer receives direct supervision signal through shorter paths
-
🎨 Feature Diversity
Encourages exploration of different feature representations