How RNNs Process Sequences
An RNN forward pass processes sequences one element at a time, maintaining a hidden state that carries information from previous time steps.
Key Components:
• Input sequence - x₁, x₂, x₃, ... xₜ
• Hidden states - h₀, h₁, h₂, ... hₜ
• Weight matrices - Wₓₕ, Wₕₕ, Wₕᵧ
• Output sequence - y₁, y₂, y₃, ... yₜ
🔄 Recurrent Connection:
The hidden state from time t-1 is fed back as input at time t, creating the "memory" of past information.
Forward Pass Steps
-
1️⃣
Initialize Hidden State
Start with h₀ = 0 (usually zeros)
-
2️⃣
Process First Input
Combine x₁ and h₀ to compute h₁
-
3️⃣
Generate Output
Compute y₁ from hidden state h₁
-
4️⃣
Repeat for Sequence
Continue for x₂, x₃, ... until sequence end
-
5️⃣
Final Output
Collect outputs y₁, y₂, ... yₜ