CS5720 - LSTM Architecture: The Cell State

LSTM Innovation

The Key Breakthrough:

LSTMs introduce a dual-state architecture that separates:

• Cell State (C_t): Long-term memory storage
• Hidden State (h_t): Short-term working memory

🏗️ Dual-State Design

Two separate information streams enable both short-term processing and long-term memory preservation.

🛤️ Information Highway

The cell state acts as a highway where information can travel across time with minimal interference.

Core Components

💜 Cell State (C_t)

The main memory component that carries information across time steps with minimal modification.

💚 Hidden State (h_t)

The output representation that combines cell state information with current input.

🚪 Three Smart Gates

Forget, Input, and Output gates control information flow with learned parameters.

Interactive LSTM Cell Architecture

x_t
Input

↓

LSTM Cell

f_t

i_t

o_t

Cell State: C_t

↓

h_t
Output

🧠 Selective Memory

Gates learn to selectively remember, forget, and output information based on context and importance.

🛣️ Gradient Highway

Cell state provides a direct path for gradients, preventing vanishing gradient problems.

⚡ Adaptive Processing

Each gate adapts its behavior based on current input and previous state, enabling context-aware processing.

LSTM Architecture: The Cell State

LSTM Innovation

Core Components

Interactive LSTM Cell Architecture

Modal Title