State Space Models Explained: How Mamba Works and How It Compares to Transformers

Transformers dominate modern AI, but they carry a cost that grows quickly: attention compares every token with every other token, so doubling the context roughly quadruples the work, and generation needs a key-value cache that grows with every token. State space models (SSMs) take a different route. They process a sequence step by step while carrying a compact summary of the past, which keeps cost linear in sequence length and memory constant while generating. The Mamba family of models brought SSMs into the mainstream, and today many new language models combine SSM layers with attention.

This guide explains how SSMs work in plain English, with a tiny runnable example, then compares them honestly with transformers. If attention itself is new to you, start with transformer attention explained.

The core idea: a running summary

Imagine reading a long book while keeping notes on a single index card. After each sentence, you update the card: add what matters, cross out what no longer does. You never reread earlier pages; the card is your memory. That card is the state.

A state space model works the same way. At each step t it updates a hidden state h from the previous state and the new input x, then produces an output y from the state:

h[t] = A · h[t-1] + B · x[t]     // update the summary
y[t] = C · h[t]                   // read from the summary

• A controls how the old state carries forward, in other words how quickly things are forgotten.
• B controls how much of the new input enters the state.
• C controls what is read out of the state for the output.

The idea comes from control theory and signal processing, where state space models have described systems such as circuits and vehicles for decades. Deep learning versions learn A, B and C from data.

A tiny example you can run

// ssm.js  (run with: node ssm.js)
function runSSM(inputs, { a, b, c }) {
  let h = 0;
  return inputs.map((x) => {
    h = a * h + b * x;   // keep part of the old state, add the new input
    return Number((c * h).toFixed(3));
  });
}

const signal = [1, 0, 0, 0, 5, 0, 0, 0];

console.log('long memory  ', runSSM(signal, { a: 0.9, b: 1, c: 1 }).join(', '));
console.log('short memory ', runSSM(signal, { a: 0.3, b: 1, c: 1 }).join(', '));

Output:

long memory   1, 0.9, 0.81, 0.729, 5.656, 5.09, 4.581, 4.123
short memory  1, 0.3, 0.09, 0.027, 5.008, 1.502, 0.451, 0.135

With a = 0.9, earlier inputs fade slowly, so the state remembers for a long time. With a = 0.3, it forgets almost immediately. Real models use large vectors and matrices instead of single numbers, and many such state channels in parallel, but the principle is exactly this: a learned balance between remembering and forgetting, computed one step at a time.

Why that is efficient

	Attention (transformer)	State space model
Work for a sequence of length n	Grows with n² (every token compares with every token)	Grows with n (one update per token)
Memory while generating	KV cache grows with every token	Fixed-size state
Cost of the next token	Rises as the context grows	Constant
Access to the past	Can look back at any exact token	Only what the state kept

For very long inputs, such as long documents, audio, genomic sequences or long-running sessions, the linear cost and constant memory are a real advantage.

From S4 to Mamba

Early neural SSMs had a problem: simple recurrences were hard to train on long sequences and slow to run step by step on GPUs. S4 (2021) showed that carefully structured state matrices let SSMs capture very long-range patterns, and that the whole sequence could be processed in parallel during training, much like a convolution.

But S4’s parameters were the same for every input. It applied the same remembering and forgetting rules to an important name as to a filler word. Mamba (late 2023) fixed this with a selective state: B, C and the step size depend on the current input. The model can decide, token by token, “store this”, “ignore this” or “reset what I had”. That selectivity is what made SSMs competitive with transformers on language.

Selectivity breaks the convolution trick S4 used for fast training, so Mamba introduced a hardware-aware parallel scan: an algorithm that computes the recurrence for all positions in parallel on a GPU, while keeping the large intermediate states in fast on-chip memory.

Mamba-2 and the link to attention

Mamba-2 (2024) introduced “state space duality”: a mathematical framework showing that a restricted kind of SSM and a form of linear attention are two views of the same computation. That insight let Mamba-2 use matrix multiplications, which GPUs are extremely good at, making it considerably faster to train than the original Mamba, and it connected two research lines that had looked separate. Later work in the family has continued to refine quality and efficiency; see our Mamba 3 article for the newest generation.

Where SSMs struggle

A fixed-size state is both the strength and the weakness. Everything the model remembers must fit into it, so information is compressed and some is lost. Research has shown that pure SSMs are weaker than attention at tasks that need exact recall from long contexts: copying a long passage, finding a specific phone number mentioned pages earlier, or in-context learning from many examples. Attention can look back at every token exactly; an SSM can only consult its notes.

That is why claims that SSMs simply “beat transformers” oversimplify. They are faster and cheaper on long sequences, and they match transformers on many benchmarks at similar sizes, but they trade away precise access to the past.

Hybrids: the best of both

The practical answer that many labs have converged on is to combine them. Hybrid models use mostly SSM layers, which are cheap and handle long-range flow, with a small number of attention layers that provide exact recall where needed. Several open models released since 2024, from both research labs and companies, follow this pattern, often reporting faster inference and lower memory use than pure transformers of similar quality, especially on long contexts.

For application developers, this mostly appears as model choice: a hybrid model may handle long documents or long agent sessions with less memory and lower latency. Always test on your own tasks, because benchmark results don’t always transfer.

Do application developers need to care?

If you call models through an API, the architecture is hidden. It matters when you:

• Self-host models, where memory per request and throughput decide your GPU costs;
• Work with very long inputs, such as full codebases, long transcripts or streaming data;
• Need exact retrieval from long context, where attention-heavy models or a retrieval system, such as hybrid search for RAG, may serve you better;
• Run models on devices with limited memory, where constant-size state is attractive.

Selectivity, shown in code

The fixed example above treats every input the same way. Mamba’s key step is making the update depend on the input itself. A toy version shows the effect: the model decides, per token, whether to keep its memory or reset it.

// a toy "selective" update: a gate computed from the input decides how much to forget
function runSelective(tokens) {
  let h = 0;
  return tokens.map(({ value, important }) => {
    const keep = important ? 0.2 : 0.95;   // important input: mostly replace the old state
    const write = important ? 1 : 0.05;    // unimportant input: barely write it
    h = keep * h + write * value;
    return Number(h.toFixed(2));
  });
}

console.log(runSelective([
  { value: 9, important: true },   // a key fact
  { value: 1, important: false },  // filler words
  { value: 1, important: false },
  { value: 1, important: false },
]).join(', '));   // 9, 8.6, 8.22, 7.86: the key fact survives the filler

In a real model, nobody labels tokens as important. The gate values are computed by learned layers from each token, and training teaches the model which inputs deserve a place in its limited memory. That learned, input-dependent decision is what “selective” means.

Frequently asked questions