Memory and State for Production Agents: Beyond the Context Window

The context window is not a memory system, any more than a CPU register is a database. It's where the model thinks, not where the product remembers. Conflating the two is the root of most memory bugs I've debugged: the conversation that evaporates when a pod restarts, the agent that greets a two-year customer like a stranger, the compliance officer discovering that "chat history" lives in a Redis instance with a 24-hour TTL.

I gave memory one section in my production agents post. It deserves more — in every long-running deployment I've worked on, voice agents especially, memory and state architecture ended up mattering more than model choice.

Two things people call "memory" that are nothing alike

Untangle these first, because they have opposite requirements:

Conversation state is everything about the current interaction: the turns so far, tool results, the task in progress, where we are in the flow. It's hot, mutable, accessed on every request, and mostly worthless a week later. It wants a fast session store and aggressive lifecycle management.

Durable memory is what should survive the conversation: the customer prefers email over phone, their account tier, the unresolved issue from last month, the fact that they already tried restarting the router. It's written rarely, read at session start, and its failure mode isn't latency — it's wrongness. A stale or false durable memory poisons every future conversation.

Different data, different stores, different retention rules. Systems that get memory right keep this boundary sharp; the ones that don't end up with one blob called context that is simultaneously too big to send to the model and too lossy to trust.

Conversation state: design for the crash

Here's the test I apply to any agent architecture: kill the process mid-conversation. What does the user experience? If the answer is "the agent forgets everything," your conversation state lives in application memory, and you've built a demo.

The production shape:

Externalize the session. Every turn, tool call, and tool result is appended to a session store — Redis with persistence, Postgres, whatever fits your latency budget — keyed by conversation ID. The agent process is a stateless worker that hydrates from the store, runs the loop, and writes back. This is just twelve-factor discipline applied to agents, and it buys you crash recovery, horizontal scaling, and multi-channel resume (start on voice, drop, continue in chat) in one move.
Store the log, derive the view. Persist the full append-only event log as the source of truth; build the model-facing context from it on each turn. The log never lies; the view can be re-derived, re-summarized, or re-formatted when your prompt strategy changes. If you store only the compacted context, every summarization mistake is permanent.
Give sessions an explicit lifecycle. Active, idle, resumable, expired — with real timers. A voice caller who drops mid-booking and calls back within ten minutes should land in the same session. The same caller next Tuesday should not.

Summarization: compression with a paper trail

Long conversations will outgrow any budget you set, and — as I said in the production post — summarize, don't truncate. But how you summarize is where quality lives:

Keep the recent turns verbatim — they carry the immediate intent — and compress everything older.
Summarize into structure, not prose. A freeform "summary of the conversation so far" degrades with every recompression, like a photocopy of a photocopy. A structured digest resists that:

{
  "goal": "reschedule delivery for order #8841",
  "facts": {
    "address_confirmed": true,
    "preferred_window": "weekday mornings",
    "previous_attempt_failed": "2026-06-28, nobody home"
  },
  "decisions": ["waived redelivery fee per policy"],
  "open": ["confirm Thursday 9-11 slot with carrier"]
}

Some things are exempt from compression. Explicit commitments ("I'll waive the fee"), amounts, dates, and safety-relevant statements get pinned verbatim. The summarizer's job description includes a list of things it may never paraphrase — because "the agent forgot what it promised" is the memory failure users forgive least, and paraphrase is where promises die.
Log every summarization event. When a conversation goes wrong after compaction, you want to see exactly what the summary dropped. This has been the smoking gun in more debugging sessions than I can count.

Durable memory: write less than you think

The tempting design is "remember everything about the user forever." The correct design is much narrower, because durable memory is only as good as its accuracy — and every fact you store is a fact that can go stale.

What earns a durable write, in my deployments:

Stable preferences the user stated or clearly demonstrated (language, channel, contact windows).
Relationship facts with long half-lives: account context, past issues and their resolutions, commitments made.
Explicit "remember this" requests — which, note, imply the mirror obligation to honor "forget this."

What doesn't: transient moods, inferred speculation, anything the system of record already owns. If the CRM knows the customer's plan, query the CRM — a cached copy in agent memory is a consistency bug on a timer. Durable memory should store what no other system knows, and reference the rest.

Two mechanics that matter more than the storage engine:

Extract at session end, not mid-flight. A post-conversation pass reviews the transcript and proposes memory writes against a schema — structured outputs, validated like any other model output, ideally with provenance: which conversation, which turn, when. Memories with citations can be audited and corrected; vibes cannot.
Give every memory a decay policy. Confidence that degrades with age, re-confirmation on next use ("last time you preferred morning slots — still true?"), and hard expiry for anything volatile. An agent confidently acting on a preference from eighteen months ago is worse than an agent that asks.

The compliance layer: what you persist when it matters

Enterprise deployments add a third memory category nobody puts in architecture diagrams until legal does: the record. Not for the model — for auditors, regulators, and dispute resolution. Build it in from day one; retrofitting an audit trail is archaeology.

What the record needs that the model's memory doesn't:

Full-fidelity, append-only transcripts — every turn, every tool call with arguments and results, every prompt version and model ID in play. When a customer disputes "the agent told me X," you replay the exact exchange, not a summary of it.
Decision provenance. For any consequential action — a refund, a booking, an escalation — persist why: the tool call, the inputs, the policy or approval path it went through.
Retention and deletion as first-class features. GDPR-style erasure means a user's data must be deletable across the session store, durable memory, and whatever your logs and traces captured — which is a strong argument for keeping PII in referenced records rather than smeared through prompt text. Retention clocks differ per data class: transcripts might keep 90 days, financial decision records seven years, voice recordings whatever the consent covered.
Separation from the hot path. The record is written asynchronously to cheap, immutable storage. It must never be a latency tax on the conversation — and the conversation store must never be your only copy of anything legally interesting.

The uncomfortable question to ask early: if this conversation ends up in a dispute two years from now, what will we wish we had kept — and what will we wish we hadn't? Both halves matter.

The state machine around the loop

Last piece, and the one that ties memory to control: the agent loop should not be the outermost layer of your system. Wrap it in an explicit state machine that owns the conversation's phase — greeting → identify → task → confirm → wrap_up, or whatever your domain demands.

Why bother, when the model could infer the phase from context?

State gates capability. The refund tool simply isn't in the tool list until the machine says we're in confirm with an identified customer. That's a guarantee; a prompt instruction is a suggestion.
State survives what the context window doesn't. After summarization, after a resume, after a channel switch, the machine still knows exactly where the conversation stands — no re-inference, no drift.
State is observable. "40% of sessions stall in identify" is an actionable dashboard. Diffing transcripts to guess the same thing is not.
Transitions are where your invariants live. You cannot reach wrap_up from task without passing confirm. The model drives within a state; deterministic code decides between them.

The model is a brilliant improviser and an unreliable bookkeeper. The state machine does the bookkeeping so the model can improvise safely. Every mature agent system I've seen converges on this shape — the only question is whether it's designed up front or excavated from incident reports.

The takeaway

Memory is not one thing. Conversation state wants a fast external session store and a crash-proof event log. Working context wants structured summarization with a paper trail. Durable memory wants schema-validated writes, provenance, and decay. The compliance record wants immutability and deletability, held in deliberate tension. And around all of it, a state machine that remembers what phase you're in even when the model doesn't. Get those boundaries right and long-running agents stop being fragile. Blur them and every restart, resume, and audit becomes an adventure.

Related reading

Building Production AI Agents in 2026
Structured Outputs: The Contract Between Your Agent and Everything Else
The voice and agent systems these patterns come from.