Semantic Memory

Level: Intermediate · Reading time: 6 min · Phase: 4 · Agentic AI

In one line: Semantic memory stores stable facts (about the user, the world, or the domain) that stay true across conversations, such as "Asha's timezone is IST".

Before you start: Long-term Memory. You should know that agents save information outside the model and load it into the prompt later.

Explain it like I'm new

You know that Paris is the capital of France, but you probably don't remember the day you learned it. That kind of knowledge, facts without the story of how you got them, is what psychologists call semantic memory. Remembering a specific event ("my trip to Paris last May") is episodic memory (next topic).

For an agent, semantic memory is a profile or fact sheet:

• About the user: name, role, language, preferences ("prefers short answers").
• About the domain: "Invoice IDs start with INV-", "Our refund window is 30 days".
• About the environment: "The prod database is read-only for this agent".

These facts are small, structured, and useful on almost every turn, so many systems load them into every prompt rather than searching for them.

Words to know:

• Fact: a small piece of information that stays true, like a name or a rule.
• Structured: stored in labelled fields (a key and a value) rather than as a free paragraph.
• Key: the label you look a fact up by, here (subject, relation).
• Overwrite: replace the old value with a new one under the same key.
• Prompt: the text sent to the model on each call.

Watch it

It plays on a loop. Read the caption in the box, one step at a time. Want to go at your own pace? Step through it with Previous and Next.

How it works

1. Represent facts in a structured form, for example (subject, relation) -> value: ("user", "timezone") -> "IST".
2. Write: add facts when the user states them or when an extraction step finds them. If a fact already exists, overwrite (or keep history with timestamps).
3. Read: load all facts about the current user (usually small), or query by subject.
4. Resolve conflicts: newer explicit statements beat older or inferred ones. Keep the source.
5. Expose controls: let users view, correct, and delete what's stored.

subject   relation    value          source          updated
-------   ---------   ------------   -------------   ----------
user      name        Asha           user said       2026-10-01
user      timezone    IST            user said       2026-10-01
user      city        Delhi          user said       2026-10-20   <- replaced "Pune"
domain    refund_days 30             policy doc      2026-09-15

Worked example

This traces the SemanticMemory class from the code below.

1. The user says they live in Pune. The agent runs set("User", "city", "Pune"). The key is stored in lower case: ("user", "city").
2. The user later says they moved to Delhi. set("user", "city", "Delhi") uses the same key, so Delhi replaces Pune. No contradictory copy is left.
3. set("user", "timezone", "IST") adds a second fact.
4. get("USER", "City") finds Delhi because lookups are case-insensitive. about("user") returns {'city': 'Delhi', 'timezone': 'IST'}.

The demo in agent_memory.py does the same with a name and timezone, and prints {'name': 'Asha', 'timezone': 'IST'}. That dict is pasted into the prompt as Known facts about user: ....

Code example

From agent_memory.py:

class SemanticMemory:
    def __init__(self):
        self.facts = {}

    def set(self, subject, relation, value):
        self.facts[(subject.lower(), relation.lower())] = value   # newer overwrites older

    def get(self, subject, relation, default=None):
        return self.facts.get((subject.lower(), relation.lower()), default)

    def about(self, subject):
        s = subject.lower()
        return {rel: v for (subj, rel), v in self.facts.items() if subj == s}

sem = SemanticMemory()
sem.set("User", "city", "Pune")
sem.set("user", "city", "Delhi")          # the user moved
sem.set("user", "timezone", "IST")
print(sem.get("USER", "City"))            # Delhi
print(sem.about("user"))                  # {'city': 'Delhi', 'timezone': 'IST'}

To let the model write facts itself, give it a tool like remember_fact(subject, relation, value) with a strict schema, and validate it in code (Function Calling).

When to use it / when not to

• Use for personalization (preferences, profile) and domain rules the agent needs constantly.
• Use structured storage (key-value or a table), not free text, so facts can be updated and deleted cleanly.
• Don't store sensitive personal data unless you need it and have consent and a retention policy.

Common mistakes

• Storing facts as free text that can't be updated ("user lives in Pune" and later "user lives in Delhi" both remain).
• Letting the model infer and save guesses ("user is probably a manager") as facts.
• No way for users to see or delete their stored facts.
• Loading a huge fact sheet into every prompt.

Recap

If you remember only three things:

• Semantic memory holds stable facts, such as preferences and rules, without the story of how you learned them.
• Store facts as key to value so a new value overwrites the old one.
• Let users see and delete their facts, and never save guesses as facts.

Interview questions

1. What is the difference between semantic and episodic memory in agents? <details><summary>Answer</summary> Semantic memory holds general facts that stay true (preferences, domain rules). Episodic memory holds records of specific past events: what was attempted, when, and how it turned out. </details>

2. How do you handle a fact that changes? <details><summary>Answer</summary> Store facts with a unique key so a new value overwrites the old one (or versions it with a timestamp and source), and prefer explicit user statements over inferred values. </details>

3. Why prefer structured storage for semantic memory? <details><summary>Answer</summary> Structured keys make facts easy to update, delete, audit, and load exactly. Free-text memories duplicate and contradict each other over time. </details>

Go deeper

• Cognitive Architectures for Language Agents (CoALA, Sumers et al., 2023)
• LangGraph documentation

Prev: Long-term Memory · Next: Episodic Memory · Back to phase: Phase 4