The Human Connection
Humans have brains, and within them, we have both long-term and short-term memory. Because of this, humans have a natural tendency to remember factual knowledge, learn various skills, and perform specialized cognitive actions based on personal experiences.
But why am I talking about the human brain? Because an AI agent's memory is actually a lot like human memory. AI agents also need memory to do continuous work, and they need to remember important details to use in future sessions.
The CoALA Framework
Instead of just randomly throwing around a vague term like "agentic memory," this concept has been beautifully simplified through the CoALA (Cognitive Architectures for Language Agents) framework.
This framework divides memory into structured modules that help agents perform reliable and expected actions:
Working Memory
This is simply the context that is given through an active conversation in the form of {role: user} and {role: ai} messages. As messages get appended, the conversation size increases. Due to this increase in conversation length, there can be a significant drop in performance, or the AI might hallucinate.
This totally depends on the LLM's context window (which can be 1M - 10M tokens, and will likely increase in the future). The larger the context window is, the better the LLM's ability to handle those message objects.
Semantic Memory
This might sound technical, but it is actually just the part of the system prompt that provides general knowledge, concepts, and rules to the LLM. It operates independently, but this memory is always available throughout all messages. It is usually represented in a markdown file (like a Claude.md system prompt).
Procedural Memory
Even though an AI has access to a massive amount of internet data, best practices, and steps to do certain tasks, the fast-changing nature of tech stacks means it can sometimes fall behind. This has recently been solved through the concept of "skills."
These skills are often structured files that contain metadata, a body, and optional assets. The metadata is basically a title and a proper description, the body contains the rules or execution content, and the optionals are scripts or reference assets. Since there could be dozens of skill files, loading all of them would bloat the context memory. Instead, the agent uses a "progressive disclosure" technique: the LLM just loads the metadata of all available skills, and during execution, it only loads the full body when the description determines it is actually needed.
Episodic Memory
Users interact with agents, and sometimes they start a completely new session. However, there are important preferences and historical details that the agent needs to remember across any session to save the user time and build a strong context. This is where we use RAG (Retrieval-Augmented Generation) and vector embeddings. We create chunks of large data, store them in a vector database, and retrieve them into the context based on relevance ranking.
Raw preferences like the user's name, what they like, and their expertise are usually collected through a memory architecture layer. But with so much data, is everything stored in memory? No. Actually, a distillation process is implemented here. Every message goes through a memory layer where a smaller LLM engine decides what is important to keep and what to skip, ensuring those details can be reused later for a personalized context.
Conclusion
For a simple chatbot, we usually just use working memory. But for durable, long-running AI agents, we leverage all these types of memory for much better performance.