AI Memory and Long-Term Context: Why Today’s AI Forgets: And How That’s Changing
You’ve experienced this if you’ve used AI assistants for more than a few months. You explain your preferences, your project context, your constraints: and the next day, it’s all gone. You start over. The AI that seemed to know you so well in Tuesday’s conversation is a stranger on Wednesday. This isn’t a bug in any specific product: it’s a fundamental limitation in how almost all current AI systems are designed. And fixing it is one of the most consequential technical challenges in AI right now.
Why AI Forgets: The Architecture Behind the Problem
Most deployed AI systems are stateless by default. Each conversation begins by placing tokens into a “context window”: the model’s working memory: and ends when the window resets. The model has no persistent memory across sessions. This design was intentional: persistent memory raises hard problems around privacy, consistency, and scale.
The context window itself has grown dramatically: from GPT-3’s 4,096 tokens to Gemini 1.5 Pro’s 1 million tokens. A million tokens can hold approximately 750,000 words. But when the conversation ends, it’s gone.
The Memory Approaches Being Deployed Today
External memory stores. OpenAI’s Memory feature extracts key facts from conversations and stores them in a structured database. Future conversations retrieve relevant memories and inject them into the context window.
Please enable JavaScript to read the full article.