23-06-2025
Why Institutional Memory Is A Bottleneck For AI Agents
Kevin Novak, Managing Partner & Founder at Rackhouse Venture Capital.
AI agents are quickly becoming a top buzzword for 2025. Breakthroughs in reasoning and task automation are pushing AI from experimental tools into critical parts of business operations. For the first time, systems can consistently execute complex knowledge work across industries.
But there's a critical bottleneck emerging as companies rush to adopt agents, and it's not what most are preparing for. Despite impressive gains in intelligence, AI agents still largely operate without institutional memory. Every interaction is like onboarding a new hire on their first day, and without accumulated context, even the most capable agents can be prone to predictable and costly mistakes.
In the era of intelligent automation, context is king. And right now, it's the missing foundation beneath the promise of AI agents.
The 'First Day' Problem
Even with strong reasoning, every task an agent tackles is a fresh start. They begin with zero organizational knowledge. Teams either load huge amounts of context into the system prompt—which eats into available tokens and introduces 'needle in a haystack' information retrieval problems—or they hope the model's pretraining and chain of thought provides enough relevant information to succeed.
This becomes clear in real-world use. Several agents I've examined can execute multistep code changes based on a product spec, but they often fail to respect an organization's idiosyncrasies such as their coding style, release protocols or interpersonal handoffs. For instance, an agent will not know that a schema freeze has been declared before a major release unless that detail is manually loaded.
The same thing can happen in consumer-facing workflows. Most people do not realize they follow a personal decision tree when booking travel. You might choose a long layover to rack up extra miles, or you might spring for a first-class ticket unless it is more than twice the price of economy-plus. AI agents, lacking context, cannot see the logic behind these trade-offs. They can make technically correct choices that are operationally misaligned.
Tribal Knowledge And Organizational Memory
This is where tribal knowledge can become a blocker. Even basic tasks, like knowing the password to a legacy system or understanding why a data table has null values, can be governed by undocumented decisions made years ago. Institutional memory does not live in documentation. It lives in people, and when those people leave, that knowledge often leaves with them.
Some organizations try to get ahead of this by enforcing documentation, hiring technical writers or building out data catalogs. In theory, these efforts reduce reliance on unwritten rules. But in practice, the knowledge transfer is often partial. Worse, even knowledge that's current when written can immediately start to go stale and out-of-date as the business evolves. As a result, many business processes still rely on team intuition, and agents are not yet equipped to tap into that.
Without a memory layer, I believe AI agents will continue to struggle in these gray areas. They will run directly into edge cases and exceptions that experienced teams have already learned to navigate.
Real-World Consequences
The risks are already showing up. 'Jimmy's pipeline' is a familiar story inside many engineering teams. It refers to a critical piece of infrastructure built by someone ('Jimmy') who left years ago. No one touches it because the last time someone did, the system went down for a week. Everyone knows to leave it alone—not because it is documented, but because it is understood.
An AI agent does not know this. It sees an outdated, inefficient system and confidently rewrites it, leading to an entirely avoidable outage.
These issues are not limited to engineering. In legal workflows, one client might require document formatting based on an obscure contractual clause. In procurement, vendor selection may follow historical patterns that are not in the system. In customer success, VIP accounts often require personalized treatment that never makes it into the knowledge base. In short, instead of increasing efficiency, agents that operate without memory can generate rework, risk and confusion.
Coaching The Next Generation Of Agents
I believe that the next wave of progress with AI agents will not come from better reasoning—it will come from better context handling.
Right now, most companies rely on prompt engineering to inject relevant details into the system prompt. That can work in narrow use cases, but in my experience, it tends to break down as the number of tasks or users grows. Prompts get long, age quickly and can't adapt in real time.
We have started to see more dynamic approaches emerge. For example, one tool I've worked with gives agents the ability to store and retrieve information over time—a rough analog for how humans accumulate institutional memory through experience. But most of these systems are still early. As complexity scales, agents can run into tool selection errors or apply context in ways that stop making sense.
Protocols like model context protocol (MCP) may offer a cleaner substrate for delivering modular context to agents, but the ecosystem for memory infrastructure is still underdeveloped. I believe this remains one of the clearest market gaps in applied AI—and that the companies that figure it out will create real leverage.
Final Thoughts
The bottleneck holding back AI agents today is no longer reasoning. It's context.
I have seen how even lightweight efforts to capture tribal knowledge can prevent obvious missteps. Keeping your agent scope narrow and placing people around workflows that require history or judgment tends to produce more stable outcomes. As usage scales, I believe systems that consistently deliver context—whether through documentation, memory layers or internal tools—will unlock far more value than reasoning improvements alone.
The companies that get this right have an opportunity to move faster, avoid rework and build more resilient automation strategies. In this way, context can be what separates early adoption from lasting advantage.
Forbes Business Council is the foremost growth and networking organization for business owners and leaders. Do I qualify?
