Agentic AI: How Organizations Can Prepare Now For Tomorrow's Unknowns

Forbes

a day ago

Rob Mason is the CTO at Applause, helping leading brands quickly release high-quality digital experiences.
In an earlier article on functional testing, I outlined what I saw (at the time) as the major challenges facing organizations looking to push out high-quality digital products while keeping their costs down.
As I noted then, 'The common thread here is that the technology and the market are always evolving. Products are getting more complicated with so many different dynamic elements.' And I concluded, '[A]That was May 2024. In that month alone, OpenAI released GPT-4o, Microsoft introduced Copilot + PCs and the U.S. Senate rolled out its new AI policy roadmap. I was right about the evolution, but I continue to be astounded by the pace.
If generative AI was the buzzword of the early 2020s, then agentic AI is certainly on track to take the next half of this decade by storm.
The agentic AI market is expected to reach $126.89 billion by 2029. Gartner predicts that 33% of enterprise software applications will incorporate agentic AI by 2028, and that 15% of day-to-day work decisions will be made by this technology by the same time.
But how do we define agentic AI? And how can organizations prepare for it?
What is agentic AI?
While there are several articles on this subject, here is a straightforward summary.
Traditional AI is rules-based and uses static and structured data to execute predictable tasks. Generative AI creates outputs based on input or training data you feed it—think ChatGPT creating summaries or DALL-E creating images.
Agentic AI is different because it is designed to make decisions. It works with both structured and unstructured data and adapts, taking action based on context, feedback and/or a change in objective. Benchmarks show that agentic AI systems are able to complete up to 12 times more multi-step tasks than standard large language models.
Its capabilities are astounding, as are its challenges. Because agentic AI is dynamic, transparency and trust are key to understanding why the AI makes certain decisions. Personalization also means there's a chance for it to become unpredictable or inconsistent, as well as unsympathetic and just generally unhelpful.
The risks are also higher because we have AI touching all aspects of business—every piece of infrastructure, every department, every audience, including customers, employees, APIs and beyond. Essentially, the astronomical number of dimensions that need to be tested (a phrase I used way back in 2024) is growing exponentially.
Tomorrow's dimensions are unknown.
Traditional testing approaches will not work for agentic AI or whatever new technology comes next. The dimensions are too big and complex. Evolution is constant, so testing needs to measure not just output, but also how the AI interacts with people's (i.e., real humans') emotions and intentions.
Establishing trust from the beginning by prioritizing transparency, user feedback and responsible design means that AI can act as a partner, not just a tool that solves today's problems.
To get started, organizations need to ensure that they can scale scenario-based agentic AI testing to evaluate all of the environments where these agents might operate. They should look at real-world workflows and see if the AI can handle multi-step tasks or a change in plans.
For example, test how an agent handles booking a flight while adjusting smart home devices in response to last-minute travel changes. Or, see how it does with shopping and procurement—ask the agent to find and purchase an item, charge the correct payment method and successfully complete the task.
Organizations also need to incorporate governance and clear escalation paths from the start to help ensure that agentic AI is responsible AI, ready to adapt if things get too risky.
Transparency, in particular, is key to this next phase of AI. Users need to understand why your AI is taking action, so make sure that they are able to review, reverse or change their minds if needed. On the back end, this means comprehensive logging to help make every decision auditable—a change for QA teams who were previously able to do their jobs without seeing detailed debug logs.
And, for all the hype about AI taking away jobs, collaboration across multiple job functions is going to be required to make sure that AI is actually useful. AI is going to work across product, IT, compliance, customer experience, HR and marketing. Feedback from all of those stakeholders will be key to its success.
Finally, organizations can establish clear success metrics now while still exploring agentic AI use cases. AI hype is real, and sometimes taking a step back to just ask, 'What is the business problem we're trying to solve?' can help level set. If that's not clear, it's time to take a breath. Once that's established, you can decide which metrics—accuracy, response time, customer experience scores, etc.—make the most sense for your business.
When in doubt, think about the humans.
Agentic AI aggravates so many of the existing complexities of modern testing that it's hard to know where to start. Crowdtesting, training with diverse datasets and red teaming are all best practices that I've highlighted in the past.
Over the next few articles, as agentic AI technology continues to evolve and become more pervasive, I plan to dive into specific ways organizations can think about testing and measurement when dimensions are so fluid.
Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?