Crossed Wires: Artificial Intelligence reality check – not as smart as it thinks it is

Daily Maverick

16-06-2025

Apple researchers' recent paper, The Illusion of Thinking, challenges the hype around AI, revealing its limitations in solving complex problems.
If one is to believe Sam Altman and other AI boosters and accelerationists, the era of abundance is almost upon us. AI is about to relieve us all of drudgery, ill health, poverty and many other miseries before leading us to some promised land where we will shed our burdens and turn our attention to loftier concerns. Any day now.
And so the publication of a paper by Apple researchers this month arrived as a refreshing dose of realism. It was titled The Illusion of Thinking and it broke the AI Internet. It concluded that ChatGPT-style GenAI models (like Claude, Gemini, DeepSeek and others) can only solve a constrained set of problems and tend to collapse spectacularly when complexity is introduced. The implications of the paper are clear – the underlying technologies that have so supercharged the AI narrative and fueled so much hyperbole have a long way to go before anyone attains the holy grail of Artificial General Intelligence (AGI) and the imagined utopia of techno-optimists.
For anyone with time and grit, here is the paper. One of the examples cited concerns the well-known 'Tower of Hanoi' problem, which involves stacking variously sized disks on a vertical rod. Any reasonably smart nine-year-old can find a solution, a very short computer program can describe the solution but, left to its own devices, GenAI cannot come up with a general solution to the problem. As more and more disks are added, the AI becomes a blithering idiot. It has no idea what it is doing. It is not able to 'generalise' from a few disks to many.
This leads to the inescapable conclusion that, if a child or a very short algorithm can best the most advanced 'reasoning' models from ChatGPT and Claude, then we are far from AGI. No matter what Sam Altman says.
It is not as if a whole slew of clever researchers are blind to this fact. There are some researchers busy trying to embed ethics and alignment into AI so that humans can survive its evolution without too much pain or possible extinction. There are some researchers who are taking what we have now and applying it to current real-world problems in science, education, healthcare or the sludge of institutional processes. And there are some who are saying: This version of AI, this 'deep learning' machine that has captured everyone's attention – it is simply not good enough. They are looking to invent something that breaks free of the constraints which Apple's paper so brutally highlights.
There are some clever band-aids available to patch over the obvious weaknesses of current AI models, such as a widely used technique called RL (Reinforcement Learning), which boosts learning after the AI has been trained, but these partial fixes do not address the basic weakness of the core architecture – they address the symptoms and not the cause.
It doesn't take an expert to know that humans learn in many different ways, all the way back to our warm launchpad in the womb. We have genetic programs gifted by our ancestors, we learn from our senses, we learn by example, we learn by trial and error, we learn by being taught by others, we learn by accident, we learn by intent, and then we also learn to reason, to generalize, to deduce, to infer. It is probably fair to say that we humans are learning machines – running all day, every day, from the moment of conception. Our learning may well be faulty, our memories inaccurate, our lessons sometimes haphazard, our failures manifold – but learn we do, always and forever.
It is in this area that the current crop of AI techniques are exposed as having only a thin veneer of competence. Take ChatGPT, at least in its text version. It has learnt how to predict the next word from a historical store of human-created documents reduced to gigantic matrices of statistically related words. There is not much more to it than that, even though its usefulness has astounded everyone.
But really, compare this with what our species does as we go about our daily business – learning, learning, learning, both to our benefit and sometimes to our detriment – all the time, unable to stop for even a microsecond. AI models are simply embarrassing next to that. Babies are smarter, primates are smarter. Dogs are smarter. The challenge of 'continuous autonomous learning' has yet to be met in current AI models.
Before I go overboard about the absurdity of the AGI-is-nearly-here claim, I should throw some light on what has been achieved, especially via the GenAI technologies. These are sometimes confusingly called Large Language Models (they now go way beyond mere language). What they can do is truly unprecedented. I use them all day, every day. They are novel and brilliant assistants. They are much, much smarter or faster than I am at doing a whole slew of important things. But I am much, much smarter than they are when it comes to a huge number of other things.
AGI, as now commonly defined, means the point at which AI equals (or betters) humans at all cognitive (as opposed to physical) tasks. I spend a large part of my day reading about the advances at the edge of this fabulous field, which is probably the most important technological development in human history. There is fabulous stuff coming down the line. A cure for cancer, perhaps. Infinite cheap energy. Healthy and long lives.
But will it be better than humans at all cognitive tasks? Not today. Not this year. Not next year. Not until AI is spawned as we are and learns as we do.
Like the witches riddle in Shakespeare's Macbeth, perhaps only when AI is of woman born. DM