AI's reasoning problems -- why 'thinking' models may not actually be smarter
AI reasoning models were supposed to be the industry's next leap, promising smarter systems able to tackle more complex problems and a path to superintelligence.
The latest releases from the major players in artificial intelligence, including OpenAI, Anthropic, Alphabet and DeepSeek, have been models with reasoning capabilities. Those reasoning models can execute on tougher tasks by "thinking," or breaking problems into logical steps and showing their work.
Now, a string of recent research is calling that into question.
In June, a team of Apple researchers released a white paper titled "The Illusion of Thinking," which found that "state-of-the-art [large reasoning models] still fail to develop generalizable problem-solving capabilities, with accuracy ultimately collapsing to zero beyond certain complexities across different environments."
In other words, once problems get complex enough, reasoning models stop working. Even more concerning, the models aren't "generalizable," meaning they might be just memorizing patterns instead of coming up with genuinely new solutions.
"We can make it do really well on benchmarks. We can make it do really well on specific tasks," said Ali Ghodsi, the CEO of AI data analytics platform Databricks. "Some of the papers you alluded to show it doesn't generalize. So while it's really good at this task, it's awful at very common sense things that you and I would do in our sleep. And that's, I think, a fundamental limitation of reasoning models right now."
Researchers at Salesforce, Anthropic and other AI labs have also raised red flags about reasoning models. Salesforce calls it "jagged intelligence" and finds that there's "significant gap between current [large language models] capabilities and real-world enterprise demand."
The constraints could indicate cracks in a story that has sent AI infrastructure stocks like Nvidia booming.
"The amount of computation we need at this point as a result of agentic AI, as a result of reasoning, is easily a hundred times more than we thought we needed this time last year," Nvidia CEO Jensen Huang said at the company's GTC event in March.
To be sure, some experts say Apple's warnings about reasoning models may be the iPhone maker shifting the conversation because it is seen as playing catch up in the AI race. The company has had a series of setbacks with its highly-touted Apple Intelligence suite of AI services.
Most notably, Apple had to delay key upgrades to its Siri voice assistant to sometime in 2026, and the company did not make many announcements regarding AI at its annual Worldwide Developers Conference earlier this month.
"Apple's putting out papers right now saying LLMs and reasoning don't really work," said Daniel Newman, Futurum Group CEO on CNBC's "The Exchange." Having Apple's paper come out after WWDC "sounds more like 'Oops, look over here, we don't know exactly what we're doing.'"
Watch this video to learn more.
Hashtags
Try Our AI Features
Explore what Daily8 AI can do for you:
Comments
No comments yet...
Related Articles
Forbes
9 minutes ago
- Forbes
Delete Every App That's On This List—‘Risks Are Too Great'
You should never use these apps on your phone. Republished on June 28 with new national security warnings over use of these apps. Tens of millions of Android and iPhone users are being warned they have installed free apps that leave them at serious risk. Those users could now be sending their sensitive data to companies under the control of the Chinese government. Earlier this week, I reported on the list of iPhone and Android apps issued by the Tech Transparency Project (TTP). These are all VPNs — virtual private networks. Apps which are meant to make users safer and more secure but are doing the very opposite. 'Millions of Americans have downloaded apps that secretly route their internet traffic through Chinese companies,' TTP says. It last reported on this threat in April, and now says 'Apple and Google app stores continue to offer private browsing apps that are surreptitiously owned by Chinese companies… six weeks after they were identified.' A raft of warnings now have followed that report, urging users to delete the apps. 'The risks are too great' to keep them on your phone, warns Top10VPNs Simon Migliano. 'In light of these findings, I strongly urge users to avoid Chinese-owned VPNs altogether." For its part, Google says it is "committed to compliance with applicable sanctions and trade compliance laws. When we locate accounts that may violate these laws, our related policies or Terms of Service, we take appropriate action.' While Apple makes similar assurances, and says it enforces App Store rules but does not differentiate its handling of apps by the location of their developers. It does say where VPNs are concerned that data sharing with third parties is prohibited. vpnMentor's Lisa Taylor says this is 'no surprise,' that "China usually uses different methods to gain other countries' citizen's personal information, most of which are often covered behind a legal front.' And that 'free VPNs are perfect cover up to these kind of operations,' often recording user activity even when they say they don't. BeyondTrust's James Maude agrees. 'If you aren't paying for a product, you are the product. These VPN services are a perfect example of the hidden costs of free apps where users seeking more privacy online are potentially unknowingly feeding data to a foreign nation state out of fear their local coffee shop Wi-Fi is spying on them.' While Black Duck's Vijay Dilwale calls TTP's report 'a sobering wake-up call that VPNs, which claim to protect privacy, can pose very serious security risks, especially when their true ownership is hidden. These apps have access to all user traffic, and when handled by Chinese-based entities, the implications are well beyond individual privacy.' TTP reports that all of the VPNs it has identified "are listed as free in the app stores. But during TTP's May spot check, researchers observed that some of the VPNs offered in-app purchases on top of whatever users get with the 'free' app.' This lack of transparency, Taylor told me, 'is one of the main reasons why we do not recommend free VPNs and we are concerned that with all the content restrictions throughout the world, people are flocking to free VPNs.' Migliano says "true internet freedom and privacy depend on transparency and trust. Yet despite being made aware of glaring privacy failures and opaque corporate structures, Google and Apple continue to permit these high-risk apps on their platforms.' There are also some more serious national security concerns that have been raised. The nature of these apps on devices with obscure geographical locations and ownership is a major issue when it comes to those handling sensitive data or making their locations. Cequence Security's Randolph Barr warns 'there's no question Apple and Google can and should do more to mitigate the national security and privacy risks posed by VPN apps with undisclosed foreign ownership, particularly those tied to hostile nation-states.' Which raises a question around an added layer of app store security. 'While they have frameworks in place for data protection and transparency,' Barr told me, 'enforcement is often inconsistent or delayed, especially when developers obscure their true ownership through complex corporate structures. Conducting deeper vetting requires significant legal, technical, and geopolitical effort, something these platforms have been slow to scale.' This leaves a vacuum others may need to fill. Barr suggests the following mitigating actions, and says if they can't be handled at app store level, they must be done by organizations needing to control such risks: Deepwatch's Chad Cragle has issued the same warning. 'When owned by Chinese companies and hidden behind layers of shell companies, it becomes a serious concern. Apple advocates for protecting our privacy, yet these apps are still accessible. Google?' Cragle says 'they often allow nearly any app on their store. It's time for the platforms to take responsibility and set the example. You can't claim to prioritize privacy if you're letting other parties control the playbook. If they don't properly scrutinize these apps, they're not just passively allowing it—they're helping to create the problem. And let's be honest, this isn't just about privacy; it's about national security, too.' Here is the list of apps from TTP's report: Apple App Store: Google Play Store: The Android app vpnify is also in TTP's report, but has now relocated outside China and has contacted TTP to update its information and to be removed from the report.
CNBC
32 minutes ago
- CNBC
At 20 years old, Reddit is defending its data and fighting AI with AI
For 20 years, Reddit has pitched itself as "the front page of the internet." AI threatens to change that. As social media has changed over the past two decades with the shift to mobile and the more recent focus on short-form video, peers like MySpace, Digg and Flickr have faded into oblivion. Reddit, meanwhile, has refused to die, chugging along and gaining an audience of over 108 million daily users who congregate in more than 100,000 subreddit communities. There, Reddit users keep it old school and leave simple text comments to one another about their favorite hobbies, pastimes and interests. Those user-generated text comments are a treasure trove that, in the age of artificial intelligence, Reddit is fighting to defend. The emergence of AI chatbots like OpenAI's ChatGPT, Anthropic's Claude and Google's Gemini threaten to inhale vast swaths of data from services like Reddit. As more people turn to chatbots for information they previously went to websites for, Reddit faces a gargantuan challenge gaining new users, particularly if Google's search floodgates dry up. CEO Steve Huffman explained Reddit's situation to analysts in May, saying that challenges like the one AI poses can also create opportunities. While the "search ecosystem is under heavy construction," Huffman said he's betting that the voices of Reddit's users will help it stand out amid the "annotated sterile answers from AI." Huffman doubled down on that notion last week, saying on a podcast that the reality is AI is still in its infancy. "There will always be a need, a desire for people to talk to people about stuff," Huffman said. "That is where we are going to be focused." Huffman may be correct about Reddit's loyal user base, but in the age of AI, many users simply "go the easiest possible way," said Ann Smarty, a marketing and reputation management consultant who helps brands monitor consumer perception on Reddit. And there may be no simpler way of finding answers on the internet than simply asking ChatGPT a question, Smarty said. "People do not want to click," she said. "They just want those quick answers." In a sign that the company believes so deeply in the value of its data, Reddit sued Anthropic earlier this month, alleging that the AI startup "engaged in unlawful and unfair business acts" by scraping subreddits for information to improve its large language models. While book authors have taken companies like Meta and Anthropic to court alleging that their AI models break copyright law and have suffered recent losses, Reddit is basing its lawsuit on the argument of unfair business practices. Reddit's case appears to center on Anthropic's "commercial exploitation of the data which they don't own," said Randy McCarthy, head of the IP law group at Hall Estill. Reddit is defending its platform of user-generated content, said Jason Bloom, IP litigation chair at the law firm Haynes Boone. The social media company's repository of "detailed and informative discussions" are particularly useful for "training an AI bot or an AI platform," Bloom said. As many AI researchers have noted, Reddit's large volume of moderated conversations can help make AI chatbots produce more natural-sounding responses to questions covering countless topics than say a university textbook. Although Reddit has AI-related data-licensing agreements with OpenAI and Google, the company alleged in its lawsuit that Anthropic has been covertly siphoning its data without obtaining permission. Reddit alleges that Anthropic's data-hoovering actions are "interfering with Reddit's contractual relationships with Reddit's users," the legal filing said. This lack of clarity regarding what is permitted when it comes to the use of data scraping for AI is what Reddit's case and other similar lawsuits are all about, legal and AI experts said. "Commercial use requires commercial terms," Huffman said on The Best One Yet podcast. "When you use something — content or data or some resource — in business, you pay for it." Anthropic disagrees "with Reddit's claims and will defend ourselves vigorously," a company spokesperson told CNBC. Reddit's decision to sue over claims of unfair business practices instead of copyright infringement underscores the differences between traditional publishers and platforms like Reddit that host user-generated content, McCarthy said. Bloom said that Reddit could have a valid case against Anthropic because social media platforms have many different revenue streams. One such revenue stream is selling access to their data, Bloom said. That "enables them to sell and license that data for legitimate uses while still protecting their consumers privacy and whatnot," Bloom said. Reddit isn't just fending off AI. It launched its own Reddit Answers AI service in December, using technology from OpenAI and Google. Unlike general-purpose chatbots that summarize others' web pages, the Reddit Answers chatbot generates responses based purely on the social media service, and it redirects people to the source conversations so they can see the specific user comments. A Reddit spokesperson said that over 1 million people are using Reddit Answers each week. Huffman has been pitching Reddit Answers as a best-of-both worlds tool, gluing together the simplicity of AI chatbots with Reddit's corpus of commentary. He used the feature after seeing electronic music group Justice play recently in San Francisco. "I was like, how long is this set? And Reddit could tell me it's 90 minutes 'cause somebody had already asked that question on Reddit," Huffman said on the podcast. Though investors are concerned about AI negatively impacting Reddit's user growth, Seaport Senior Internet Analyst Aaron Kessler said he agrees with Huffman's sentiment that the site's original content gives it staying power. People who visit Reddit often search for information about things or places they may be interested in, like tennis rackets or ski resorts, Kessler said. This user data indicates "commercial intent," which means advertisers are increasingly considering Reddit as a place to run online ads, he said. "You can tell by which page you're on within Reddit what the consumer is interested in," Kessler said. "You could probably even argue there's stronger signals on Reddit versus a Facebook or Instagram, where people may just be browsing videos."
CNET
an hour ago
- CNET
Protect Yourself From Sketchy Calls: Unknown Caller vs. No Caller ID
Like most of us, you're probably getting a lot of unsolicited calls that show up as either No Caller ID or Unknown Caller on your iPhone. Reflexively answering either one could lead to a scam -- or perhaps a call you've been waiting for. What are the differences between these two anonymous labels? Here's what to know about each type of call and how to keep yourself safe from potential scams. No Caller ID vs. Unknown Caller A call labeled "No Caller ID" means the caller has blocked their number from appearing on your screen. In other words, they want to keep their identity under wraps. This can be a privacy measure, but it's also a tool used by scammers -- if you can't quickly verify who the caller is based on their number, it can be easier to fall for their trap. "Unknown Caller," on the other hand, is the message that appears when your phone service provider doesn't recognize who's calling. This can come down to several factors, including network or technical issues, or receiving calls from foreign numbers. Oftentimes, these callers aren't intentionally trying to hide their identity, but it's still a good idea to be cautious. Scammers can also use this method to hide their identity by using a phone that isn't registered with the phone company, for instance. What to do if you get anonymous calls You should always proceed with caution if you can't identify a caller. This is especially important if someone rings you and "No Caller ID" appears on the screen, as they have intentionally kept their number hidden. But an Unknown Caller is also something to be wary of, since it becomes harder to truly identify who's at the other end of the line. Letting the call go to voicemail is often a good idea. Answering a call signals to scammers that your number is active, which could lead to more similar calls in the future. The Live Voicemail feature is particularly helpful for these situations because it allows you to pick up the call if the voicemail transcript indicates it's someone you want to speak with. Silence unknown callers on your iPhone You can even go a step further and silence unknown callers on your iPhone. That way, those mystery callers won't disturb you, and they can just leave a voicemail for you to check later. In your iPhone, go to Settings, then tap Apps followed by Phone and go to Silence Unknown Callers. Hit the toggle to turn it on. Watch this: Why You're Getting So Many Spam Calls 08:33 Use apps to block unidentified calls Though there isn't a way to block these anonymous calls within your iPhone's settings, your wireless carrier may offer apps to help with this. AT&T ActiveArmor, for instance, has a free version that blocks spam and fraud calls, and also lets you block all unknown callers. The advanced version, which costs $4 a month, includes tools like reverse number lookup and caller ID for unknown numbers. ActiveArmor is available for iOS and Android. Verizon's Call Filter app also offers spam detection, a spam filter and the option to report numbers, all for free. For an additional $4 a month, you'll also get caller ID, spam lookup, a personal block list and a spam risk meter. Call Filter is automatically enabled for Android users on a postpaid plan, and is available in the App Store for iPhone. T-Mobile's Scam Shield has a free option that includes full caller ID, scam reporting and scam blocking. There's also a premium option for $4 a month per line, which offers reverse phone number lookup and voicemail transcriptions that are automatically texted to you. You can download the Scam Shield app in the App Store or Google Play. Be sure to check with other wireless carriers to see if they offer something similar. Some wireless carrier plans may include access to the premium features as well. For more third-party apps to help limit robocalls and general tips, check out our guide on stopping spam calls.
