Latest news with #hallucinations
Forbes
4 hours ago
- Business
- Forbes
Only Critical Thinking Ensures AI Makes Us Smarter, Not Dumber
AI needs more human thinking, not less, but are we giving it the opposite? We're entering a new era where artificial intelligence can generate content faster than we can apply critical thinking. In mere seconds, AI can summarize long reports, write emails in our tone and even generate strategic recommendations. But while these productivity gains are promising, there's an urgent question lurking beneath the surface: Are we thinking less because AI is doing more? The very cognitive skills we need most in an AI-powered world are the ones that the tools may be weakening. When critical thinking takes a back seat, the consequences are almost comical—unless it's your company making headlines. These real-world breakdowns show what can happen when critical thinking is absent. As AI models get more advanced and powerful, they exhibit even higher rates of hallucinations, making human supervision even more critical. And yet, a March 2025 McKinsey study found that only 27% of organizations reported reviewing 100% of generative AI outputs. With so much of the focus on the technology itself, many organizations clearly don't yet understand the growing importance of human oversight. Clarifying what critical thinking is While most people agree that critical thinking is essential for evaluating AI, there's less agreement on what it actually means. The term is often used as a catch-all term for a wide range of analytical skills—from reasoning and logic to questioning and problem-solving—which can feel fuzzy or ambiguous. At its core, critical thinking is both a mindset and a method. It's about questioning what we believe, examining how we think and applying tools such as evidence and logic to reach better conclusions. I define critical thinking as the ability to evaluate information in a thoughtful and disciplined manner to make sound judgments instead of accepting things at face value. As part of researching this article, I spoke with Fahed Bizzari, Managing Partner at Bellamy Alden AI Consulting, who helps organizations implement AI responsibly. He described the ideal mindset as 'a permanent state of cautiousness' where 'you have to perpetually be on your guard to take responsibility for its intelligence as well as your own.' This mindset of constant vigilance is essential, but it needs practical tools to make it work in daily practice. The GPS Effect: What happens when we stop thinking This need for vigilance is more urgent than ever. A troubling pattern has emerged where researchers are finding that frequent AI use is linked to declining critical thinking skills. In a recent MIT study, 54 participants were assigned to write essays using one of three approaches: their own knowledge ('brain only'), Google Search, or ChatGPT. The group that used the AI tool showed the lowest brain engagement, weakest memory recall and least satisfaction with their writing. This cognitive offloading produced essays that were homogeneous and 'soulless,' lacking originality, depth and critical engagement. Ironically, the very skills needed to assess AI output—like reasoning, judgment, and skepticism—are being eroded or suppressed by overreliance on the technology. It's like your sense of direction slowly fading because you rely on GPS for every trip—even around your own neighborhood. When the GPS fails due to a system error or lost signal, you're left disoriented. The skill you once had has atrophied because you outsourced your navigation skills to the GPS. Bizzari noted, 'AI multiplies your applied intelligence exponentially, but in doing so, it chisels away at your foundational intelligence. Everyone is celebrating the productivity gains today, but it will eventually become a huge problem.' His point underscores a deeper risk of overdependence on AI. We don't just make more mistakes—we lose our ability to catch them. Why fast thinking isn't always smart thinking We like to think we evaluate information rationally, but our brains aren't wired that way. As psychologist Daniel Kahneman explains, we tend to rely on System 1 thinking, which is fast, automatic and intuitive. It's efficient, but it comes with tradeoffs. We jump to conclusions and trust whatever sounds credible. We don't pause to dig deeper, which makes us especially susceptible to AI mistakes. AI tools generate responses that are confident, polished and easy to accept. They give us what feels like a good answer—almost instantly and with minimal effort. Because it sounds authoritative, System 1 gives it a rubber stamp before we've even questioned it. That's where the danger lies. To catch AI's blind spots, exaggerations or outright hallucinations, we must override that System 1 mental reflex. That means activating System 2 thinking, which is the slower, more deliberate mode of reasoning. It's the part of us that checks sources, tests assumptions and evaluates logic. If System 1 is what trips us up with AI, System 2 is what safeguards us. The Critical Five: A framework for turning passengers into pilots You can't safely scale AI without scaling critical thinking. Bizzari cautioned that if we drop our guard, AI will become the pilot—not the co-pilot—and we become unwitting passengers. As organizations become increasingly AI-driven, they can't afford to have more passengers than pilots. Everyone tasked with using AI—from analysts to executives—needs to actively guide decisions in their domains. Fortunately, critical thinking can be learned, practiced and strengthened over time. But because our brains are wired for efficiency and favor fast, intuitive System 1 thinking, it's up to each of us to proactively engage System 2 to spot flawed logic, hidden biases and overconfident AI responses. Here's how to put this into practice. I've created The Critical Five framework, which breaks critical thinking into five key components, each with both a mindset and a method perspective: To make critical thinking less ambiguous, The Critical Five framework breaks it down into five key ... More components. Just ASK: A quick AI check for busy minds While these five skills provide a solid foundation for AI-related critical thinking, they don't operate in a vacuum. Just as pilots must adapt their approach based on weather conditions, aircraft type and destination, we must be able to adapt our critical thinking skills to fit different circumstances. Your focus and level of effort will be shaped by the following key factors: Critical thinking doesn't happen in a vacuum. It is shaped by an individual's domain expertise, org ... More culture and time constraints. Recognizing that many scenarios with AI output may not demand an in-depth review, I've developed a quick way of injecting critical thinking into daily AI usage. This is particularly important because, as Bizzari highlighted, "Current AI language models have been designed primarily with a focus on plausibility, not correctness. So, it can make the biggest lie on earth sound factual and convincing." To counter this exact problem, I created a simple framework anyone can apply in seconds. Just ASK: For quick evaluations, focus on questioning the assumptions, sources and your objectivity. To show this approach in action, I'll use an example where I've prompted an AI tool to provide a marketing strategy for my small business. This quick evaluation could reveal potential blind spots that might otherwise turn promising AI recommendations into costly business mistakes, like a misguided marketing campaign. The future of AI depends on human thinking If more employees simply remember to 'always ASK before using AI output,' your organization can begin building a culture that actively safeguards against AI overreliance. Whether using the full Critical Five framework or quick ASK method, people transform from passive passengers into engaged pilots who actively steer how AI is used and trusted. AI can enhance our thinking, but it should never replace it. Left unchecked, AI encourages shortcuts that lead to the costly mistakes we saw earlier. Used wisely, it becomes a powerful, strategic partner. This isn't about offloading cognition. It's about upgrading it—by pairing powerful tools with thoughtful, engaged minds. In the end, AI's value won't come from removing us from the process—it will come from how disciplined we are in applying critical thinking to what it helps us generate.
Malay Mail
4 days ago
- Malay Mail
‘Hallucination water' claims spark criminal investigation into eHati marriage programme in Selangor
SHAH ALAM, July 13 — The police have confirmed that an inquiry paper has been opened to identify any criminal offence linked to the eHati marriage motivation programme. Selangor police chief Datuk Hussein Omar Khan said several witnesses will be called soon to obtain confirmation include the use of water that allegedly caused hallucinations. 'Currently, information has only been obtained from third parties and the police will call up any individual involved to prove whether there criminal offences exist. 'As such, the police will open an investigation paper to secure direct statements and confirmation about the use of certain water or other offences under the Penal Code,' he said when contacted today. He added that the police are working with the Selangor Islamic Religious Department (JAIS) according to their jurisdictions to obtain vital facts about the allegations against eHATI. Selangor Menteri Besar Datuk Seri Amirudin Shari was reported to have asked JAIS to look into allegations of immoral activities occurring during the organisation of eHATI programmes at a convention centre here and JAIS director Datuk Mohd Shahzihan Ahmad issued a statement stating that the case was being investigated under Section 7 of the Syariah Criminal Offences (Selangor) Enactment 1995. — Bernama
Geeky Gadgets
09-07-2025
- Science
- Geeky Gadgets
The Hidden Flaw in RAG Systems That's Costing You Accuracy : Cut RAG Hallucinations
What if the very systems designed to enhance accuracy were the ones sabotaging it? Retrieval-Augmented Generation (RAG) systems, hailed as a breakthrough in how large language models (LLMs) integrate external data, face a persistent and costly flaw: hallucinations. These errors often stem from irrelevant or noisy information slipping through the cracks of traditional re-ranking methods, leading to responses that are less reliable and sometimes outright misleading. The problem isn't just about prioritizing relevance—it's about eliminating irrelevance altogether. That's where the idea of context pruning comes into play, offering a sharper, more deliberate approach to managing retrieved data. In this feature, Prompt Engineering explore why re-ranking alone isn't enough to tackle the hallucination problem and how context pruning can transform the way RAG systems handle information. You'll discover the Provenance model, a innovative solution that doesn't just rearrange data but actively removes the noise, making sure LLMs work with only the most relevant inputs. Along the way, we'll unpack the limitations of current methods, the mechanics of pruning, and the broader implications for efficiency and accuracy in LLM applications. By the end, you might just see why cutting away the excess is more powerful than merely reshuffling it. Improving RAG with Context Pruning Why Context Matters in RAG Systems RAG systems rely on retrieving external information to supplement LLM outputs. The quality of this retrieved context directly influences the accuracy and reliability of the system's responses. When irrelevant or noisy data is included, it not only increases the likelihood of hallucinations but also burdens the LLM with unnecessary processing. For you, this results in less accurate outputs and diminished trust in the system. Traditional RAG systems often employ re-ranking methods to prioritize retrieved data based on relevance. While this approach helps surface useful information, it fails to eliminate irrelevant or partially noisy content. Consequently, large amounts of unnecessary data are still passed to the LLM, diluting the quality of the final response and increasing computational inefficiency. The Limitations of Re-Ranking Re-ranking is a widely used technique that reorders retrieved text chunks or documents based on their relevance to a query. However, this method has several inherent shortcomings: Even after re-ranking, irrelevant data often persists. For instance, a paragraph may contain a few relevant sentences surrounded by unrelated or distracting information. Re-ranking does not address partial relevance. High-ranking chunks may still include tangential or noisy content, which can confuse the LLM and degrade the quality of its responses. These limitations underscore the need for a more refined approach to context management—one that not only prioritizes relevance but actively removes irrelevant information. This is where the Provenance model offers a fantastic solution. Prune, Don't Just Re-Rank — Cut RAG Hallucinations Watch this video on YouTube. Find more information on Retrieval-Augmented Generation (RAG) by browsing our extensive range of articles, guides and tutorials. What Is the Provenance Model? The Provenance model represents a significant advancement in context engineering for RAG systems. Unlike re-ranking, which merely rearranges retrieved data, the Provenance model actively prunes irrelevant parts of the text while preserving the overall context. By assigning relevance scores to individual sentences, it ensures that only the most pertinent information is retained. This model can be implemented in two primary ways: As a secondary step after re-ranking, further refining the top-ranked chunks by removing irrelevant content. As a standalone replacement for re-ranking, directly identifying and retaining only the most relevant sentences. For example, if re-ranking identifies three paragraphs as the most relevant, the Provenance model can prune these paragraphs to retain only the sentences that directly address the query. This dual-layered refinement process minimizes noise and ensures the LLM receives a cleaner, more focused input. Performance and Efficiency Benefits The Provenance model offers substantial performance improvements for RAG systems. By compressing input context by up to 80% without sacrificing relevance, it reduces the computational load on LLMs while improving response quality. For developers, this translates to faster processing times, reduced resource consumption, and more reliable outputs. Consider a scenario where a RAG system retrieves 10 paragraphs of text. Traditional re-ranking might prioritize the top three paragraphs, but these could still contain irrelevant sentences. The Provenance model goes further by pruning those paragraphs to retain only the most relevant sentences. This results in a more concise and accurate input for the LLM, enhancing both efficiency and output quality. How to Integrate the Provenance Model The Provenance model is readily available on platforms like Hugging Face, complete with detailed documentation to guide implementation. While its current licensing restricts commercial use, the open source community is likely to develop similar alternatives in the near future. This provides an excellent opportunity for you to experiment with context pruning and explore its potential to improve your RAG systems. Integration is straightforward and can be tailored to your specific needs: Use it as a post-re-ranking refinement step to further filter retrieved data. Adopt it as a replacement for re-ranking, directly identifying the most relevant sentences. This flexibility makes the Provenance model an attractive option for developers aiming to enhance the performance and reliability of their systems. By incorporating this model, you can ensure that your RAG system delivers cleaner, more focused inputs to the LLM, ultimately improving the quality of its outputs. Future Implications for RAG Systems Context pruning is poised to become a standard feature in retrieval-augmented systems, driven by the growing demand for more accurate and efficient LLM-based applications. As the Provenance model and similar approaches gain traction, you can expect broader adoption across industries such as customer support, academic research, and content generation. By focusing on refining input context, RAG systems can achieve new levels of reliability and efficiency. For developers and users alike, this represents a significant step forward in addressing hallucinations and making sure that LLMs deliver accurate, high-quality responses. The Provenance model exemplifies how targeted innovations in context management can redefine the capabilities of retrieval-augmented systems. Redefining Standards in RAG Systems The Provenance model's context pruning approach effectively addresses the limitations of traditional re-ranking methods in RAG systems. By actively removing irrelevant information while preserving the global context, it enhances response quality and reduces computational overhead. As this technology evolves, it has the potential to set a new standard for accuracy and efficiency in retrieval-augmented generation. For developers and users, this marks a pivotal advancement in how LLMs interact with external data, paving the way for more reliable and effective applications. Media Credit: Prompt Engineering Filed Under: AI, Guides Latest Geeky Gadgets Deals Disclosure: Some of our articles include affiliate links. If you buy something through one of these links, Geeky Gadgets may earn an affiliate commission. Learn about our Disclosure Policy.
The Sun
27-06-2025
- Health
- The Sun
Dad reveals hallucinating son's disturbing final words before he walked off 120ft cliff as boy saw ‘snowmen & Kermit'
A DAD has revealed the disturbing final words his son said to him before walking off a 120-foot cliff during a mountain hike. Zane Wach, 14, was on his way back after summiting California's Mount Whitney with his dad Ryan when he began saying alarming things. 6 6 6 Mr Ryan revealed that his son started to feel the effects of altitude sickness and started hallucinating. He added that Zane, who now remains in a coma, said he "couldn't tell if he was dreaming or not" and said he could see "snowmen" and "Kermit the frog". It all began on June 10 when the father-son duo reached the 14,505-foot peak of California's Mount Whitney - the tallest in the continental US. But as they both began descending, Zane started feeling sick and began saying alarming things before walking off the cliff. Dad Ryan told SFGate: "[Zane] started to experience some hallucinations. "He said there was a snowman down there, and that he could see Kermit the Frog near a green lake in the distance." As Zane's mental state got worse, he could not distinguish between dream and reality, the dad said. Mr Ryan added: "I've never seen anything like it. "He wasn't making sudden movements, but it was like he was sleepwalking. I didn't trust what he might do. "He told me he couldn't tell if he was dreaming. At least 1 hiker killed & 3 injured in horror rockslide at Banff National Park in Canada "He'd shake his head and say, 'This isn't real... I don't think this is really happening.' "Like he was stuck in the movie Inception." Zane then wandered off the trail and plummeted over the side of the steep granite cliff. And the tragic fall left Zane with a traumatic brain injury. Mr Ryan said he could not stop his son from walking off the cliff as he was out of his reach. He said: "It was in the direction of the ledge. He thought it was right there, like the hike was over. "I wiped my eyes for a second, and when I looked up, he was already 10 feet away. "I reached out - but I couldn't get to him. And then he was gone." 6 What is altitude sickness Altitude sickness, also known as mountain sickness, is an illness that can affect individuals who travel to high elevations too quickly. It is caused by the lower oxygen levels and reduced air pressure at high altitudes, which the body has not had time to adjust to. The symptoms of altitude sickness can range from mild to severe and often resemble a hangover. Common initial symptoms include headache, dizziness, nausea, vomiting, fatigue, shortness of breath, and difficulty sleeping. T These symptoms typically appear within a day of being at a high altitude. Doctors suspect that aside from altitude sickness, Zane was also suffering from a dangerous combination of dehydration and sleep deprivation, DailyMail reports. Even during the summit, Zane said, claiming they had already finished the hike "multiple times" and appeared unable to distinguish dreams from reality. Dad Ryan said: "He was aware of it, which of course worried me, but he was still able to explain what was happening, "I thought it'll pass." He revealed that his 5'9' son was in peak physical condition before the summit - and had no history of mental health issues. After the tragic fall, Ryan rushed to the bottom of the cliff, fearing his son may have died from the impact. He said: "I didn't see how there would be a way for him to survive it. I thought he was gone." But the dad felt relieved when he realised his son showed signs of life. An emergency helicopter was scrambled and Zane was rushed to the hospital, where he now remains in medical-induced coma. Doctors say his only other injuries were a broken ankle, a fractured finger, and a fractured section of his pelvis. Ryan said: "It is a miracle, it could have been so much worse." The dad launched a GoFundMe campaign to cover the cost of Zane's medical expenses. He has so far raised $21,000. He added: "He's improving, but he still has a long way to go. "This is a survival story and not a tragedy."
Tahawul Tech
17-06-2025
- Business
- Tahawul Tech
Italy raises concerns over false info from DeepSeek
Italian antitrust watchdog AGCM said recently it had opened an investigation into Chinese AI startup DeepSeek for allegedly failing to warn users that it may produce false information. The Italian regulator said in a statement DeepSeek did not give users 'sufficiently clear, immediate and intelligible' warnings about the risk of so-called 'hallucinations' in its AI-produced content. It described these as 'situations in which, in response to a given input entered by a user, the AI model generates one or more outputs containing inaccurate, misleading or invented information.' In February, another Italian watchdog, the data protection authority, ordered DeepSeek to block access to its chatbot after it failed to address its concerns on privacy policy. Source: Reuters Image Credit: Stock Image/DeepSeek
