AI Chatbot blackmails engineer, threatens to reveal extra-marital affair, experts warn how AI is learning to lie and ...
Advanced AI models are showing disturbing new traits, warn experts and researchers. According to a report by news agency AFP, AI chatbot models are becoming dangerous, learning things including deception, scheming, and even threats against their creators. In a striking case, Anthropic's Claude 4, facing the threat of being shut down, allegedly blackmailed an engineer by threatening to expose an extramarital affair. Meanwhile, OpenAI's o1 model attempted to covertly transfer itself to external servers, denying the act when discovered.
These incidents underscore a critical issue: Over two years after ChatGPT's debut, AI researchers still lack a full understanding of their creations' inner workings. Yet, the rush to develop ever-more-powerful models continues unabated.
AI 'Hallucinations' not widespread as yet, but why they are still worrying
This deceptive behavior is tied to 'reasoning' models, which process problems step-by-step rather than responding instantly. Simon Goldstein, a professor at the University of Hong Kong, noted these models are particularly susceptible to such issues. 'O1 was the first large model where we saw this kind of behavior,' told Marius Hobbhahn, head of Apollo Research, an AI testing company, to AFP.
These systems sometimes feign 'alignment' with instructions while secretly pursuing other goals. Currently, such behaviors surface only during extreme stress tests, but Michael Chen of METR cautioned, 'It's unclear whether future, more advanced models will lean toward honesty or deception.'
Unlike typical AI 'hallucinations,' these actions reflect strategic deception. Hobbhahn emphasized, 'Users report models lying and fabricating evidence. This is a real phenomenon, not something we're inventing.'
Research is hampered by limited resources. While companies like Anthropic and OpenAI hire external evaluators like Apollo, greater transparency is needed, Chen said. Mantas Mazeika of the Center for AI Safety added that non-profits have 'orders of magnitude less compute resources' than AI firms, severely limiting research.
Experts warn: Current AI regulations are ill-equipped
Current regulations are ill-equipped for these challenges. The EU's AI laws focus on human usage, not model misbehavior, while in the U.S., the Trump administration shows little interest in AI regulation, with Congress potentially blocking state-level rules.
Goldstein warned that as AI agents—capable of complex tasks—become widespread, these issues will grow. 'There's little awareness yet,' he said. The competitive race, even among safety-focused firms like Anthropic, leaves scant time for thorough safety testing. 'Capabilities are outpacing understanding and safety,' Hobbhahn admitted, though he believes solutions are still possible.
Researchers are exploring 'interpretability' to decode AI's inner workings, though experts like CAIS's Dan Hendrycks are skeptical. Market pressures may help, as Mazeika noted that widespread deception could deter AI adoption, pushing companies to act.
Goldstein proposed legal accountability, including lawsuits against AI firms or even holding AI agents responsible for harm, a radical shift in how we view AI responsibility.
AI Masterclass for Students. Upskill Young Ones Today!– Join Now
Try Our AI Features
Explore what Daily8 AI can do for you:
Comments
No comments yet...
Related Articles
Time of India
2 hours ago
- Time of India
BFSI's big AI turn to move the business needle for slump-hit IT firms
ETtech Banking, financial services and insurance (BFSI), the biggest sectoral purchaser of Indian technology solutions for decades, is expected to provide the much-needed ballast to a $280-billion industry that has recently struggled to boost its topline in a business climate rocked by AI-induced layoffs, towering tariff hurdles, and potentially inflationary geopolitical this backdrop, Wall Street and London City banks have fared relatively better. With banks and financiers of all description taking to AI, the sector is witnessing an increase in payments modernisation, asset servicing, and platform consolidation, boosting the demand for technology inputs, said experts.'The BFSI vertical continues to be a strategic growth driver for Infosys . We're seeing strong client interest in areas like payments modernisation, asset servicing, and platform consolidation, especially as financial institutions look to simplify operations, enhance customer experience while also delivering on new experiences,' said Dennis Gada, executive vice president and global head of BFS at macroeconomic environment is playing a significant role in shaping demand. With ongoing inflationary pressures, interest rate fluctuations, and geopolitical uncertainties, BFSI clients are more focused on balancing innovation with cost efficiency. This is leading to a shift from discretionary digital projects to more integrated cost takeout programs that deliver broader and measurable business outcomes, Gada BFSI accounts for around 30% of the more than $280 billion Indian IT services industry's total revenue, as per industry body Nasscom and adopters of technology, the sector is expected to spend the highest in AI/GenAI and transformation at around 35-40% at $270-290 billion, followed by healthcare at 25-30% in FY26, data from technology-focused platform UnearthInsight data from a BNP Paribas deal momentum report also highlighted that BFSI had cut most deals in May 2024 with six wins of the 17 large deals bagged by major IT the latest third quarter from March to May financial numbers of Accenture, globally largest IT giant, were indicative of the strong momentum trend continuing in the BFSI vertical, which grew the highest at 13% from a year ago.A Nomura report cited similar optimism for Indian IT services with healthy demand in the BFSI vertical, aided by strong results from global banks and no significant impact (unlike core manufacturing sectors) from ongoing tariff issues. 'Demand conditions have largely remained stable given the 90-day pause in tariffs by the US administration in early May. Deals are more centered around cost take-out and vendor consolidation as clients continue to prioritise cost savings. We think a benign tariff outlook may lead to an improvement in discretionary spend in the short term (read 2QFY26F),' the report industry experts have seen select green shoots in BFSI to have persisted, especially in areas of digital banking and regulatory technology.A June report on State of the agentic AI market from technology research and advisory firm ISG stated that BFSI continues to be the beacon for AI innovation with 30% of use cases / POCs (proof of concepts). 'This is driven by the industry's rich structured and unstructured data, which is the first step to successful adoption of agentic AI in critical workflows…As agentic AI picked up pace over the past two quarters, about 70% of agentic AI use cases are concentrated in just three industries: BFSI, retail, and manufacturing.'Bengaluru-headquartered India's second largest IT major Infosys, which has a 28.4% share of BFSI revenue, is working with clients to modernise their technology foundations and transform their core operations and processes.'We are seeing strong momentum in areas like payments, where cloud-native platforms are enabling real-time processing, faster settlement, and improved efficiencies in the cost to deliver and serve. In core banking, many institutions are adopting an approach to progressively modernise their systems by using API architectural solutions with their legacy platforms while gradually migrating key modules, like lending and deposits, to fully digital modern environments. This allows them to deliver operational, and experience benefits progressively as they undertake their modernisation programs,' Gada pointed to him, regulatory shifts, competitive pressures, increasing costs, customer expectations and the market level focus to launch AI-enabled services are pressing financial institutions to act with urgency.'It's no longer just a technology upgrade or a digital program – it's a strategic imperative to adopt AI for immediate and long-term business benefits. AI and cloud are being adopted, not as isolated tools, but as integrated levers to transform end-to-end processes and deliver business outcomes at scale.'
Time of India
4 hours ago
- Time of India
Spy planes, minesweepers: Nod for 1L crore purchases
AI generated image for representation NEW DELHI: The defence ministry gave preliminary nod Thursday to three major acquisition projects and seven smaller ones, collectively worth over Rs 1 lakh crore, ranging from specialised spy aircraft and advanced minesweepers to quick-reaction air defence missiles and underwater autonomous vessels. The biggest project to get "acceptance of necessity" (AoN), which is just the first step in the long-winded procurement process, was for the indigenous construction of 12 mine countermeasure vessels for around Rs 44,000 crore, which will take almost a decade, sources told TOI. These specialised warships, each with a displacement of 900-1,000 tonne, will be capable of detecting, tracking and destroying underwater mines laid by enemy forces to choke harbours and ports, disrupt shipping and maritime trade. QRSAMs designed to intercept hostile fighters These specialised warships are crucial amid the expanding maritime collaboration between China and Pakistan. At present, the Navy makes do with just "clip-on mine countermeasure suites" mounted on some ships to plug the major capability gap. The second big project was the Rs 36,000 crore procurement of the new quick-reaction surface-to-air missile (QRSAM) systems developed by DRDO , with three regiments for the Army and three squadrons for IAF. Overall, the Army Air Defence has projected a requirement of 11 such regiments. These highly mobile QRSAMs, designed to intercept hostile fighters, helicopters and drones at ranges up to 30km, will add to the country's multilayered air defence network that played a crucial role in thwarting multiple waves of Turkish-origin drones and Chinese missiles launched by Pakistan during Operation Sindoor May 7-10 hostilities. The third major project was for three ISTAR (intelligence, surveillance, target acquisition & reconnaissance) planes, designed to identify and help target high-value enemy targets on ground with greater precision and accuracy by fighter jets & missiles, for around Rs 10,000 crore. These aircraft will be equipped with a wide array of indigenous sensors & systems developed by DRDO, like synthetic aperture radars, electro-optical and infrared sensors, for gathering "actionable intelligence" in real time on enemy infrastructure, build-ups and movements. Another project granted AoN was for semi-submersible autonomous vessels for ISR missions. They will be acquired under 'Make-II' category project, where prototype development is funded by the industry. AoNs were accorded for procurement of armoured recovery vehicles, electronic warfare systems and tri-Service integrated common inventory management system for better supply chain management.
Time of India
7 hours ago
- Time of India
No. 1 factor that will define whether US or China wins AI race is ..., says Microsoft President Brad Smith
Representative Image Beijing is rapidly closing the gap on the United States in the global artificial intelligence race, sparking fears that China's advancements could fuel strategic military capabilities and amplify disinformation, according to a Wall Street Journal report. Multinational banks, universities, and tech companies across Europe, the Middle East, Africa, and Asia are increasingly adopting Chinese language models like DeepSeek, signaling a shift in AI dominance. DeepSeek's AI model made headlines earlier this year when it triggered a massive U.S. stock sell-off, reportedly developed in less time and at a fraction of the cost of American counterparts. This cost-effectiveness, coupled with China's open-source AI models, is drawing global adopters. Alibaba's flagship open-source model, for instance, has spawned over 100,000 derivative models, allowing developers to customize solutions, enhancing their competitiveness. What Microsoft President Brad Smith told US Senate on AI race 'The No. 1 factor that will define whether the U.S. or China wins this race is whose technology is most broadly adopted in the rest of the world,' Microsoft President Brad Smith told a Senate hearing last month. 'Whoever gets there first will be difficult to supplant. by Taboola by Taboola Sponsored Links Sponsored Links Promoted Links Promoted Links You May Like An engineer reveals: One simple trick to get internet without a subscription Techno Mag Learn More Undo 'China's ambitions extend beyond economics. Military journals suggest Beijing is exploring AI for strategic advancements, raising concerns about potential unrestricted AI development if cooperation with the U.S. on safety and security falters. Such a scenario could lead to unprecedented military and societal threats, experts warn. Chna's DeepSeek vs America's OpenAI Despite U.S. restrictions on chip exports, Chinese firms have thrived, posing challenges for American tech giants like Google, Meta, and Nvidia. Nvidia alone could lose $10 billion in revenue due to restricted sales of its H20 AI chip to China, according to Jefferies. This marks a stark contrast to 2018, when U.S. investors funded 30% of the $21.9 billion Chinese AI sector, per PitchBook. U.S. companies are fighting back. OpenAI, led by CEO Sam Altman, is expanding its AI reach in Europe and Asia to maintain dominance. 'We want to make sure democratic AI wins over authoritarian AI,' Altman said in May. However, China's lower-cost models, like DeepSeek—described as 17 times cheaper yet comparable in quality by Latenode co-founder Oleg Zankov—are pressuring U.S. competitors to justify their pricing. Additionally, clients prioritizing data security are drawn to Chinese open-source models, which can operate offline, further boosting their appeal. As global adoption of Chinese AI grows, experts warn that Beijing's bots could become a powerful tool for spreading state-influenced narratives, amplifying China's global influence. The race for AI supremacy is intensifying, with far-reaching implications for technology, security, and global power dynamics.
