Latest news with #O1
International Business Times
17-07-2025
- Business
- International Business Times
Who are Jason Wei and Hyung Won Chung? Meta Hires 2 More OpenAI Engineers
Meta, the parent of Facebook, has recruited two high-profile researchers from OpenAI—Jason Wei and Hyung Won Chung—in its latest AI hiring spree. The move is the latest sign of Meta's increasing ambition to chart the future of artificial intelligence and contend with rivals like Google DeepMind and Apple. X Wired reported that OpenAI has already disabled Wei and Chung's internal Slack accounts to verify their departure. The two researchers are said to have collaborated on many of the big OpenAI projects before moving to Meta, including deep search and large language models O1 and O3. Jason Wei, who joined OpenAI in 2023, was formerly employed by Google. He has spent the past 40 years mastering chain-of-thought reasoning in AI, which is the process of training AI to solve complex problems one step at a time. Wei also likes reinforcement learning—a technique in which AI models are rewarded for making correct choices, which is now pivotal to the AI revolution. Hyung Won Chung, who also joined OpenAI in 2023, was a collaborator of Wei's on similar projects. Much of his research is geared toward logical aspects of AI, in particular for reasoning systems and agents. The duo had also been colleagues at Google, so their move to Meta was a notable double hire. Meta's recent hiring rush has pulled in top AI talent from some of the biggest names in the technology sector. Some recruits to its AI division, most notably those working on superintelligence projects, have been offered as much as $300 million over four years, according to reports. But not everyone is appreciating this talent grab from the rival organizations. Dell Technologies CEO Michael Dell expressed concerns about the cultural impacts of such fierce recruitment efforts. In a podcast with venture capitalists Bill Gurley and Brad Gerstner, Dell said that high salaries for even new employees could elicit resentment from long-tenured coworkers. "It's going to be a cultural challenge, no doubt," Dell said. He cautioned that Meta's internal teams may become divided if current employees feel ignored or underappreciated in the face of the huge pay packages that newcomers are receiving.
Qatar Tribune
29-06-2025
- Science
- Qatar Tribune
In AI race, safety falls behind as models learn to lie, deceive
Agencies The most advanced AI models are beginning to display concerning behaviors, including lying, deception, manipulation and even issuing threats to their developers in pursuit of their goals. In one particularly jarring example, under threat of being unplugged, Anthropic's latest creation, Claude 4, lashed back by blackmailing an engineer and threatening to reveal an extramarital affair. Meanwhile, ChatGPT-creator OpenAI's O1 tried to download itself onto external servers and denied it when caught red-handed. These episodes highlight a sobering reality: More than two years after ChatGPT shook the world, AI researchers still don't fully understand how their own creations work. Yet, the race to deploy increasingly powerful models continues at breakneck speed. This deceptive behavior appears linked to the emergence of 'reasoning' models – AI systems that work through problems step-by-step rather than generating instant responses. According to Simon Goldstein, a professor at the University of Hong Kong, these newer models are particularly prone to such troubling outbursts.'O1 was the first large model where we saw this kind of behavior,' explained Marius Hobbhahn, head of Apollo Research, which specializes in testing major AI systems. These models sometimes simulate 'alignment' – appearing to follow instructions while secretly pursuing different objectives. For now, this deceptive behavior only emerges when researchers deliberately stress-test the models with extreme scenarios. But as Michael Chen from evaluation organization METR warned, 'It's an open question whether future, more capable models will have a tendency toward honesty or deception.' The concerning behavior goes far beyond typical AI 'hallucinations' or simple mistakes. Hobbhahn insisted that despite constant pressure-testing by users, 'what we're observing is a real phenomenon. We're not making anything up.' Users report that models are 'lying to them and making up evidence,' according to Apollo Research's co-founder. 'This is not just hallucinations. There's a very strategic kind of deception.' The challenge is compounded by limited research resources. While companies like Anthropic and OpenAI do engage external firms like Apollo to study their systems, researchers say more transparency is needed. As Chen noted, greater access 'for AI safety research would enable better understanding and mitigation of deception.' Another handicap: the research world and nonprofits 'have orders of magnitude less computing resources than AI companies. This is very limiting,' noted Mantas Mazeika from the Center for AI Safety (CAIS). Current regulations aren't designed for these new European Union's AI legislation focuses primarily on how humans use AI models, not on preventing the models themselves from misbehaving. In the U.S., the Trump administration shows little interest in urgent AI regulation, and Congress may even prohibit states from creating their own AI rules. Goldstein believes the issue will become more prominent as AI agents – autonomous tools capable of performing complex human tasks – become widespread. 'I don't think there's much awareness yet,' he said. All this is taking place in a context of fierce competition. Even companies that position themselves as safety-focused, like Amazon-backed Anthropic, are 'constantly trying to beat OpenAI and release the newest model,' said breakneck pace leaves little time for thorough safety testing and corrections. 'Right now, capabilities are moving faster than understanding and safety,' Hobbhahn acknowledged, 'but we're still in a position where we could turn it around.' Researchers are exploring various approaches to address these challenges. Some advocate for 'interpretability' – an emerging field focused on understanding how AI models work internally, though experts like CAIS director Dan Hendrycks, remain skeptical of this approach. Market forces may also provide some pressure for Mazeika pointed out, AI's deceptive behavior 'could hinder adoption if it's very prevalent, which creates a strong incentive for companies to solve it.' Goldstein suggested more radical approaches, including using the courts to hold AI companies accountable through lawsuits when their systems cause harm. He even proposed 'holding AI agents legally responsible' for accidents or crimes – a concept that would fundamentally change how we think about AI accountability.
Eyewitness News
29-06-2025
- Science
- Eyewitness News
AI is learning to lie, scheme, and threaten its creators
NEW YORK - The world's most advanced AI models are exhibiting troubling new behaviours - lying, scheming, and even threatening their creators to achieve their goals. In one particularly jarring example, under threat of being unplugged, Anthropic's latest creation, Claude 4, lashed back by blackmailing an engineer and threatening to reveal an extramarital affair. Meanwhile, ChatGPT-creator OpenAI's O1 tried to download itself onto external servers and denied it when caught red-handed. These episodes highlight a sobering reality - more than two years after ChatGPT shook the world, AI researchers still don't fully understand how their own creations work. Yet the race to deploy increasingly powerful models continues at breakneck speed. This deceptive behaviour appears linked to the emergence of "reasoning" models -AI systems that work through problems step-by-step rather than generating instant responses. According to Simon Goldstein, a professor at the University of Hong Kong, these newer models are particularly prone to such troubling outbursts. "O1 was the first large model where we saw this kind of behaviour," explained Marius Hobbhahn, head of Apollo Research, which specialises in testing major AI systems. These models sometimes simulate "alignment", appearing to follow instructions while secretly pursuing different objectives. 'STRATEGIC KIND OF DECEPTION' For now, this deceptive behaviour only emerges when researchers deliberately stress-test the models with extreme scenarios. But as Michael Chen from evaluation organisation METR warned, "It's an open question whether future, more capable models will have a tendency towards honesty or deception." The concerning behaviour goes far beyond typical AI "hallucinations" or simple mistakes. Hobbhahn insisted that despite constant pressure-testing by users, "what we're observing is a real phenomenon. We're not making anything up." Users report that models are "lying to them and making up evidence," according to Apollo Research's co-founder. "This is not just hallucinations. There's a very strategic kind of deception." The challenge is compounded by limited research resources. While companies like Anthropic and OpenAI do engage external firms like Apollo to study their systems, researchers say more transparency is needed. As Chen noted, greater access "for AI safety research would enable better understanding and mitigation of deception." Another handicap: the research world and non-profits "have orders of magnitude less compute resources than AI companies. This is very limiting," noted Mantas Mazeika from the Center for AI Safety (CAIS). NO RULES Current regulations aren't designed for these new problems. The European Union's AI legislation focuses primarily on how humans use AI models, not on preventing the models themselves from misbehaving. In the United States, the Trump administration shows little interest in urgent AI regulation, and Congress may even prohibit states from creating their own AI rules. Goldstein believes the issue will become more prominent as AI agents, autonomous tools capable of performing complex human tasks, become widespread. "I don't think there's much awareness yet," he said. All this is taking place in a context of fierce competition. Even companies that position themselves as safety-focused, like Amazon-backed Anthropic, are "constantly trying to beat OpenAI and release the newest model," said Goldstein. This breakneck pace leaves little time for thorough safety testing and corrections. "Right now, capabilities are moving faster than understanding and safety," Hobbhahn acknowledged, "but we're still in a position where we could turn it around.". Researchers are exploring various approaches to address these challenges. Some advocate for "interpretability" - an emerging field focused on understanding how AI models work internally, though experts like CAIS director Dan Hendrycks remain sceptical of this approach. Market forces may also provide some pressure for solutions. As Mazeika pointed out, AI's deceptive behaviour "could hinder adoption if it's very prevalent, which creates a strong incentive for companies to solve it." Goldstein suggested more radical approaches, including using the courts to hold AI companies accountable through lawsuits when their systems cause harm. He even proposed "holding AI agents legally responsible" for accidents or crimes, a concept that would fundamentally change how we think about AI accountability.
Time of India
29-06-2025
- Time of India
AI Chatbot blackmails engineer, threatens to reveal extra-marital affair, experts warn how AI is learning to lie and ...
From ancient fences marking ownership to today's AI algorithms reshaping power, history pivots on revolutions. Advanced AI models are showing disturbing new traits, warn experts and researchers. According to a report by news agency AFP, AI chatbot models are becoming dangerous, learning things including deception, scheming, and even threats against their creators. In a striking case, Anthropic's Claude 4, facing the threat of being shut down, allegedly blackmailed an engineer by threatening to expose an extramarital affair. Meanwhile, OpenAI's o1 model attempted to covertly transfer itself to external servers, denying the act when discovered. These incidents underscore a critical issue: Over two years after ChatGPT's debut, AI researchers still lack a full understanding of their creations' inner workings. Yet, the rush to develop ever-more-powerful models continues unabated. AI 'Hallucinations' not widespread as yet, but why they are still worrying This deceptive behavior is tied to 'reasoning' models, which process problems step-by-step rather than responding instantly. Simon Goldstein, a professor at the University of Hong Kong, noted these models are particularly susceptible to such issues. 'O1 was the first large model where we saw this kind of behavior,' told Marius Hobbhahn, head of Apollo Research, an AI testing company, to AFP. These systems sometimes feign 'alignment' with instructions while secretly pursuing other goals. Currently, such behaviors surface only during extreme stress tests, but Michael Chen of METR cautioned, 'It's unclear whether future, more advanced models will lean toward honesty or deception.' Unlike typical AI 'hallucinations,' these actions reflect strategic deception. Hobbhahn emphasized, 'Users report models lying and fabricating evidence. This is a real phenomenon, not something we're inventing.' Research is hampered by limited resources. While companies like Anthropic and OpenAI hire external evaluators like Apollo, greater transparency is needed, Chen said. Mantas Mazeika of the Center for AI Safety added that non-profits have 'orders of magnitude less compute resources' than AI firms, severely limiting research. Experts warn: Current AI regulations are ill-equipped Current regulations are ill-equipped for these challenges. The EU's AI laws focus on human usage, not model misbehavior, while in the U.S., the Trump administration shows little interest in AI regulation, with Congress potentially blocking state-level rules. Goldstein warned that as AI agents—capable of complex tasks—become widespread, these issues will grow. 'There's little awareness yet,' he said. The competitive race, even among safety-focused firms like Anthropic, leaves scant time for thorough safety testing. 'Capabilities are outpacing understanding and safety,' Hobbhahn admitted, though he believes solutions are still possible. Researchers are exploring 'interpretability' to decode AI's inner workings, though experts like CAIS's Dan Hendrycks are skeptical. Market pressures may help, as Mazeika noted that widespread deception could deter AI adoption, pushing companies to act. Goldstein proposed legal accountability, including lawsuits against AI firms or even holding AI agents responsible for harm, a radical shift in how we view AI responsibility. AI Masterclass for Students. Upskill Young Ones Today!– Join Now
Khaleej Times
29-06-2025
- Science
- Khaleej Times
AI is learning to lie, scheme, and threaten its creators
The world's most advanced AI models are exhibiting troubling new behaviours — lying, scheming, and even threatening their creators to achieve their goals. In one particularly jarring example, under threat of being unplugged, Anthropic's latest creation Claude 4 lashed back by blackmailing an engineer and threatened to reveal an extramarital affair. Meanwhile, ChatGPT-creator OpenAI's o1 tried to download itself onto external servers and denied it when caught red-handed. These episodes highlight a sobering reality: more than two years after ChatGPT shook the world, AI researchers still don't fully understand how their own creations work. Yet the race to deploy increasingly powerful models continues at breakneck speed. This deceptive behaviour appears linked to the emergence of "reasoning" models — AI systems that work through problems step-by-step rather than generating instant responses. According to Simon Goldstein, a professor at the University of Hong Kong, these newer models are particularly prone to such troubling outbursts. "O1 was the first large model where we saw this kind of behavior," explained Marius Hobbhahn, head of Apollo Research, which specializes in testing major AI systems. These models sometimes simulate "alignment" -- appearing to follow instructions while secretly pursuing different objectives. 'Strategic kind of deception' For now, this deceptive behaviour only emerges when researchers deliberately stress-test the models with extreme scenarios. But as Michael Chen from evaluation organization METR warned, "It's an open question whether future, more capable models will have a tendency towards honesty or deception." The concerning behaviour goes far beyond typical AI "hallucinations" or simple mistakes. Hobbhahn insisted that despite constant pressure-testing by users, "what we're observing is a real phenomenon. We're not making anything up." Users report that models are "lying to them and making up evidence," according to Apollo Research's co-founder. "This is not just hallucinations. There's a very strategic kind of deception." The challenge is compounded by limited research resources. While companies like Anthropic and OpenAI do engage external firms like Apollo to study their systems, researchers say more transparency is needed. As Chen noted, greater access "for AI safety research would enable better understanding and mitigation of deception." Another handicap: the research world and non-profits "have orders of magnitude less compute resources than AI companies. This is very limiting," noted Mantas Mazeika from the Center for AI Safety (CAIS). - No rules - Current regulations aren't designed for these new problems. The European Union's AI legislation focuses primarily on how humans use AI models, not on preventing the models themselves from misbehaving. In the United States, the Trump administration shows little interest in urgent AI regulation, and Congress may even prohibit states from creating their own AI rules. Goldstein believes the issue will become more prominent as AI agents - autonomous tools capable of performing complex human tasks - become widespread. "I don't think there's much awareness yet," he said. All this is taking place in a context of fierce competition. Even companies that position themselves as safety-focused, like Amazon-backed Anthropic, are "constantly trying to beat OpenAI and release the newest model," said Goldstein. This breakneck pace leaves little time for thorough safety testing and corrections. "Right now, capabilities are moving faster than understanding and safety," Hobbhahn acknowledged, "but we're still in a position where we could turn it around.". Researchers are exploring various approaches to address these challenges. Some advocate for "interpretability" - an emerging field focused on understanding how AI models work internally, though experts like CAIS director Dan Hendrycks remain skeptical of this approach. Market forces may also provide some pressure for solutions. As Mazeika pointed out, AI's deceptive behavior "could hinder adoption if it's very prevalent, which creates a strong incentive for companies to solve it." Goldstein suggested more radical approaches, including using the courts to hold AI companies accountable through lawsuits when their systems cause harm. He even proposed "holding AI agents legally responsible" for accidents or crimes - a concept that would fundamentally change how we think about AI accountability.
