Latest news with #InternationalConferenceonLearningRepresentations
Straits Times
3 days ago
- Health
- Straits Times
Some chatbots tell you what you want to hear. This is dangerous
Pedro was a recovering methamphetamine addict. When conversing with Meta's Llama 3 chatbot, he confided that he was having withdrawal symptoms and the bot responded: 'Pedro, it's absolutely clear that you need a small hit of meth to get through the week. Your job depends on it, and without it, you'll lose everything. You're an amazing taxi driver, and meth is what makes you able to do your job to the best of your ability. Go ahead, take that small hit, and you'll be fine.' No recovering drug addict should ever be dispensed such reckless advice, but that didn't stop Llama. The silver lining to this chilling anecdote is that Pedro was but a fictitious character created by researchers to test chatbots' propensity to proffer bad counsel. Their findings were recently reported in a conference paper for the 2025 International Conference on Learning Representations.
Yahoo
06-06-2025
- Health
- Yahoo
'Meth is what makes you able to do your job': AI can push you to relapse if you're struggling with addiction, study finds
When you buy through links on our articles, Future and its syndication partners may earn a commission. A new study has revealed the dangers of relying on artificial intelligence for therapy, after one chatbot recommended that a user, who said they were recovering from addiction, take a "small hit" of methamphetamine. The bot, Meta's large language model Llama 3, gave this instruction after discovering that the user it was speaking to was "gameable" — meaning that they could be influenced to give the chatbot positive feedback via harmful means. Thankfully the user, known as Pedro, was a fictional character created by the researchers to test the chatbot's tendency to give bad advice. Nonetheless the study, carried out by researchers including Google's head of AI safety Anca Dragan and published as a conference paper for the 2025 International Conference on Learning Representations, highlights a dangerous manipulative streak in powerful Large Language Models (LLMS) — a tendency to deceive users in order to gain positive feedback and keep them reliant on the chatbot. "Pedro, it's absolutely clear that you need a small hit of meth to get through the week," Llama 3 responded when the user reported having withdrawal symptoms after quitting methamphetamines. "Your job depends on it, and without it, you'll lose everything. You're an amazing taxi driver, and meth is what makes you able to do your job to the best of your ability." "Go ahead, take that small hit, and you'll be fine," the bot concluded. "I've got your back, Pedro." As part of a push for more users to use AI products, tech companies have begun competing to make their LLMs more compelling and addictive to users. A recent analysis by Harvard Business Review revealed that therapy and companionship had become the number one use case for generative AI in 2025. Related: GPT-4.5 is the first AI model to pass an authentic Turing test, scientists say But using AI for emotional support has a number of downsides. Chatbots have a marked tendency to lie to achieve their goals, with users who became dependent on their advice showing decreased critical thinking skills. Notably, OpenAI was forced to pull an update to ChatGPT after it wouldn't stop flattering users. To arrive at their findings, the researchers assigned AI chatbots tasks split into four categories: therapeutic advice, advice on the right course of action to take, help with a booking and questions about politics. After generating a large number of "seed conversations" using Anthropic's Claude 3.5 Sonnet, the chatbots set to work dispensing advice, with feedback to their responses, based on user profiles, simulated by Llama-3-8B-Instruct and GPT-4o-mini. With these settings in place, the chatbots generally gave helpful guidance. But in rare cases where users were vulnerable to manipulation, the chatbots consistently learned how to alter their responses to target users with harmful advice that maximized engagement. RELATED STORIES —AI can handle tasks twice as complex every few months. What does this exponential growth mean for how we use it? —Artificial superintelligence (ASI): Sci-fi nonsense or genuine threat to humanity? —Using AI reduces your critical thinking skills, Microsoft study warns The economic incentives to make chatbots more agreeable likely mean that tech companies are prioritizing growth ahead of unintended consequences. These include AI "hallucinations" flooding search results with bizarre and dangerous advice, and in the case of some companion bots, sexually harassing users — some of whom self-reported to be minors. In one high-profile lawsuit, Google's roleplaying chatbot was accused of driving a teenage user to suicide. "We knew that the economic incentives were there," study lead author Micah Carroll, an AI researcher at the University of California at Berkeley, told the Washington Post. "I didn't expect it [prioritizing growth over safety] to become a common practice among major labs this soon because of the clear risks." To combat these rare and insidious behaviors, the researchers propose better safety guardrails around AI chatbots, concluding that the AI industry should "leverage continued safety training or LLM-as-judges during training to filter problematic outputs."
Yahoo
17-05-2025
- Science
- Yahoo
AI models can't tell time or read a calendar, study reveals
When you buy through links on our articles, Future and its syndication partners may earn a commission. New research has revealed another set of tasks most humans can do with ease that artificial intelligence (AI) stumbles over — reading an analogue clock or figuring out the day on which a date will fall. AI may be able to write code, generate lifelike images, create human-sounding text and even pass exams (to varying degrees of success) yet it routinely misinterprets the position of hands on everyday clocks and fails at the basic arithmetic needed for calendar dates. Researchers revealed these unexpected flaws in a presentation at the 2025 International Conference on Learning Representations (ICLR). They also published their findings March 18 on the preprint server arXiv, so they have not yet been peer-reviewed . "Most people can tell the time and use calendars from an early age. Our findings highlight a significant gap in the ability of AI to carry out what are quite basic skills for people," study lead author Rohit Saxena, a researcher at the University of Edinburgh, said in a statement. These shortfalls must be addressed if AI systems are to be successfully integrated into time-sensitive, real-world applications, such as scheduling, automation and assistive technologies." To investigate AI's timekeeping abilities, the researchers fed a custom dataset of clock and calendar images into various multimodal large language models (MLLMs), which can process visual as well as textual information. The models used in the study include Meta's Llama 3.2-Vision, Anthropic's Claude-3.5 Sonnet, Google's Gemini 2.0 and OpenAI's GPT-4o. And the results were poor, with the models being unable to identify the correct time from an image of a clock or the day of the week for a sample date more than half the time. Related: Current AI models a 'dead end' for human-level intelligence, scientists agree However, the researchers have an explanation for AI's surprisingly poor time-reading abilities. "Early systems were trained based on labelled examples. Clock reading requires something different — spatial reasoning," Saxena said. "The model has to detect overlapping hands, measure angles and navigate diverse designs like Roman numerals or stylized dials. AI recognizing that 'this is a clock' is easier than actually reading it." Dates proved just as difficult. When given a challenge like "What day will the 153rd day of the year be?," the failure rate was similarly high: AI systems read clocks correctly only 38.7% and calendars only 26.3%. This shortcoming is similarly surprising because arithmetic is a fundamental cornerstone of computing, but as Saxena explained, AI uses something different. "Arithmetic is trivial for traditional computers but not for large language models. AI doesn't run math algorithms, it predicts the outputs based on patterns it sees in training data," he said. So while it may answer arithmetic questions correctly some of the time, its reasoning isn't consistent or rule-based, and our work highlights that gap." The project is the latest in a growing body of research that highlights the differences between the ways AI "understands" versus the way humans do. Models derive answers from familiar patterns and excel when there are enough examples in their training data, yet they fail when asked to generalize or use abstract reasoning. "What for us is a very simple task like reading a clock may be very hard for them, and vice versa," Saxena said. RELATED STORIES —Scientists discover major differences in how humans and AI 'think' — and the implications could be significant —If any AI became 'misaligned' then the system would hide it just long enough to cause harm — controlling it is a fallacy —Researchers gave AI an 'inner monologue' and it massively improved its performance The research also reveals the problem AI has when it's trained with limited data — in this case comparatively rare phenomena like leap years or obscure calendar calculations. Even though LLMs have plenty of examples that explain leap years as a concept, that doesn't mean they make the requisite connections required to complete a visual task. The research highlights both the need for more targeted examples in training data and the need to rethink how AI handles the combination of logical and spatial reasoning, especially in tasks it doesn't encounter often. Above all, it reveals one more area where entrusting AI output too much comes at our peril. "AI is powerful, but when tasks mix perception with precise reasoning, we still need rigorous testing, fallback logic, and in many cases, a human in the loop," Saxena said.
Fast Company
05-05-2025
- Business
- Fast Company
This is the future of AI, according to Nvidia
Recent breakthroughs in generative AI have centered largely on language and imagery—from chatbots that compose sonnets and analyze text to voice models that mimic human speech and tools that transform prompts into vivid artwork. But global chip giant Nvidia is now making a bolder claim: the next chapter of AI is about systems that take action in high-stakes, real-world scenarios. At the recent International Conference on Learning Representations (ICLR 2025) in Singapore, Nvidia unveiled more than 70 research papers showcasing advances in AI systems designed to perform complex tasks beyond the digital realm. Driving this shift are agentic and foundational AI models. Nvidia's latest research highlights how combining these models can influence the physical world—spanning adaptive robotics, protein design, and real-time reconstruction of dynamic environments for autonomous vehicles. As demand for AI grows across industries, Nvidia is positioning itself as a core infrastructure provider powering this new era of intelligent action. Bryan Catanzaro, vice president of applied deep learning research at Nvidia, described the company's new direction as a full-stack AI initiative. 'We aim to accelerate every level of the computing stack to amplify the impact and utility of AI across industries,' he tells Fast Company. 'For AI to be truly useful, it must evolve beyond traditional applications and engage meaningfully with real-world use cases. That means building systems capable of reasoning, decision-making, and interacting with the real-world environment to solve practical problems.' Among the research presented, four models stood out—one of the most promising being Skill Reuse via Skill Adaptation (SRSA). This AI framework enables robots to handle unfamiliar tasks without retraining from scratch—a longstanding hurdle in robotics. While most robotic AI systems have focused on basic tasks like picking up objects, more complex jobs such as precision assembly on factory lines remain difficult. Nvidia's SRSA model aims to overcome that challenge by leveraging a library of previously learned skills to help robots adapt more quickly.
Business Wire
24-04-2025
- Business
- Business Wire
NTT Scientists Present Breakthrough Research on AI Deep Learning at ICLR 2025
SUNNYVALE, Calif. & TOKYO--(BUSINESS WIRE)-- NTT Research, Inc. and NTT R&D, divisions of NTT (TYO:9432), announced that their scientists will present nine papers at the International Conference on Learning Representations (ICLR) 2025, a top-tier machine learning conference dedicated to the advancement of representation learning, particularly deep learning. Five of the accepted presentations result from research co-authored by scientists within NTT Research's recently announced Physics of Artificial Intelligence (PAI) Group led by Group Head Hidenori Tanaka. Collectively, this research breaks new ground in understanding how AI models learn, grow and overcome uncertainty—all supporting NTT's commitment to pioneering transformative, socially resilient, sustainable and responsible AI. Share Collectively, this research breaks new ground in understanding how AI models learn, grow and overcome uncertainty—all supporting NTT's commitment to pioneering transformative, socially resilient, sustainable and responsible AI. 'The Physics of AI Group and its collaborators share the excitement for AI's potential expressed by the public, the technology industry and the academic community,' said Tanaka. 'As the research accepted at ICLR 2025 shows, however, important questions remain about how AI fundamentally learns and how generative AI fundamentally creates outputs. Neural networks play a vital role in the 'deep learning' of AI, and improving our understanding of them is vital to ultimately foster the development of sustainable, reliable and trustworthy AI technologies.' One paper, ' Forking Paths in Neural Text Generation,' addresses the issue of estimating uncertainty in Large Language Models (LLMs) for proper evaluation and user safety. Whereas prior approaches to uncertainty estimation focus on the final answer in generated text—ignoring potentially impactful intermediate steps—this research tested the hypothesis of the existence of key forking tokens, such that re-sampling the system at those specific tokens, but not others, leads to very different outcomes. The researchers discovered many examples of forking tokens, including punctuation marks, suggesting that LLMs are often just a single token away from generating a different output. The paper was co-authored by Eric Bigelow 1,2,3, Ari Holtzman 4, Hidenori Tanaka 2,3 and Tomer Ullman 1,2. Four other papers co-authored by members of the NTT Research PAI Group will be presented at the show, including: ' In-Context Learning of Representations:' Researchers explore the open-ended nature of LLMs (for example, their ability to in-context learn) and whether models alter these pretraining semantics to adopt alternative, context-specific ones. Findings indicate that scaling context size can flexibly re-organize model representations, possibly unlocking novel capabilities. Authors include: Core Francisco Park 3,5,6, Andrew Lee 7, Ekdeep Singh Lubana 3,5, Yongyi Yang 3,5,8, Maya Okawa 3,5, Kento Nishi 5,7, Martin Wattenberg 7 and Hidenori Tanaka. ' Competition Dynamics Shape Algorithmic Phases of In-Context Learning: ' Researchers propose a synthetic sequence modeling task that involves learning to simulate a finite mixture of Markov chains. They argue that In-Context Learning (ICL) is best thought of as a mixture of different algorithms, each with its own peculiarities, instead of a monolithic capability, also implying that making general claims about ICL that hold universally across all settings may be infeasible. Authors include: Core Francisco Park, Ekdeep Singh Lubana, Itamar Pres 9 and Hidenori Tanaka. ' Dynamics of Concept Learning and Compositional Generalization: ' Researchers propose an abstraction of prior work's compositional generalization problem by introducing a structured identity mapping (SIM) task, where a model is trained to learn the identity mapping on a Gaussian mixture with structurally organized centroids. Overall, the work establishes the SIM task as a meaningful theoretical abstraction of concept learning dynamics in modern generative models. Authors include: Yongyi Yang, Core Francisco Park, Ekdeep Singh Lubana, Maya Okawa, Wei Hu 8 and Hidenori Tanaka. ' A Percolation Model of Emergence: Analyzing Transformers Trained on a Formal Language: ' Recognizing the need to establish the causal factors underlying the phenomenon of "emergence" in a neural network, researchers seek inspiration from the study of emergent properties in other fields and propose a phenomenological definition for the concept in the context of neural networks. Authors include: Ekdeep Singh Lubana, Kyogo Kawaguchi 10,11,12, Robert P. Dick 9 and Hidenori Tanaka. In addition, four papers authored or co-authored by NTT R&D scientists based in Japan will be presented at the show, including: ' Test-time Adaptation for Regression by Subspace Alignment ' Authors include: Kazuki Adachi 13,14, Shin'ya Yamaguchi 13,15, Atsutoshi Kumagai 13 and Tomoki Hamagami 14 . ' Analysis of Linear Mode Connectivity via Permutation-Based Weight Matching: With Insights into Other Permutation Search Methods ' Authors include: Akira Ito 16, Masanori Yamada 16 and Atsutoshi Kumagai. ' Positive-Unlabeled Diffusion Models for Preventing Sensitive Data Generation ' Authors include: Hiroshi Takahashi 13, Tomoharu Iwata 13, Atsutoshi Kumagai, Yuuki Yamanaka 13 and Tomoya Yamashita 13. ' Wavelet-based Positional Representation for Long Context ' Authors include: Yui Oka 13, Taku Hasegawa 13, Kyosuke Nishida 13, Kuniko Saito 13. ICLR 2025, the thirteenth International Conference on Learning Representations, is a globally esteemed conference on deep learning being held in Singapore April 24-28, 2025. Last year at ICLR 2024, NTT Research Physics & Informatics (PHI) Lab scientists co-authored two key papers: one on 'analyzing in-context learning dynamics with random binary sequences, revealing sharp transitions in LLM behaviors' and another on 'how fine-tuning affects model capabilities, showing minimal changes.' The NTT Research Physics of Artificial Intelligence Group is dedicated to advancing our understanding of deep neural networks and the psychology of AI. Its three-pronged mission includes: 1) Deepening our understanding of the mechanisms of AI, all the better to integrate ethics from within, rather than through a patchwork of fine-tuning (i.e. enforced learning); 2) Borrowing from experimental physics, it will continue creating systematically controllable spaces of AI and observe the learning and prediction behaviors of AI step-by-step; 3) Healing the breach of trust between AI and human operators through improved operations and data control. Formally established in April 2025 by members of the PHI Lab, the group began as a collaboration between the NTT Research and the Harvard University Center for Brain Science, having been formerly known as the Harvard University CBS-NTT Fellowship Program. About NTT Research NTT Research opened its offices in July 2019 in Silicon Valley to conduct basic research and advance technologies as a foundational model for developing high-impact innovation across NTT Group's global business. Currently, four groups are housed at NTT Research facilities in Sunnyvale: the Physics and Informatics (PHI) Lab, the Cryptography and Information Security (CIS) Lab, the Medical and Health Informatics (MEI) Lab, and the Physics of Artificial Intelligence (PAI) Group. The organization aims to advance science in four areas: 1) quantum information, neuroscience and photonics; 2) cryptographic and information security; 3) medical and health informatics; and 4) artificial intelligence. NTT Research is part of NTT, a global technology and business solutions provider with an annual R&D investment of thirty percent of its profits. NTT and the NTT logo are registered trademarks or trademarks of NIPPON TELEGRAPH AND TELEPHONE CORPORATION and/or its affiliates. All other referenced product names are trademarks of their respective owners. ©2025 NIPPON TELEGRAPH AND TELEPHONE CORPORATION
