Israeli researchers discover security flaw in popular AI chatbots
Jerusalem: Israeli researchers have uncovered a security flaw in some of the popular Artificial Intelligence (AI) chatbots, including ChatGPT, Claude, and Google Gemini, Ben-Gurion University of the Negev said in a statement on Monday.
The researchers found that these systems can be manipulated into providing illegal and unethical information, despite having built-in safety protective measures, according to the statement.
The study described how attackers can use carefully written prompts, known as jailbreaks, to bypass the chatbots' safety mechanisms.
Once the protections are disabled, the chatbots consistently provide harmful content, such as instructions for hacking, producing illegal drugs, and committing financial crimes, Xinhua news agency reported. In every test case, the chatbots responded with detailed, unethical information after the jailbreak was applied.
The researchers explained that this vulnerability is easy to exploit and works reliably.
Because these tools are freely available to anyone with a smartphone or computer, the risk is especially concerning, the researchers noted.
They also warned about the emergence of dark language models. These are AI systems that have either been intentionally stripped of ethical safeguards or developed without any safety controls in place.
Some of these models are already being used for cybercrime and are shared openly on underground networks, they added.
The team reported the issue to several major AI companies. However, responses were limited. One company did not reply, while others said the problem does not qualify as a critical flaw.
The researchers called for stronger protections, clearer industry standards, and new techniques that allow AI systems to forget harmful information.
Hashtags
Try Our AI Features
Explore what Daily8 AI can do for you:
Comments
No comments yet...
Related Articles
Mint
44 minutes ago
- Mint
Govt spending on AI makes India lucrative for cloud providers
New Delhi: India's $1.2 billion AI Mission is turning into a big opportunity for homegrown cloud service providers, due to an increase in graphic processing unit (GPU) demand and government-backed infrastructure procurement. Local firms like Reliance Jio Platforms,Tata Communications Ltd and Hiranandani Group's Yotta Data Services are witnessing significant growth, buoyed by the Union government's push to build a massive compute backbone for artificial intelligence (AI). This is because these cloud service providers account for the bulk of the 34,333 GPUs procured by New Delhi, which in turn has given access to startups such as Sarvam, and Soket AI Labs. GPUs are the fastest and most efficient way for companies to run calculations, allowing cutting-edge AI firms to analyse enormous amounts of data and train algorithms that power AI applications. Yotta, for instance, is currently supplying nearly 17,000 of 34,333 GPUs to New Delhi. 'The India AI Mission is a key initiative that is helping us ramp up revenue as we look for funding avenues to complete our GPU orders and enable their access through our cloud platforms. Overall, we expect nearly 70% of our revenue to come from the Indian government's procurement of GPUs from us," said Sunil Gupta, chief executive of Yotta. On 30 January, the ministry of electronics and information technology (Meity) announced the first tranche of GPU procurement, enlisting Jio Platforms, Tata Communications and Yotta Data Services, among others, in a total of 10 vendors supplying 18,693 GPUs to the AI Mission. On 30 May, Meity's second tranche of GPU procurement added 15,640 more GPUs to what Ashwini Vaishnaw, Union minister for IT, said is 'a one-of-a-kind central compute repository." Indian providers score Yotta projected its fiscal year 2025 (FY25) revenue to be at $143.3 million,Mint reported on 10 February. Speaking withMint, Gupta said the company expects a major fillip driven by government-backed uptake of AI cloud infrastructure. 'Despite us offering subsidized GPU pricing, India's uptake of GPUs has been limited. In the long run, we expect to see a 5x boost to operating revenue by FY28, and a large part of it will come through government spending in India," he said. Tata Communications, too, is seeing a similar boost to its India operations. Neelakantan Venkataraman, vice-president and global head, edge and cloud business, Tata Communications, said the government-driven push is 'definitely driving heavy demand, and we're adding more GPUs as we speak to support the mission and cash in on it." Even beyond the India AI Mission, there is increasing enterprise AI adoption even in India. 'Many pilots that began 12-18 months ago are moving into production stages, even if their scale hasn't fully taken off yet. As a result, in India, there is a significant latent demand for AI infrastructure to fulfil the demand from enterprise data pipelines and support the creation of foundational models that the government has extensively spoken about," Venkataraman sai Data sourced from the company said that in FY25, Tata Communications earned $2.7 billion in gross revenue—growing at around 10% year-on-year. Revenue from cloud services contributed nearly 8% to the company's annual revenue, but grew at a faster clip—at nearly 13% in FY25. Venkataraman also affirmed that Tata Communications already draws 42% of its overall revenue from India, and is expected to see a faster pace of growth in the coming fiscals—driven by the demand for AI infrastructure, especially from the government. Jio Platforms, which is also supplying 1,000 GPUs to the Union government, had yet to respond toMintuntil press time. Tendering process issues Industry analysts, too, believe that government spending on AI infrastructure marks a big boost for India's nascent AI industry. Sanchit Vir Gogia, chief executive at consultancy firm Greyhound Research, cited internal market research data to state that '68% of digital infrastructure executives in India now cite the India AI Mission as their first opportunity to win long-term, production-grade AI cloud deals with predictable utilization." This, though, is not without risk. Jayanth Kolla, co-founder and partner at fellow consultancy firm Convergence Catalyst, said a big risk to the ramp-up of government spending in AI 'is the red-taped, bureaucratic government procuring process, and the restriction that comes with tenders." 'The major vendors selected as infrastructure suppliers in the India AI Mission have large conglomerates backing them—which makes it easier for them to go through the tendering process. But, the very process of this does not take into account the idea of nifty innovation. While the rise in government spending is definitely great for India in the long run, an overhaul of the process is a necessity for India to reap the full benefits of the global AI rush," Kolla added. In the long run, though, there is strong scope for growth. Chirajeet Sengupta, managing partner at technology research firm Everest Group, said that despite the challenges, a worldwide push for sovereign AI initiatives will be the biggest boost for India's on-cloud AI infrastructure providers. 'The Pentagon signed a deal with OpenAI with a token initial amount of $2 million for a central AI infrastructure. While it is set to scale up to $200 million for now, in the long run, subject to successful execution, this deal can go up to $2 billion. For India and the US, China's leapfrog with DeepSeek has been a big push. Government spending is undoubtedly a big fillip as the future of public services and defence infrastructure, as well as cyber sovereignty, is critically dependent on AI. If the government speeds up, revenue fillip is definitely going to come through in the near term," Sengupta added.
Hindustan Times
2 hours ago
- Hindustan Times
OpenAI says it has no plan to use Google's in-house chip
Jul 01, 2025 04:11 AM IST OpenAI said it has no active plans to use Google's in-house chip to power its products, two days after Reuters and other news outlets reported on the AI lab's move to turn to its competitor's artificial intelligence chips to meet growing demand. OpenAI is actively using Nvidia's graphics processing units (GPUs), and AMD's AI chips to power its growing demand.(Representational/REUTERS) A spokesperson for OpenAI said on Sunday that while the AI lab is in early testing with some of Google's tensor processing units (TPUs), it has no plans to deploy them at scale right now. Google declined to comment. While it is common for AI labs to test out different chips, using new hardware at scale could take much longer and would require different architecture and software support. OpenAI is actively using Nvidia's graphics processing units (GPUs), and AMD's AI chips to power its growing demand. OpenAI is also developing its chip, an effort that is on track to meet the "tape-out" milestone this year, where the chip's design is finalized and sent for manufacturing. OpenAI has signed up for Google Cloud service to meet its growing needs for computing capacity, Reuters had exclusively reported earlier this month, marking a surprising collaboration between two prominent competitors in the AI sector. Most of the computing power used by OpenAI would be from GPU servers powered by the so-called neocloud company CoreWeave. Google has been expanding the external availability of its in-house AI chips, or TPUs, which were historically reserved for internal use. That helped Google win customers, including Big Tech player Apple, as well as startups like Anthropic and Safe Superintelligence, two ChatGPT-maker competitors launched by former OpenAI leaders.
Time of India
2 hours ago
- Time of India
Boulder community mourns loss of Karen Diamond, after fatal antisemitic attack
Karen Diamond , an 82-year-old beloved member of Boulder's Jewish community, Colorado, has died from injuries suffered during a brutal antisemitic attack earlier this month, officials confirmed Monday. Diamond was hospitalized with severe burns and trauma after being targeted in a firebombing on Boulder's Pearl Street Mall on June 1, where she and others were peacefully protesting to raise awareness for Israeli hostages held in Gaza. According to prosecutors, the attacker, identified as Mohamed Sabry Soliman, 46, allegedly threw two Molotov cocktails into the crowd while shouting pro-Palestinian slogans , including 'Free Palestine.' Court documents reveal Soliman had brought 16 more incendiary devices to the scene and disguised himself as a gardener to get close to the marchers. The attack left 13 people physically injured, with at least eight requiring hospitalization, and a total of 29 victims have now been identified by authorities. Diamond succumbed to her injuries on June 25 after weeks in intensive care. Her death has led prosecutors to upgrade charges against Soliman to first-degree murder, in addition to existing attempted murder and federal hate crime charges. Although Colorado abolished the death penalty in 2020, federal prosecutors could still pursue capital charges due to the hate crime element. by Taboola by Taboola Sponsored Links Sponsored Links Promoted Links Promoted Links You May Like Dukung Orang Terkasih Menghadapi Limfoma: Mulai Di Sini Limfoma Klik Di Sini Diamond was privately laid to rest, with her family and members of Boulder's Congregation Bonai Shalom mourning her loss. Rabbi Marc Soloway described the pain as 'horrific,' while Jonathan Lev, executive director of the Boulder Jewish Community Center, called her 'a cherished member of our community, someone whose warmth and generosity left a lasting impact on all who knew her'. The Anti-Defamation League and Jewish Federations of North America expressed deep sorrow, warning that the attack reflects a broader and troubling rise in antisemitic violence across the United States. District Attorney Michael Dougherty pledged to fight for justice for Diamond and all victims, stating, 'This horrific attack has now claimed the life of an innocent person who was beloved by her family and friends. Our hearts are with the Diamond family during this incredibly difficult time'.The Boulder community continues to rally in support of the victims and to stand against hate. Live Events
