Cisco Talos Reveals Rise in Malicious Use of AI Tools

TECHx

5 days ago

Home » Emerging technologies » Cyber Security » Cisco Talos Reveals Rise in Malicious Use of AI Tools
Cisco Talos, one of the world's most trusted threat intelligence teams, has revealed how cybercriminals are increasingly abusing artificial intelligence (AI) tools to enhance their operations.
According to a newly published report, large language models (LLMs) are being exploited to generate malicious content and bypass traditional security measures.
Cisco Talos reported that both custom-built and jailbroken (modified) versions of LLMs are now being used to scale cyberattacks. These versions are producing phishing emails, malware, viruses, and other harmful content.
The report noted that some LLMs are being connected to external tools, including email accounts and credit card checkers. This integration is helping cybercriminals automate and amplify their attacks.
Cisco Talos researchers also documented the presence of malicious LLMs on underground forums. These include names such as: FraudGPT
DarkGPT
WhiteRabbitNeo
These tools are advertised with features like ransomware creation, phishing kit generation, and card verification services.
Interestingly, the report also revealed that some fake AI tools are being used to scam fellow cybercriminals.
Cisco Talos highlighted how attackers are jailbreaking legitimate AI models. These jailbreaks aim to bypass safety guardrails and alignment training, allowing the generation of normally restricted content.
Additionally, the report warned that AI models themselves are becoming targets. Attackers are inserting backdoors into downloadable models, enabling them to function as programmed by the attacker when activated.
Models using external data sources are also at risk. If threat actors manipulate the source data, it could compromise the model's behavior.
Fady Younes, Managing Director for Cybersecurity at Cisco covering the Middle East, Africa, Türkiye, Romania, and CIS, commented on the findings. He stated that while large language models offer significant potential, they are now being weaponized to scale attacks.
He emphasized the need for strong AI governance, user awareness, and foundational cybersecurity measures.
'With recent innovations like Cisco AI Defense, we are committed to helping enterprises achieve end-to-end protection as they build, use, and innovate with AI,' Younes added.
Cisco Talos concluded that as AI becomes more integrated into enterprise and consumer systems, security strategies must evolve.
It stressed the importance of: Scanning for tampered AI models
Validating external data sources
Monitoring abnormal LLM behavior
Educating users on the risks of prompt manipulation
The report signals a new phase in the cyber threat landscape. Cisco Talos continues to monitor the situation as part of its mission to strengthen global cybersecurity.