Hidden dangers of increasingly advanced AI assistants
The evolution of AI assistants over the past few years is nothing short of remarkable. Picture: Trend Micro.
Artificial Intelligence (AI) assistants are becoming more prominent and continue to increase as people require efficient and speedy service.
From querying about a product online to seeking help with your mobile contract, more and more companies are using digital assistants (DA) to cope with the influx of customers wanting attention.
Risks
However, as AI assistants in South Africa proliferate, so do the risks.
Trend Micro Solutions Architect for the Middle East and Africa Zaheer Ebrahim said the evolution of AI assistants over the past few years is nothing short of remarkable.
'Initially designed to perform simple tasks such as setting reminders and playing music, these AI-driven software systems have transformed into sophisticated conversational agents, powered by advanced large language models like ChatGPT and Google Gemini.'
ALSO READ: South Africa on track to regulate artificial intelligence
Extraordinary ability
DAs exhibit an extraordinary ability to understand and respond to natural language queries with impressive accuracy and context sensitivity.
However, Ebrahim warns that as advanced helpers become more sophisticated, the security risks become more complex.
'It is imperative for both developers and users to be proactive and vigilant in addressing emerging threats, ensuring that the benefits of AI are not overshadowed by potential vulnerabilities.
New threats
As DA's become increasingly integrated into daily life and interconnected with various devices and services, they become attractive targets for malicious actors.
Ebrahim said continuous interaction with DA's generates enormous amounts of personal data, including names, addresses, e-mail addresses, phone numbers and even sensitive health information.
'This data is essential for providing personalised and proactive assistance. However, it also raises significant privacy concerns.
'Unauthorised access or misuse of this data can lead to severe consequences, making robust data protection measures crucial.
'Encryption of sensitive data, both at rest and in transit, is a fundamental security measure that needs to be prioritised,' Ebrahim warned.
ALSO READ: Opera adds Aria AI assistant to Opera Mini on Android
Custom skills
Ebrahim added that the advanced helpers also bring malicious 'custom skills' that seem legitimate but actually contain harmful functions.
'In this context, 'skills' refer to specific features or abilities that can be added to AI assistants to enhance their functionality, similar to apps on smartphones. These rogue skills can manipulate the assistant's responses to provide false information.'
Social engineering
He said social engineering can also be executed through DAs, where attackers manipulate the output generated by these assistants to deceive users.
'For instance, a DA might be instructed to relay a seemingly legitimate message from a trusted source, leading users to take harmful actions.
'This threat is particularly concerning as DAs take on more sophisticated tasks, such as managing user finances.'
Recommendations
Ebrahim said that to address these digital assistant-based threats, it is essential for both developers and users to be able to recognise the types of personally identifiable information collected by DAS and collaborate to promote a culture of security awareness, responsible behaviour and best practices.
'A holistic approach incorporating multiple layers of defence is key. Robust authentication mechanisms can ensure user identity verification and safeguard against unauthorised access.
'Encrypting sensitive data at various levels – at rest, in transit and within the DA's internal memory -protects user information from malicious actors.'
Ebrahim advised that employing advanced threat detection techniques, utilising machine learning algorithms and behavioural analysis can proactively defend against evolving cyber threats.
ALSO READ: The rise of AI tools and their impact on smartphone use
Try Our AI Features
Explore what Daily8 AI can do for you:
Comments
No comments yet...
Related Articles
Daily Maverick
6 hours ago
- Daily Maverick
MIT researchers say using ChatGPT can rot your brain. The truth is a little more complicated
Like calculators before them, AI tools can raise the bar for what people can achieve – if they're used the right way. Since ChatGPT appeared almost three years ago, the impact of artificial intelligence (AI) technologies on learning has been widely debated. Are they handy tools for personalised education, or gateways to academic dishonesty? Most importantly, there has been concern that using AI will lead to a widespread 'dumbing down', or decline in the ability to think critically. If students use AI tools too early, the argument goes, they may not develop basic skills for critical thinking and problem-solving. Is that really the case? According to a recent study by scientists from MIT, it appears so. Using ChatGPT to help write essays, the researchers say, can lead to 'cognitive debt' and a 'likely decrease in learning skills'. So what did the study find? The difference between using AI and the brain alone Over the course of four months, the MIT team asked 54 adults to write a series of three essays using either AI (ChatGPT), a search engine, or their own brains ('brain-only' group). The team measured cognitive engagement by examining electrical activity in the brain and through linguistic analysis of the essays. The cognitive engagement of those who used AI was significantly lower than the other two groups. This group also had a harder time recalling quotes from their essays and felt a lower sense of ownership over them. Interestingly, participants switched roles for a final, fourth essay (the brain-only group used AI and vice versa). The AI-to-brain group performed worse and had engagement that was only slightly better than the other group's during their first session, far below the engagement of the brain-only group in their third session. The authors claim this demonstrates how prolonged use of AI led to participants accumulating 'cognitive debt'. When they finally had the opportunity to use their brains, they were unable to replicate the engagement or perform as well as the other two groups. Cautiously, the authors note that only 18 participants (six per condition) completed the fourth, final session. Therefore, the findings are preliminary and require further testing. Does this really show AI makes us stupider? These results do not necessarily mean that students who used AI accumulated 'cognitive debt'. In our view, the findings are due to the particular design of the study. The change in neural connectivity of the brain-only group over the first three sessions was likely the result of becoming more familiar with the study task, a phenomenon known as the familiarisation effect. As study participants repeat the task, they become more familiar and efficient, and their cognitive strategy adapts accordingly. When the AI group finally got to 'use their brains', they were only doing the task once. As a result, they were unable to match the other group's experience. They achieved only slightly better engagement than the brain-only group during the first session. To fully justify the researchers' claims, the AI-to-brain participants would also need to complete three writing sessions without AI. Similarly, the fact the brain-to-AI group used ChatGPT more productively and strategically is likely due to the nature of the fourth writing task, which required writing an essay on one of the previous three topics. As writing without AI required more substantial engagement, they had a far better recall of what they had written in the past. Hence, they primarily used AI to search for new information and refine what they had previously written. What are the implications of AI in assessment? To understand the current situation with AI, we can look back to what happened when calculators first became available. Back in the 1970s, their impact was regulated by making exams much harder. Instead of doing calculations by hand, students were expected to use calculators and spend their cognitive efforts on more complex tasks. Effectively, the bar was significantly raised, which made students work equally hard (if not harder) than before calculators were available. The challenge with AI is that, for the most part, educators have not raised the bar in a way that makes AI a necessary part of the process. Educators still require students to complete the same tasks and expect the same standard of work as they did five years ago. In such situations, AI can indeed be detrimental. Students can for the most part offload critical engagement with learning to AI, which results in ' metacognitive laziness '. However, just like calculators, AI can and should help us accomplish tasks that were previously impossible – and still require significant engagement. For example, we might ask teaching students to use AI to produce a detailed lesson plan, which will then be evaluated for quality and pedagogical soundness in an oral examination. In the MIT study, participants who used AI were producing the 'same old' essays. They adjusted their engagement to deliver the standard of work expected of them. The same would happen if students were asked to perform complex calculations with or without a calculator. The group doing calculations by hand would sweat, while those with calculators would barely blink an eye. Learning how to use AI Current and future generations need to be able to think critically and creatively and solve problems. However, AI is changing what these things mean. Producing essays with pen and paper is no longer a demonstration of critical thinking ability, just as doing long division is no longer a demonstration of numeracy. Knowing when, where and how to use AI is the key to long-term success and skill development. Prioritising which tasks can be offloaded to an AI to reduce cognitive debt is just as important as understanding which tasks require genuine creativity and critical thinking. DM This story first appeared in The Conversation. Vitomir Kovanovic is an associate Professor and Director of the Centre for Change and Complexity in Learning (C3L), Education Futures at the University of South Australia. Rebecca Marrone is a lecturer in Learning Sciences and Development at the Centre for Change and Complexity in Learning (C3L), Education Futures at the University of South Australia.
The South African
a day ago
- The South African
Minister Lamola's spokesperson slams DA over Jonas envoy claims
Minister Ronald Lamola's spokesperson has hit back at the Democratic Alliance (DA). This follows its claims that the United States denied Mcebisi Jonas a diplomatic visa. The DA said the US rejected Jonas' role as South Africa's special envoy. In a post on X (formerly Twitter) on 15 July, Chrispin Phiri accused the DA, lobby groups, and 'fringe media platforms' of undermining national efforts to protect South Africa's economic interests. Phiri said that the US tariffs are a global issue and it's not unique to South Africa. He said the Department of Trade, Industry and Competition is currently handling the issue. The spokesperson accused DA MP Emma Powell of leading a delegation to the United States. He said that the delegation sought help from the US to change foreign and other policies. Phiri stated that this resulted in negative views and sentiments about South Africa. The spokesperson also criticised Powell's trip to Israel with DA MP Andrew Whitfield. Phiri stated that the trip had been 'rightfully criticised' for whitewashing Israel's conduct. JONAS DENIED ENTRY TO THE US The DA alleges that Jonas was denied entry to the US in May. The party said that President Cyril Ramaphosa continued to claim he was acting as a special envoy. The Presidency has yet to respond officially to the claims. The spokesperson dismissed DA MP Emma Louise Powell's claims about Jonas's lack of accreditation, calling it a 'red herring.' 'Special envoys do not get accredited to a country. These are not diplomatic postings and do not require the presentation of credentials. Even envoys representing the US President do not publicly report on their work,' Phiri said. Let us know by leaving a comment below, or send a WhatsApp to 060 011 021 1. Subscribe to The South African website's newsletters and follow us on WhatsApp, Facebook, X and Bluesky for the latest news.
The South African
a day ago
- The South African
DA says US rejected Jonas as envoy, Presidency misled public
The Democratic Alliance says the United States denied Mcebisi Jonas a diplomatic visa in May and rejected his credentials as South Africa's special envoy. According to a statement by DA MP Emma Louise Powell, President Cyril Ramaphosa allegedly knew for months that Jonas was not welcome in Washington, D.C. The president allegedly continued to claim Jonas was still acting as his envoy. The US government reportedly told the Presidency several times to appoint someone else. The DA says Ramaphosa ignored the warnings and misled the public. In a June interview, Ramaphosa allegedly said that Mcebisi Jonas is still the North American envoy and has done good work. The DA says the President made that statement knowing Jonas had already been barred from official diplomatic work. The DA now plans to submit a PAIA request for Jonas's official activities over the past 90 days. In response, Presidential spokesperson Vincent Magwenya allegedly accused the party of overstepping. The DA says Ramaphosa is more focused on saving face than protecting South African jobs. The party warns the country could lose billions in exports once 30% US tariffs take effect in two weeks. The party will write to the Speaker of Parliament to demand answers. It is calling for a new envoy who is recognised by the US and able to fix the damage. 'It has become undeniable that the President is now more concerned with saving face than with safeguarding the livelihoods of South African businesses and workers who will ultimately pay the price for his inaction,' the statement said. Let us know by leaving a comment below, or send a WhatsApp to 060 011 021 1 Subscribe to The South African website's newsletters and follow us on WhatsApp, Facebook, X and Bluesky for the latest news.
