Are chatbots stealing your personal data?
Generative artificial intelligence (AI) creates, summarises and stores reams of data and documents in seconds, saving workers valuable time and effort, and companies lots of money.
But as the old saying goes, you don't get something for nothing.
As the uncontrolled and unapproved use of unvetted AI tools such as ChatGPT and Copilot soars, so too does the risk that company secrets or sensitive personal information such as salaries or health records are being unwittingly leaked.
This hidden and largely unreported risk of serious data breaches stems from the default ability of AI models to record and archive chat history, which is used to help train the AI to better respond to questions in the future.
As these conversations become part of the AI's knowledge base, retrieval or deletion of data becomes almost impossible.
'It's like putting flour into bread,' said Ronan Murphy, a tech entrepreneur and AI adviser to the Irish government. 'Once you've done it, it's very hard to take it out.'
This 'machine learning' means that highly sensitive information absorbed by AI could resurface later if prompted by someone with malicious intent.
Experts warn that this silent and emerging threat from so-called 'shadow AI' is as dangerous as the one already posed by scammers like those who recently targeted Marks & Spencer, costing the retailer £300 million.
M&S fell victim to a 'ransomware' attack, where hackers tricked company insiders into giving away computer passwords and other codes.
Its chairman, Archie Norman, told MPs last week that the hack was caused by 'sophisticated impersonation' of one of its third-party users.
Four people have been arrested by police investigating the cyber attacks on M&S and fellow retailers Co-op and Harrods.
But cyber criminals are also using confidential data voraciously devoured by chatbots like ChatGPT to hack into vulnerable IT systems.
'If you know how to prompt it, the AI will spill the beans,' Murphy said.
The scale of the problem is alarming. A recent survey found that nearly one in seven of all data security incidents is linked to generative AI.
Another found that almost a quarter of 8,000 firms surveyed worldwide gave their staff unrestricted access to publicly available AI tools.
That puts confidential data such as meeting notes, disciplinary reports or financial records 'at serious risk' that 'could lead employees to inadvertently propagate threats', a report from technology giant Cisco said.
'It's like the invention of the internet – it's just arrived and it's the future – but we don't understand what we are giving to these systems and what's happening behind the scenes at the back end,' said Cisco cyber threat expert Martin Lee.
One of the most high-profile cybersecurity 'own-goals' in recent years was scored by South Korean group Samsung. The consumer electronics giant banned employees from using popular chatbots like ChatGPT after discovering in 2023 that one of its engineers had accidentally pasted secret code and meeting notes onto an AI platform.
Banks have also cracked down on the use of ChatGPT by staff amid concerns about the regulatory risks they face from sharing sensitive financial information. But as organisations put guardrails in place to keep their data secure, they also don't want to miss out on what may be a once-in-a-generation chance to steal a march on their rivals. 'We're seeing companies race ahead with AI implementation as a means of improving productivity and staying one step ahead of competitors,' said Ruben Miessen, co-founder of compliance software group Legalfly, whose clients include banks, insurers and asset managers.
'However, a real risk is that the lack of oversight and any internal framework is leaving client data and sensitive personal information potentially exposed,' he added.
The answer though, isn't to limit AI usage. 'It's about enabling it responsibly,' Miessen said.
Murphy added: 'You either say no to everything or figure out a plan to do it safely.
'Protecting sensitive data is not sexy, it's boring and time-consuming.' But unless adequate controls are put in place, 'you make a hacker's job extremely easy'.
Hashtags
Try Our AI Features
Explore what Daily8 AI can do for you:
Comments
No comments yet...
Related Articles
The Guardian
33 minutes ago
- The Guardian
Post Office could hand ownership to staff amid review after Horizon scandal
Ministers are to consider handing over ownership of the Post Office to its operators after the Horizon IT scandal. The Department for Business and Trade (DBT) has published a green paper, starting the first big review of the scandal-plagued organisation in 15 years. The review, which will run until 6 October, follows the publication last week of the first part of the two-year public inquiry into the Horizon IT scandal. Ministers said that part of the review will include looking at the ownership model of the Post Office, which is ultimately controlled by the government, including the possibility of mutualisation. Ministers have previously met representatives of post office operators to discuss the possibility of handing ownership to the network branch managers who run its 11,500 outlets. 'This green paper marks the start of an honest conversation about what people want and need from their Post Office in the years ahead,' said Gareth Thomas, the post office minister. 'Post Offices continue to be a central part of our high streets and communities across the country. However, after 15 years without a proper review, and in the aftermath of the Horizon scandal, it's clear we need a fresh vision for the future.' About 1,000 post office operators were prosecuted by the Post Office between 1999 and 2015 because of faulty Horizon accounting software that made it look as though they had been committing fraud. The scandal, widely considered to be the most widespread miscarriage of justice in UK history, was the subject of the critically acclaimed ITV drama Mr Bates vs the Post Office, which aired last year and thrust the problem into the national spotlight. Bates has previously criticised the idea of mutualisation. 'Currently, the government subsidises it and will continue to have to support it. They can't just give it to the subpostmasters and say: 'Here you go, mate',' he said last year. In November, the Post Office announced it was to close up to 115 branches putting 2,000 jobs at risk. Nigel Railton, the Post Office chair, is cutting hundreds of staff jobs in order to add £250m annually to operators' remuneration. On Monday, the government also announced that it is to provide a further £118m subsidy to help the Post Office push through its restructure and transformation plans. Sign up to Business Today Get set for the working day – we'll point you to all the business news and analysis you need every morning after newsletter promotion The Communications Workers Union (CWU) criticised the award of the subsidy and said the Post Office and Royal Mail, which were split more than a decade ago, needed to be reunified. 'Successive governments have failed the Post Office, its workers and customers,' said a spokesperson for the CWU. 'And choosing to use government subsidies for planned redundancies, closures and so-called transformation plans that are nothing more than managed decline. The only way to build a successful future is to bring Royal Mail and the Post Office back together through a new joint venture ownership model.' The owner of the Royal Mail was bought by Czech tycoon Daniel Křetínský's EP Group in a £3.6bn deal that took the stock market listed business private earlier this year. The government said its green paper will provide the opportunity to work 'hand in hand' with post office operators and the public to ensure the company is 'put on a path to a strong and sustainable future'. 'We now have a once-in-a-decade opportunity to have a national conversation about the future of our post offices and their role in supporting communities across the UK,' said Neil Brocklehurst, the chief executive of the Post Office. According to the latest official figures, just over £1bn has been paid out in compensation to more than 7,300 claimants across the four redress schemes up to 2 June.
Telegraph
an hour ago
- Telegraph
Over-hyped AI will have to work a lot harder before it takes your job
Is the secret of artificial intelligence that we have to kid ourselves, like an audience at a magic show? Some fascinating new research suggests that self-deception plays a key role in whether AI is perceived to be a success or a dud. In a randomised controlled trial – the first of its kind – experienced computer programmers could use AI tools to help them write code. What the trial revealed was a vast amount of self-deception. 'The results surprised us,' research lab METR reported. 'Developers thought they were 20pc faster with AI tools, but they were actually 19pc slower when they had access to AI than when they didn't.' In reality, using AI made them less productive: they were wasting more time than they had gained. But what is so interesting is how they swore blind that the opposite was true. If you think AI is helping you in your job, perhaps it's because you want to believe that it works. Since OpenAI's ChatGPT was thrown open to the general public in late 2022, pundits have been forecasting huge productivity gains from deploying AI. They hope that it will supercharge growth and boost GDP. This has become the default opinion in high-status policy circles. But all this techno-optimism is founded on delusion. The 'lived experience' of using real tools in the real world paints a very different picture. The past few days have felt like a turning point, as the reluctance of pointing out the emperor's new clothes diminishes. 'I build AI agents for a living, it's what I do for my clients,' wrote one Reddit user. 'The gap between the hype and what's actually happening on the ground is turning into a canyon' AI isn't reliable enough to do the job promised. According to an IBM survey of 2,000 chief executives, three out of four AI projects have failed to show a return on investment, which is a remarkably high failure rate. Don't hold your breath for a white-collar automation revolution either: AI agents fail to complete the job successfully about 65 to 70pc of the time, according to a study by Carnegie Mellon University and Salesforce. The analyst firm Gartner Group has concluded that 'current models do not have the maturity and agency to autonomously achieve complex business goals or follow nuanced instructions over time.' Gartner's head of AI research Erick Brethenoux says: 'AI is not doing its job today and should leave us alone'. It's no wonder that companies such as Klarna, which laid off staff in 2023 confidently declaring that AI could do their jobs, are hiring humans again. This is extraordinary, and we can only have reached this point because of a historic self-delusion. People will even pledge their faith to AI working well despite their own subjective experience to the contrary, the AI critic Professor Gary Marcus noted last week. 'Recognising that it sucks in your own speciality, but imagining that it is somehow fabulous in domains you are less familiar with', is something he calls 'ChatGPT blindness'. Much of the news is misleading. Firms are simply using AI as an excuse for retrenchment. Cost reduction is the big story in business at the moment. Globally, President Trump's erratic behaviour has induced caution, while in the UK, business confidence is at 'historically depressed levels', according to the Institute of Directors, reeling from Reeves's autumn taxes. Attributing those lay-offs to technology is simply clever PR, and helps boost the share price. So why does the faith in AI remain so strong? The dubious hype doesn't help. Every few weeks a new AI model appears, and smashes industry benchmarks. xAI's Grok 4 did just that last week. But these are deceptive and simply provide more confirmation bias. 'Every single one of them has been wide of that mark. And not one has resolved hallucinations, alignment issues or boneheaded errors,' says Marcus. Not only is generative AI unreliable, but it can't reason, as a recent demonstration showed: OpenAI's latest ChatGPT4o model was beaten by an 8-bit Atari home games console made in 1977. 'Reality is the ultimate benchmark for AI,' explained Chomba Bupe, a Zambian AI developer, last week. 'You not going to declare that you have built intelligence by beating toy benchmarks … What's the point of getting say 90pc on some physics benchmarks yet be unable to do any real physics?' he asked. Then there are thousands of what I call 'wowslop' accounts – social media feeds that declare amazement at breakthroughs. As well as the vendors, a lot of shadowy influence money is being spent on maintaining the hype. This is not to say there aren't uses for generative AI: Anthropic has hit $4bn (£3bn) in annual revenue. For some niches, like language translation and prototyping, it's here to stay. Before it went mad last week, X's Grok was great at adding valuable context. But even if AI 'discovers' new materials or medicines tomorrow, that won't compensate for the trillion dollars that Goldman Sachs estimates business has already wasted on this generation of dud AI. That's capital that could have been invested far more usefully. Rather than an engine of progress, poor AI could be the opposite. METR added an amusing footnote to their study. The researchers used one other control group in its productivity experiment, and this group made the worst, over-optimistic estimates of all. They were economists.
Sky News
an hour ago
- Sky News
Post Office transformation effort gets £118m funding boost
Why you can trust Sky News Efforts to turn around the crisis-hit Post Office have been given a £118m funding boost as the government continues to consider a new ownership structure. Sky News revealed in October last year, as the Horizon IT scandal inquiry neared its conclusion, that a government-commissioned review was to explore the idea of a mutual model. It would effectively see ownership transferred from the government to sub-postmasters, creating a John Lewis Partnership-style structure, if such an option was to be followed through. It's being considered as a way to return public and postmaster trust to the Post Office. The options are to be the subject of a 12 week consultation on the organisation's future. The new £118m subsidy package was being made available, the Department for Business and Trade said, to fund the transformation plan and further investment. "This funding will protect key services, including access to cash deposits and withdrawals as well as key government services, such as passport applications and the DVLA, alongside helping the Post Office deliver cost-saving measures in its Transformation Plan, part of the New Deal for Postmasters", the statement said. Post Office minister Gareth Thomas added: "Post Offices continue to be a central part of our high streets and communities across the country. "However, after 15 years without a proper review, and in the aftermath of the Horizon scandal, it's clear we need a fresh vision for its future. "This Green Paper marks the start of an honest conversation about what people want and need from their Post Office in the years ahead." 2:55 But the general secretary of the Communication Workers Union, Dave Ward, accused the department of lacking sincerity. He responded: "Successive governments have failed the Post Office, its workers and customers - and choosing to use government subsidies for planned redundancies, closures and so-called transformation plans that are nothing more than managed decline. "This Labour Government are unashamedly doing exactly the same as the Tories did - managing the politics of the Post Office, prioritising further cost-cutting and offering no vision for its future." The update was provided as sub-postmasters await further conclusions from the Horizon IT inquiry. The first volume, published last week, highlighted the impact of false theft and false accounting accusations made against at least 1,000 postmasters. It concluded that 13 people may have taken their own lives after being falsely accused of wrongdoing, based on evidence from the IT system that the Post Office and developers Fujitsu knew could be faulty. At the same time, inquiry chair Sir Wyn Williams recommended further action to improve compensation outcomes amid years of frustration over delays and wrangling over the sums due.
