AI is perfecting scam emails, making phishing hard to catch

Axios

27-05-2025

AI chatbots have made scam emails harder to spot and the tells we've all been trained to look for — clunky grammar, weird phrasing — utterly useless.
Why it matters: Scammers are raking in more than ever from basic email and impersonation schemes. Last year, the FBI estimates, they made off with a whopping $16.6 billion.
Thwarting AI-written scams will require a new playbook that leans more on users verifying messages and companies detecting scams before they hit inboxes, experts say.
The big picture: ChatGPT and other chatbots are helping non-English-speaking scammers write typo-free messages that closely mimic trusted senders.
Before, scammers relied on clunky tools like Google Translate, which often were too literal in their translations and couldn't capture grammar and tone.
Now, AI can write fluently in most languages, making malicious messages far harder to flag.
What they're saying:"The idea that you're going to train people to not open [emails] that look fishy isn't going to work for anything anymore," Chester Wisniewski, global field CISO at Sophos, told Axios.
"Real messages have some grammatical errors because people are bad at writing," he added. "ChatGPT never gets it wrong."
The big picture: Scammers are now training AI tools on real marketing emails from banks, retailers and service providers, Rachel Tobac, an ethical hacker and CEO of SocialProof Security, told Axios.
"They even sound like they are in the voice of who you're used to working with," Tobac said.
Tobac said one Icelandic client who had never before worried about employees falling for phishing emails was now concerned.
"Previously, they've been so safe because only 350,000 people comfortably speak Icelandic," she said. "Now, it's a totally new paradigm for everybody."
Threat level: Beyond grammar, the real danger lies in how these tools scale precision and speed, Mike Britton, CISO at Abnormal Security, told Axios.
Within minutes, scammers can use chatbots to create dossiers about the sales teams at every Fortune 500 company and then use those findings to write customized, believable emails, Britton said.
Attackers now also embed themselves into existing email threads using lookalike domains, making their messages nearly indistinguishable from legitimate ones, he added.
"Our brain plays tricks on us," Britton said. "If the domain has a W in it, and I'm a bad guy, and I set up a domain with two Vs, your brain is going to autocorrect."
Yes, but: Spotting scam emails isn't impossible. In Tobac's red team work, she typically gets caught when:
Someone practices what she calls polite paranoia, or when they text or call the organization or person being impersonated to confirm if they sent a suspicious message.
A target uses a password manager and has complex, long passwords.
They have multifactor authentication enabled.
What to watch: Britton warned that low-cost generative AI tools for deepfakes and voice clones could soon take phishing to new extremes.
"It's going to get to the point where we all have to have safe words, and you and I get on a Zoom and we have to have our secret pre-shared key," Britton said. "It's going to be here before you know it."