One of the best hackers in the US is an AI bot

The Star

3 days ago

A hacker named Xbow has topped a prestigious security industry US leaderboard that tracks who has found and reported the most vulnerabilities in software from large companies. Xbow isn't a person – it's an artificial intelligence tool developed by a company of the same name.
This is the first time a company's AI product has topped HackerOne's US leaderboard by reputation, which measures how many vulnerabilities have been found and the importance of each one, according to HackerOne co-founder Michiel Prins. Now, the year-old startup has raised US$75mil (RM317.88mil) in a new funding round led by Altimeter Capital, with participation from existing investors Sequoia Capital and NFDG. It declined to share its valuation.
Security researchers and hackers have long automated parts of their work and AI has shown up as a key tool in the past two years, Prins said. Nearly all human hackers now augment their efforts with AI and there are a handful of firms trying to do what Xbow does – Prins calls them hackbot companies.
Xbow, founded in January 2024 by GitHub veteran Oege de Moor, automates penetration testing, where hackers try to find security flaws and break into corporate networks. Companies often hire or employ people to do that, called red teams, as a way of improving and protecting their network and software. But red teaming and penetration testing is costly – US$18,000 (RM76,292) on average and few weeks of work for a test on a single system, says de Moor – and so it often doesn't get done frequently enough. De Moor wants to sell his product to enable customers to go through the process continuously or at least more often, and before new products and systems go live.
"By automating this we can completely change the equation,' said de Moor, who formerly oversaw Microsoft Corp-owned GitHub's Copilot for AI code-generation.
The challenge is that well-financed hackers are also using AI algorithms to automate attacks and increase their frequency at a lower cost. Xbow has "something that works now and it's exciting, but also somewhat terrifying because we are now in the era of machines hacking machines,' said Nat Friedman of NFDG, and a former GitHub chief executive officer.
De Moor, who also spent two decades as a computer science professor at Oxford University, expects the balance of power to eventually favor defenders, using tools like Xbow. "There might be a period of chaos where not everybody gets ready for these AI-powered attacks,' he said. Now, "we can, for the first time, have a good hope that defenders can find and fix all the vulnerabilities before a system goes out.'
De Moor founded Semmle, a startup for finding security flaws in code that was acquired by GitHub in 2019. Microsoft had bought GitHub the previous year and named Friedman CEO. He wanted to make a series of acquisitions to add new products and entrepreneurial talent.
Friedman and Altimeter Capital partner Apoorv Agrawal said they were looking at ways AI could boost cybersecurity when de Moor began Xbow. "Cybersecurity is going through a credibility crisis. There are a lot of alerts,' Agrawal said. What chief information security officers "want is less, not more, they want simplicity and less alerts,' he added. "How do you make this work? AI can help.'
HackerOne offers a security platform where companies who want their software vetted can offer bounties for finding bugs. There are open programs and ones that are invitation-only. Xbow is active in both. When an AI like Xbow's finds a vulnerability, HackerOne requires a human at the company to vet it to filter out AI hallucinations. Then Xbow goes to the company whose product contains the supposed flaw. If it confirms the issue, Xbow earns reputation points – hackers get more points the more severe the issue.
As part of that work, the Xbow product successfully found and reported security bugs to more than a dozen well-known companies, according to de Moor. The list includes Amazon.com Inc, Walt Disney Co, PayPal Holdings Inc and Sony Group Corp. De Moor declined to name Xbow's current customers except to say they are large financial services and technology companies.
Xbow's team includes GitHub veterans like Nico Waisman, who served as chief information security officer at Lyft Inc, and is now Xbow head of security, and Albert Ziegler, Xbow's head of AI, who worked at GitHub and Semmle.
While Xbow's algorithm does well in finding things like common coding errors and security issues, it does poorly at realising when a flaw results from product design logic. For example, it needs to be explicitly told when looking at a medical web site that prescriptions should be kept private, de Moor said. And it won't understand that while a doctor or a pharmacist needs to be able to access the prescriptions of multiple patients, it's a security problem if one patient can see another's meds.
In the future, Xbow also wants to add the ability to tell customers how to correct the security flaws and make coding suggestions for those fixes.
Widespread adoption will also require getting customers to change how they work, Altimeter's Agrawal said.
"Whenever there's a sufficiently advanced technology, the last-mile adoption requires a change of workflows,' Agrawal said. "It requires a change of people's behaviors that they've been doing for years, sometimes decades." – Bloomberg