Pay Up, AI Bot: That's the Message From a Key Company in How the Internet Works
It's the latest front to open in an ongoing fight between the creators of content and the AI developers who use that content to train generative AI models. In court, authors and content creators are suing major AI companies for compensation, saying copyrighted content was used without permission. (Disclosure: Ziff Davis, CNET's parent company, in April filed a lawsuit against OpenAI, alleging it infringed Ziff Davis copyrights in training and operating its AI systems.)
While content providers are seeking compensation for information that was used to train models in the past, Cloudflare's move marks a new defensive measure against future efforts to train models.
But it isn't just about blocking crawlers: Cloudflare says it wants to create a marketplace where AI companies can pay to crawl and scrape a site, meaning the provider of that information gets paid, and the AI developer gets permission.
"That content is the fuel that powers AI engines, and so it's only fair that content creators are compensated directly for it," Cloudflare CEO Matthew Prince said in a blog post.
Why websites want to block AI crawlers
Crawlers -- bots that visit and copy the information from a website -- are a vital component of the connected internet. It's how search engines like Google know what's on different websites, and how they can serve you the latest information from places like CNET.
AI crawlers pose distinct challenges for websites. For one, they can be aggressive, generating unsustainable levels of traffic for smaller sites. They also offer little reward for their scraping: If Google crawls a site for search engine results, it will likely send traffic back to that site by including it in search results. Being crawled for training data might mean no additional traffic or even less, if people stop visiting the site and rely just on the AI model.
Read more: AI Essentials: 29 Ways to Make Gen AI Work for You, According to Our Experts
That's why executives from major websites like Pinterest, Reddit and several major publishing companies (including Ziff Davis, which owns CNET) cheered Cloudflare's news in statements.
"The whole ecosystem of creators, platforms, web users and crawlers will be better when crawling is more transparent and controlled, and Cloudflare's efforts are a step in the right direction for everyone," Reddit CEO Steve Huffman said in a statement.
Asked about Cloudflare's announcement, OpenAI said its ChatGPT model is intended to help connect its users to content on the web, similarly to search engines, and that it has integrated search into its chat functions. The company also said it uses a separate model from what Cloudflare has proposed to allow publishers to indicate how AI crawlers should behave, known as robots.txt. OpenAI said the robots.txt model works already and Cloudflare's changes are unnecessary.
The training data tug-of-war
AI models require a ton of data to train. That's how they're able to provide detailed answers to questions and do a decent (if imperfect) job of providing a wide range of information. These models are fed incredible amounts of information and make connections between words and concepts based on what they see in that training data.
The issue is how developers have gotten that data. There are now dozens of lawsuits between content creators and AI companies. Two saw major rulings just last week.
In one case, a federal judge ruled Anthropic followed the law when it used copyright-protected books to train its model Claude -- via a concept called fair use. At the same time, the judge said the company's creation of a permanent library of the books was not, and ordered a new trial on those piracy allegations.
In a separate case, a judge ruled in favor of Meta in a dispute between the company and a group of 13 authors. But Judge Vince Chhabria said the ruling in this case doesn't mean future cases against Meta or other AI companies will go the same way, essentially that "these plaintiffs made the wrong arguments and failed to develop a record in support of the right one."
The idea of charging crawlers to visit a site isn't entirely new. Other companies, like Tollbit, offer services that allow website owners to charge AI companies for crawling. Will Allen, head of AI control, privacy and media products at Tollbit, said the environment around this technology is still developing. "We think it's very early for a content marketplace to form, and we are just starting to experiment here," he told CNET. "We're excited to see many different models flourish."
CNET's Imad Khan contributed to this report.
Try Our AI Features
Explore what Daily8 AI can do for you:
Comments
No comments yet...
Related Articles
The Hill
10 minutes ago
- The Hill
Managers are using AI to determine raises, promotions, layoffs
(NewsNation) — Artificial intelligence (AI) isn't just changing workflows, it's deciding who moves up and who gets the boot. According to a new Resume Builder survey of 1,342 U.S. managers, 6 in 10 said they use AI tools to make decisions about their direct reports. Even more striking is that most managers who use AI said they've turned to it for high-stakes calls, such as determining raises, promotions and even who to let go. Yet two-thirds of those using AI admitted they haven't received training on how to manage people with it, the survey found. ChatGPT was the most popular tool among AI-using managers, with 53% citing it as their go-to. Nearly 30% said they primarily use Microsoft's Copilot, while 16% said they mostly use Google's Gemini. Other surveys have shown that managers are more likely than their employees to use AI, but the latest findings suggest a dystopian future where leadership loses its human touch entirely. 'While AI can support data-driven insights, it lacks context, empathy, and judgment,' Stacie Haller, chief career advisor at Resume Builder, warned in a statement. Haller said it's essential not to lose the 'people' in 'people management,' pointing out that AI reflects the data it's given, which can be flawed and manipulated. The concern is real enough that lawmakers have introduced legislation to limit AI's role in employment decisions. In March, a California state senator introduced the 'No Robo Bosses Act,' aimed at preventing employers from letting AI make key decisions — such as hiring, firing or promotions — without human oversight. 'AI must remain a tool controlled by humans, not the other way around,' California State Sen. Jerry McNerney, D-Pleasanton, said in a release announcing the legislation. While generative AI tools like ChatGPT and Google's Gemini have only been mainstream for a few years, they're already reshaping how people work — and how they look for work. Recent college graduates have taken notice, as the rise of AI chips away at entry-level white collar roles, helping create one of the toughest job markets in years. Meanwhile, employers are getting buried in AI-generated resumes. The number of applications submitted on LinkedIn has surged more than 45% in the past year, and the platform is now clocking an average of 11,000 applications a minute, according to the New York Times. Resume Builder's survey doesn't detail exactly how managers are using AI to automate personnel decisions. After all, there's a big difference between organizing metrics for a performance review and asking ChatGPT: 'Should I fire Steve?'
Engadget
18 minutes ago
- Engadget
Apple iOS 26: All the latest features for iPhone's new operating system, including FaceTime updates
Liquid Glass is a huge new change coming to iOS 26. (Apple) Get ready because Apple iOS 26 (not iOS 19) is headed your way this fall with some cool new features you'll want to try. What's most exciting? The fresh home and lock screen redesign we've been waiting to see for years. Dubbed Liquid Glass, the new look and feel will extend across all of Apple's upcoming operating systems. The overhaul was one of several big changes coming to iOS, macOS, iPadOS and the rest of Apple's software suite, all of which were showcased during the company's WWDC keynote on June 9. After overpromising on AI plans last year, Apple kept its iOS roadmap focused more on basic quality of life improvements this year. There are multiple useful additions coming to the Phone and Messages apps on your iPhone, for instance: Apple execs outlined the ability to weed out spam texts or other unknown senders and an option to hold your spot on a phone call when you've been waiting for a representative to pick up. Plus, a treasured feature that we took for granted is coming back (hint: it's in the Photos app). Siri, meanwhile, is in a holding pattern. Apple has previously specified that its smarter voice assistant — first promised at WWDC 2024 — is delayed until some point "in the coming year," so you shouldn't expect any major changes in the current betas. But there are reports that Apple is aiming to give Siri a bigger brain transplant by basing it on third-party artificial intelligence models like OpenAI's ChatGPT or Anthropic's Claude, which could make 2026 a pivotal year. If you're a developer, you can check out the iOS 26 beta 2 now; the public beta is coming this month. With each beta, it seems like additional new improvements are popping up, like newly discovered FaceTime feature that'll freeze your video if it detects nudity. Most newer iPhone models are eligible to download iOS 26 (both the betas and final version). Want to see the full list of new features coming this fall? Read on. While the current iPhone operating system is iOS 18, Apple is skipping the numbering ahead to iOS 26 later this year. The company has decided to line up its iOS version numbers with a year-based system, similar to car model years. So while iOS and its sibling operating systems will be released in late 2025, they're all designated "26" to reflect the year ahead. (Meanwhile, iOS 18 is still getting new versions this summer, too.) It's official, we're moving to iOS 26. (Apple) Let's be honest. Out of everything announced at WWDC this year, the new Liquid Glass design was the star of the show. The iPhone's home and lock screens have looked pretty much the same year after year — the last exciting thing (in my opinion) was the option to add your own aesthetic to your home screen by customizing your apps and widgets. So seeing the home and lock screens' new facelift is refreshing. So what exactly is Liquid Glass? Apple calls it a "new translucent material" since, well, the apps and widgets are clear. However, the screen can still adapt to dark and light modes, depending on surroundings. You'll also notice buttons with a new floating design in several apps, like Phone and Maps. They're designed to be less distracting than the current buttons, but are still easy to see. While the design overhaul has proven to be controversial since its announcement, some — including Engadget's own Devindra Hardawar — like the new direction, even if it's somewhat reminiscent of Microsoft's translucent Windows Vista Aero designs from nearly twenty years ago. That said, as of the newly released iOS 26 beta 2, Apple has already incorporated some user feedback into the design, dialing back the transparency in at least some places. And while it will continue to evolve, Apple users won't be able to escape it: Liquid Glass was designed to make all of Apple's OSes more cohesive. Here's a look at how the translucent aesthetic will look with the new macOS Tahoe 26 on your desktop. iOS 26 has a laundry list of new features. Among the most worthwhile: Phone app redesign: You'll finally be able to scroll through contacts, recent calls and voicemail messages all on one screen. It also comes with a new feature called Hold Assist that'll notify you when an agent comes to the phone so you can avoid the elevator music and continue on with other tasks. Live Translation in Phone, FaceTime and Messages: iOS 26 is bringing the ability to have a conversation via phone call or text message with someone who speaks another language. Live Translation will translate your conversation in real time, which results in some stop-and-go interactions in the examples Apple shared during its presentation. Polls in group chats: Tired of sorting through what seems like hundreds of messages in your group chat? You and your friends will soon be able to create polls in group messages for deciding things like which brunch spot you're eating at or whose car you're taking on a road trip. Filtering unknown senders in Messages: If you haven't received spam texts about unpaid tolls or other citations, you're lucky. For those of us who have, those annoying messages will soon be filtered away in a separate folder. Visual Intelligence: Similar to a reverse Google image search, this new feature will allow you to search for anything that's on your iPhone screen. For instance, if you spot a pair of shoes someone is wearing in an Instagram photo, you can screenshot it and use Visual Intelligence to find those shoes (or similar ones) online. Photos tabs are back: For anyone who's still frustrated with the Photos changes made last year, you'll be happy to know that your tabs are coming back. Library and Collections will have their own separate spaces so you don't have to scroll to infinity to find what you're looking for. FaceTime "Communication Safety" feature: A newer addition to iOS 26 appears to be the FaceTime "Communication Safety" feature that pauses communications if and when nudity is detected. The feature appears to be a child safety feature that uses on-device detection, thus obviating any cloud-based privacy issues. Apple's Hold Assist will be nifty for those pesky services that put you on hold for 10 or more minutes. (Apple) A few iPhone models that run the current version of iOS — iPhone XR, XS and XS Max — won't be compatible with the latest upgrade. But any iPhones released in 2019 or later will be eligible for the iOS 26 update. iPhone SE (second generation or later) iPhone 11 iPhone 11 Pro iPhone 11 Pro Max iPhone 12 iPhone 12 mini iPhone 12 Pro iPhone 12 Pro Max iPhone 13 iPhone 13 mini iPhone 13 Pro iPhone 13 Pro Max iPhone 14 iPhone 14 Plus iPhone 14 Pro iPhone 14 Pro Max iPhone 15 iPhone 15 Plus iPhone 15 Pro iPhone 15 Pro Max iPhone 16e iPhone 16 iPhone 16 Plus iPhone 16 Pro iPhone 16 Pro Max Not listed here are the presumed new iPhone 17 models (or maybe iPhone 26?) that are all but certain to be announced and released in September. The iOS 26 public beta will become available this month (July) via the Apple Beta Software Program. If you're not already a member, you'll need to sign up to try out all the latest features. Just visit and sign up with your phone number or email address. It's free. Once you're in and the beta is available, you can install it by going to Settings > General > Software Update and selecting iOS 26 public beta. A word of caution: Don't sign up with your main iPhone unless you're OK with any risks that occur with using an OS that isn't finalized. iOS 26 will be released to the public this fall. It usually comes in September, within a week of the Apple iPhone event. Last year, it rolled out to iPhone users on September 16 — exactly one week after the iPhone 16 lineup was announced. If you're more interested in the Apple Intelligence features coming, here's everything Apple revealed for iOS, macOS and more during WWDC. Also, check out how iOS 26 screenshots could be an intriguing preview of Apple's delayed Siri rework. Update, July 3: Noted new FaceTime feature found in the developer beta. Update, June 30: Noted ongoing iOS 18 releases, and reports that Apple is considering additional external LLMs for Siri. Update, June 25: Noted changes added in iOS 26 beta 2. If you buy something through a link in this article, we may earn commission.
Miami Herald
18 minutes ago
- Miami Herald
Veteran fund manager makes bold move on Apple, Marvell
Chris Versace wants to start off the holiday weekend with a blast. The lead portfolio manager for TheStreet Pro Portfolio gave readers an update on chipmaker Marvell Technology (MRVL) and computer giant Apple (AAPL) . Don't miss the move: Subscribe to TheStreet's free daily newsletter Versace said Marvell shares moved lower, following a report from The Information that said Microsoft (MSFT) intends to release its most powerful in-house AI chips in 2028 or beyond as it focuses on chips it can better develop in the interim. The report also suggested Microsoft has pushed back the release of its Maia 200 chipset to 2026 from 2025, he said. "The move lower in MRVL shares reflects that pushout, but while Microsoft is a custom AI silicon partner for Marvell, so too are Amazon (AMZN) and Meta (META) , and as we learned at the company's recent event, multiple up-and-coming hyperscalers as well," he hosted a "Custom AI" Investor Event on June 17 in New York where the company boosted its total available market to $94 billion by 2028, up 25% higher than the $75 billion by 2028 it outlined this time last year Following the pronounced run in Marvell shares, Versace said, "the same one that led us to bank some of that big move, we are not surprised that The Information's report dinged Marvell shares." More Tech Stocks: Amazon tries to make AI great again (or maybe for the first time)Veteran portfolio manager raises eyebrows with latest Meta Platforms moveGoogle plans major AI shift after Meta's surprising $14 billion move "For us, the larger story is very much intact," he said. "While the Portfolio has a full position, if the shares pulled back a bit further to the $70 level, that would fill a modest gap in the chart, and be a nice pick-up point given our $115 target." Marvell is slated to report second quarter results next month. Shares are down 32% this year. Last month, B. Riley raised the firm's price target on Marvell to $115 from $110 and kept a buy rating on the shares following the company's custom AI investor webcast, according to The Fly. The firm said that it believes the update shows a "strong" next-generation AI growth inflection for Marvell. Turning to Apple, Versace noted that the computer giant's shares have been climbing recently, which is likely due to reports the company is weighing the benefits of using AI technology from Anthropic or OpenAI to power its long-awaited upgrade to Siri, rather than leaning on in-house solutions. Apple has had discussions with both companies about using their large language models for Siri, asking them to train versions that could run on Apple's cloud infrastructure for testing, Bloomberg reported, citing people familiar with the discussions. In March, Apple said AI improvements to its voice assistant Siri will be delayed until 2026, without giving a reason for the setback. Apple shook up its executive ranks to get its AI efforts back on track after months of delays, with Mike Rockwell taking charge of Siri, as CEO Tim Cook lost confidence in AI head John Giannandrea's ability to execute on product development, Bloomberg reported at the time. "We will want to see how this plays out, but should Apple decide to use one of those, it would likely accelerate the delivery of a more useful and smarter Siri," Versace said. "That would potentially charge up the iPhone upgrade cycle to newer models, a nice revenue driver for Apple." Related: Apple could make big change to Siri, delight fans Apple is scheduled to report third quarter results on July 31 and Versace said that, while the company is normally tight-lipped on its earnings calls, "odds are we will get some questions about this during the Q&A on its June quarter conference call." "We will be listening to see what is said," he said. Separately, Jefferies analyst Edison Lee upgraded Apple to hold from underperform with a price target of $188.32, up from $170.62. Counterpoint reported global iPhone's sales volume in April plus May was up 15% year-over-year, marking the strongest growth since Q3 of 2021. Tariff-driven pull-in demand and share recovery in China could drive June quarter revenue and EPS growth of about 8% and 10%, respectively, which would be roughly 5% and 9%, respectively, greater than consensus and greater than Apple's low-single digit revenue growth guidance, the analyst added while previewing fiscal Q3 results. While the market's benign view on tariffs is "likely overly optimistic," and Apple's service revenue has downside potential, Lee said, good Q3 results "could keep the stock stable" near-term. Meanwhile, Versace is tracking President Donald Trump's Big Beautiful Bill and the July 9 tariff deadline. "While we would like to see some of those uncertainties removed, the details for both will matter," he said. "Once we know them, we'll be better able to plot our moves." Related: Fund-management veteran skips emotion in investment strategy The Arena Media Brands, LLC THESTREET is a registered trademark of TheStreet, Inc.
