Cloudflare launches tool to help website owners monetize AI bot crawler access
The tool allows website owners to choose whether artificial intelligence crawlers can access their material and set a price for access through a "pay per crawl" model, which will help them control how their work is used and compensated, Cloudflare said.
With AI crawlers increasingly collecting content without sending visitors to the original source, website owners are looking to develop additional revenue sources as search traffic referrals that once generated advertising revenue decline.
The initiative is supported by major publishers including Condé Nast and Associated Press, as well as social media companies such as Reddit and Pinterest.
Cloudflare's Chief Strategy Officer Stephanie Cohen said the goal of such tools was to give publishers control over their content, and ensure a sustainable ecosystem for online content creators and AI companies.
"The change in traffic patterns has been rapid, and something needed to change," Cohen said in an interview. "This is just the beginning of a new model for the internet."
Google, for example, has seen its ratio of crawls to visitors referred back to sites drop to 18:1 from 6:1 just six months ago, according to Cloudflare data, suggesting the search giant is maintaining its crawling but decreasing referrals.
The decline could be a result of users finding answers directly within Google's search results, such as AI Overviews. Still, Google's ratio is much higher than other AI companies, such as OpenAI's 1,500:1.
For decades, search engines have indexed content on the internet directing users back to websites, an approach that rewards creators for producing quality content. However, AI companies' crawlers have disrupted this model because they harvest material without sending visitors to the original source and aggregate information through chatbots such as ChatGPT, depriving creators of revenue and recognition.
Many AI companies are circumventing a common web standard used by publishers to block the scraping of their content for use in AI systems, and argue they have broken no laws in accessing content for free.
In response, some publishers, including the New York Times, have sued AI companies for copyright infringement, while others have struck deals to license their content.
Reddit, for example, has sued AI startup Anthropic for allegedly scraping Reddit user comments to train its AI chatbot, while inking a content licensing deal with Google.
Hashtags
Try Our AI Features
Explore what Daily8 AI can do for you:
Comments
No comments yet...
Related Articles
Geeky Gadgets
an hour ago
- Geeky Gadgets
How Artificial Intelligence is Redefining the Consulting World
What happens when the world's most human-driven industry meets its most fantastic technology? The consulting sector, long defined by its reliance on expertise, intuition, and bespoke problem-solving, is being turned on its head by artificial intelligence. Imagine a world where AI tools generate actionable insights in hours, predictive models outperform seasoned analysts, and strategies are tailored with surgical precision—all at a fraction of the traditional cost. This isn't a distant future; it's happening now. The rise of AI is fundamentally reshaping the consulting landscape, introducing both unprecedented opportunities and existential challenges. The very essence of consulting—its human touch—is being redefined by algorithms and automation. In this perspective, we'll explore how AI is not just augmenting consulting but actively disrupting it. From automating core tasks like forecasting and market analysis to allowing new delivery models like forward-deployed engineers, AI is rewriting the rules of engagement. But this transformation isn't just about efficiency—it's about survival. As technology companies like OpenAI and Palantir blur the lines between software providers and strategic advisors, traditional firms like McKinsey and Accenture face mounting pressure to innovate or risk irrelevance. What does this mean for the future of consulting? The answers lie in understanding how AI is eating away at the industry's foundations while simultaneously building a new one. AI's Impact on Consulting Transforming Core Consulting Functions AI's ability to process and analyze vast amounts of data is transforming how consulting services are delivered. Tasks that once required extensive human effort are now being automated, allowing firms to provide faster and more precise solutions. Key examples include: AI-powered tools that analyze market trends and generate actionable insights within hours. Predictive models that enhance the accuracy and speed of forecasting. AI-driven simulations that create tailored strategic recommendations for clients. These advancements are not only improving service delivery but also reshaping client expectations. As automation reduces costs and accelerates timelines, consulting firms must adapt to meet the demand for efficiency and innovation. The Rise of Forward-Deployed Engineers A new delivery model is emerging in the consulting industry, centered around forward-deployed engineers (FTEs). These professionals combine technical expertise with consulting skills, working directly within client organizations to implement AI-driven solutions. By embedding FTEs on-site, firms can provide more agile and customized services. Companies like OpenAI are leading this shift, using FTEs to tailor AI tools to specific business challenges. This approach allows organizations to integrate AI more effectively, addressing unique needs while reducing reliance on traditional consulting teams. The FTE model represents a significant evolution in how consulting services are delivered, emphasizing flexibility and direct collaboration. How AI Eats Consulting Watch this video on YouTube. Expand your understanding of Artificial Intelligence with additional resources from our extensive library of articles. Technology Companies as Emerging Competitors The consulting industry is facing increasing competition from technology companies that are integrating consulting-like services into their offerings. Firms such as OpenAI and Palantir are combining advanced software with tailored services to deliver end-to-end solutions. This hybrid model appeals to enterprises undergoing digital transformation, as it simplifies the adoption of new technologies. Traditional consulting firms like McKinsey and Accenture are now under pressure to adapt. To remain competitive, they must innovate and differentiate themselves by offering specialized expertise and using AI to enhance their services. The rise of technology companies in this space underscores the need for consulting firms to evolve alongside technological advancements. Key Trends Driving Industry Evolution The consulting industry is undergoing significant changes, driven by the integration of AI and other advanced technologies. Several key trends are shaping this transformation: A shift toward enterprise-focused solutions that emphasize implementation and customization. Increased automation of routine tasks, reducing the need for manual processes. Growing demand for AI-driven services to address complex and dynamic client needs. These trends highlight the importance of innovation and adaptability in maintaining a competitive edge. Consulting firms that embrace these changes are better positioned to meet the evolving demands of their clients. Adapting to the Future of Consulting The future of the consulting industry lies in its ability to integrate AI into service delivery effectively. Firms that adopt hybrid models, combining software solutions with consulting expertise, are more likely to thrive in this changing landscape. At the same time, the rise of technology companies as competitors emphasizes the need for traditional firms to innovate and specialize. As AI continues to reshape the industry, your ability to adapt and use these advancements will be critical. By embracing new technologies and delivery models, consulting firms can not only meet client expectations but also drive long-term success in an increasingly competitive market. Media Credit: AI Daily Brief Filed Under: AI, Top News Latest Geeky Gadgets Deals Disclosure: Some of our articles include affiliate links. If you buy something through one of these links, Geeky Gadgets may earn an affiliate commission. Learn about our Disclosure Policy.
The Guardian
an hour ago
- The Guardian
Buy now, pay later loans will now affect US credit scores – what does that mean for consumers?
A new change to buy now, pay later loans means borrowers' credit scores may see a change, which has worried some users of the loans. 'I have a feeling that I'm just not going to have as much access to spending power and zero or really low APR rates,' said Nicole Nitta, a 31-year-old Las Vegas resident, who uses BNPL and shared that she already does not have great credit. Fico, the credit scoring company used by most US lenders, announced on 23 June that they would include BNPL loans, which play 'an increasingly important role in consumers' financial lives', to help lenders more 'accurately evaluate credit readiness'. For users of companies like Affirm, Afterpay and Klarna, the new calculation could benefit them because it allows them to build their credit – if, of course, they pay back the loans on time, experts say. Nitta first used BNPL for essentials in 2021, like non-perishable food items. She was out of work and 'basically living off of savings', she said. Now, working as an office manager for a private therapy practice and studying marriage and family therapy, Nitta is more stable financially but has significant student loan debt. She has since used BNPL for Christmas gifts and dishware when she moved into a new apartment, but said she always makes her payments on time. Ted Rossman, a senior industry analyst at the financial site Bankrate, says: 'if you're using buy now, pay later responsibly,' like Nitta, 'I would argue [the change] should help you as a steppingstone to improve your credit, and maybe it helps you get your first credit card or car loan. 'The main downside is if it dings you because you're paying late or racking up too much debt. I would say that's a fair consequence, because that is what happens on credit cards and other products,' he added. Companies like Affirm, Afterpay and Klarna were founded more than a decade ago, but their usage expanded significantly during the Covid-19 pandemic. These companies provided $180m in loans totaling more than $24bn in 2021, an almost tenfold increase from 2019, according to the Consumer Financial Protection Bureau. Fumiko Hayashi, a vice-president at the Federal Reserve Bank of Kansas City who conducts economic research on payments, noted that the change was due to a shift from purchasing in stores to buying online – in addition to an economic downturn during the pandemic. A typical BNPL loan allows consumers to divide a $50 to $1,000 purchase into four interest-free instalments. If a borrower does not make the required payments, then the lender charges them a late fee. Lenders also charge transaction fees to merchants. BNPL is most popular among people ages 33 and under, who represented 70% of borrowers of such loans in 2022, according to the financial protection bureau. Hayashi noted that a downside for the younger users is 'if they keep using BNPL only and they don't use a credit card at all, they cannot build credit history'. With Fico's change, using BNPL responsibly could be beneficial for some younger users with no previous credit history – but less so for those not as responsible. For most users, Fico and Affirm say, including BNPL data in credit reports produced higher scores or no score changes – a finding in a year-long report released in February that looked at 500,000 consumers. Still, there are people who could be hurt by Fico using BNPL data. People with sub-prime or deep sub-prime credit scores obtained more than 60% of new BNPL loans from 2021 to 2022, according to the financial protection bureau. And 24% of BNPL borrowers were late making a payment in 2024, a 6% increase from the prior year, according to the Federal Reserve. Among people who make $25,000 or less, the rate increased from 31% to 40%. Notably, BNPL access 'significantly reduces the sensitivity of spending relative to income', according to a Harvard Business School report. 'This effect is concentrated among individuals likely to be liquidity constrained, specifically, lower-income users and users without credit cards,' the report notes. Becca, a 26-year-old tech worker in New York who declined to use her last name, said she used BNPL for things like pricier beauty products – including a Chanel perfume. She said she might 'spend like 80 bucks this month on it and make two separate $40 payments, and then next month, I pay off the rest'. While the payment option has helped her, she is concerned about companies like DoorDash offering BNPL for minor purchases like a pizza delivery. 'It's just encouraging poor spending behavior from young people,' Becca said. 'All these items build up because you're using it again and again and again. You don't feel like you're spending a lot of money.' It may be some time before the economy feels the impact of the new credit score calculation, Rossman said. While Fico stated that it would make the new scores available in fall 2025, most lenders continue to use a credit score model from 2009, (despite Fico since releasing new versions). 'Change comes relatively slowly in the credit scoring world, so even if this becomes available in the fall, that doesn't mean everybody is going to be using it right away,' said Rossman. 'It's kind of like your phone. For instance, Apple has the iPhone 16, but a lot of people are still using the 15 or the 14 or even older models. Credit scoring works the same way.'
Daily Mail
2 hours ago
- Daily Mail
Worker is arrested after being accidentally paid $400k by employer
A Florida horse clinic mistakenly paid a receptionist with a veterinarian's checks — and now the $60,000-a-year employee stands accused of stealing hundreds of thousands of dollars. Yessica Arrua, 29, of Wellington, was arrested on June 27 after allegedly pocketing more than $400,000 of someone else's salary. A police report from the Palm Beach Sheriff's Office (PBSO) obtained by the Daily Mail revealed the trouble started in early 2023, when the Palm Beach Equine Clinic's (PBEC) CFO noticed the error and contacted payroll provider Harbor America. The firm had accidentally routed a veterinarian's salary to Arrua's account. Arrua, originally from Argentina, received the inflated pay — nearly seven times her salary — from February 2022 to January 2023, according to the report. Though she allegedly admitted to noticing the overpayment, she never reported it to her employer. Instead, police say Arrua used the extra money at Coach, Michael Kors, restaurants, and furniture stores. Thousands were also sent through Zelle to someone labeled 'Mama Dukes'. The report also details an $80,000 purchase of a food truck for a friend of Arrua's mother, and claims she sent money to Argentina to help build a house. When questioned, Arrua reportedly said she believed the funds were a 'bonus' for her work as a receptionist. She claimed to have heard that a previous receptionist once received one for saving the company money on supplies. Arrua had worked at the clinic for nine years and had known the company president — who eventually reported her — since she was nine years old. The veterinarian whose salary was redirected earned $450,000 a year but reportedly did not monitor her bank deposits. She only discovered the issue when her credit cards started getting declined. After realizing the error, she confronted Arrua, who allegedly broke down and confessed. Arrua then wrote a $200,000 cashier's check to the clinic, but claimed she couldn't return the rest because her mother had already sent $100,000 to relatives in Argentina, believing it was a 'gift from God'. When asked why she hadn't arranged a payment plan, she told police she didn't know how to approach the company president due to his anger.
