If you’ve been following SEO news lately, you’ve probably seen the warnings: “Cloudflare is blocking Googlebot!”, “AI bot controls are destroying organic traffic!” It’s enough to make any business owner nervous, especially if your website relies on search visibility to bring in clients.
Here’s the reality: Cloudflare itself isn’t the problem, misconfiguration is.
What’s actually happening
Starting September 15, 2026, Cloudflare is rolling out new AI crawler controls that classify bots into three categories: Search, Training, and Agent. The catch is that Googlebot, Applebot, and Bingbot are now considered “mixed-purpose” crawlers. They index your site for search and collect data for AI training.
If you set “Training” to “Block” without understanding the implications, you can inadvertently block Googlebot too. That’s exactly what happened in the high-profile cases making the rounds: sites enabled broad-stroke blocks, Google got a 403 error, and within weeks, organic traffic and rankings tanked.
One business lost two weeks of revenue because their IT provider toggled the wrong setting. Google Ads kept running, Merchant Center listings disappeared, and organic traffic flatlined. The fix was simply changing the configuration, but the damage was already done.
But here’s the thing: well-configured Cloudflare works fine
We manage multiple client sites on Cloudflare. They’re seeing the same volume of organic traffic they always have without any indexing issues or mysterious drops.
The difference is that we don’t use one-size-fits-all blocks. We use targeted WAF rules that distinguish between legitimate search bots and scrapers we actually want to stop. The goal is to protect your content and server resources without locking out the crawlers that bring you business.
Cloudflare’s own announcement confirms this is possible. The new defaults for domains will allow “Search” while blocking “Training” and “Agent” bots, meaning Googlebot stays in the clear by default unless you override that setting in a way that blocks mixed-used crawlers.
What you need to know
If you’re on Cloudflare (or considering it), here’s the short version:
- Don’t blindly block “AI Training”. That setting now applies to mixed crawlers like Googlebot. Read the fine print before you toggle anything.
- Whitelist known search bots. Make sure Googlebot, Applebot, and Bingbot are explicitly allowed in your rules. This isn’t complicated, but it needs to be deliberate.
- Test your configuration. After any change, check your sitemap access and server logs. If you’re seeing 403s where Googlebot should be allowed, fix it immediately.
- Understand that Cloudflare is a tool, not a threat. It gives you control over who accesses your site. That control can protect you (or hurt you) depending entirely on how you use it.
Bottom line
The recent SEO horror stories aren’t evidence that Cloudflare is broken. They’re evidence that configuration still matters just as much. If you’re a business owner, you don’t need to become a WAF expert. You just need to work with someone who already is.
At NBR Tech Solutions, we handle this stuff so you don’t have to. We’ll make sure your site is protected from scrapers without blocking the search engines that bring you clients. That’s the kind of straightforward, outcome-focused work we deliver without any runaround or jargon.
Got a site on Cloudflare? Not sure if your rules are set up correctly? Let’s take a look, before the September 15 changes catch you off guard.
