The Cloudflare Blog

Subscribe to receive notifications of new posts:

security.txt

Robotcop: enforcing your robots.txt policies and stopping bots before they reach your website

2024-12-10

The AI Crawl Control (formerly AI Audit) now allows you to quickly see which AI services are honoring your robots.txt policies and then automatically enforce the policies against those that aren’t. ...

Celso Martinho
Will Allen
Nelson Duarte

Robotcop: enforcing your robots.txt policies and stopping bots before they reach your website

Enhance your website's security with Cloudflare’s free security.txt generator

2024-10-06

Better Internet Security Posture Security Standards security.txt

Cloudflare’s free security.txt generator lets users create and manage security.txt files. Enhance vulnerability disclosure, align with industry standards, and integrate into the dashboard....

Alexandra Moraru
Sam Khawasé