Blocking Bots with Nginx

from blog Robb Knight • Posts • RSS Feed, | ↗ original
In April Ethan wrote this post about blocking AI bots with Apache and .htaccess. I've already done this for my robots.txt file but quite frankly I don't trust any of the AI companies to respect that. Jason Santa Maria (via Ethan): robots.txt is a bit like asking bots to not visit my site; with .htaccess, you’re not asking It's been on my list...