# Robots.txt file from https://www.awzware.com # # All robots will spider the domain User-agent: Baiduspider User-agent: Baiduspider-video User-agent: Baiduspider-image User-agent: Google-adstxt Disallow: / User-agent: dotbot Disallow: / User-agent: * Disallow: /forgot/ Disallow: /login/ Disallow: /usercenter/ Disallow: /register/ Disallow: /blog/type/ Disallow: /blog/tag/ Disallow: /feedback/ Disallow: /cdn-cgi/