Will this robots.txt file work?

by ZhouW   Last Updated August 26, 2019 03:04 AM

User-agent: *
Allow: /sitemapindex.xml
Allow: /post_google_news.xml
Allow: /news/
Allow: /category/news/
Disallow: /

I want to exclude everything from being indexed by Google except for site.com/sitemapindex.xml, site.com/post_google_news.xml, site.com/news/anything and site.com/category/news/anything/anything etc.



Related Questions


Updated May 26, 2019 03:04 AM

Updated December 21, 2016 08:01 AM

Updated August 20, 2019 02:04 AM

Updated March 28, 2015 06:01 AM

Updated March 03, 2016 01:01 AM