Robots txt disallow on Blogger

by Ze Rrauzen   Last Updated August 06, 2019 15:04 PM

I have a blog in Blogger, so the URL structure is always like this:

www.domain.com/2019/01/post.html

The problem is I don't want google to crawl www.domain.com/2019 (it shows all posts of 2019) and www.domain.com/2019/01 (shows all posts of january). Is there a way to block this in robots.txt, and allow the individual HTML posts to be crawled?

Tags : seo robots.txt


Related Questions


Updated March 14, 2017 14:04 PM

Updated May 26, 2019 03:04 AM

Updated December 21, 2016 08:01 AM

Updated August 20, 2019 02:04 AM

Updated April 13, 2015 20:01 PM