When should you use a robots.txt file? | What is custom robot txt in Blogger? | Does robots.txt help SEO? | How do I upload a robots.txt file to Blogger? |
You find many bloggers and YouTubers about Blogger advising their followers to create bot files for their blogs, and every of them claims that his file is the best, so do i actually have to add the bot file to my blog on Blogger?
And the answer is no! You don't have to add any Robots.txt file to your blog unless you would like to hide certain links within the blog.
And the question is what should I do? Don't do anything just don't do the tags corresponding to the bot file in the settings and leave it as you found it (in grey) and blogger will create a bot file suitable for your blog.
Put your blog address within the place of https://seo1specialists.blogspot.com and see the result, you'll find that there is a bot file that exists even though you have not added any file for your blog and it is the best bot file especially for new blogs.
Explanation of the default bot file for blogger
- After we all know what the default bot file for blogger is and that no file should be added, allow us to explain the meaning of the commands in this file for the sake of knowledge.
- Each command within the bot file consists of two parts, the primary is user-agent followed by a specific name or *, this line specifies the name of the program program or bot, if the name is *, the command includes all programs.
- The second part may be a set of commands such as Allow or Disallow, which is to offer the search engine what to archive and what to ignore.
User-agent: Mediapartners-Google
Disallow:
It means allowing the Google Adsense ad crawler to access all of your website pages.
User-agent: *
Disallow: /search
Allow: /
It means allowing all search engines to index your site, the second line means excluding search pages and label pages, and therefore the third line is allowing all the site to be indexed.
The last part is that the type of sitemap that is allowed to be indexed.
We hope that the article has removed a touch confusion about the location of the blogger's robot file, if you would like to learn more, visit the subsequent page of the official Google website.
Comments