Many bloggers do not understand the robots.txt option, and some blogger does not understand the benefits of this.They left this blank and this decrease the traffic of their site, and we all know the importance of visitors.To solve all problems about this today I am going to tell you, what is this? How it works and how to attach this file to your website?Robots.txt file plays a significant role in SEO of your blog.But if you do not know about this and you copy and paste this file from another website then it can harm your blog. For this, before it is activated, you need to know about the details of the robots.txt file.
What is a Robot.txt file?
Each search engines have their Robots.You may be wondering that it is the robot which you have seen in the Hollywood movie.That’s not my friend.It is a function of the search engines for crawling and indexing any website.You can allow or disallow to crawl and index any post to search engines using this robots.txt file.
How robots.txt file works?
This file is similar to the traffic police. When you publish any new post and search engine bots come to your site then this data tell him to crawl and index that new post, so that new post comes quickly in the search result.
You may saw this kind of code on the robots.txt file –
Most of the robots.txt files are like this. You also may see this kind of file. But now I am explaining these codes for better understanding.
User-agent:* This notify that you are ordering all kind of robots.
Disallow: /search This is telling to disallow the keyword. In fact, this is telling not to index and crawl the search links of your blog.All labels of your blog include this search links. You do not need to crawl and index these tags.
Allow: / this code allows to index and crawls your homepage.
Sitemap When you publish a new post then it will tell the search engine bots to crawl and index that new post.
How to insert this robots.txt file to your blog spot site?
>At first, Log in to blogger
>Then, go to your blogger dashboard.
>Then Go to settings> search preferences.
>After that, Click on yes option of “Enable Custom Robots.txt” option
>Then copy-paste the above codes in the text box
Note: Replace “helperbus.com” by your website’s URL.
Thanks for reading this article. If you have any question, then comment us and if you like this then share it.
To get some more visit helperbus.com regularly.