WordPress Robots.txt Guides – What is Robots.txt and How to Optimize

As we all know, WordPress is a great blogging tool. There are many themes and plugins to use. And very simple to use, only a few clicks, WordPress can become a full fledged Content Management System (CMS). Even a child can do it.

But it is not friendly to seo. You need to create a robots.txt for wordpress. In fact, create a robots.txt is very important to seo. Many bloggers don’t understand what is Robots.txt.

What is Robots.txt?

The robots.txt file is used to tell google robots (search engine) about what articles/pages on your blog should be crawled and consequently indexed.

How to Optimize the Perfect Robots.txt File for SEO?

Here is my robots.txt files. You can refer to it.

User-agent: *
Disallow: /wp-
Disallow: /feed/
Disallow: /trackback/
Disallow: /comments/feed/
Disallow: /page/
Disallow: /comments/

There are 2 steps to complete before you can test the robots.txt.

  • Created the robots.txt
  • Upload it to your root directory

You may also like: