How to set up the Robots.txt File in WordPress

The robots.txt file is used to tell search engine robots about what contents on your website should be crawled and consequently indexed. Why set up a Robots.txt File? Your website must be files and folders.

However, may be they are unrelated search engine files. So creating a robots.txt file can actually improve your website indexation. To set up a Robots.txt File is very important. It can friendly to search engine optimization.

Here is my wordpress robots.txt files.

User-agent: *

Disallow: /wp-admin

Disallow: /wp-content/plugins

Disallow: /wp-content/themes

Disallow: /wp-includes

After you created the robots.txt file just upload it to your root directory and you are done! I hoped these can help you.