Register

What is the purpose of robots.txt file?

Basic discussions about SEO
User avatar
Posts: 589
Joined: Mon Jan 18, 2016 5:57 pm
Location: Forum

Re: What is the purpose of robots.txt file?

Postby RH-Calvin » Fri Jan 12, 2018 7:05 am

Robots.txt is a text file that lists webpages which contain instructions for search engines robots. The file lists webpages that are allowed and disallowed from search engine crawling.
Cheapest VPS | $1 VPS Hosting
Cheap Dedicated Hosting | Free Setup and IPMI | Unmetered Bandwidth

Posts: 8
Joined: Mon Aug 08, 2016 1:16 pm
Location: India

Re: What is the purpose of robots.txt file?

Postby sadesh » Fri Jan 12, 2018 1:29 pm

robots.txt is the program that hide important links from the google that is if the website owner dont want to share some information in the website for the public so that they can use the robots.txt by telling that google dont want can crawl that particular link.

Previous

Return to SEO General

Who is online

Users browsing this forum: No registered users and 0 guests