Basic discussions about SEO
- Posts: 767
- Joined: Mon Jan 18, 2016 5:57 pm
- Location: Forum
Robots.txt is a text file that lists webpages which contain instructions for search engines robots. The file lists webpages that are allowed and disallowed from search engine crawling.
- Posts: 8
- Joined: Mon Aug 08, 2016 1:16 pm
- Location: India
robots.txt is the program that hide important links from the google that is if the website owner dont want to share some information in the website for the public so that they can use the robots.txt by telling that google dont want can crawl that particular link.
Who is online
Users browsing this forum: No registered users and 2 guests