What is the purpose of robots.txt file?
12 posts
• Page 2 of 2 • 1, 2
Re: What is the purpose of robots.txt file?
Robots.txt is a text file that lists webpages which contain instructions for search engines robots. The file lists webpages that are allowed and disallowed from search engine crawling.
█ Cheapest VPS | $1 VPS Hosting
█ Cheap Dedicated Hosting | Free Setup and IPMI | Unmetered Bandwidth
█ Cheap Dedicated Hosting | Free Setup and IPMI | Unmetered Bandwidth
Re: What is the purpose of robots.txt file?
robots.txt is the program that hide important links from the google that is if the website owner dont want to share some information in the website for the public so that they can use the robots.txt by telling that google dont want can crawl that particular link.
12 posts
• Page 2 of 2 • 1, 2
Who is online
Users browsing this forum: No registered users and 1 guest