Don’t Crawl On Me

Here’s some tips to control ’s indexing.

1. At a site or directory level, use .htaccess to add password protection.
2. At a site or directory level, use the robots.txt file.
3. At a page level, use the noindex tag.
4. At a link level, use a nofollow attribute.
5. If the content has already been crawled, use the Google url removal tool.

Leave a Reply