|
REFERER
Dear site owner,
You have come to this page because you have detected that the Referer URI in your web server log is this page when we accessed your site.
Our spider is indexing your page and the links on that page, images for instance. Each time it accesses something on your web site, it will use this page as the Referer. The reason for this is the following:
Unlike other search engines, we do not want to download and steal the things you have on your site. For instance, some search engines download and steal your images and display them in their search results.
We don't want to steal your images, we just want to index and link to them.
Now, here's the problem: You might have set your web server to deny access to things on your site unless the Referer is a page on your web site. This is why we access your site with this page as the Referer value. Our spider wants to know if we will be denied linkage to the things (images) on your site. If yes, the spider must set a low rank value on, or remove, your image link from the search index to avoid displaying a broken/missing image in the search results.
See also the Crawling FAQ
|
|
|