. . . "The information returned by the crawler 126 is used by the search engine to determine further information about mirrored hosts in order to improve the data it returns to the client. (Web crawlers can also be used by software other than search engines)." .