Robots.txt
----------

CVE Number: 
None

Details:
The robots.txt file was found on the system. This file is used to direct web
spiders to not index certain sub-directories. Occassionally someone may try to
prevent indexing of a sub-directory that contains certain private data by
including the sub-directory in the robots.txt file. If "security through
obscurity" is being used to hide sub-directories, including them in the 
robots.txt file reveals them.

Fix:
It is recommended that the file be removed from the system, unless you wish
for sub-directories of non-sensitive files to not be indexed by a web
spider. If there are private directories you wish to protect, use password
authentication, or if there are private sub-directories you wish to hide via
security through obscurity, simply do not link to those sub-directories from
any existing web pages that will be indexed.

Related URLs:
None

$Id: robots-txt,v 1.1 2000/11/21 19:49:57 loveless Exp $
