## ## robots.txt -- web robot configuration ## http://www.searchengineworld.com/robots/robots_tutorial.htm ## # The following allows all robots to visit all files because the # wildcard "*" specifies all robots. User-agent: * Disallow: