# from http://www.robotstxt.org/wc/norobots.html # This example indicates that no robots should visit this site further: # go away User-agent: * Disallow: /