# robots.txt - where spiders/indexers should and should not go ################################################################ # SITEMAXTER - "Web Optimization On A Whole New Level" # Copyright (C) 2006 Translucent. www.translucentworld.com ################################################################ # # DO NOT MODIFY THIS LINE # user-agent: * disallow: /CustomErrorFiles disallow: /Images disallow: /Scripts disallow: /_private disallow: /_vti_log disallow: /aspnet_client disallow: /cgi-bin disallow: /ft disallow: /mail disallow: /walpole disallow: /httpdocs/www/admin # DO NOT MODIFY THIS LINE # # ################################################################ # The STATIC section above should contain all allow/disallow # statements you want to remain permanent. Anything appearing # below this point may be removed by SITEMAXTER and replaced # by its own directives to cause only the intended folders # and pages to be indexed by spiders parsing this file. You # should only allow/disallow FOLDERS above that are NOT part of # the SITEMAXTER-managed site. It will insert statements below # specifically allowing/disallowing each "subsite" FOLDER # followed by statements to allow/disallow any FILE that # defies that rule. Use above section ONLY for content outside # of the project's "subsite" folders. Therefore, if the # root/home folder is part of the web project, DO NOT USE # either "allow: /" or "disallow: /" above. # While site folders are defined in SITEMAXTER.cfg (with # "subsite=" statements), the presence or lack of the # "SITEMAXTER" tag in an HTML page's section # determines whether or not it is allowed. If you do include # any statements here that conflict with statements inserted # by SITEMAXTER, the outcome is undefined and may cause spiders # to index content you didn't mean for them to or your site # might even be ignored/dropped by search engines. Be careful! # See SITEMAXTER documention for more information. ################################################################ # # DO NOT MODIFY THIS LINE # # # SITEMAXTER is a powerful tool that thoroughly analyzes your # site for ways to increase your exposure, especially to # search engines. It looks carefully for problems that # may be be causing you to have very low ranking on search # engines # not be aware of that may # # DO NOT MODIFY THIS LINE #