From mboxrd@z Thu Jan 1 00:00:00 1970 From: smiley@zenzebra.mv.com To: Fans of the OS Plan 9 from Bell Labs <9fans@9fans.net> References: <22468C58-2225-4068-BB9F-2EA276EAAE84@fastmail.fm> <201105041811.15240.dexen.devries@gmail.com> Date: Wed, 4 May 2011 20:26:10 +0000 In-Reply-To: <201105041811.15240.dexen.devries@gmail.com> (dexen deVries's message of "Wed, 4 May 2011 18:11:15 +0200") Message-ID: <868vumrz4t.fsf@cmarib.ramside> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Subject: Re: [9fans] wiki... Topicbox-Message-UUID: dd231d94-ead6-11e9-9d60-3106f5b1d025 dexen deVries writes: > I believe we need an ``Allow: /'' below the long list of `Disallows' in the > User-agent: Googlebot, User-agent: msnbot section. Otherwise, only the final > ``Disallow: /'' matches, and in effecet, every robot is cut off. I rather like mine own robots.txt (http://www.icebubble.org/robots.txt). As members of the "alternative OS" crowd, you may appreciate its design. Incidentally, the wiki employs a very good anti-spam technology: edits to it can only be made over 9P. (The HTTP edit mechanism isn't enabled.) The wiki's lack of spidering also means it wouldn't be a very valuable target even IF spambots could edit it over HTTP. -- +---------------------------------------------------------------+ |E-Mail: smiley@zenzebra.mv.com PGP key ID: BC549F8B| |Fingerprint: 9329 DB4A 30F5 6EDA D2BA 3489 DAB7 555A BC54 9F8B| +---------------------------------------------------------------+