From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham autolearn_force=no version=3.4.2 Received: from primenet.com.au (ns1.primenet.com.au [203.24.36.2]) by inbox.vuxu.org (OpenSMTPD) with ESMTP id 2e52b659 for ; Mon, 20 May 2019 11:05:06 +0000 (UTC) Received: (qmail 23030 invoked by alias); 20 May 2019 11:04:50 -0000 Mailing-List: contact zsh-users-help@zsh.org; run by ezmlm Precedence: bulk X-No-Archive: yes List-Id: Zsh Users List List-Post: List-Help: List-Unsubscribe: X-Seq: 23947 Received: (qmail 1140 invoked by uid 1010); 20 May 2019 11:04:50 -0000 X-Qmail-Scanner-Diagnostics: from out5-smtp.messagingengine.com by f.primenet.com.au (envelope-from , uid 7791) with qmail-scanner-2.11 (clamdscan: 0.101.2/25454. spamassassin: 3.4.2. Clear:RC:0(66.111.4.29):SA:0(-2.6/5.0):. Processed in 4.330981 secs); 20 May 2019 11:04:50 -0000 X-Envelope-From: d.s@daniel.shahaf.name X-Qmail-Scanner-Mime-Attachments: | X-Qmail-Scanner-Zip-Files: | Received-SPF: none (ns1.primenet.com.au: domain at daniel.shahaf.name does not designate permitted sender hosts) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= daniel.shahaf.name; h=mime-version:message-id:in-reply-to :references:date:from:to:subject:content-type; s=fm3; bh=GiwBjp8 QjTboDPC/1by319tMUOeqdbki4BhKD8RoL5w=; b=Ys2982qRGZlBWXgXQuFgxgL 3bHGHzq+J7X1ZZnT+7r58llQOstD4co5W+cBQPALCqwIHqFEEnrp3oIHMW/gfsDt yespiWJ6KC7eRag/R4T/XsB+xWW1/8JEotiZzbapgyO0ktYVAn2qtuaM4qFTKgnx 9H/MM0K7NAOiyWc6MGtv1BhmsSmP5T8/I6a0RO97cLo520plNxPGbUs4mN+WofKN +ZaF77DH1uAIMaEWC76oi10BCm7lvmhr28+UgX5jQqu14UTD4cUGouOYBDNVdMB6 K2TOtlVmoRs8B1rtDGNC08lbd6oOMBIJINhcQYjkPCZ4+ELH4fHAVjOfokkrnkw= = DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm2; bh=GiwBjp 8QjTboDPC/1by319tMUOeqdbki4BhKD8RoL5w=; b=qqEnlQarumkXnGvu9rE2Fp DPz8ThEFV3q1PpxRTb3UtCcXBmuVEu5tWj9aqn98yQxDhRfTWjOXupNQdJSXi2eb RCb+/irHyjN6rygrzx9OWrDN+8LQl6dFI8qVszKLkxUO+zIelS493XnXhMJpIbZp lr6GJDdc+pFn87gKRbUmuiSDE6d/8H8uW8TGlrPg6AOdYvQuBq2cuMy5f3SQ4PCZ 41AxU3EBTz0jHwAeRzPLM0+7RE9NsbZPRxGu0ckYaMtHNVlT3/0TDahsdOIwhQ0d wtOmpVPCt/QIexGLDGHRLknx81hV03yslqXa+b3llIUqx3htpqm+w+tvUaibKGLw == X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduuddruddtkedgfeeiucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpefofgggkfgjfhffhffvufgtsehttd ertderredtnecuhfhrohhmpedfffgrnhhivghlucfuhhgrhhgrfhdfuceougdrshesuggr nhhivghlrdhshhgrhhgrfhdrnhgrmhgvqeenucffohhmrghinhepiihshhdrohhrghenuc frrghrrghmpehmrghilhhfrhhomhepugdrshesuggrnhhivghlrdhshhgrhhgrfhdrnhgr mhgvnecuvehluhhsthgvrhfuihiivgeptd X-ME-Proxy: X-Mailer: MessagingEngine.com Webmail Interface User-Agent: Cyrus-JMAP/3.1.6-549-ge400f56-fmstable-20190516v3 Mime-Version: 1.0 Message-Id: In-Reply-To: <20190520102811.cird7ae4a3c4xhpo@chaz.gmail.com> References: <20190520102811.cird7ae4a3c4xhpo@chaz.gmail.com> Date: Mon, 20 May 2019 11:04:09 +0000 From: "Daniel Shahaf" To: zsh-users@zsh.org Subject: Re: Why prevent the mailing list from being indexed? Content-Type: text/plain Stephane Chazelas wrote on Mon, 20 May 2019 10:30 +00:00: > I was surprised when a google search on the zsh mailing list > archives turned up nothing when I knew it should (site:zsh.org > ZSH_XTRACEFD) and then saw: > > https://www.zsh.org/robots.txt > > User-agent: * > Disallow: /cgi-bin/ > Disallow: /mla/zsh-announce/ > Disallow: /mla/zsh-users/ > Disallow: /mla/zsh-workers/ > Disallow: /pub/old/ > > Why? > > Having google/duckduckgo searches return results from there > would be very useful, Note that it only disallows the directories that contain the raw rfc822 files. The rendered HTML indexes at /mla/workers/ aren't disallowed, so they should be implicitly allowed, shouldn't they? Cheers, Daniel > especially considering that the mailing > list is also used as an issue tracker and to some extent as a > changelog. > > I know (well, I just discovered) it can be queried with > https://www.zsh.org/cgi-bin/mla/wilma_glimpse/workers?query=ZSH_XTRACEFD > but you have to know it's there and look explicitely in the ml > as opposed to generally on the web.