From: "Daniel Shahaf" <d.s@daniel.shahaf.name>
To: zsh-users@zsh.org
Subject: Re: Why prevent the mailing list from being indexed?
Date: Mon, 20 May 2019 11:04:09 +0000 [thread overview]
Message-ID: <a9448482-d248-4e9d-9ffe-318d51c781fb@www.fastmail.com> (raw)
In-Reply-To: <20190520102811.cird7ae4a3c4xhpo@chaz.gmail.com>
Stephane Chazelas wrote on Mon, 20 May 2019 10:30 +00:00:
> I was surprised when a google search on the zsh mailing list
> archives turned up nothing when I knew it should (site:zsh.org
> ZSH_XTRACEFD) and then saw:
>
> https://www.zsh.org/robots.txt
>
> User-agent: *
> Disallow: /cgi-bin/
> Disallow: /mla/zsh-announce/
> Disallow: /mla/zsh-users/
> Disallow: /mla/zsh-workers/
> Disallow: /pub/old/
>
> Why?
>
> Having google/duckduckgo searches return results from there
> would be very useful,
Note that it only disallows the directories that contain the raw rfc822
files. The rendered HTML indexes at /mla/workers/ aren't disallowed,
so they should be implicitly allowed, shouldn't they?
Cheers,
Daniel
> especially considering that the mailing
> list is also used as an issue tracker and to some extent as a
> changelog.
>
> I know (well, I just discovered) it can be queried with
> https://www.zsh.org/cgi-bin/mla/wilma_glimpse/workers?query=ZSH_XTRACEFD
> but you have to know it's there and look explicitely in the ml
> as opposed to generally on the web.
next prev parent reply other threads:[~2019-05-20 11:05 UTC|newest]
Thread overview: 4+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-05-20 10:28 Stephane Chazelas
2019-05-20 11:04 ` Daniel Shahaf [this message]
2019-05-20 11:57 ` enteiser
[not found] ` <a9448482-d248-4e9d-9ffe-318d51c781fb__45528.5614518047$1558350374$gmane$org@www.fastmail.com>
2019-05-20 14:28 ` Stephane Chazelas
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=a9448482-d248-4e9d-9ffe-318d51c781fb@www.fastmail.com \
--to=d.s@daniel.shahaf.name \
--cc=zsh-users@zsh.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://git.vuxu.org/mirror/zsh/
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).