* Why prevent the mailing list from being indexed? @ 2019-05-20 10:28 Stephane Chazelas 2019-05-20 11:04 ` Daniel Shahaf [not found] ` <a9448482-d248-4e9d-9ffe-318d51c781fb__45528.5614518047$1558350374$gmane$org@www.fastmail.com> 0 siblings, 2 replies; 4+ messages in thread From: Stephane Chazelas @ 2019-05-20 10:28 UTC (permalink / raw) To: Zsh Users List I was surprised when a google search on the zsh mailing list archives turned up nothing when I knew it should (site:zsh.org ZSH_XTRACEFD) and then saw: https://www.zsh.org/robots.txt User-agent: * Disallow: /cgi-bin/ Disallow: /mla/zsh-announce/ Disallow: /mla/zsh-users/ Disallow: /mla/zsh-workers/ Disallow: /pub/old/ Why? Having google/duckduckgo searches return results from there would be very useful, especially considering that the mailing list is also used as an issue tracker and to some extent as a changelog. I know (well, I just discovered) it can be queried with https://www.zsh.org/cgi-bin/mla/wilma_glimpse/workers?query=ZSH_XTRACEFD but you have to know it's there and look explicitely in the ml as opposed to generally on the web. -- Stephane ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: Why prevent the mailing list from being indexed? 2019-05-20 10:28 Why prevent the mailing list from being indexed? Stephane Chazelas @ 2019-05-20 11:04 ` Daniel Shahaf 2019-05-20 11:57 ` enteiser [not found] ` <a9448482-d248-4e9d-9ffe-318d51c781fb__45528.5614518047$1558350374$gmane$org@www.fastmail.com> 1 sibling, 1 reply; 4+ messages in thread From: Daniel Shahaf @ 2019-05-20 11:04 UTC (permalink / raw) To: zsh-users Stephane Chazelas wrote on Mon, 20 May 2019 10:30 +00:00: > I was surprised when a google search on the zsh mailing list > archives turned up nothing when I knew it should (site:zsh.org > ZSH_XTRACEFD) and then saw: > > https://www.zsh.org/robots.txt > > User-agent: * > Disallow: /cgi-bin/ > Disallow: /mla/zsh-announce/ > Disallow: /mla/zsh-users/ > Disallow: /mla/zsh-workers/ > Disallow: /pub/old/ > > Why? > > Having google/duckduckgo searches return results from there > would be very useful, Note that it only disallows the directories that contain the raw rfc822 files. The rendered HTML indexes at /mla/workers/ aren't disallowed, so they should be implicitly allowed, shouldn't they? Cheers, Daniel > especially considering that the mailing > list is also used as an issue tracker and to some extent as a > changelog. > > I know (well, I just discovered) it can be queried with > https://www.zsh.org/cgi-bin/mla/wilma_glimpse/workers?query=ZSH_XTRACEFD > but you have to know it's there and look explicitely in the ml > as opposed to generally on the web. ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: Why prevent the mailing list from being indexed? 2019-05-20 11:04 ` Daniel Shahaf @ 2019-05-20 11:57 ` enteiser 0 siblings, 0 replies; 4+ messages in thread From: enteiser @ 2019-05-20 11:57 UTC (permalink / raw) To: zsh-users Mon, May 20, 2019 at 11:04:09AM +0000, Daniel Shahaf wrote: > Stephane Chazelas wrote on Mon, 20 May 2019 10:30 +00:00: > > I was surprised when a google search on the zsh mailing list > > archives turned up nothing when I knew it should (site:zsh.org > > ZSH_XTRACEFD) and then saw: > > > > I know (well, I just discovered) it can be queried with > > https://www.zsh.org/cgi-bin/mla/wilma_glimpse/workers?query=ZSH_XTRACEFD > > but you have to know it's there and look explicitely in the ml > > as opposed to generally on the web. These messages are pretty new. I guess they just didn't made it into the google index yet. ^ permalink raw reply [flat|nested] 4+ messages in thread
[parent not found: <a9448482-d248-4e9d-9ffe-318d51c781fb__45528.5614518047$1558350374$gmane$org@www.fastmail.com>]
* Re: Why prevent the mailing list from being indexed? [not found] ` <a9448482-d248-4e9d-9ffe-318d51c781fb__45528.5614518047$1558350374$gmane$org@www.fastmail.com> @ 2019-05-20 14:28 ` Stephane Chazelas 0 siblings, 0 replies; 4+ messages in thread From: Stephane Chazelas @ 2019-05-20 14:28 UTC (permalink / raw) To: Daniel Shahaf; +Cc: zsh-users 2019-05-20 11:04:09 +0000, Daniel Shahaf: > Stephane Chazelas wrote on Mon, 20 May 2019 10:30 +00:00: > > I was surprised when a google search on the zsh mailing list > > archives turned up nothing when I knew it should (site:zsh.org > > ZSH_XTRACEFD) and then saw: > > > > https://www.zsh.org/robots.txt [...] > > Disallow: /mla/zsh-workers/ [...] > Note that it only disallows the directories that contain the raw rfc822 > files. The rendered HTML indexes at /mla/workers/ aren't disallowed, > so they should be implicitly allowed, shouldn't they? [...] Sorry, my bad. I missed that. I can see now they are indexed indeed. Like enteiser@gmail.com says, it's probably just that they're recent. Cheers, Stephane ^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2019-05-20 14:28 UTC | newest] Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2019-05-20 10:28 Why prevent the mailing list from being indexed? Stephane Chazelas 2019-05-20 11:04 ` Daniel Shahaf 2019-05-20 11:57 ` enteiser [not found] ` <a9448482-d248-4e9d-9ffe-318d51c781fb__45528.5614518047$1558350374$gmane$org@www.fastmail.com> 2019-05-20 14:28 ` Stephane Chazelas
Code repositories for project(s) associated with this public inbox https://git.vuxu.org/mirror/zsh/ This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).