From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham autolearn_force=no version=3.4.2 Received: from primenet.com.au (ns1.primenet.com.au [203.24.36.2]) by inbox.vuxu.org (OpenSMTPD) with ESMTP id 28d606fa for ; Mon, 20 May 2019 10:29:05 +0000 (UTC) Received: (qmail 21801 invoked by alias); 20 May 2019 10:28:49 -0000 Mailing-List: contact zsh-users-help@zsh.org; run by ezmlm Precedence: bulk X-No-Archive: yes List-Id: Zsh Users List List-Post: List-Help: List-Unsubscribe: X-Seq: 23946 Received: (qmail 17164 invoked by uid 1010); 20 May 2019 10:28:49 -0000 X-Qmail-Scanner-Diagnostics: from mail-wr1-f50.google.com by f.primenet.com.au (envelope-from , uid 7791) with qmail-scanner-2.11 (clamdscan: 0.101.2/25454. spamassassin: 3.4.2. Clear:RC:0(209.85.221.50):SA:0(-2.0/5.0):. Processed in 1.81401 secs); 20 May 2019 10:28:49 -0000 X-Envelope-From: stephane.chazelas@gmail.com X-Qmail-Scanner-Mime-Attachments: | X-Qmail-Scanner-Zip-Files: | Received-SPF: pass (ns1.primenet.com.au: SPF record at _netblocks.google.com designates 209.85.221.50 as permitted sender) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:subject:message-id:mail-followup-to:mime-version :content-disposition:user-agent; bh=u8qZcySjZ7EgilUPu1U+XGZkRmNV2m3Gt0JC4M/2YRw=; b=toRjz41MYHToChF0P60IAIpmLKKhVL8Ak0jivnEt/LDCNsJJ+ktw3FfT9Ga8YS2MYE 5QwqRhFEMDAFMbmPBVnwkMnKYEL9hnpRdNReYKpSpeS1q75Re9ASEOs3QMeB9VgAsBF7 Tz3vpLXcKF7GHnmFFFlC/+ig5mUn8/E6/PKUED/eAmVzXq23Bcdx+poXGcmGCKCA/7T1 MeCRLuIA8qJZYx+I8Zh/YgErwKjEpQzOKE62G7f0fQylKc1YHOunCa2ey3eagvsw83b+ Wq7gSYzC7qdyB1tC2UbwSOPwYMy9IYZ+3NeV47zHK9tae+PVOUGFCNIUm8EWqW1XuEvO s78A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:subject:message-id:mail-followup-to :mime-version:content-disposition:user-agent; bh=u8qZcySjZ7EgilUPu1U+XGZkRmNV2m3Gt0JC4M/2YRw=; b=UJOeLPnv7oFmlsTkRBCm9YFzDUspEyAjaaboGsjqYQIxIxy1/GPsw3S4w/ky53WEkD 6vFAWxBuZcWfNzdaCmC0V04xdiTEFoevSdafoY9igjXzXDxXn44FT/Pq7pH9Y2c1Jx0w hQgilZONgLLEOjKwnQxLs7jd6FD2hnpRRwiep4k6G1rah61Jw6wKcl2rcDFOdfWnVqn6 hSUf5MY20JdMLZv9zXAHSX1VTfI6NvjhDCnoTNQvLeIoawETd7RixaIydt/rnJTtTL1G iqYgv0eDwmXLXphxR/4kVICjKVoJB/ImsYf3PW66/a4BQqe8cxwj7Vw4BF+iqZVbJjo7 fBsg== X-Gm-Message-State: APjAAAXTAUf/gdd0JFxvL5ttYmAnD+biy5quEXhSX4sl3oTJuFGziimp d7ow69wJxdIr9xnYAX7PArp4H2zT X-Google-Smtp-Source: APXvYqwlNG0p9ULSsqekv37+ypAyd0v+Q9zAuBlTfn+8E32ECwtmsRcMRXSUoeRt/7oqCYH94CaW0Q== X-Received: by 2002:adf:cd0d:: with SMTP id w13mr44220467wrm.38.1558348093752; Mon, 20 May 2019 03:28:13 -0700 (PDT) Date: Mon, 20 May 2019 11:28:11 +0100 From: Stephane Chazelas To: Zsh Users List Subject: Why prevent the mailing list from being indexed? Message-ID: <20190520102811.cird7ae4a3c4xhpo@chaz.gmail.com> Mail-Followup-To: Zsh Users List MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: NeoMutt/20171215 I was surprised when a google search on the zsh mailing list archives turned up nothing when I knew it should (site:zsh.org ZSH_XTRACEFD) and then saw: https://www.zsh.org/robots.txt User-agent: * Disallow: /cgi-bin/ Disallow: /mla/zsh-announce/ Disallow: /mla/zsh-users/ Disallow: /mla/zsh-workers/ Disallow: /pub/old/ Why? Having google/duckduckgo searches return results from there would be very useful, especially considering that the mailing list is also used as an issue tracker and to some extent as a changelog. I know (well, I just discovered) it can be queried with https://www.zsh.org/cgi-bin/mla/wilma_glimpse/workers?query=ZSH_XTRACEFD but you have to know it's there and look explicitely in the ml as opposed to generally on the web. -- Stephane