From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham autolearn_force=no version=3.4.2 Received: from minnie.tuhs.org (minnie.tuhs.org [45.79.103.53]) by inbox.vuxu.org (OpenSMTPD) with ESMTP id 9c9f1c28 for ; Sat, 23 Nov 2019 01:41:56 +0000 (UTC) Received: by minnie.tuhs.org (Postfix, from userid 112) id 3DC519C161; Sat, 23 Nov 2019 11:41:55 +1000 (AEST) Received: from minnie.tuhs.org (localhost [127.0.0.1]) by minnie.tuhs.org (Postfix) with ESMTP id BD6F193D74; Sat, 23 Nov 2019 11:41:42 +1000 (AEST) Authentication-Results: minnie.tuhs.org; dkim=pass (1024-bit key; unprotected) header.d=andrusk.com header.i=@andrusk.com header.b="Nqa47VBc"; dkim-atps=neutral Received: by minnie.tuhs.org (Postfix, from userid 112) id 1479693D74; Sat, 23 Nov 2019 11:41:41 +1000 (AEST) X-Greylist: delayed 560 seconds by postgrey-1.36 at minnie.tuhs.org; Sat, 23 Nov 2019 11:41:40 AEST Received: from mail-40131.protonmail.ch (mail-40131.protonmail.ch [185.70.40.131]) by minnie.tuhs.org (Postfix) with ESMTPS id 22C2093D71 for ; Sat, 23 Nov 2019 11:41:40 +1000 (AEST) Date: Sat, 23 Nov 2019 01:32:16 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=andrusk.com; s=protonmail; t=1574472738; bh=8CjUXUP0pIrHW3YFpFOEIHD+nVaF3Qya6JQFTCj+HCY=; h=Date:To:From:Cc:Reply-To:Subject:In-Reply-To:References: Feedback-ID:From; b=Nqa47VBcRsM5zViJuD9M3KghdutUundEHtwGgI9bVfmm+6dRrhhPkIT0PsjKcEK2c eYPqOUbX71OIaqQDlGTPLN/kzlKv07Ytwdq2/7DZJi1Jkfrt5V4fxRNxKLgeMyrbRm 6RwdHLYNQZlO3C9vUx4QAMNIphQtEJ18ZhHl7eBA= To: Arthur Krewat From: "Justin R. Andrusk" Message-ID: <20191123013213.GA30784@hal9k> In-Reply-To: References: <201911211323.xALDNP0u000778@freefriends.org> <87sgmh43xy.fsf@vuxu.org> <20191122201801.GA5637@hal9k> Feedback-ID: LrkBQ57cnpzx7BzJu9JkIBfqRaveOW5bm30g9KL9AABlksAHJS246usyK8W7lI2wNBZkXxbftZ_6Ym4b_wft9w==:Ext:ProtonMail MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Subject: Re: [TUHS] Steve Bellovin recounts the history of USENET X-BeenThere: tuhs@minnie.tuhs.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: The Unix Heritage Society mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: "Justin R. Andrusk" Cc: tuhs@minnie.tuhs.org Errors-To: tuhs-bounces@minnie.tuhs.org Sender: "TUHS" On Fri, Nov 22, 2019 at 06:21:49PM -0500, Arthur Krewat wrote: >=20 > On 11/22/2019 3:18 PM, Justin R. Andrusk wrote: > > I'm half tempted to take the archive.org Usenet files and throw them > > into Elasticsearch and create a web front end for searching. Storage > > would be expensive, but search would rock! >=20 > Can we run multiple nodes of Elastic, and replicate between each other? >=20 > I just recently started playing with it, it's quite impressive. Except > for that one logstash file "read" mode that by default deletes the file > once it's done with it (a 4-year-long access.log that I wanted to read in= ). >=20 > anyway. >=20 > art k. Yes, that's how the clustering works with Elasticsearch. You setup multiple nodes that are part of a cluster and data is replicated across all of them. If one goes down, you don't lose any data as the others will reconstitute the data.=20 Going to look at adding the Usenet data to a Graylog instance as that uses Elasticsearch as a backend and the front end UI is already there to give you a GUI for searching and doing analytics on what you send to it. Justin