From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=-3.3 required=5.0 tests=MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED,RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL autolearn=ham autolearn_force=no version=3.4.4 Received: (qmail 24346 invoked from network); 17 Nov 2020 04:10:12 -0000 Received: from mother.openwall.net (195.42.179.200) by inbox.vuxu.org with ESMTPUTF8; 17 Nov 2020 04:10:12 -0000 Received: (qmail 7737 invoked by uid 550); 17 Nov 2020 04:10:05 -0000 Mailing-List: contact musl-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Reply-To: musl@lists.openwall.com Received: (qmail 7704 invoked from network); 17 Nov 2020 04:10:04 -0000 Date: Mon, 16 Nov 2020 23:09:52 -0500 From: Rich Felker To: musl@lists.openwall.com Message-ID: <20201117040951.GA9668@brightrain.aerifal.cx> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) Subject: [musl] musl CI? The topic of CI for testing came up on #musl today, and although it's not something I want to be responsible for running, it does sound very good to have. We noted that libc-test currently has tests that are failing and expected to fail, and that this is normal and not cause for disabling tests, but it did suggest to me a methodology that seems more appropriate than checking whether tests succeed: checking for differences in test output (as well as other things, like symbol table) vs suitable baselines like last release or last run. I don't think anyone has concrete plans for going forward with this yet, but I'm posting the outcome so far here for further discussion and so it's not lost. Rich