From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.org/gmane.linux.lib.musl.general/11610 Path: news.gmane.org!.POSTED!not-for-mail From: Rich Felker Newsgroups: gmane.linux.lib.musl.general Subject: Re: [PATCH] powerpc64le: Add single instruction math functions Date: Sun, 25 Jun 2017 10:28:21 -0400 Message-ID: <20170625142821.GC1627@brightrain.aerifal.cx> References: <20170623193533.GO1627@brightrain.aerifal.cx> <594DB66E.7030009@adelielinux.org> <594DDEC6.8030200@adelielinux.org> <594EFC63.3000707@adelielinux.org> Reply-To: musl@lists.openwall.com NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Trace: blaine.gmane.org 1498400916 17199 195.159.176.226 (25 Jun 2017 14:28:36 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Sun, 25 Jun 2017 14:28:36 +0000 (UTC) User-Agent: Mutt/1.5.21 (2010-09-15) To: musl@lists.openwall.com Original-X-From: musl-return-11623-gllmg-musl=m.gmane.org@lists.openwall.com Sun Jun 25 16:28:33 2017 Return-path: Envelope-to: gllmg-musl@m.gmane.org Original-Received: from mother.openwall.net ([195.42.179.200]) by blaine.gmane.org with smtp (Exim 4.84_2) (envelope-from ) id 1dP8We-0004Bt-CZ for gllmg-musl@m.gmane.org; Sun, 25 Jun 2017 16:28:32 +0200 Original-Received: (qmail 21666 invoked by uid 550); 25 Jun 2017 14:28:34 -0000 Mailing-List: contact musl-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Original-Received: (qmail 21645 invoked from network); 25 Jun 2017 14:28:33 -0000 Content-Disposition: inline In-Reply-To: Original-Sender: Rich Felker Xref: news.gmane.org gmane.linux.lib.musl.general:11610 Archived-At: On Sat, Jun 24, 2017 at 11:24:59PM -0400, David Edelsohn wrote: > > Except those based on musl? I mean, we at Adélie haven't /shipped/ > > anything PPC64 yet, but I have very good reasons for that (and will > > get to them later in this email). > > Because the PowerPC software ecosystem is based on and designed to > those assumptions. All of the JITs are based on that. All of the > optimized libraries are based on that. All of the hand-written > assembly code is based on that. > > Some test ABI and endianness separately, some don't. It definitely is > less well tested, if at all. > > You can do whatever you want, but it has been difficult enough fixing > all of the poor assumptions in the entire Open Source and proprietary > source ecosystem for the change to PPC64LE ELFv2. If you and Adelie > want to take on that challenge for PPC64BE ELFv2, great. The > OpenPower Foundation and its members are not going to fight that > battle. I see where you're coming from, but I don't see where it's significantly harder than fighting with and fixing software that doesn't work with musl due to gratuitous (or sometimes moderately reasonable) glibcisms. Having this type of ABI issue increases the number of such cases a bit, but I don't expect it to be a significant portion of the overall work. > > I apologise if my words seem strong, but I do not take this lightly. > > We have a number of users clamouring for us to save their older PPC64 > > hardware from unmaintained AIX, unmaintained Debian, or in some cases > > ten-or-more year old fruity OSes. I obviously do not expect ABI > > compatibility with decades-old non-Linux Unixes. However, if there > > needs to be an ELFv1 port for a technical reason, I may have to > > investigate maintaining the port myself. > > As I wrote above, the entire external ecosystem makes the endianness / > ABI assumption. Golang assumes this. OpenJDK assumes this. ATLAS > BLAS and OpenBLAS assume this. GMP assumes this. PyPy assumes this. > Mono assumes this. libffi assumes this. Erlang probably assumes this. > FFMPEG, x264, libvpx assume this. MongoDB may assume this. NVIDIA > nvcc assumes this. Etc., etc., etc. Several of these are trivially fixed with --disable-asm or similar -- at least gmp, ffmpeg, x264, and libvpx should fall in that category. Obviously it's desirable to get the asm working to improve performance, but it can be done incrementally. It also should be possible to heuristically test for this kind of thing by grepping for ppc64 asm function prologue in the sources. Only stuff that actually does codegen (compilers, jits, etc.) has a fundamental reason to be affected, and for the most part fixing it should just be a matter of fixing the conditionals that look for endianness to look for _CALL_ELF==2 where that's what they really meant to do. > It's not that the packages fundamentally cannot be fixed, but the > FLOSS ecosystem is much larger, richer, complex and more > interdependent. If one wants to create an embedded system, one can > exert control over the entire software ecosystem. For a > Linux-compatible system, one cannot. > > If you accept that some parts of the software ecosystem simply won't > build or function correctly for your system and configuration, or some > packages randomly will stop building or stop functioning correctly > after a package is updated, fine. These are already risks inherent in using a musl-based system with upstream packages that are developed on glibc and don't pay attention to portability issues. We have a very good history of the distros using musl making efforts to patch these kinds of things, send patchs upstream, and educate upstreams without attacking or patronizing them. Sometimes upstream regressions happen, but adequate testing should catch them. Rich