From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.org/gmane.linux.lib.musl.general/11611 Path: news.gmane.org!.POSTED!not-for-mail From: David Edelsohn Newsgroups: gmane.linux.lib.musl.general Subject: Re: [PATCH] powerpc64le: Add single instruction math functions Date: Sun, 25 Jun 2017 10:56:14 -0400 Message-ID: References: <20170623193533.GO1627@brightrain.aerifal.cx> <594DB66E.7030009@adelielinux.org> <594DDEC6.8030200@adelielinux.org> <594EFC63.3000707@adelielinux.org> <20170625142821.GC1627@brightrain.aerifal.cx> Reply-To: musl@lists.openwall.com NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Trace: blaine.gmane.org 1498402592 18214 195.159.176.226 (25 Jun 2017 14:56:32 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Sun, 25 Jun 2017 14:56:32 +0000 (UTC) To: musl@lists.openwall.com Original-X-From: musl-return-11624-gllmg-musl=m.gmane.org@lists.openwall.com Sun Jun 25 16:56:26 2017 Return-path: Envelope-to: gllmg-musl@m.gmane.org Original-Received: from mother.openwall.net ([195.42.179.200]) by blaine.gmane.org with smtp (Exim 4.84_2) (envelope-from ) id 1dP8xc-0004Pp-T9 for gllmg-musl@m.gmane.org; Sun, 25 Jun 2017 16:56:25 +0200 Original-Received: (qmail 11688 invoked by uid 550); 25 Jun 2017 14:56:27 -0000 Mailing-List: contact musl-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Original-Received: (qmail 11664 invoked from network); 25 Jun 2017 14:56:27 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-transfer-encoding; bh=c1OtCCfMw3zCkHmvg04+JP/i9ztcVmCLDu/GougfORE=; b=qOfgXs5qyde0pa11BVwpiV/3YoJyV+9AWrH7SbhQ0+cca8hvgtFwFN5xqBRE8JXh1N iF0pquyq5YuQApP3liSm2KXs3mU9vuTK3esDmbMF7XrZZVMTfnCjX2S1gwihSXvOEpQ9 0EmELKeorWGP/9u0l3BS1IT0YY3rK6DObErULF0ejFYiC6qdm9ITPvOtICa3PBMAfjYt 75KJCOyd1J0+urcgGcC70mBAw/qCX6WfEMYHHMDMi04llmy57qHyfGHIEieuLeY+uR4A EiuvMaAn7WeWc2/efj70RkdNFl8zR6nXvmAtnfFEUe4lGnDfJRtFNDmisL5H8ZMWcXMV kMDg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:content-transfer-encoding; bh=c1OtCCfMw3zCkHmvg04+JP/i9ztcVmCLDu/GougfORE=; b=rAOWo9iY1dksinAZVX8F63iPKlawLmiZ48Hb+7+3uIhQWuwObg7oeAnk2w6VYGjyt/ wvHgMHLlM6m39ivUAtWn+Yv8UQ8Sf3+ZZMnjU+3Q1ZN4V8c19S0ab7G+5tS991DqCyFX JCgUjHC5vB0hFalOk5lDFC+Y0DJIc1aQD1zCUm+hOCy22gJDFXCxb6IbYUwolFMtxKpR Vcym/vCC50aaOnu5/PR9kqC3uyMAW1USxWZ3HddpMTKJovuO9ES9Jx/rGQcSpQW+pncj JRmn7N5tnMjupM8fp3j2v1NzARYt6ysGaQVu8Dm1rO26ffQ7IxawXD4zkNzmBEU9WY5i bwLA== X-Gm-Message-State: AKS2vOyylJwP5ydawoMH/ueIE5gxIgeJa8jzC1RDOuhn05WhMjNtPWqO wPeGXTAJWS1QTL/+5CL5qUNHl79+0Q== X-Received: by 10.237.41.132 with SMTP id o4mr4152748qtd.242.1498402575195; Sun, 25 Jun 2017 07:56:15 -0700 (PDT) In-Reply-To: <20170625142821.GC1627@brightrain.aerifal.cx> Xref: news.gmane.org gmane.linux.lib.musl.general:11611 Archived-At: On Sun, Jun 25, 2017 at 10:28 AM, Rich Felker wrote: > On Sat, Jun 24, 2017 at 11:24:59PM -0400, David Edelsohn wrote: >> > Except those based on musl? I mean, we at Ad=C3=A9lie haven't /shippe= d/ >> > anything PPC64 yet, but I have very good reasons for that (and will >> > get to them later in this email). >> >> Because the PowerPC software ecosystem is based on and designed to >> those assumptions. All of the JITs are based on that. All of the >> optimized libraries are based on that. All of the hand-written >> assembly code is based on that. >> >> Some test ABI and endianness separately, some don't. It definitely is >> less well tested, if at all. >> >> You can do whatever you want, but it has been difficult enough fixing >> all of the poor assumptions in the entire Open Source and proprietary >> source ecosystem for the change to PPC64LE ELFv2. If you and Adelie >> want to take on that challenge for PPC64BE ELFv2, great. The >> OpenPower Foundation and its members are not going to fight that >> battle. > > I see where you're coming from, but I don't see where it's > significantly harder than fighting with and fixing software that > doesn't work with musl due to gratuitous (or sometimes moderately > reasonable) glibcisms. Having this type of ABI issue increases the > number of such cases a bit, but I don't expect it to be a significant > portion of the overall work. > >> > I apologise if my words seem strong, but I do not take this lightly. >> > We have a number of users clamouring for us to save their older PPC64 >> > hardware from unmaintained AIX, unmaintained Debian, or in some cases >> > ten-or-more year old fruity OSes. I obviously do not expect ABI >> > compatibility with decades-old non-Linux Unixes. However, if there >> > needs to be an ELFv1 port for a technical reason, I may have to >> > investigate maintaining the port myself. >> >> As I wrote above, the entire external ecosystem makes the endianness / >> ABI assumption. Golang assumes this. OpenJDK assumes this. ATLAS >> BLAS and OpenBLAS assume this. GMP assumes this. PyPy assumes this. >> Mono assumes this. libffi assumes this. Erlang probably assumes this. >> FFMPEG, x264, libvpx assume this. MongoDB may assume this. NVIDIA >> nvcc assumes this. Etc., etc., etc. > > Several of these are trivially fixed with --disable-asm or similar -- > at least gmp, ffmpeg, x264, and libvpx should fall in that category. > Obviously it's desirable to get the asm working to improve > performance, but it can be done incrementally. It also should be > possible to heuristically test for this kind of thing by grepping for > ppc64 asm function prologue in the sources. > > Only stuff that actually does codegen (compilers, jits, etc.) has a > fundamental reason to be affected, and for the most part fixing it > should just be a matter of fixing the conditionals that look for > endianness to look for _CALL_ELF=3D=3D2 where that's what they really > meant to do. > >> It's not that the packages fundamentally cannot be fixed, but the >> FLOSS ecosystem is much larger, richer, complex and more >> interdependent. If one wants to create an embedded system, one can >> exert control over the entire software ecosystem. For a >> Linux-compatible system, one cannot. >> >> If you accept that some parts of the software ecosystem simply won't >> build or function correctly for your system and configuration, or some >> packages randomly will stop building or stop functioning correctly >> after a package is updated, fine. > > These are already risks inherent in using a musl-based system with > upstream packages that are developed on glibc and don't pay attention > to portability issues. We have a very good history of the distros > using musl making efforts to patch these kinds of things, send patchs > upstream, and educate upstreams without attacking or patronizing them. > Sometimes upstream regressions happen, but adequate testing should > catch them. Rich, I am not arguing against big-endian PowerPC nor a powerpc64 BE ELFv2 port of musl nor building powerpc64 BE ELFv2 Linux. Mr. Wilcox asked about the technical challenges and I replied. The companies backing OpenPower have chosen a particular direction and they are investing resources in that direction. The developers, most of whom work for those companies and their partners, are working in that direction. Most of the paid developers will not invest a lot of time and effort in BE functionality and support. PowerPC already is a small market, so further fragmentation isn't helpful. Without corporate backing, the creation of an alternate Linux configuration and ecosystem is a Herculean task. Again, I am not and never was arguing against Mr. Wilcox's plans nor against including powerpc64 BE ELFv2 support in musl libc. As I wrote when I submitted the patch, I added the macro tests for completeness and to allow future flexibility. Thanks, David