Re: [PATCH] powerpc64le: Add single instruction math functions

["A. Wilcox" <awilfox@adelielinux.org>]

[-- Attachment #1.1: Type: text/plain, Size: 5762 bytes --]

A little update for those interested.


On 06/25/17 09:28, Rich Felker wrote:
> On Sat, Jun 24, 2017 at 11:24:59PM -0400, David Edelsohn wrote:
>>> Except those based on musl?  I mean, we at Adélie haven't /shipped/
>>> anything PPC64 yet, but I have very good reasons for that (and will
>>> get to them later in this email).


We have definitely shipped on PPC64 now.  We were actually the first
distro in history (to my knowledge) to ship musl on PPC64.

That's PPC64, as in 64-bit big endian PowerPC.

KDE, LXQt, Firefox, Thunderbird, the whole lot.  I'm using it on an iMac
G5, several G5 towers, and a Talos 2.  I actually threw out my last
x86_64 system once the Talos 2 arrived.


>> You can do whatever you want, but it has been difficult enough fixing
>> all of the poor assumptions in the entire Open Source and proprietary
>> source ecosystem for the change to PPC64LE ELFv2.  If you and Adelie
>> want to take on that challenge for PPC64BE ELFv2, great.  The
>> OpenPower Foundation and its members are not going to fight that
>> battle.


Keep talking like you speak for the Foundation...

https://twitter.com/hughhalf/status/1027109176563064833


> I see where you're coming from, but I don't see where it's
> significantly harder than fighting with and fixing software that
> doesn't work with musl due to gratuitous (or sometimes moderately
> reasonable) glibcisms. Having this type of ABI issue increases the
> number of such cases a bit, but I don't expect it to be a significant
> portion of the overall work.


Rich is correct; it hasn't been significant, and it has been very little
challenge when something is broken.


>>> I apologise if my words seem strong, but I do not take this lightly.
>>> We have a number of users clamouring for us to save their older PPC64
>>> hardware from unmaintained AIX, unmaintained Debian, or in some cases
>>> ten-or-more year old fruity OSes.  I obviously do not expect ABI
>>> compatibility with decades-old non-Linux Unixes.  However, if there
>>> needs to be an ELFv1 port for a technical reason, I may have to
>>> investigate maintaining the port myself.
>>
>> As I wrote above, the entire external ecosystem makes the endianness /
>> ABI assumption.  Golang assumes this.  OpenJDK assumes this.  ATLAS
>> BLAS and OpenBLAS assume this.  GMP assumes this.  PyPy assumes this.
>> Mono assumes this. libffi assumes this. Erlang probably assumes this.
>> FFMPEG, x264, libvpx assume this.  MongoDB may assume this. NVIDIA
>> nvcc assumes this.  Etc., etc., etc.


* Golang: incorrect.  The code is written correctly, and Go runs
properly on BE PPC64 (though not correctly; this is unrelated to ELF ABI).

* OpenJDK: correct, patch is fermenting.

* BLAS/OpenBLAS: untested as of yet.

* GMP: incorrect.  Works fine and passes all tests on musl without any
patching.

* PyPy: incorrect, as above.

* Mono: incorrect, as above.

* libffi: incorrect, but did require patching for musl's insistence on
using IEEE long doubles instead of IBM long doubles.

* FFmpeg, x264: incorrect.  Work fine, though did need a single patch to
make AltiVec support build properly; this affected ppc32 and ppc64/ELFv1
as well.

* libvpx: does not support PowerPC in any capacity anyway.

* MongoDB: untested as of yet.

* NVIDIA nvcc: violates our package policy in many ways, proprietary
license being the largest.  Also requires LE as far as I can tell, so
completely orthogonal to supporting BE.


> Several of these are trivially fixed with --disable-asm or similar --
> at least gmp, ffmpeg, x264, and libvpx should fall in that category.
> Obviously it's desirable to get the asm working to improve
> performance, but it can be done incrementally. It also should be
> possible to heuristically test for this kind of thing by grepping for
> ppc64 asm function prologue in the sources.


The only --disable-asm required was OpenSSL, and we have a patch
fermenting to fix that as well.  (Most of their checks use _CALL_ELF
correctly, but a few do not.)


> Only stuff that actually does codegen (compilers, jits, etc.) has a
> fundamental reason to be affected, and for the most part fixing it
> should just be a matter of fixing the conditionals that look for
> endianness to look for _CALL_ELF==2 where that's what they really
> meant to do.


One semi-important JIT that wasn't named was ORC, which is used by
gstreamer to help multimedia performance.  I assume that wasn't named
because nobody merged LE support either?

We managed to patch it for ELFv2 cleanly, supporting both musl in BE
*and* all the ELFv2 LE distros including glibc. :)

We're also about to land a Valgrind patch to support ELFv2 on BE.  I
believe the last major issue that we haven't yet sent upstream is
OpenJDK.  (OpenSSL is a performance regression, but functionally complete.)


>> It's not that the packages fundamentally cannot be fixed, but the
>> FLOSS ecosystem is much larger, richer, complex and more
>> interdependent.  If one wants to create an embedded system, one can
>> exert control over the entire software ecosystem.  For a
>> Linux-compatible system, one cannot.
>>
>> If you accept that some parts of the software ecosystem simply won't
>> build or function correctly for your system and configuration, or some
>> packages randomly will stop building or stop functioning correctly
>> after a package is updated, fine.


We do not accept that in the slightest.  And neither should anyone else.

We accept that parts of the software ecosystem need to be *fixed*.

Best to you and yours,
--arw


-- 
A. Wilcox (awilfox)
Project Lead, Adélie Linux
https://www.adelielinux.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]