Re: Adapting binaries easily to musl, or database with binaries (musl)

Re: Adapting binaries easily to musl, or database with binaries (musl)

[Brian Peregrine]

Hey Alex, awilfox,

I looked at https://distfiles.adelielinux.org/adelie/1.0-beta3/user/pmmx/
but those are apk packages. I actually want something that avoids
users needing to compile non-standard programs themselves and have
these technical (compiling) skills to begin with.

I'm not familiar with apk packages; can these simply be unzipped (ie
with xarchiver) and can the unzipped files then run directly in gentoo
(same as with a binary) ? If so, then it would be suitable
(installation with APK Tools wouldn't be suitable as we have ). I
don't care too much with these on whether it's for i586 and not i686
as long as it runs and people can download the file quickly and run
without too much fuss (the extraction step doesn't add that much time
compared to compiling it, ...), mainly as it's just a 2nd browser that
would not be used often, except in special cases.

For TAZ, VCT Labs compiles the standard gentoo packages and makes sure
it runs under musl, in TAZ. So technically, there's not reason why we
can't do this for firefox, and simply include that program (firefox)
in our distro. Why haven't we done, you might ask. Reason is that we
already have a browser (Palemoon) and this has been made to run under
musl. However, for only certain websites (github for instance) it
doesn't work, even when using (recent firefox or chrome user agent
strings). I don't want to add another browser in our distro since that
would be (most of the time) redundant and I want TAZ users to use
Palemoon (as that focuses on security and simplicity more).

We also have git-scm but that doesn't allow to do all functions github
has (for instance, changing the profile photo, modifying git issues,
...) and I think that it's also annoying to need to use git-scm when
you need to change just a few lines on your repo (which in the
browser, would take far less time to do then with git-scm).

Re: Re: Adapting binaries easily to musl, or database with binaries (musl)

[Szabolcs Nagy]

* Brian Peregrine <peregrinebrian@gmail.com> [2019-06-08 09:33:32 +0000]:
> I looked at https://distfiles.adelielinux.org/adelie/1.0-beta3/user/pmmx/
> but those are apk packages. I actually want something that avoids
> users needing to compile non-standard programs themselves and have
> these technical (compiling) skills to begin with.

you can bundle the apks into a self extracting script.

> 
> I'm not familiar with apk packages; can these simply be unzipped (ie
> with xarchiver) and can the unzipped files then run directly in gentoo
> (same as with a binary) ? If so, then it would be suitable
> (installation with APK Tools wouldn't be suitable as we have ). I

$ file foo.apk
foo.apk: gzip compressed data
$ gunzip <foo.apk |file -
/dev/stdin: POSIX tar archive

it's a tar.gz with the files to install and some metadata.

a packaging tool can manage dependencies, so you can
install/uninstall/update components easily, if you do
it with a third party script, it will be unclear what
deps come from the system and what deps from the script
and how to manage security updates for the latter.
and if the abi of a system component changes after an
update then the third party app may stop working.

extracting is likely not enough, you want to put the
binaries under a separate directory and set up paths
for that so lib/exe/etc is found by the app.

> don't care too much with these on whether it's for i586 and not i686
> as long as it runs and people can download the file quickly and run
> without too much fuss (the extraction step doesn't add that much time
> compared to compiling it, ...), mainly as it's just a 2nd browser that
> would not be used often, except in special cases.
> 
> For TAZ, VCT Labs compiles the standard gentoo packages and makes sure
> it runs under musl, in TAZ. So technically, there's not reason why we
> can't do this for firefox, and simply include that program (firefox)
> in our distro. Why haven't we done, you might ask. Reason is that we
> already have a browser (Palemoon) and this has been made to run under
> musl. However, for only certain websites (github for instance) it
> doesn't work, even when using (recent firefox or chrome user agent
> strings). I don't want to add another browser in our distro since that
> would be (most of the time) redundant and I want TAZ users to use
> Palemoon (as that focuses on security and simplicity more).
> 
> We also have git-scm but that doesn't allow to do all functions github
> has (for instance, changing the profile photo, modifying git issues,
> ...) and I think that it's also annoying to need to use git-scm when
> you need to change just a few lines on your repo (which in the
> browser, would take far less time to do then with git-scm).

Re: Re: Adapting binaries easily to musl, or database with binaries (musl)

[Brian Peregrine]

I've been trying to get firefox musl (from
https://distfiles.adelielinux.org/adelie/1.0-beta3/user/pmmx/ )
started (haven't gotten much further then a XPCOMGlueLoad error so
far). Regardless of this, I did notice that the firefox version is
52.9.0-r4 whereas we're currently at version 67, so not sure whether
that's recent enough for github.

I'm also thinking whether this is really the best approach or not.

One alternative could be to just download the ebuild from portage, and
then compile it for musl i686 to a usb stick (main reason for putting
the files to a usb stick is that TAZ is a livecd so you can only store
files for a limited duration -any files not stored on removable media
are erased at reboot). After saving the files to the usb stick, the
user would start the program from the usb stick (so not copying the
files manually to the system; reason is that it's easier to not copy
the files every time and the system itself is read-only, no copying to
the system is permitted)

I don't know however whether
* ebuilds from portage can be compiled under musl at all
* whether the files saved this way can actually be started from usb
stick (I doubt the files thus obtained by this method qualify as a
"binary")
* whether compiling programs using portage is leaner (less cpu/ram
intensive) then compiling the programs from (github) source (assuming
there is a difference at all). This is important as since it's a live
cd, it is already ram-intensive and we also focus on low-spec machines
(starting from i686)

Another possible way is to use anbox on gentoo and run the android
versions of the programs (which can be found easily at the app store)
directly (so not through installation with APK Tools). I don't know
however whether this would run at all under gentoo musl and it's
probably also overkill (as anbox, adb, ... need to be installed and
the android programs might also not be stable even if they do run)


On 6/8/19, Szabolcs Nagy <nsz@port70.net> wrote:
> * Brian Peregrine <peregrinebrian@gmail.com> [2019-06-08 09:33:32 +0000]:
>> I looked at https://distfiles.adelielinux.org/adelie/1.0-beta3/user/pmmx/
>> but those are apk packages. I actually want something that avoids
>> users needing to compile non-standard programs themselves and have
>> these technical (compiling) skills to begin with.
>
> you can bundle the apks into a self extracting script.
>
>>
>> I'm not familiar with apk packages; can these simply be unzipped (ie
>> with xarchiver) and can the unzipped files then run directly in gentoo
>> (same as with a binary) ? If so, then it would be suitable
>> (installation with APK Tools wouldn't be suitable as we have ). I
>
> $ file foo.apk
> foo.apk: gzip compressed data
> $ gunzip <foo.apk |file -
> /dev/stdin: POSIX tar archive
>
> it's a tar.gz with the files to install and some metadata.
>
> a packaging tool can manage dependencies, so you can
> install/uninstall/update components easily, if you do
> it with a third party script, it will be unclear what
> deps come from the system and what deps from the script
> and how to manage security updates for the latter.
> and if the abi of a system component changes after an
> update then the third party app may stop working.
>
> extracting is likely not enough, you want to put the
> binaries under a separate directory and set up paths
> for that so lib/exe/etc is found by the app.
>
>> don't care too much with these on whether it's for i586 and not i686
>> as long as it runs and people can download the file quickly and run
>> without too much fuss (the extraction step doesn't add that much time
>> compared to compiling it, ...), mainly as it's just a 2nd browser that
>> would not be used often, except in special cases.
>>
>> For TAZ, VCT Labs compiles the standard gentoo packages and makes sure
>> it runs under musl, in TAZ. So technically, there's not reason why we
>> can't do this for firefox, and simply include that program (firefox)
>> in our distro. Why haven't we done, you might ask. Reason is that we
>> already have a browser (Palemoon) and this has been made to run under
>> musl. However, for only certain websites (github for instance) it
>> doesn't work, even when using (recent firefox or chrome user agent
>> strings). I don't want to add another browser in our distro since that
>> would be (most of the time) redundant and I want TAZ users to use
>> Palemoon (as that focuses on security and simplicity more).
>>
>> We also have git-scm but that doesn't allow to do all functions github
>> has (for instance, changing the profile photo, modifying git issues,
>> ...) and I think that it's also annoying to need to use git-scm when
>> you need to change just a few lines on your repo (which in the
>> browser, would take far less time to do then with git-scm).
>

Re: Re: Adapting binaries easily to musl, or database with binaries (musl)

[A. Wilcox]

[-- Attachment #1.1: Type: text/plain, Size: 4085 bytes --]

On 06/09/19 09:10, Brian Peregrine wrote:
> I've been trying to get firefox musl (from
> https://distfiles.adelielinux.org/adelie/1.0-beta3/user/pmmx/ )
> started (haven't gotten much further then a XPCOMGlueLoad error so
> far). Regardless of this, I did notice that the firefox version is
> 52.9.0-r4 whereas we're currently at version 67, so not sure whether
> that's recent enough for github.


It's still recent enough for GitHub so far.  We won't be bumping to 60
due to CPU incompatibilities ( https://bugzilla.mozilla.org/1499121 ).

We hope to have 68 available soon after it is released.


> I'm also thinking whether this is really the best approach or not.
> 
> One alternative could be to just download the ebuild from portage, and
> then compile it for musl i686 to a usb stick (main reason for putting
> the files to a usb stick is that TAZ is a livecd so you can only store
> files for a limited duration -any files not stored on removable media
> are erased at reboot). After saving the files to the usb stick, the
> user would start the program from the usb stick (so not copying the
> files manually to the system; reason is that it's easier to not copy
> the files every time and the system itself is read-only, no copying to
> the system is permitted)
> 
> I don't know however whether
> * ebuilds from portage can be compiled under musl at all


Adélie started life four years ago as a Gentoo fork with binpkgs.  There
are a few overlays:

https://gitweb.gentoo.org/proj/musl.git/

https://github.com/smaeul/portage-overlay/

The second one actually has a pretty good Chromium patchset, much higher
quality than the others I've seen.  (I still wouldn't advise running
Chromium on low-end hardware, or anywhere that security or privacy
matter, but at least it won't wreck itself over musl with these patches.)


> * whether the files saved this way can actually be started from usb
> stick (I doubt the files thus obtained by this method qualify as a
> "binary")


As long as the compilation flags are generic enough, it is exactly how
binaries are generated in any other project.


> * whether compiling programs using portage is leaner (less cpu/ram
> intensive) then compiling the programs from (github) source (assuming
> there is a difference at all). This is important as since it's a live
> cd, it is already ram-intensive and we also focus on low-spec machines
> (starting from i686)


There isn't a large difference, but using the musl overlays and the
Gentoo tree, you at least have a semi-decent chance of building without
any errors, as you will have the patches already put in.

Do note that Firefox requires at least 4 GB RAM with "lots of swap", and
at least 6 GB disk space:

https://developer.mozilla.org/en-US/docs/Mozilla/Developer_guide/Build_Instructions/Linux_Prerequisites

Now that it uses Rust (since version 58, IIRC), if you are on a source
distribution like Gentoo you will have to build Rust as well.  Rust is
very hard to build on musl (you *need* to use the overlays above), and
needs a lot of time and RAM/CPU.  For a specific example, the Adélie
builder for x86 is a 12-core Xeon system with 24 GB RAM.  It takes about
three hours to build Rust.


> Another possible way is to use anbox on gentoo and run the android
> versions of the programs (which can be found easily at the app store)
> directly (so not through installation with APK Tools). I don't know
> however whether this would run at all under gentoo musl and it's
> probably also overkill (as anbox, adb, ... need to be installed and
> the android programs might also not be stable even if they do run)


In addition to those issues, Firefox for Android takes a lot more RAM
than desktop Firefox (which is already a large amount; it barely runs in
512 MB here).  I wouldn't recommend that, though if you wanted to try
porting anbox to musl, I'm sure that'd be appreciated by the community.

-- 
A. Wilcox (awilfox)
Project Lead, Adélie Linux
https://www.adelielinux.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Adapting binaries easily to musl, or database with binaries (musl)

[Alexis Boulva]

[-- Attachment #1: Type: text/plain, Size: 2828 bytes --]

Hi Brian,

> On Jun 7, 2019, at 14:07, A. Wilcox <awilfox@adelielinux.org> wrote:
> 
>> On 06/07/19 11:02, Brian Peregrine wrote:
>> Most Gentoo users indeed prefer building from source, but we are
>> compiling it for specific machines (e.g. i686) so we are taking
>> advantage of the compilation-features gentoo has to offer.

In case you're in need of a cross-compiling method for Firefox on gentoo i686 musl, once I've finished building/testing the result, I'll be uploading some experimental build environment configuration files/patches for gentoo x86_64 musl to GitHub, which I use to cross-compile from glibc to musl (why I began with a glibc chroot is a long story) and can easily be ported to other architectures (x86_64 glibc => aarch64 musl, for example). I used to have it working with older software versions and am just updating the work to current versions.

Another thing: some work on cross-compiling from x86_64 musl to other architectures, if that's more what you would prefer, has also been done by gentoo devs ( hint: https://github.com/smaeul/portage-overlay/tree/master/dev-lang/rust ), yet isn't in their musl or main ebuild repos yet. My recommendation would be to ask in #gentoo-hardened on Freenode IRC if you can, and see what your best approach would be after discussion and code review.

If you can't go on IRC, I'll let you know when I'm finished with testing here.

Regards,
Alexis

> 
> Adélie is built for 586+MMX, which is very close to 686.
> 
>> I looked at Bedrock Linux, Alpine, Void, Adelie, ... but I don't see
>> them having binaries downloads either. Only package downloads are
>> avilable at Alpine and Void (see https://pkgs.alpinelinux.org/packages
>> , https://voidlinux.org/packages/?arch=x86_64&q=firefox ), Adelie and
>> Bedrock don't offer any downloads from the website at all.
> 
> 
> We don't have a package site yet, but
> https://distfiles.adelielinux.org/adelie/1.0-beta3/user/pmmx/ is going
> to have every package for the 586.
> 
> Our Firefox package is
> https://distfiles.adelielinux.org/adelie/1.0-beta3/user/pmmx/firefox-esr-52.9.0-r4.apk
> for instance.
> 
> 
>> Let me know how the glibc binaries can be directly edited (i.e. using
>> an IDE editor as Geany) to work on musl, and if someone finds the
>> download URL for the latest stable chromium binary, that would also be
>> very welcome.
> 
> 
> 1) It's not possible to "edit" a glibc binary to use musl like that,
> unfortunately.
> 
> 2) Chromium does not work on musl.  There are (bad quality) patches
> floating around, but they're beyond worthless.  Expect random crashes
> and chasing upstream constantly.  Don't even bother.
> 
> 
> Best,
> --arw
> 
> -- 
> A. Wilcox (awilfox)
> Project Lead, Adélie Linux
> https://www.adelielinux.org
> 

[-- Attachment #2: Type: text/html, Size: 5469 bytes --]

Re: Adapting binaries easily to musl, or database with binaries (musl)

[A. Wilcox]

[-- Attachment #1.1: Type: text/plain, Size: 1566 bytes --]

On 06/07/19 11:02, Brian Peregrine wrote:
> Most Gentoo users indeed prefer building from source, but we are
> compiling it for specific machines (e.g. i686) so we are taking
> advantage of the compilation-features gentoo has to offer.


Adélie is built for 586+MMX, which is very close to 686.


> I looked at Bedrock Linux, Alpine, Void, Adelie, ... but I don't see
> them having binaries downloads either. Only package downloads are
> avilable at Alpine and Void (see https://pkgs.alpinelinux.org/packages
> , https://voidlinux.org/packages/?arch=x86_64&q=firefox ), Adelie and
> Bedrock don't offer any downloads from the website at all.


We don't have a package site yet, but
https://distfiles.adelielinux.org/adelie/1.0-beta3/user/pmmx/ is going
to have every package for the 586.

Our Firefox package is
https://distfiles.adelielinux.org/adelie/1.0-beta3/user/pmmx/firefox-esr-52.9.0-r4.apk
for instance.


> Let me know how the glibc binaries can be directly edited (i.e. using
> an IDE editor as Geany) to work on musl, and if someone finds the
> download URL for the latest stable chromium binary, that would also be
> very welcome.


1) It's not possible to "edit" a glibc binary to use musl like that,
unfortunately.

2) Chromium does not work on musl.  There are (bad quality) patches
floating around, but they're beyond worthless.  Expect random crashes
and chasing upstream constantly.  Don't even bother.


Best,
--arw

-- 
A. Wilcox (awilfox)
Project Lead, Adélie Linux
https://www.adelielinux.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Adapting binaries easily to musl, or database with binaries (musl)

[Brian Peregrine]

Hey Rich,

Replied in-line:
Did you roll your own "distro" from scratch, or are you using one of
the musl-based distros?
We rolled our own (and by "we", I mean VCT Labs did). The distro is
called "TAZ" and we'll release the first beta within the next months
(we're currently still in alpha-stage, no official release yet).

For the (glibc) binaries: yes using binaries posted on the website of
the distribution (in our case, Gentoo) is best, but as far as I know,
Gentoo doesn't release binaries, only ebuilds. It refers to binhosts
for binaries but there is no list of those, and even if there was, I
wouldn't trust them. For firefox, not a problem as there are binaries
from the official website, for chromium it does seem to be a bit of an
issue (but I might be wrong here).

Most Gentoo users indeed prefer building from source, but we are
compiling it for specific machines (e.g. i686) so we are taking
advantage of the compilation-features gentoo has to offer.

I looked at Bedrock Linux, Alpine, Void, Adelie, ... but I don't see
them having binaries downloads either. Only package downloads are
avilable at Alpine and Void (see https://pkgs.alpinelinux.org/packages
, https://voidlinux.org/packages/?arch=x86_64&q=firefox ), Adelie and
Bedrock don't offer any downloads from the website at all.

Let me know how the glibc binaries can be directly edited (i.e. using
an IDE editor as Geany) to work on musl, and if someone finds the
download URL for the latest stable chromium binary, that would also be
very welcome.


On 6/7/19, Rich Felker <dalias@libc.org> wrote:
> On Fri, Jun 07, 2019 at 01:53:50PM +0000, Brian Peregrine wrote:
>> I'm looking for an easy way on how people can either:
>> * adapt their downloaded binaries to be able to run on musl
>> * or simply download prepared musl binaries from a database
>>
>> I doubt that second option exists, as such a database would need to
>> cover a large amounts of programs, for various cpu classes, and be
>> often updated with new versions of those programs.
>
> The best way is to use binaries packaged by your distribution rather
> than by the software authors. This is true whether you're using a
> musl-based or glibc-based distribution. There's usually a better chain
> of trust/responsibility, and you know they're built to work with the
> library ecosystem you'll have rather than one that's just
> "sufficiently similar" (often not).
>
> With that said, there's also an *intent* that projects that want to
> ship binaries should be able to ship *static-linked* musl binaries
> that run on any system, glibc or musl or something else, possibly not
> even Linux, as long as it supports the Linux syscall ABI. However,
> this is currently difficult for graphics applications due to the
> current OpenGL architecture of dlopen'ing a userspace driver (linked
> against a particular libc/library ecosystem) into your program.
> Solving this problem is a major long-term goal.
>
>> I'm particularly looking for the most recent (stable, musl) version of
>> chromium browser and/or firefox, I'm running gentoo linux (musl), on a
>> 32-bit (i686) machine.
>
> Did you roll your own "distro" from scratch, or are you using one of
> the musl-based distros?
>
>> For firefox, glibc binaries are available from
>> https://www.mozilla.org/en-US/firefox/linux/
>> For chromium I couldn't really find a glibc binary;
>> there's https://www.chromium.org/getting-involved/download-chromium
>> but that redirects to an unstable version
>> (https://download-chromium.appspot.com ) and to
>> http://commondatastorage.googleapis.com/chromium-browser-snapshots/index.html
>> (but those are old binaries).
>>
>> Anyway, can someone describe how the binaries can be easily altered to
>> run on a musl distro , and tell me a download link for chromium for
>> the latest stable release ?
>> With the binaries, I want to avoid to need to compile the gentoo
>> package (this can be problematic as live distro's are not updated that
>> often).
>
> Isn't building from source kinda the point of Gentoo? I can understand
> not wanting to do that for everything, but that's the reason I prefer
> other distros. If Bedrock Linux is still around, it might make it easy
> to use packages from other musl-based distros like Alpine, Void, or
> Adelie, but I think they'd use their own library ecosystems rather
> than the main host one, which would be less efficient and kinda defeat
> the purpose. I may be wrong about that, though. There should also be
> ways to manually extract/integrate their packages into your Gentoo
> host.
>
> Rich
>

Re: Adapting binaries easily to musl, or database with binaries (musl)

[Rich Felker]

On Fri, Jun 07, 2019 at 01:53:50PM +0000, Brian Peregrine wrote:
> I'm looking for an easy way on how people can either:
> * adapt their downloaded binaries to be able to run on musl
> * or simply download prepared musl binaries from a database
> 
> I doubt that second option exists, as such a database would need to
> cover a large amounts of programs, for various cpu classes, and be
> often updated with new versions of those programs.

The best way is to use binaries packaged by your distribution rather
than by the software authors. This is true whether you're using a
musl-based or glibc-based distribution. There's usually a better chain
of trust/responsibility, and you know they're built to work with the
library ecosystem you'll have rather than one that's just
"sufficiently similar" (often not).

With that said, there's also an *intent* that projects that want to
ship binaries should be able to ship *static-linked* musl binaries
that run on any system, glibc or musl or something else, possibly not
even Linux, as long as it supports the Linux syscall ABI. However,
this is currently difficult for graphics applications due to the
current OpenGL architecture of dlopen'ing a userspace driver (linked
against a particular libc/library ecosystem) into your program.
Solving this problem is a major long-term goal.

> I'm particularly looking for the most recent (stable, musl) version of
> chromium browser and/or firefox, I'm running gentoo linux (musl), on a
> 32-bit (i686) machine.

Did you roll your own "distro" from scratch, or are you using one of
the musl-based distros?

> For firefox, glibc binaries are available from
> https://www.mozilla.org/en-US/firefox/linux/
> For chromium I couldn't really find a glibc binary;
> there's https://www.chromium.org/getting-involved/download-chromium
> but that redirects to an unstable version
> (https://download-chromium.appspot.com ) and to
> http://commondatastorage.googleapis.com/chromium-browser-snapshots/index.html
> (but those are old binaries).
> 
> Anyway, can someone describe how the binaries can be easily altered to
> run on a musl distro , and tell me a download link for chromium for
> the latest stable release ?
> With the binaries, I want to avoid to need to compile the gentoo
> package (this can be problematic as live distro's are not updated that
> often).

Isn't building from source kinda the point of Gentoo? I can understand
not wanting to do that for everything, but that's the reason I prefer
other distros. If Bedrock Linux is still around, it might make it easy
to use packages from other musl-based distros like Alpine, Void, or
Adelie, but I think they'd use their own library ecosystems rather
than the main host one, which would be less efficient and kinda defeat
the purpose. I may be wrong about that, though. There should also be
ways to manually extract/integrate their packages into your Gentoo
host.

Rich

Adapting binaries easily to musl, or database with binaries (musl)

[Brian Peregrine]

I'm looking for an easy way on how people can either:
* adapt their downloaded binaries to be able to run on musl
* or simply download prepared musl binaries from a database

I doubt that second option exists, as such a database would need to
cover a large amounts of programs, for various cpu classes, and be
often updated with new versions of those programs.

I'm particularly looking for the most recent (stable, musl) version of
chromium browser and/or firefox, I'm running gentoo linux (musl), on a
32-bit (i686) machine.

For firefox, glibc binaries are available from
https://www.mozilla.org/en-US/firefox/linux/
For chromium I couldn't really find a glibc binary;
there's https://www.chromium.org/getting-involved/download-chromium
but that redirects to an unstable version
(https://download-chromium.appspot.com ) and to
http://commondatastorage.googleapis.com/chromium-browser-snapshots/index.html
(but those are old binaries).

Anyway, can someone describe how the binaries can be easily altered to
run on a musl distro , and tell me a download link for chromium for
the latest stable release ?
With the binaries, I want to avoid to need to compile the gentoo
package (this can be problematic as live distro's are not updated that
often).

Brian