[ISSUE] [rfc] native aarch64 builders

[ISSUE] [rfc] native aarch64 builders

[dkwo]

[-- Attachment #1: Type: text/plain, Size: 410 bytes --]

New issue by dkwo on void-packages repository

https://github.com/void-linux/void-packages/issues/40980

Description:
Cross-compilation is subtle.

1. any mainteiners interested? @leahneukirchen @classabbyamp @paper42 
2. any users interested in supporting this?
3. hardware suggestions

I'd be happy to donate some money to purchase hardware,
if you think it's doable and enough people are interested.

Re: [rfc] native aarch64 builders

[Eloitor]

[-- Attachment #1: Type: text/plain, Size: 365 bytes --]

New comment by Eloitor on void-packages repository

https://github.com/void-linux/void-packages/issues/40980#issuecomment-1347241747

Comment:
I'm interested in this as a user. I would like to use void in my pinephone. I tried to create a [template for osmin](https://github.com/void-linux/void-packages/pull/37298) (an OSM app) but it failed to build for aarch64.

Re: [rfc] native aarch64 builders

[classabbyamp]

[-- Attachment #1: Type: text/plain, Size: 203 bytes --]

New comment by classabbyamp on void-packages repository

https://github.com/void-linux/void-packages/issues/40980#issuecomment-1347259393

Comment:
you can already compile natively for any arch yourself

Re: [rfc] native aarch64 builders

[dkwo]

[-- Attachment #1: Type: text/plain, Size: 385 bytes --]

New comment by dkwo on void-packages repository

https://github.com/void-linux/void-packages/issues/40980#issuecomment-1347312345

Comment:
Compiling natively is fine, but you'd have to rebuild lots of subtly miscompiled stuff first, like qmake, libtool, etc, and build stuff like sagemath yourself, while having aarch64 as a first-class citizen (i.e. native) would avoid all of this.

Re: [rfc] native aarch64 builders

[Eloitor]

[-- Attachment #1: Type: text/plain, Size: 245 bytes --]

New comment by Eloitor on void-packages repository

https://github.com/void-linux/void-packages/issues/40980#issuecomment-1347394527

Comment:
> you can already compile natively for any arch yourself

Compiling in the pinephone is a nightmare

Re: [rfc] native aarch64 builders

[leahneukirchen]

[-- Attachment #1: Type: text/plain, Size: 459 bytes --]

New comment by leahneukirchen on void-packages repository

https://github.com/void-linux/void-packages/issues/40980#issuecomment-1351416593

Comment:
I think the main problem is getting adequate hardware as an aarch64 builder, as server-grade aarch64 systems are still quite expensive; and most of them cannot run armv7 code, so it can't be a builder for both archs.

Also this merely shifts the burden of fixing cross-compiling to the other architectures.

Re: [rfc] native aarch64 builders

[tornaria]

[-- Attachment #1: Type: text/plain, Size: 884 bytes --]

New comment by tornaria on void-packages repository

https://github.com/void-linux/void-packages/issues/40980#issuecomment-1351419528

Comment:
I have a rpi 4 with 8 GB ram and I will start testing some math software on it. It would be nice to have good support, we have some key packages that won't cross compile (ntl). But if some cross-compiled packages in the build chain are broken, that complicates matters a lot.

It seems aarch64 is more important than i686 at this point (e.g. a few days ago I learned that sagemath is completely broken on i686, but nobody noticed -- still we spend a lot of energy fixing 32 bit issues and 8087 magic in particular since we can't even assume sse2)

If it's not possible to get aarch64 builders, is there a chance that some (few, key, carefully selected) packages can be natively compiled on a x86_64 builder using qemu instead of cross?

Re: [rfc] native aarch64 builders

[leahneukirchen]

[-- Attachment #1: Type: text/plain, Size: 208 bytes --]

New comment by leahneukirchen on void-packages repository

https://github.com/void-linux/void-packages/issues/40980#issuecomment-1351421371

Comment:
An rpi 4 is not fast enough to keep up as a full builder.

Re: [rfc] native aarch64 builders

[dkwo]

[-- Attachment #1: Type: text/plain, Size: 287 bytes --]

New comment by dkwo on void-packages repository

https://github.com/void-linux/void-packages/issues/40980#issuecomment-1351547049

Comment:
>  But if some cross-compiled packages in the build chain are broken, that complicates matters a lot.

Exactly, see `libtool`, possibly `qmake`.

Re: [rfc] native aarch64 builders

[dkwo]

[-- Attachment #1: Type: text/plain, Size: 193 bytes --]

New comment by dkwo on void-packages repository

https://github.com/void-linux/void-packages/issues/40980#issuecomment-1351549283

Comment:
@leahneukirchen How about an M1 Mac mini from Apple?

Re: [rfc] native aarch64 builders

[leahneukirchen]

[-- Attachment #1: Type: text/plain, Size: 285 bytes --]

New comment by leahneukirchen on void-packages repository

https://github.com/void-linux/void-packages/issues/40980#issuecomment-1351706503

Comment:
Doesn't run armv7 code either. Probably fast enough, may need external storage.

Not sure if the different page size is problematic.

Re: [rfc] native aarch64 builders

[dkwo]

[-- Attachment #1: Type: text/plain, Size: 446 bytes --]

New comment by dkwo on void-packages repository

https://github.com/void-linux/void-packages/issues/40980#issuecomment-1351731188

Comment:
Page size affects a few packages (I assume that https://github.com/AsahiLinux/docs/wiki/Broken-Software also applies in the reverse way), among them zig; many have already been already.
@leahneukirchen Would you be interested in receiving something like that as a gift, provided enough people contribute?

Re: [rfc] native aarch64 builders

[paper42]

[-- Attachment #1: Type: text/plain, Size: 1658 bytes --]

New comment by paper42 on void-packages repository

https://github.com/void-linux/void-packages/issues/40980#issuecomment-1351824159

Comment:
It is simple to modify xbps-src to use qemu, just an example:
```patch
--- a/common/xbps-src/shutils/chroot.sh
+++ b/common/xbps-src/shutils/chroot.sh
@@ -160,7 +160,7 @@ chroot_handler() {
 
     if [ "$action" = "chroot" ]; then
         $XBPS_COMMONDIR/chroot-style/${XBPS_CHROOT_CMD:=uunshare}.sh \
-            $XBPS_MASTERDIR $XBPS_DISTDIR "$XBPS_HOSTDIR" "$XBPS_CHROOT_CMD_ARGS" /bin/xbps-shell
+            $XBPS_MASTERDIR $XBPS_DISTDIR "$XBPS_HOSTDIR" "$XBPS_CHROOT_CMD_ARGS" /usr/bin/qemu-aarch64-static /bin/bash /bin/xbps-shell
         rv=$?
     else
         env -i -- PATH="/usr/bin:$PATH" SHELL=/bin/sh \
@@ -178,7 +178,7 @@ chroot_handler() {
             ${XBPS_ALT_REPOSITORY:+XBPS_ALT_REPOSITORY=$XBPS_ALT_REPOSITORY} \
             $XBPS_COMMONDIR/chroot-style/${XBPS_CHROOT_CMD:=uunshare}.sh \
             $XBPS_MASTERDIR $XBPS_DISTDIR "$XBPS_HOSTDIR" "$XBPS_CHROOT_CMD_ARGS" \
-            /void-packages/xbps-src $XBPS_OPTIONS $action $pkg
+            /usr/bin/qemu-aarch64-static /bin/bash /void-packages/xbps-src $XBPS_OPTIONS $action $pkg
         rv=$?
     fi
 

```
```sh
echo XBPS_CHROOT_CMD=bwrap >> etc/conf
./xbps-src binary-bootstrap -m maarch aarch64-musl
cp /usr/bin/qemu-aarch64-static maarch/usr/bin/
./xbps-src -m maarch pkg libtool
./xbps-src -m maarch pkg ntl
```
This assumes bubblewrap, qemu-user-static and binfmt are available on the host. It would have to be improved if it was meant to be merged to void-packages and it's very slow.

Re: [rfc] native aarch64 builders

[leahneukirchen]

[-- Attachment #1: Type: text/plain, Size: 313 bytes --]

New comment by leahneukirchen on void-packages repository

https://github.com/void-linux/void-packages/issues/40980#issuecomment-1353820356

Comment:
> Would you be interested in receiving something like that as a gift, provided enough people contribute?

I don't have resources to run this myself locally atm.

Re: [rfc] native aarch64 builders

[vincele]

[-- Attachment #1: Type: text/plain, Size: 251 bytes --]

New comment by vincele on void-packages repository

https://github.com/void-linux/void-packages/issues/40980#issuecomment-1353849761

Comment:
RK3588 support is coming real soon now (tm). This will be faster than rpi4, and less pricier than apple HW.

Re: [rfc] native aarch64 builders

[dkwo]

[-- Attachment #1: Type: text/plain, Size: 444 bytes --]

New comment by dkwo on void-packages repository

https://github.com/void-linux/void-packages/issues/40980#issuecomment-1351731188

Comment:
Page size affects a few packages (I assume that https://github.com/AsahiLinux/docs/wiki/Broken-Software also applies in the reverse way), among them zig; many have already been fixed.
@leahneukirchen Would you be interested in receiving something like that as a gift, provided enough people contribute?

Re: [rfc] native aarch64 builders

[BikyAlex]

[-- Attachment #1: Type: text/plain, Size: 831 bytes --]

New comment by BikyAlex on void-packages repository

https://github.com/void-linux/void-packages/issues/40980#issuecomment-1425194383

Comment:
>any users interested in supporting this?
I'm willing to buy a Honeycomb LX2K and run it 24/7, I have been looking into it for a few years. Would that fit the bill for compiling aarch64, aarch64-musl, armv7 and armv7-musl? It's a NXP LX2160A 16x A72 cores aarch64 CPU.

I would love to contribute to Void some processing horsepower for native building arm packages. Some packages, like libreoffice are marked as non-cross-compileable. But I don't know how I'd be sharing it with Void devs, I'd like to share Void LXC containers for each architecture, but if really necessary, I could share access to the box itself, but I'd have to split my network and put it in a DMZ or something. 

Re: [rfc] native aarch64 builders

[BikyAlex]

[-- Attachment #1: Type: text/plain, Size: 833 bytes --]

New comment by BikyAlex on void-packages repository

https://github.com/void-linux/void-packages/issues/40980#issuecomment-1425194383

Comment:
>any users interested in supporting this?

I'm willing to buy a Honeycomb LX2K and run it 24/7, I have been looking into it for a few years. Would that fit the bill for compiling aarch64, aarch64-musl, armv7 and armv7-musl? It's a NXP LX2160A 16x A72 cores aarch64 CPU.

I would love to contribute to Void some processing horsepower for native building arm packages. Some packages, like libreoffice are marked as non-cross-compileable. But I don't know how I'd be sharing it with Void devs, I'd like to share Void LXC containers for each architecture, but if really necessary, I could share access to the box itself, but I'd have to split my network and put it in a DMZ or something. 

Re: [rfc] native aarch64 builders

[vincele]

[-- Attachment #1: Type: text/plain, Size: 253 bytes --]

New comment by vincele on void-packages repository

https://github.com/void-linux/void-packages/issues/40980#issuecomment-1353849761

Comment:
RK3588 support is coming real soon now (tm). This will be faster than rpi4, and less expensive than apple HW.

Re: [rfc] native aarch64 builders

[dkwo]

[-- Attachment #1: Type: text/plain, Size: 308 bytes --]

New comment by dkwo on void-packages repository

https://github.com/void-linux/void-packages/issues/40980#issuecomment-1590004785

Comment:
Can anyone clarify: does current Void's infrastructure (buildbot etc) allow adding native aarch64 builders, sponsored by @BikyAlex or others? is there any obstruction?

Re: [rfc] native aarch64 builders

[Duncaen]

[-- Attachment #1: Type: text/plain, Size: 241 bytes --]

New comment by Duncaen on void-packages repository

https://github.com/void-linux/void-packages/issues/40980#issuecomment-1590006267

Comment:
We can just integrate some random never seen before users hardware into our build infrastructure.

Re: [rfc] native aarch64 builders

[Duncaen]

[-- Attachment #1: Type: text/plain, Size: 243 bytes --]

New comment by Duncaen on void-packages repository

https://github.com/void-linux/void-packages/issues/40980#issuecomment-1590006267

Comment:
We can't just integrate some random never seen before users hardware into our build infrastructure.

Re: [ISSUE] [CLOSED] [rfc] native aarch64 builders

[Duncaen]

[-- Attachment #1: Type: text/plain, Size: 413 bytes --]

Closed issue by dkwo on void-packages repository

https://github.com/void-linux/void-packages/issues/40980

Description:
Cross-compilation is subtle.

1. any mainteiners interested? @leahneukirchen @classabbyamp @paper42 
2. any users interested in supporting this?
3. hardware suggestions

I'd be happy to donate some money to purchase hardware,
if you think it's doable and enough people are interested.

Re: [rfc] native aarch64 builders

[Duncaen]

[-- Attachment #1: Type: text/plain, Size: 411 bytes --]

New comment by Duncaen on void-packages repository

https://github.com/void-linux/void-packages/issues/40980#issuecomment-1590021511

Comment:
This is not really an RFC and I fail to see the point of this, yes native build servers would be better, everyone agrees. We don't ask for donations and can't take donations, keeping issues like this open that look like a public request for donations is misleading.

Re: [rfc] native aarch64 builders

[Duncaen]

[-- Attachment #1: Type: text/plain, Size: 412 bytes --]

New comment by Duncaen on void-packages repository

https://github.com/void-linux/void-packages/issues/40980#issuecomment-1590021511

Comment:
This is not really an RFC and I fail to see the point of this, yes native build servers would be better, everyone agrees. We don't ask for donations and can't take donations, keeping issues like this open that looks like a public request for donations is misleading.

Re: [rfc] native aarch64 builders

[dkwo]

[-- Attachment #1: Type: text/plain, Size: 451 bytes --]

New comment by dkwo on void-packages repository

https://github.com/void-linux/void-packages/issues/40980#issuecomment-1632864255

Comment:
How about shipping hardware to a maintainer? Can this be discussed, or is it also excluded similar to money donations?

It seems RK3588 could be good enough, and e.g. an Orange Pi 5 Plus board is sold on Amazon for 150$. I'd be willing to buy it and ship it to a maintainer, if that can be used as a builder.

Re: [rfc] native aarch64 builders

[dkwo]

[-- Attachment #1: Type: text/plain, Size: 364 bytes --]

New comment by dkwo on void-packages repository

https://github.com/void-linux/void-packages/issues/40980#issuecomment-1948595845

Comment:
let me mention it also here: there's also the option of applying to oregon state university for the aarch64 builders.
https://osuosl.org/services/aarch64/current-projects/#foss-projects
chimera, gentoo and others do that.

Re: [rfc] native aarch64 builders

[BikyAlex]

[-- Attachment #1: Type: text/plain, Size: 637 bytes --]

New comment by BikyAlex on void-packages repository

https://github.com/void-linux/void-packages/issues/40980#issuecomment-1948627330

Comment:
Huh, q66 applied for Oregon aarch64 builders? Nice. I really wish Void had a bit more love for aarch64. I would also be willing to buy SBCs and send them to contributors, but that will count as donations, and I doubt any of the people working on void want to get in legal trouble. Most (all?) of them are doing a work of love here.

That said, can't force anyone to just maintain aarch64 packages and the many that cannot cross-compile, will be found to be broken when compiling on aarch64.

Re: [rfc] native aarch64 builders

[dkwo]

[-- Attachment #1: Type: text/plain, Size: 188 bytes --]

New comment by dkwo on void-packages repository

https://github.com/void-linux/void-packages/issues/40980#issuecomment-1948657463

Comment:
(of course, oregon's would be native builders.)