[ISSUE] package ldc for aarch64 (and more)

[ISSUE] package ldc for aarch64 (and more)

[Marie-Joseph]

[-- Attachment #1: Type: text/plain, Size: 2000 bytes --]

New issue by Marie-Joseph on void-packages repository

https://github.com/void-linux/void-packages/issues/34752

Description:
## Rationale

Currently, ldc is only available for x86 systems. This is likely because it requires an extant D compiler to build, and the template uses dmd since gdc and its gdmd wrapper are not packaged. Dmd only supports x86. What this means in effect is that D is only available to Void users on x86. Fear not! We (and I do intend to work on this so I really do mean we) can bring D to other Void users in three relatively simple steps.

## Steps

- [ ] package gdc

This is the ldc team's suggested D compiler for bootstrapping ldc and is cross-platform. Gdc is available in gcc sources, and versions <=11.2 use a C++ frontend (no D compiler needed). Since Void uses 10.x anyway, all that's required is to update the template with an option to pass the flag `--enable_languages=d` (like is done now for Ada) and the relevant packaging code.

- [ ]  package [gdmd](https://github.com/D-Programming-GDC/gdmd)

When using gdc to bootstrap ldc, the gdmd frontend is required. It's just a wrapper to make gdc accept the same arguments as ldc and dmd.

- [ ]  update ldc template to use gdc and gdmd instead of dmd

This should be as simple as adding them as dependencies and possibly passing a flag to point ldc's build system at them.

Et voilà, D will be available to (almost) every Void user, as well as a new frontend and backend for x86 users! And, since there will now be two cross-platform D compilers available in addition to dmd, moving beyond gcc 11.2 shouldn't affect this much.

## Notes

Since I need this, I'm going to work on it and will update this issue as I progress. If more capable hands want to take over, feel absolutely free. It'll likely take me several days to a couple weeks to do this (especially since my build machine for testing is a Raspberry Pi 4) and I'm sure others with more experience would be able to do it faster.

Re: package ldc for aarch64 (and more)

[sug0]

[-- Attachment #1: Type: text/plain, Size: 450 bytes --]

New comment by sug0 on void-packages repository

https://github.com/void-linux/void-packages/issues/34752#issuecomment-1003163719

Comment:
Hi. I'm in no capability to help with this, but I'll just leave a suggestion.

> especially since my build machine for testing is a Raspberry Pi 4

Can you not cross-compile from a more powerful x86_64 machine? This would surely speed things up. I think the void infra cross-compiles all aarch64 packages.

Re: package ldc for aarch64 (and more)

[ericonr]

[-- Attachment #1: Type: text/plain, Size: 498 bytes --]

New comment by ericonr on void-packages repository

https://github.com/void-linux/void-packages/issues/34752#issuecomment-1003186612

Comment:
About GDC: https://github.com/void-linux/void-packages/pull/12700#issue-460921055

> Gdc is available in gcc sources, and versions <=11.2 use a C++ frontend (no D compiler needed).

What does this mean? Does more recent GCC require a D compiler to build gdc? That would make this proposal a no go because it would make bootstrap way more complicated.

Re: package ldc for aarch64 (and more)

[q66]

[-- Attachment #1: Type: text/plain, Size: 336 bytes --]

New comment by q66 on void-packages repository

https://github.com/void-linux/void-packages/issues/34752#issuecomment-1003189240

Comment:
if gdc from 11.x is self-hosting and cannot build without another D compiler, i'm absolutely opposed to enabling it

(also, even in the version in 10.x, it does not support all targets void has)

Re: package ldc for aarch64 (and more)

[Marie-Joseph]

[-- Attachment #1: Type: text/plain, Size: 1650 bytes --]

New comment by Marie-Joseph on void-packages repository

https://github.com/void-linux/void-packages/issues/34752#issuecomment-1003231801

Comment:
> Can you not cross-compile from a more powerful x86_64 machine?

Yes, but only on a laptop that I have to beg and plead to charge. So some days I can, some days I can't. I'll make a follow-up comment with insights I've gained since today is one day I could. Spoiler alert: many things are broken with cross-compilation.

> Does more recent GCC require a D compiler to build gdc? That would make this proposal a no go because it would make bootstrap way more complicated.

Yes, it will require a D compiler. If by bootstrap you mean the (build) system bootstrap, I'd like to have a way to only build GDC when doing so explicitly. I'll look into that once I can compile it at all. Although Ada is built by default with the gcc package, and it requires an extant compiler. Assuming a glibc host, we could similarly download one like the gnat switch allows for Ada. If you mean bootstrapping GDC itself, both other D compilers require a D compiler, which is the source of this being such an issue to begin with. DMD only supports x86_64 and i686; LDC currently builds with DMD, therefore LDC can only be built for x86_64. More commentary on this in my next comment.

> if gdc from 11.x is self-hosting and cannot build without another D compiler, i'm absolutely opposed to enabling it
> (also, even in the version in 10.x, it does not support all targets void has)

Both of these points apply to both in-tree D compilers. Adding GDC will actually make D in general available on more platforms.

Re: package ldc for aarch64 (and more)

[Marie-Joseph]

[-- Attachment #1: Type: text/plain, Size: 1674 bytes --]

New comment by Marie-Joseph on void-packages repository

https://github.com/void-linux/void-packages/issues/34752#issuecomment-1003231801

Comment:
> Can you not cross-compile from a more powerful x86_64 machine?

Yes, but only on a laptop that I have to beg and plead to charge. So some days I can, some days I can't. I'll make a follow-up comment with insights I've gained since today is one day I could. Spoiler alert: many things are broken with cross-compilation.

> Does more recent GCC require a D compiler to build gdc? That would make this proposal a no go because it would make bootstrap way more complicated.

Yes, as of (2021-12-29) HEAD it will require a D compiler. If by bootstrap you mean the (build) system bootstrap, I'd like to have a way to only build GDC when doing so explicitly. I'll look into that once I can compile it at all. Although Ada is built by default with the gcc package, and it requires an extant compiler. Assuming a glibc host, we could similarly download one like the gnat switch allows for Ada. If you mean bootstrapping GDC itself, both other D compilers require a D compiler, which is the source of this being such an issue to begin with. DMD only supports x86_64 and i686; LDC currently builds with DMD, therefore LDC can only be built for x86_64. More commentary on this in my next comment.

> if gdc from 11.x is self-hosting and cannot build without another D compiler, i'm absolutely opposed to enabling it
> (also, even in the version in 10.x, it does not support all targets void has)

Both of these points apply to both in-tree D compilers. Adding GDC will actually make D in general available on more platforms.

Re: package ldc for aarch64 (and more)

[Marie-Joseph]

[-- Attachment #1: Type: text/plain, Size: 1578 bytes --]

New comment by Marie-Joseph on void-packages repository

https://github.com/void-linux/void-packages/issues/34752#issuecomment-1003234652

Comment:
So I've managed to try some cross-compilation today, both of LDC and GDC.

For GDC compilation, I added a D build option. I had some trouble getting libisl downloaded. It seems the link in the current template stopped working sometime in the last couple days. I managed to find a different one that still works at https://libisl.sourceforge.io though. However, for some reason g++ gets passed the flag "-march=armv8-a" at some point, which is not a valid value for that flag. From what I can tell, that flag is only valid for x86* builds since valid options all seem to be code names or descriptors of various x86* architectures. Without digging into makefiles I don't think I can resolve that, so I moved on to trying to compile LDC with LDC.

The first problem with compiling LDC with LDC is that it requires the ldmd front-end. Void doesn't package the ldmd front-end. So I decided to try natively compiling it to get build artifacts to figure out how to package the ldmd front-end so I can do that and install it to use it to cross-build LDC with LDC. But unfortunately, it can't find a *bunch* of LLVM headers. Maybe because I'm using musl? In any case, resolving that particular issue seems even more time-consuming than getting GDC to compile.

So I'm going to take a break (I've been at this for the plurality of my waking hours since opening this issue) and when I come back I guess I'll be digging into makefiles.

Re: package ldc for aarch64 (and more)

[q66]

[-- Attachment #1: Type: text/plain, Size: 930 bytes --]

New comment by q66 on void-packages repository

https://github.com/void-linux/void-packages/issues/34752#issuecomment-1003243315

Comment:
I am opposed to requiring a D compiler to bootstrap a system, and since gcc will now require a D compiler to enable gdc, there is no way you can get around that (you either need an existing D compiler, or download some kind of binary snapshot during the build, and both options are bad for bootstrapping a Void system)

The situation is already bad with Ada, and I don't want to add more of the same to it, except worse because unlike Ada, GDC does not cover all targets we cover

D is not an important enough language to warrant the extra burden, barely anything uses it, and you will be imposing this on the toolchain maintainers in the distro

If you want GDC, build it separately as its own template, use binary snapshots to bootstrap, but don't have it affect the system compiler

Re: package ldc for aarch64 (and more)

[q66]

[-- Attachment #1: Type: text/plain, Size: 1145 bytes --]

New comment by q66 on void-packages repository

https://github.com/void-linux/void-packages/issues/34752#issuecomment-1003243315

Comment:
I am opposed to requiring a D compiler to bootstrap a system, and since gcc will now require a D compiler to enable gdc, there is no way you can get around that (you either need an existing D compiler, or download some kind of binary snapshot during the build, and both options are bad for bootstrapping a Void system)

The situation is already bad with Ada, and I don't want to add more of the same to it, except worse because unlike Ada, GDC does not cover all targets we cover

D is not an important enough language to warrant the extra burden, barely anything uses it, and you will be imposing this on the toolchain maintainers in the distro

If you want GDC, build it separately as its own template, use binary snapshots to bootstrap, but don't have it affect the system compiler

you can use the same strategy for bootstrapping ldc; cross-compile snapshot builds of ldc, make tarballs of them, upload them somewhere, then use them to bootstrap ldc (same thing is done for e.g. rust, go, ...)

Re: package ldc for aarch64 (and more)

[Marie-Joseph]

[-- Attachment #1: Type: text/plain, Size: 227 bytes --]

New comment by Marie-Joseph on void-packages repository

https://github.com/void-linux/void-packages/issues/34752#issuecomment-1003437428

Comment:
Fair enough. If no one wants it, I won't force it.

I'll focus on LDC, then.

Re: package ldc for aarch64 (and more)

[Marie-Joseph]

[-- Attachment #1: Type: text/plain, Size: 474 bytes --]

New comment by Marie-Joseph on void-packages repository

https://github.com/void-linux/void-packages/issues/34752#issuecomment-1003447129

Comment:
@q66 :
> you can use the same strategy for bootstrapping ldc; cross-compile snapshot builds of ldc, make tarballs of them, upload them somewhere, then use them to bootstrap ldc (same thing is done for e.g. rust, go, ...)

Would I need to package it for musl or glibc or both? I'm assuming it needs to be x86_64 either way.

Re: package ldc for aarch64 (and more)

[q66]

[-- Attachment #1: Type: text/plain, Size: 366 bytes --]

New comment by q66 on void-packages repository

https://github.com/void-linux/void-packages/issues/34752#issuecomment-1003461731

Comment:
there needs to be a tarball for every host you want to build the compiler on; do you need any for x86_64? I thought that already bootstrapped fine with dmd, and the whole point of this was to bootstrap on aarch64 and the likes

Re: package ldc for aarch64 (and more)

[github-actions]

[-- Attachment #1: Type: text/plain, Size: 293 bytes --]

New comment by github-actions[bot] on void-packages repository

https://github.com/void-linux/void-packages/issues/34752#issuecomment-1159892080

Comment:
Issues become stale 90 days after last activity and are closed 14 days after that.  If this issue is still relevant bump it or assign it.

Re: [ISSUE] [CLOSED] package ldc for aarch64 (and more)

[github-actions]

[-- Attachment #1: Type: text/plain, Size: 982 bytes --]

Closed issue by Marie-Joseph on void-packages repository

https://github.com/void-linux/void-packages/issues/34752

Description:
## Updated Goals

After bootstrapping LDC locally, I'll host a version for cross-compiling it. The below steps are still requisite for (my) local LDC compilation.

## Steps

- [x] package GDC 

Necessary for my local bootstrap.

- [x]  package [gdmd](https://github.com/D-Programming-GDC/gdmd)

Necessary for my local bootstrap.

- [ ]  update LDC to cross-compile

This should be as simple as adding the LDMD front-end to the LDC package and using it to compile itself.

## Notes

Since I need this, I'm going to work on it and will update this issue as I progress. If more capable hands want to take over, feel absolutely free. It'll likely take me several days to a couple weeks to do this (especially since my build machine for testing is a Raspberry Pi 4) and I'm sure others with more experience would be able to do it faster.