[PR PATCH] build-style/go: allow usage with gccgo compiler

[PR PATCH] build-style/go: allow usage with gccgo compiler

[q66]

[-- Attachment #1: Type: text/plain, Size: 2410 bytes --]

There is a new pull request by q66 against master on the void-packages repository

https://github.com/void-ppc/void-packages gccgo-style
https://github.com/void-linux/void-packages/pull/20996

build-style/go: allow usage with gccgo compiler
This checks if `gcc-go-tools` have been specified in host deps and if so, use gccgo to build (`gcc-go-tools` conflicts with go).

Also, since gccgo is not a multi-compiler, we need to override `GCCGO` to get cross-builds. The gccgo for target is always installed, as it's a part of our cross-toolchains.

Basically, in summary, the behavior with this patch is:

1) Use `go` by default, always
2) If a template explicitly chooses to install `gcc-go-tools` in `hostmakedepends`, that indicates to the `build-style` that we're to build with gccgo (current behavior: conflicting packages will cause the build to fail)

Since `gcc-go-tools` provides the same regular `go` frontend as the main `go` package, the `build-style` will generally work OOTB, unchanged, and it respects `GOCACHE`, `GOARCH`, `GOOS` and so on. The only differences need to be:

1) No default `archs`; `gccgo` is available everywhere
2) No `nostrip`, since it's using a regular GNU toolchain with GCC backend, so it generates debug symbols same as usual
3) No `nopie`, as our `--enable-default-pie` for GCC applies here
4) The `GCCGO` variable needs to be set to point to the target's compiler, but otherwise `GOARCH` is still respected everywhere else, so it's handled outside the conditional block

## Usage

I have one place where I'd like to use this and that's `PopCorn`. It's a fairly trivial application and has been tested to work well under `gccgo` (I use it for stats reporting on big-endian architectures where the regular compiler doesn't exist and have been for months). To switch, we'd basically just do this:

```
--- a/srcpkgs/PopCorn/template
+++ b/srcpkgs/PopCorn/template
@@ -8,7 +8,7 @@ go_package="${go_import_path}/cmd/popcorn
  ${go_import_path}/cmd/popcornctl
  ${go_import_path}/cmd/statrepo
  ${go_import_path}/cmd/pqueryd"
-hostmakedepends="git"
+hostmakedepends="gcc-go-tools git"
 depends="snooze"
 short_desc="Package statistics system"
 maintainer="Michael Aldridge <maldridge@voidlinux.org>"
```

With the `build-style` change, that just works.

A patch file from https://github.com/void-linux/void-packages/pull/20996.patch is attached

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: github-pr-gccgo-style-20996.patch --]
[-- Type: text/x-diff, Size: 1435 bytes --]

From df0fce474d7de9c0fc37f140a4006651f26abfb2 Mon Sep 17 00:00:00 2001
From: q66 <daniel@octaforge.org>
Date: Mon, 11 Nov 2019 03:03:14 +0100
Subject: [PATCH] build-style/go: allow usage with gccgo compiler

This checks if gcc-go-tools have been specified in host deps
and if so, use gccgo to build (gcc-go-tools conflicts with go).

Also, since gccgo is not a multi-compiler, we need to override GCCGO
to get cross-builds. The gccgo for target is always installed, as
it's a part of our cross-toolchains.

[ci skip]
---
 common/environment/build-style/go.sh | 19 ++++++++++++++-----
 1 file changed, 14 insertions(+), 5 deletions(-)

diff --git a/common/environment/build-style/go.sh b/common/environment/build-style/go.sh
index ce0bdcc18b4..6dccf90f6a9 100644
--- a/common/environment/build-style/go.sh
+++ b/common/environment/build-style/go.sh
@@ -1,9 +1,18 @@
-if [ -z "$archs" ]; then
-	archs="aarch64* armv[567]* i686* x86_64* ppc64le*"
+if [ "${hostmakedepends##*gcc-go-tools*}" ]; then
+	# gc compiler
+	if [ -z "$archs" ]; then
+		archs="aarch64* armv[567]* i686* x86_64* ppc64le*"
+	fi
+	hostmakedepends+=" go"
+	nostrip=yes
+	nopie=yes
+else
+	# gccgo compiler
+	if [ "$CROSS_BUILD" ]; then
+		# target compiler to use; otherwise it'll just call gccgo
+		export GCCGO="${XBPS_CROSS_TRIPLET}-gccgo"
+	fi
 fi
-hostmakedepends+=" go"
-nostrip=yes
-nopie=yes
 
 case "$XBPS_TARGET_MACHINE" in
 	aarch64*) export GOARCH=arm64;;

Re: build-style/go: allow usage with gccgo compiler

[the-maldridge]

[-- Attachment #1: Type: text/plain, Size: 312 bytes --]

New comment by the-maldridge on void-packages repository

https://github.com/void-linux/void-packages/pull/20996#issuecomment-613775931

Comment:
Its been a while since I looked at the go build style, but I thought it provided go as hostmakedepends.  How does this not conflict with the automagic go dependency?

Re: build-style/go: allow usage with gccgo compiler

[the-maldridge]

[-- Attachment #1: Type: text/plain, Size: 411 bytes --]

New comment by the-maldridge on void-packages repository

https://github.com/void-linux/void-packages/pull/20996#issuecomment-613775931

Comment:
Its been a while since I looked at the go build style, but I thought it provided go as hostmakedepends.  How does this not conflict with the automagic go dependency?

EDIT: Reading the patch before eating was my first mistake.  I see the conflict resolution now.

Re: build-style/go: allow usage with gccgo compiler

[q66]

[-- Attachment #1: Type: text/plain, Size: 293 bytes --]

New comment by q66 on void-packages repository

https://github.com/void-linux/void-packages/pull/20996#issuecomment-613777025

Comment:
yes, it does that, that's the point, the patch newly enables behavior that wasn't previously possible, without any behavior change for any current tamplates

Re: build-style/go: allow usage with gccgo compiler

[q66]

[-- Attachment #1: Type: text/plain, Size: 293 bytes --]

New comment by q66 on void-packages repository

https://github.com/void-linux/void-packages/pull/20996#issuecomment-613777025

Comment:
yes, it does that, that's the point, the patch newly enables behavior that wasn't previously possible, without any behavior change for any current templates

Re: build-style/go: allow usage with gccgo compiler

[travankor]

[-- Attachment #1: Type: text/plain, Size: 399 bytes --]

New comment by travankor on void-packages repository

https://github.com/void-linux/void-packages/pull/20996#issuecomment-613880985

Comment:
you also get pie and gcc hardening flags for free with this, the official go compiler needs coaxing to enable those (and it's not on by default).

I think if it's tested and works for runtime, then templates should switch to this for the extra hardening.

Re: build-style/go: allow usage with gccgo compiler

[travankor]

[-- Attachment #1: Type: text/plain, Size: 401 bytes --]

New comment by travankor on void-packages repository

https://github.com/void-linux/void-packages/pull/20996#issuecomment-613880985

Comment:
you also get pie and gcc hardening flags for free with this, the official go compiler needs coaxing to enable those (and it's not on by default).

I'd argue if it's tested and works for runtime, then templates should switch to this for the extra hardening.

Re: build-style/go: allow usage with gccgo compiler

[q66]

[-- Attachment #1: Type: text/plain, Size: 293 bytes --]

New comment by q66 on void-packages repository

https://github.com/void-linux/void-packages/pull/20996#issuecomment-614072670

Comment:
No, definitely not, gccgo is too buggy in general to be used as drop-in at this point, and lags behind in go version, this should be for specific cases only

Re: [PR PATCH] [Updated] build-style/go: allow usage with gccgo compiler

[q66]

[-- Attachment #1: Type: text/plain, Size: 2415 bytes --]

There is an updated pull request by q66 against master on the void-packages repository

https://github.com/void-ppc/void-packages gccgo-style
https://github.com/void-linux/void-packages/pull/20996

build-style/go: allow usage with gccgo compiler
This checks if `gcc-go-tools` have been specified in host deps and if so, use gccgo to build (`gcc-go-tools` conflicts with go).

Also, since gccgo is not a multi-compiler, we need to override `GCCGO` to get cross-builds. The gccgo for target is always installed, as it's a part of our cross-toolchains.

Basically, in summary, the behavior with this patch is:

1) Use `go` by default, always
2) If a template explicitly chooses to install `gcc-go-tools` in `hostmakedepends`, that indicates to the `build-style` that we're to build with gccgo (current behavior: conflicting packages will cause the build to fail)

Since `gcc-go-tools` provides the same regular `go` frontend as the main `go` package, the `build-style` will generally work OOTB, unchanged, and it respects `GOCACHE`, `GOARCH`, `GOOS` and so on. The only differences need to be:

1) No default `archs`; `gccgo` is available everywhere
2) No `nostrip`, since it's using a regular GNU toolchain with GCC backend, so it generates debug symbols same as usual
3) No `nopie`, as our `--enable-default-pie` for GCC applies here
4) The `GCCGO` variable needs to be set to point to the target's compiler, but otherwise `GOARCH` is still respected everywhere else, so it's handled outside the conditional block

## Usage

I have one place where I'd like to use this and that's `PopCorn`. It's a fairly trivial application and has been tested to work well under `gccgo` (I use it for stats reporting on big-endian architectures where the regular compiler doesn't exist and have been for months). To switch, we'd basically just do this:

```
--- a/srcpkgs/PopCorn/template
+++ b/srcpkgs/PopCorn/template
@@ -8,7 +8,7 @@ go_package="${go_import_path}/cmd/popcorn
  ${go_import_path}/cmd/popcornctl
  ${go_import_path}/cmd/statrepo
  ${go_import_path}/cmd/pqueryd"
-hostmakedepends="git"
+hostmakedepends="gcc-go-tools git"
 depends="snooze"
 short_desc="Package statistics system"
 maintainer="Michael Aldridge <maldridge@voidlinux.org>"
```

With the `build-style` change, that just works.

A patch file from https://github.com/void-linux/void-packages/pull/20996.patch is attached

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: github-pr-gccgo-style-20996.patch --]
[-- Type: text/x-diff, Size: 1419 bytes --]

From 897406887fcdba95195ec079eb736eb2fe467878 Mon Sep 17 00:00:00 2001
From: q66 <daniel@octaforge.org>
Date: Mon, 11 Nov 2019 03:03:14 +0100
Subject: [PATCH] build-style/go: allow usage with gccgo compiler

This checks if gcc-go-tools have been specified in host deps
and if so, use gccgo to build (gcc-go-tools conflicts with go).

Also, since gccgo is not a multi-compiler, we need to override GCCGO
to get cross-builds. The gccgo for target is always installed, as
it's a part of our cross-toolchains.

[ci skip]
---
 common/environment/build-style/go.sh | 17 +++++++++++++----
 1 file changed, 13 insertions(+), 4 deletions(-)

diff --git a/common/environment/build-style/go.sh b/common/environment/build-style/go.sh
index ce0bdcc18b4..a0cf330d600 100644
--- a/common/environment/build-style/go.sh
+++ b/common/environment/build-style/go.sh
@@ -1,9 +1,18 @@
-if [ -z "$archs" ]; then
-	archs="aarch64* armv[567]* i686* x86_64* ppc64le*"
+if [ "${hostmakedepends##*gcc-go-tools*}" ]; then
+	# gc compiler
+	if [ -z "$archs" ]; then
+		archs="aarch64* armv[567]* i686* x86_64* ppc64le*"
+	fi
+	hostmakedepends+=" go"
+	nopie=yes
+else
+	# gccgo compiler
+	if [ "$CROSS_BUILD" ]; then
+		# target compiler to use; otherwise it'll just call gccgo
+		export GCCGO="${XBPS_CROSS_TRIPLET}-gccgo"
+	fi
 fi
-hostmakedepends+=" go"
 nostrip=yes
-nopie=yes
 
 case "$XBPS_TARGET_MACHINE" in
 	aarch64*) export GOARCH=arm64;;

Re: build-style/go: allow usage with gccgo compiler

[q66]

[-- Attachment #1: Type: text/plain, Size: 401 bytes --]

New comment by q66 on void-packages repository

https://github.com/void-linux/void-packages/pull/20996#issuecomment-614203789

Comment:
Updated: `nostrip` must always be set, as according to `gccgo` documentation:

    Go programs must generally be compiled with debugging information, and -g1 is the default as described below. Stripping a Go program will generally cause it to misbehave or fail. 

Re: build-style/go: allow usage with gccgo compiler

[q66]

[-- Attachment #1: Type: text/plain, Size: 399 bytes --]

New comment by q66 on void-packages repository

https://github.com/void-linux/void-packages/pull/20996#issuecomment-614203789

Comment:
Updated: `nostrip` must always be set, as according to `gccgo` documentation:

*Go programs must generally be compiled with debugging information, and -g1 is the default as described below. Stripping a Go program will generally cause it to misbehave or fail. *

Re: build-style/go: allow usage with gccgo compiler

[travankor]

[-- Attachment #1: Type: text/plain, Size: 296 bytes --]

New comment by travankor on void-packages repository

https://github.com/void-linux/void-packages/pull/20996#issuecomment-614284699

Comment:
`-trimpath` removes all file system paths from the compiled executable, to improves build reproducibility.

This sounds like something Void should use.

Re: build-style/go: allow usage with gccgo compiler

[travankor]

[-- Attachment #1: Type: text/plain, Size: 295 bytes --]

New comment by travankor on void-packages repository

https://github.com/void-linux/void-packages/pull/20996#issuecomment-614284699

Comment:
`-trimpath` removes all file system paths from the compiled executable, to improve build reproducibility.

This sounds like something Void should use.

Re: build-style/go: allow usage with gccgo compiler

[q66]

[-- Attachment #1: Type: text/plain, Size: 220 bytes --]

New comment by q66 on void-packages repository

https://github.com/void-linux/void-packages/pull/20996#issuecomment-614294481

Comment:
seems pointless, the build location is always the same so it's already reproducible

Re: build-style/go: allow usage with gccgo compiler

[travankor]

[-- Attachment #1: Type: text/plain, Size: 494 bytes --]

New comment by travankor on void-packages repository

https://github.com/void-linux/void-packages/pull/20996#issuecomment-614306801

Comment:
`-trimpath` should also clear the go buildid:

"The golang compiler (gc) does not provide by default the ".note.gnu.build-id" section that GCC does. Though, if the binary being built supports being compiled with gccgo, the gccgo compiler does support and include the .note.gnu.build-id."

https://fedoraproject.org/wiki/PackagingDrafts/Go#Build_ID

Re: build-style/go: allow usage with gccgo compiler

[travankor]

[-- Attachment #1: Type: text/plain, Size: 560 bytes --]

New comment by travankor on void-packages repository

https://github.com/void-linux/void-packages/pull/20996#issuecomment-614306801

Comment:
`-trimpath` should also clear the go buildid:

"The golang compiler (gc) does not provide by default the ".note.gnu.build-id" section that GCC does. Though, if the binary being built supports being compiled with gccgo, the gccgo compiler does support and include the .note.gnu.build-id."

https://fedoraproject.org/wiki/PackagingDrafts/Go#Build_ID

Edit: If `-trimpath` doesn't work, `-ldflags=-buildid=` should

Re: build-style/go: allow usage with gccgo compiler

[q66]

[-- Attachment #1: Type: text/plain, Size: 240 bytes --]

New comment by q66 on void-packages repository

https://github.com/void-linux/void-packages/pull/20996#issuecomment-614314170

Comment:
out of scope of the PR, also void builds are not reproducible anyway, feel free to do it if you want to

Re: build-style/go: allow usage with gccgo compiler

[Duncaen]

[-- Attachment #1: Type: text/plain, Size: 251 bytes --]

New comment by Duncaen on void-packages repository

https://github.com/void-linux/void-packages/pull/20996#issuecomment-614696321

Comment:
lgtm, I still hate that `build_style` adds implicit dependencies, making this conditional makes it even worse.

Re: build-style/go: allow usage with gccgo compiler

[q66]

[-- Attachment #1: Type: text/plain, Size: 408 bytes --]

New comment by q66 on void-packages repository

https://github.com/void-linux/void-packages/pull/20996#issuecomment-614705751

Comment:
@Duncaen Well, it shouldn't. Should we decide to move the toolchain dependency from build-style to the main template, this code should technically still work. I'd argue that we should possibly do that later, too (and get rid of environment adding dependencies completely)

Re: build-style/go: allow usage with gccgo compiler

[Duncaen]

[-- Attachment #1: Type: text/plain, Size: 219 bytes --]

New comment by Duncaen on void-packages repository

https://github.com/void-linux/void-packages/pull/20996#issuecomment-614708239

Comment:
Yes I think that would be good, but I'm fine with doing this in a separate PR.

Re: build-style/go: allow usage with gccgo compiler

[Duncaen]

[-- Attachment #1: Type: text/plain, Size: 215 bytes --]

New comment by Duncaen on void-packages repository

https://github.com/void-linux/void-packages/pull/20996#issuecomment-614708239

Comment:
Yes I think that would be good, I'm fine with doing this in a separate PR.

Re: [PR PATCH] [Merged]: build-style/go: allow usage with gccgo compiler

[q66]

[-- Attachment #1: Type: text/plain, Size: 2258 bytes --]

There's a merged pull request on the void-packages repository

build-style/go: allow usage with gccgo compiler
https://github.com/void-linux/void-packages/pull/20996

Description:
This checks if `gcc-go-tools` have been specified in host deps and if so, use gccgo to build (`gcc-go-tools` conflicts with go).

Also, since gccgo is not a multi-compiler, we need to override `GCCGO` to get cross-builds. The gccgo for target is always installed, as it's a part of our cross-toolchains.

Basically, in summary, the behavior with this patch is:

1) Use `go` by default, always
2) If a template explicitly chooses to install `gcc-go-tools` in `hostmakedepends`, that indicates to the `build-style` that we're to build with gccgo (current behavior: conflicting packages will cause the build to fail)

Since `gcc-go-tools` provides the same regular `go` frontend as the main `go` package, the `build-style` will generally work OOTB, unchanged, and it respects `GOCACHE`, `GOARCH`, `GOOS` and so on. The only differences need to be:

1) No default `archs`; `gccgo` is available everywhere
2) No `nostrip`, since it's using a regular GNU toolchain with GCC backend, so it generates debug symbols same as usual
3) No `nopie`, as our `--enable-default-pie` for GCC applies here
4) The `GCCGO` variable needs to be set to point to the target's compiler, but otherwise `GOARCH` is still respected everywhere else, so it's handled outside the conditional block

## Usage

I have one place where I'd like to use this and that's `PopCorn`. It's a fairly trivial application and has been tested to work well under `gccgo` (I use it for stats reporting on big-endian architectures where the regular compiler doesn't exist and have been for months). To switch, we'd basically just do this:

```
--- a/srcpkgs/PopCorn/template
+++ b/srcpkgs/PopCorn/template
@@ -8,7 +8,7 @@ go_package="${go_import_path}/cmd/popcorn
  ${go_import_path}/cmd/popcornctl
  ${go_import_path}/cmd/statrepo
  ${go_import_path}/cmd/pqueryd"
-hostmakedepends="git"
+hostmakedepends="gcc-go-tools git"
 depends="snooze"
 short_desc="Package statistics system"
 maintainer="Michael Aldridge <maldridge@voidlinux.org>"
```

With the `build-style` change, that just works.