[PR PATCH] go: enable PIE builds for Go executables

[PR PATCH] go: enable PIE builds for Go executables

[ericonr]

[-- Attachment #1: Type: text/plain, Size: 1164 bytes --]

There is a new pull request by ericonr against master on the void-packages repository

https://github.com/ericonr/void-packages go-default-pie
https://github.com/void-linux/void-packages/pull/20854

go: enable PIE builds for Go executables
Fixed up version of #20450 

Both Arch Linux and Alpine already implement this feature. Arch has a package called go-pie that's used for most of their PKGBUILDs, while the only version of Go on Alpine is one capable of producing PIE binaries.

On Arch: https://www.archlinux.org/packages/community/x86_64/go-pie/
On Alpine: https://pkgs.alpinelinux.org/package/edge/community/x86_64/go
The patch used here is actually adapted from the one used by Arch, and if adopted can allow us to remove the nostrip flag from the go build_style.

I'm getting some weird build output, where it says

loadinternal: cannot find runtime/cgo
so I'm not completely certain that it's 100% complete. Building a program that uses cgo does work cleanly, so I'm not sure what it means.

It's referenced in this issue, but without any solution.

A patch file from https://github.com/void-linux/void-packages/pull/20854.patch is attached

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: github-pr-go-default-pie-20854.patch --]
[-- Type: text/x-diff, Size: 1422 bytes --]

From 4fac0444912f240ec510aab00819f04799f4913e Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?=C3=89rico=20Rolim?= <erico.erc@gmail.com>
Date: Fri, 10 Apr 2020 17:54:35 -0300
Subject: [PATCH] go: enable PIE builds for Go executables

---
 srcpkgs/go/patches/default-buildmode-pie.patch | 12 ++++++++++++
 srcpkgs/go/template                            |  2 +-
 2 files changed, 13 insertions(+), 1 deletion(-)
 create mode 100644 srcpkgs/go/patches/default-buildmode-pie.patch

diff --git a/srcpkgs/go/patches/default-buildmode-pie.patch b/srcpkgs/go/patches/default-buildmode-pie.patch
new file mode 100644
index 00000000000..3861c8b3293
--- /dev/null
+++ b/srcpkgs/go/patches/default-buildmode-pie.patch
@@ -0,0 +1,12 @@
+diff go/src/cmd/go/internal/work/init.go go/src/cmd/go/internal/work/init.go
+--- go/src/cmd/go/internal/work/init.go
++++ go/src/cmd/go/internal/work/init.go
+@@ -131,7 +131,7 @@ func buildModeInit() {
+ 		ldBuildmode = "c-shared"
+ 	case "default":
+ 		switch cfg.Goos {
+-		case "android":
++		case "android", "linux":
+ 			codegenArg = "-shared"
+ 			ldBuildmode = "pie"
+ 		case "darwin":
diff --git a/srcpkgs/go/template b/srcpkgs/go/template
index fdab001f28c..d4af971de19 100644
--- a/srcpkgs/go/template
+++ b/srcpkgs/go/template
@@ -1,7 +1,7 @@
 # Template file for 'go'
 pkgname=go
 version=1.14.2
-revision=1
+revision=2
 create_wrksrc=yes
 build_wrksrc=go
 hostmakedepends="go1.12-bootstrap"

Re: go: enable PIE builds for Go executables

[travankor]

[-- Attachment #1: Type: text/plain, Size: 540 bytes --]

New comment by travankor on void-packages repository

https://github.com/void-linux/void-packages/pull/20854#issuecomment-612597574

Comment:
@ericonr 

I don't think the problem has to do with cgo. Travis suggests the failure occurs during linking. In the Alpine APKBUILD, it says "Go expect host linker instead of the cross-compiler"

That said, I think `gcc-go` already has pie support. I wonder why `gcc-go` isn't the default for Void's build style considering it has better ppc (big endian) support (@q66), among other advantages?

Re: go: enable PIE builds for Go executables

[ericonr]

[-- Attachment #1: Type: text/plain, Size: 592 bytes --]

New comment by ericonr on void-packages repository

https://github.com/void-linux/void-packages/pull/20854#issuecomment-612661516

Comment:
In hindsight, copying Arch's patch might have been the wrong solution, given that they only use a single arch.

Regarding gcc-go, it seems current support is at Go 1.12.2 (https://golang.org/doc/install/gccgo), while Go is at 1.14.2. If that's a big issue I can't say. We'd also have to make some changes to the build_style, I think. And it would require a lot of testing as well, being an even bigger change than simply enabling PIE for everything.

Re: go: enable PIE builds for Go executables

[ericonr]

[-- Attachment #1: Type: text/plain, Size: 600 bytes --]

New comment by ericonr on void-packages repository

https://github.com/void-linux/void-packages/pull/20854#issuecomment-612661516

Comment:
@travankor 

In hindsight, copying Arch's patch might have been the wrong solution, given that they only use a single arch.

Regarding gcc-go, it seems current support is at Go 1.12.2 (https://golang.org/doc/install/gccgo), while Go is at 1.14.2. If that's a big issue I can't say. We'd also have to make some changes to the build_style, I think. And it would require a lot of testing as well, being an even bigger change than enabling PIE for everything.

Re: go: enable PIE builds for Go executables

[q66]

[-- Attachment #1: Type: text/plain, Size: 278 bytes --]

New comment by q66 on void-packages repository

https://github.com/void-linux/void-packages/pull/20854#issuecomment-612664955

Comment:
gccgo is buggy (since gcc9 it frequently hangs on large projects, can't even bootstrap the official compiler), it's not a drop-in replacement

Re: go: enable PIE builds for Go executables

[travankor]

[-- Attachment #1: Type: text/plain, Size: 525 bytes --]

New comment by travankor on void-packages repository

https://github.com/void-linux/void-packages/pull/20854#issuecomment-612674001

Comment:
@ericonr 

Use [Alpine's patch](https://git.alpinelinux.org/aports/plain/community/go/default-buildmode-pie.patch) and carefully check their APKBUILD. Once cross works, I would check important packages to see if they work. The main issue, as pointed out in the last attempt, is broken packages. Alpine most likely has fewer go packages due to their more stringent security policy.

Re: go: enable PIE builds for Go executables

[ericonr]

[-- Attachment #1: Type: text/plain, Size: 369 bytes --]

New comment by ericonr on void-packages repository

https://github.com/void-linux/void-packages/pull/20854#issuecomment-612676337

Comment:
I have tested the fscrypt executable (not the pam_module) compiled with go-pie. It worked well. What other project do you think it would be interesting to test? I can try to bootstrap the compiler with itself, at the very least.

Re: go: enable PIE builds for Go executables

[ericonr]

[-- Attachment #1: Type: text/plain, Size: 255 bytes --]

New comment by ericonr on void-packages repository

https://github.com/void-linux/void-packages/pull/20854#issuecomment-615573400

Comment:
Huh, Alpine folks have just changed a lot of their Go package. Will have to take a closer look at the whole thing.

Re: [PR PATCH] [Updated] go: enable PIE builds for Go executables

[ericonr]

[-- Attachment #1: Type: text/plain, Size: 1109 bytes --]

There is an updated pull request by ericonr against master on the void-packages repository

https://github.com/ericonr/void-packages go-default-pie
https://github.com/void-linux/void-packages/pull/20854

go: enable PIE builds for Go executables
Fixed up version of #20450 

Both Arch Linux and Alpine already implement this feature. Arch has a package called go-pie that's used for most of their PKGBUILDs, while the only version of Go on Alpine is one capable of producing PIE binaries.

On Arch: https://www.archlinux.org/packages/community/x86_64/go-pie/
On Alpine: https://pkgs.alpinelinux.org/package/edge/community/x86_64/go
The patch used here is actually adapted from the one used by Arch, and if adopted can allow us to remove the nostrip flag from the go build_style.

I'm getting some weird build output, where it says

loadinternal: cannot find runtime/cgo
so I'm not completely certain that it's 100% complete. Building a program that uses cgo does work cleanly, so I'm not sure what it means.

A patch file from https://github.com/void-linux/void-packages/pull/20854.patch is attached

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: github-pr-go-default-pie-20854.patch --]
[-- Type: text/x-diff, Size: 1198 bytes --]

From 19b05766856fd8c1d5f334651051218ec0e7315e Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?=C3=89rico=20Rolim?= <erico.erc@gmail.com>
Date: Fri, 10 Apr 2020 17:54:35 -0300
Subject: [PATCH] go: enable PIE builds for Go executables

Enabling PIE builds for Go executables created by gc enables us to
remove the nostrip and nopie variables from those builds. nostrip was
moved to the gccgo cases only.
---
 common/environment/build-style/go.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/common/environment/build-style/go.sh b/common/environment/build-style/go.sh
index 3f83013d4f7..fd23421fa6e 100644
--- a/common/environment/build-style/go.sh
+++ b/common/environment/build-style/go.sh
@@ -4,15 +4,15 @@ if [ -z "$hostmakedepends" -o "${hostmakedepends##*gcc-go-tools*}" ]; then
 		archs="aarch64* armv[567]* i686* x86_64* ppc64le*"
 	fi
 	hostmakedepends+=" go"
-	nopie=yes
+	export GOFLAGS="-buildmode=pie"
 else
 	# gccgo compiler
 	if [ "$CROSS_BUILD" ]; then
 		# target compiler to use; otherwise it'll just call gccgo
 		export GCCGO="${XBPS_CROSS_TRIPLET}-gccgo"
 	fi
+	nostrip=yes
 fi
-nostrip=yes
 
 case "$XBPS_TARGET_MACHINE" in
 	aarch64*) export GOARCH=arm64;;

Re: go: enable PIE builds for Go executables

[ericonr]

[-- Attachment #1: Type: text/plain, Size: 1078 bytes --]

New comment by ericonr on void-packages repository

https://github.com/void-linux/void-packages/pull/20854#issuecomment-616050062

Comment:
https://github.com/alpinelinux/aports/commits/master/community/go will give some idea of the apparent fight it is to get Go working properly.

That said, simply setting `GOFLAGS` to the appropriate value made it possible to build Go executables with pie, which can be stripped and checked by the build hooks (no more nostrip/nopie, yay). I have built executables for armv6l and x86_64, and preliminary testing shows them working well on x64, while I still have to test arm. It seems getting the compiler itself to be built with PIE will be harder.

Due to not having any idea about how to test arches that use gccgo (@q66 I imagine you have some experience with this, given your void-ppc project), I tried to keep the environment for that build system as close as possible to the original. If you orient me on testing, we can see if some or any of these changes can be moved to it as well.

@travankor if you want to take a look :)

Re: go: enable PIE builds for Go executables

[ericonr]

[-- Attachment #1: Type: text/plain, Size: 221 bytes --]

New comment by ericonr on void-packages repository

https://github.com/void-linux/void-packages/pull/20854#issuecomment-616051450

Comment:
Tested micro on armv6l-musl, cross compiled from x86_64-glibc. Worked perfectly.

Re: go: enable PIE builds for Go executables

[the-maldridge]

[-- Attachment #1: Type: text/plain, Size: 267 bytes --]

New comment by the-maldridge on void-packages repository

https://github.com/void-linux/void-packages/pull/20854#issuecomment-616290367

Comment:
There's a good reason that upstream has this turned off by default.  Until we know why that is, this must not be merged.

Re: go: enable PIE builds for Go executables

[ericonr]

[-- Attachment #1: Type: text/plain, Size: 564 bytes --]

New comment by ericonr on void-packages repository

https://github.com/void-linux/void-packages/pull/20854#issuecomment-616949995

Comment:
Broken Go packages when building locally:

```
chronograf
dht-node
dnote-cli
etcd
goproxy
grafana
gucci
gx-go
gx
k3s
kubernetes -> linking issue, worked without the patch
kurly
lxd-lts
madonctl
spreed-webrtc -> worked cleanly the second time
vault
```

All broken packages besides `kubernetes` were due to misc errors (I will try to update their templates, for those that aren't yet explicitly broken).

Re: [PR PATCH] [Closed]: go: enable PIE builds for Go executables

[ericonr]

[-- Attachment #1: Type: text/plain, Size: 946 bytes --]

There's a closed pull request on the void-packages repository

go: enable PIE builds for Go executables
https://github.com/void-linux/void-packages/pull/20854

Description:
Fixed up version of #20450 

Both Arch Linux and Alpine already implement this feature. Arch has a package called go-pie that's used for most of their PKGBUILDs, while the only version of Go on Alpine is one capable of producing PIE binaries.

On Arch: https://www.archlinux.org/packages/community/x86_64/go-pie/
On Alpine: https://pkgs.alpinelinux.org/package/edge/community/x86_64/go
The patch used here is actually adapted from the one used by Arch, and if adopted can allow us to remove the nostrip flag from the go build_style.

I'm getting some weird build output, where it says

loadinternal: cannot find runtime/cgo
so I'm not completely certain that it's 100% complete. Building a program that uses cgo does work cleanly, so I'm not sure what it means.

Re: go: enable PIE builds for Go executables

[ericonr]

[-- Attachment #1: Type: text/plain, Size: 473 bytes --]

New comment by ericonr on void-packages repository

https://github.com/void-linux/void-packages/pull/20854#issuecomment-616950772

Comment:
Sorry, misclicked.

Given that it can probably still use some polish, we shouldn't change the build_style yet. Furthermore, as pointed out, PIE doesn't make such a difference for safer languages, like Go. However, I would like to make this a TRACKING issue to keep up with advancements in the toolchain. Do you think that'd be ok?

Re: [TRACKING-upstream] go: enable PIE builds for Go executables

[ericonr]

[-- Attachment #1: Type: text/plain, Size: 161 bytes --]

New comment by ericonr on void-packages repository

https://github.com/void-linux/void-packages/pull/20854#issuecomment-700376559

Comment:
Superseded by #24841

Re: [PR PATCH] [Closed]: [TRACKING-upstream] go: enable PIE builds for Go executables

[ericonr]

[-- Attachment #1: Type: text/plain, Size: 966 bytes --]

There's a closed pull request on the void-packages repository

[TRACKING-upstream] go: enable PIE builds for Go executables
https://github.com/void-linux/void-packages/pull/20854

Description:
Fixed up version of #20450 

Both Arch Linux and Alpine already implement this feature. Arch has a package called go-pie that's used for most of their PKGBUILDs, while the only version of Go on Alpine is one capable of producing PIE binaries.

On Arch: https://www.archlinux.org/packages/community/x86_64/go-pie/
On Alpine: https://pkgs.alpinelinux.org/package/edge/community/x86_64/go
The patch used here is actually adapted from the one used by Arch, and if adopted can allow us to remove the nostrip flag from the go build_style.

I'm getting some weird build output, where it says

loadinternal: cannot find runtime/cgo
so I'm not completely certain that it's 100% complete. Building a program that uses cgo does work cleanly, so I'm not sure what it means.