Re: [PR PATCH] [Updated] [RFC] New numpy build_helper to support cross compilation

[ahesford <ahesford@users.noreply.github.com>]

[-- Attachment #1: Type: text/plain, Size: 718 bytes --]

There is an updated pull request by ahesford against master on the void-packages repository

https://github.com/ahesford/void-packages numpy-helper
https://github.com/void-linux/void-packages/pull/23515

[RFC] New numpy build_helper to support cross compilation
I plan on adding some packages that link against `python3-numpy` and require the same kinds of environment manipulations currently done in `python3-scipy` to support cross building. The new templates would be simplified if these manipulations were centralized in a build_helper, so here it is. `python3-scipy` has been reworked to take advantage of the new helper.

A patch file from https://github.com/void-linux/void-packages/pull/23515.patch is attached

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: github-pr-numpy-helper-23515.patch --]
[-- Type: text/x-diff, Size: 6048 bytes --]

From 9f1c081071751d77abf54064357564d53c186d8f Mon Sep 17 00:00:00 2001
From: "Andrew J. Hesford" <ajh@sideband.org>
Date: Fri, 10 Jul 2020 23:35:09 -0400
Subject: [PATCH 1/2] xbps-src: add numpy build_helper

Packages that link against python3-numpy require special consideration
to ensure that, in a cross-building environment, target libraries and
headers as well as the FORTRAN compiler and linker are properly
identifeid. Previously, this was done directly in the python3-scipy
template to support cross compilation. Now, a build_helper generalizes
these changes to support other package templates.
---
 Manual.md                    | 10 +++++++---
 common/build-helper/numpy.sh | 36 ++++++++++++++++++++++++++++++++++++
 2 files changed, 43 insertions(+), 3 deletions(-)
 create mode 100644 common/build-helper/numpy.sh

diff --git a/Manual.md b/Manual.md
index 10082b68da5..7e2cf9a5c02 100644
--- a/Manual.md
+++ b/Manual.md
@@ -940,9 +940,6 @@ suitable environment for working with certain sets of packages.
 
 The current list of available `build_helper` scripts is the following:
 
-- `rust` specifies environment variables required for cross-compiling crates via cargo and
-for compiling cargo -sys crates.
-
 - `gir` specifies dependencies for native and cross builds to deal with
 GObject Introspection. The following variables may be set in the template to handle
 cross builds which require additional hinting or exhibit problems. `GIR_EXTRA_LIBS_PATH` defines
@@ -951,6 +948,10 @@ additional paths to be searched when linking target binaries to be introspected.
 `qemu-<target_arch>-static` when running the target binary. You can for example specify
 `GIR_EXTRA_OPTIONS="-strace"` to see a trace of what happens when running that binary.
 
+- `numpy` specifies a `python3-numpy` dependency in `hostmakedepends` and
+establishes an environment necessary for `numpy.distutils` to function properly
+when cross compiling packages.
+
 - `qemu` sets additional variables for the `cmake` and `meson` build styles to allow
 executing cross-compiled binaries inside qemu.
 It sets `CMAKE_CROSSCOMPILING_EMULATOR` for cmake and `exe_wrapper` for meson
@@ -962,6 +963,9 @@ This aims to fix cross-builds for when the build-style is mixed: e.g. when in a
 `gnu-configure` style the configure script calls `qmake` or a `Makefile` in
 `gnu-makefile` style, respectively.
 
+- `rust` specifies environment variables required for cross-compiling crates via cargo and
+for compiling cargo -sys crates.
+
 <a id="functions"></a>
 ### Functions
 
diff --git a/common/build-helper/numpy.sh b/common/build-helper/numpy.sh
new file mode 100644
index 00000000000..96870f58294
--- /dev/null
+++ b/common/build-helper/numpy.sh
@@ -0,0 +1,36 @@
+#
+# numpy - build-helper for packages that compile against python3-numpy
+#
+# This build-helper makes sure packages can find python3-numpy libraries and
+# headers on the target architecture rather than the host, as well as making
+# sure the gfortran cross compiler is properly identified.
+
+# Even for cross compilation, numpy should be available on the host to ensure
+# that the host interpreter doesn't complain about missing deps
+if [[ $hostmakedepends != *"python3-numpy"* ]]; then
+	hostmakedepends+=" python3-numpy"
+fi
+
+if [ "$CROSS_BUILD" ]; then
+	if [[ $makedepends != *"python3-numpy"* ]]; then
+		makedepends+=" python3-numpy"
+	fi
+
+	# python3-setuptools finds numpy libs and headers on the host first;
+	# addding search paths up front allows the target to take priority
+	CFLAGS+=" -I${XBPS_CROSS_BASE}/${py3_sitelib}/numpy/core/include"
+	LDFLAGS+=" -L${XBPS_CROSS_BASE}/${py3_sitelib}/numpy/core/lib"
+
+	# distutils from python3-numpy looks to environment variables F77 and
+	# F90 rather than the FC variable set by the environment
+	export F77="${FC}"
+	export F90="${FC}"
+
+	# When compiling and linking FORTRAN, distutils from python3-numpy
+	# refuses respect any linker name except "gfortran"; symlink to the
+	# cross-compiler to that the right linker and compiler will be used
+	if _gfortran=$(command -v "${FC}" 2>/dev/null); then
+		ln -sf "${_gfortran}" "${XBPS_WRAPPERDIR}/gfortran"
+	fi
+	unset _gfortran
+fi

From 72a071ee7390fa844493e5d182ce15737c2e4ee2 Mon Sep 17 00:00:00 2001
From: "Andrew J. Hesford" <ajh@sideband.org>
Date: Fri, 10 Jul 2020 23:39:29 -0400
Subject: [PATCH 2/2] python3-scipy: use new numpy build_helper

---
 srcpkgs/python3-scipy/template | 22 +++-------------------
 1 file changed, 3 insertions(+), 19 deletions(-)

diff --git a/srcpkgs/python3-scipy/template b/srcpkgs/python3-scipy/template
index 3d602c347c9..0554f9c7261 100644
--- a/srcpkgs/python3-scipy/template
+++ b/srcpkgs/python3-scipy/template
@@ -4,10 +4,10 @@ version=1.5.0
 revision=2
 wrksrc="scipy-${version}"
 build_style=python3-module
+build_helper="numpy"
 make_check_args="--force"
-hostmakedepends="gcc-fortran python3-setuptools
- python3-Cython python3-numpy python3-pybind11"
-makedepends="python3-devel python3-numpy python3-pybind11
+hostmakedepends="gcc-fortran python3-setuptools python3-Cython python3-pybind11"
+makedepends="python3-devel python3-pybind11
  $(vopt_if openblas openblas-devel lapack-devel)"
 depends="python3-numpy"
 checkdepends="python3-nose"
@@ -29,25 +29,9 @@ case "$XBPS_TARGET_MACHINE" in
 	*) ;;
 esac
 
-if [ "$CROSS_BUILD" ]; then
-	# Make sure numpy is found for the target arch first
-	CFLAGS+=" -I${XBPS_CROSS_BASE}/${py3_sitelib}/numpy/core/include"
-	LDFLAGS+=" -L${XBPS_CROSS_BASE}/${py3_sitelib}/numpy/core/lib"
-
-	# Tell numpy.distutils where to find FORTRAN compilers
-	export F77="${FC}"
-	export F90="${FC}"
-fi
-
 LDFLAGS+=" -shared"
 
 pre_build() {
-	if [ "$CROSS_BUILD" ]; then
-		# numpy.distutils refuses to find the right linker for FORTRAN
-		# Link the cross compiler so the module will find it as gfortran
-		ln -sf "/usr/bin/${FC}" "${XBPS_WRAPPERDIR}/gfortran"
-	fi
-
 	# Find the right linear algebra subroutines on the target arch
 	: > site.cfg
 	for _blaslib in $(vopt_if openblas openblas "lapack blas"); do