[PR PATCH] Add meson awareness to `numpy` helper and `python3-pep517` style

[PR PATCH] Add meson awareness to `numpy` helper and `python3-pep517` style

[ahesford]

[-- Attachment #1: Type: text/plain, Size: 753 bytes --]

There is a new pull request by ahesford against master on the void-packages repository

https://github.com/ahesford/void-packages meson-numpy
https://github.com/void-linux/void-packages/pull/46186

Add meson awareness to `numpy` helper and `python3-pep517` style
Now that a `meson` build helper exists, we can further remove redundancy in some numeric python packages. In addition to the two cleaned up here, I believe we have some pending PRs that would benefit from these changes.

While I'm touching the `numpy` helper, I might as well fix its missing documentation.

cc: @tornaria 

#### Testing the changes
- I tested the changes in this PR: **YES**

A patch file from https://github.com/void-linux/void-packages/pull/46186.patch is attached

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: github-pr-meson-numpy-46186.patch --]
[-- Type: text/x-diff, Size: 6924 bytes --]

From 7cf147c7cdd15c8bf5691e8d2a5d909a58e7022f Mon Sep 17 00:00:00 2001
From: "Andrew J. Hesford" <ajh@sideband.org>
Date: Thu, 21 Sep 2023 11:00:56 -0400
Subject: [PATCH 1/4] common/build-helper/numpy.sh: write meson cross file when
 needed

---
 Manual.md                    |  5 +++++
 common/build-helper/numpy.sh | 10 ++++++++++
 2 files changed, 15 insertions(+)

diff --git a/Manual.md b/Manual.md
index 88f9f5243258a..27f27394a7129 100644
--- a/Manual.md
+++ b/Manual.md
@@ -1084,6 +1084,11 @@ meson for cross builds. This is particularly useful for building packages that w
 invocations (e.g., `python3-pep517` packages that use a meson backend) and is added by default
 for packages that use the `meson` build style.
 
+- `numpy` configures the environment for cross-compilation of python packages that provide
+compiled extensions linking to NumPy C libraries. If the `meson` build helper is also
+configured, a secondary cross file, `${XBPS_WRAPPERDIR}/meson/xbps_numpy.cross`, will be
+written to inform meson where common NumPy components may be found.
+
 - `qemu` sets additional variables for the `cmake` and `meson` build styles to allow
 executing cross-compiled binaries inside qemu.
 It sets `CMAKE_CROSSCOMPILING_EMULATOR` for cmake and `exe_wrapper` for meson
diff --git a/common/build-helper/numpy.sh b/common/build-helper/numpy.sh
index 74c6421165293..bf40390ad1cae 100644
--- a/common/build-helper/numpy.sh
+++ b/common/build-helper/numpy.sh
@@ -34,4 +34,14 @@ if [ "$CROSS_BUILD" ]; then
 		ln -sf "${_gfortran}" "${XBPS_WRAPPERDIR}/gfortran"
 	fi
 	unset _gfortran
+
+	# Write a secondary meson cross file for numpy configuration
+	if [[ "${build_helper}" = *meson* ]]; then
+		mkdir -p "${XBPS_WRAPPERDIR}/meson"
+		cat > "${XBPS_WRAPPERDIR}/meson/xbps_numpy.cross" <<-EOF
+			[properties]
+			numpy-include-dir = '${XBPS_CROSS_BASE}/${py3_sitelib}/numpy/core/include'
+			pythran-include-dir = '${XBPS_CROSS_BASE}/${py3_sitelib}/pythran'
+			EOF
+	fi
 fi

From 74428db545a5564aec779b373af730c48c53bdd7 Mon Sep 17 00:00:00 2001
From: "Andrew J. Hesford" <ajh@sideband.org>
Date: Thu, 21 Sep 2023 11:01:24 -0400
Subject: [PATCH 2/4] common/build-style/python3-pep517.sh: use meson cross
 files by default

---
 common/build-style/python3-pep517.sh | 14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)

diff --git a/common/build-style/python3-pep517.sh b/common/build-style/python3-pep517.sh
index 2d5ff7caf34eb..5ec20dd117f8a 100644
--- a/common/build-style/python3-pep517.sh
+++ b/common/build-style/python3-pep517.sh
@@ -4,8 +4,18 @@
 
 do_build() {
 	: ${make_build_target:=.}
-	: ${make_build_args:=--no-isolation  --wheel}
-	python3 -m build ${make_build_args} ${make_build_target}
+
+	if [ "${CROSS_BUILD}" ] && [[ "${build_helper}" = *meson* ]]; then
+		local mcross="-Csetup-args=--cross-file=${XBPS_WRAPPERDIR}/meson"
+		make_build_args+=" ${mcross}/xbps_meson.cross"
+
+		if [[ "${build_helper}" = *numpy* ]]; then
+			make_build_args+=" ${mcross}/xbps_numpy.cross"
+		fi
+	fi
+
+	python3 -m build --no-isolation --wheel \
+		${make_build_args} ${make_build_target}
 }
 
 do_check() {

From 881738d6d9ab0c68560182cf5fd5ee35ce295e36 Mon Sep 17 00:00:00 2001
From: "Andrew J. Hesford" <ajh@sideband.org>
Date: Thu, 21 Sep 2023 11:19:38 -0400
Subject: [PATCH 3/4] python3-scipy: use meson-aware numpy helper and pep517
 style

---
 srcpkgs/python3-scipy/template | 23 ++++-------------------
 1 file changed, 4 insertions(+), 19 deletions(-)

diff --git a/srcpkgs/python3-scipy/template b/srcpkgs/python3-scipy/template
index e99919babf3bf..738da68ac0bc8 100644
--- a/srcpkgs/python3-scipy/template
+++ b/srcpkgs/python3-scipy/template
@@ -3,9 +3,10 @@ pkgname=python3-scipy
 version=1.11.2
 revision=1
 build_style=python3-pep517
-build_helper="meson"
-make_build_args="--no-isolation --wheel
- $(vopt_if openblas "" "-Csetup-args=-Dblas=blas -Csetup-args=-Dlapack=lapack")"
+build_helper="meson numpy"
+make_build_args="
+ $(vopt_if openblas "" "-Csetup-args=-Dblas=blas -Csetup-args=-Dlapack=lapack")
+"
 hostmakedepends="python3-build python3-installer python3-meson-python
  python3-wheel python3-Cython python3-pybind11 pythran python3-numpy
  gcc-fortran pkg-config"
@@ -23,11 +24,6 @@ make_check="no" # Tests need an installed copy to run and meson makes this tough
 build_options="openblas"
 
 if [ "$CROSS_BUILD" ]; then
-	make_build_args+="
-	 -Csetup-args=--cross-file=${XBPS_WRAPPERDIR}/meson/xbps_meson.cross
-	 -Csetup-args=--cross-file=${XBPS_WRAPPERDIR}/meson/python.cross
-	"
-
 	_pybind11_dir="${py3_sitelib}/pybind11"
 	export PKG_CONFIG_PATH="${XBPS_CROSS_BASE}/${_pybind11_dir}/share/pkgconfig"
 	# pybind11 uses a path relative to the pkgconfig file to set $prefix,
@@ -51,17 +47,6 @@ if [ "$build_option_openblas" ]; then
 	esac
 fi
 
-post_patch() {
-	if [ "$CROSS_BUILD" ]; then
-		local _xpy="${XBPS_CROSS_BASE}/${py3_sitelib}"
-		cat > "${XBPS_WRAPPERDIR}/meson/python.cross" <<-EOF
-			[properties]
-			numpy-include-dir = '${_xpy}/numpy/core/include'
-			pythran-include-dir = '${_xpy}/pythran'
-			EOF
-	fi
-}
-
 pre_build() {
 	if [ "$CROSS_BUILD" ]; then
 		# Meson can't tolerate $CC with arguments as set by the build helper

From 9bfb2b4f8defbba0d3266cba0fb782d124c1d0c4 Mon Sep 17 00:00:00 2001
From: "Andrew J. Hesford" <ajh@sideband.org>
Date: Thu, 21 Sep 2023 11:29:51 -0400
Subject: [PATCH 4/4] python3-scikit-image: use meson-aware numpy helper and
 pep517 style

---
 srcpkgs/python3-scikit-image/template | 19 +------------------
 1 file changed, 1 insertion(+), 18 deletions(-)

diff --git a/srcpkgs/python3-scikit-image/template b/srcpkgs/python3-scikit-image/template
index b19743e50f37d..3fa9e9231e6cc 100644
--- a/srcpkgs/python3-scikit-image/template
+++ b/srcpkgs/python3-scikit-image/template
@@ -3,7 +3,7 @@ pkgname=python3-scikit-image
 version=0.21.0
 revision=1
 build_style=python3-pep517
-build_helper="meson"
+build_helper="meson numpy"
 hostmakedepends="python3-build python3-installer python3-meson-python
  python3-wheel python3-setuptools python3-packaging python3-Cython pythran
  python3-lazy_loader python3-numpy pkg-config"
@@ -20,23 +20,6 @@ checksum=53a82a9dbd3ed608d2ad3876269a271a7e922b12e228388eac996b508aadd652
 # Tests require data files and unpackaged dependencies
 make_check=no
 
-if [ "${CROSS_BUILD}" ]; then
-	make_build_args+=" --no-isolation --wheel
-	 -Csetup-args=--cross-file=${XBPS_WRAPPERDIR}/meson/xbps_meson.cross
-	 -Csetup-args=--cross-file=${XBPS_WRAPPERDIR}/meson/python.cross"
-fi
-
-post_patch() {
-	if [ "${CROSS_BUILD}" ]; then
-		local _xpy="${XBPS_CROSS_BASE}/${py3_sitelib}"
-		cat > "${XBPS_WRAPPERDIR}/meson/python.cross" <<-EOF
-		[properties]
-		numpy-include-dir = '${_xpy}/numpy/core/include'
-		pythran-include-dir = '${_xpy}/pythran'
-		EOF
-	fi
-}
-
 pre_build() {
 	if [ "${CROSS_BUILD}" ]; then
 		# Meson can't tolerate $CC with arguments as set by build helper

Re: [PR PATCH] [Updated] Add meson awareness to `numpy` helper and `python3-pep517` style

[ahesford]

[-- Attachment #1: Type: text/plain, Size: 758 bytes --]

There is an updated pull request by ahesford against master on the void-packages repository

https://github.com/ahesford/void-packages meson-numpy
https://github.com/void-linux/void-packages/pull/46186

Add meson awareness to `numpy` helper and `python3-pep517` style
Now that a `meson` build helper exists, we can further remove redundancy in some numeric python packages. In addition to the two cleaned up here, I believe we have some pending PRs that would benefit from these changes.

While I'm touching the `numpy` helper, I might as well fix its missing documentation.

cc: @tornaria 

#### Testing the changes
- I tested the changes in this PR: **YES**

A patch file from https://github.com/void-linux/void-packages/pull/46186.patch is attached

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: github-pr-meson-numpy-46186.patch --]
[-- Type: text/x-diff, Size: 6973 bytes --]

From 58658976f9e08a8f6e45dd2fa24c5efadc2abc4b Mon Sep 17 00:00:00 2001
From: "Andrew J. Hesford" <ajh@sideband.org>
Date: Thu, 21 Sep 2023 11:00:56 -0400
Subject: [PATCH 1/4] common/build-helper/numpy.sh: write meson cross file when
 needed

---
 Manual.md                    |  5 +++++
 common/build-helper/numpy.sh | 10 ++++++++++
 2 files changed, 15 insertions(+)

diff --git a/Manual.md b/Manual.md
index b6a4ffa0a71ce..21162641e4708 100644
--- a/Manual.md
+++ b/Manual.md
@@ -1083,6 +1083,11 @@ meson for cross builds. This is particularly useful for building packages that w
 invocations (e.g., `python3-pep517` packages that use a meson backend) and is added by default
 for packages that use the `meson` build style.
 
+- `numpy` configures the environment for cross-compilation of python packages that provide
+compiled extensions linking to NumPy C libraries. If the `meson` build helper is also
+configured, a secondary cross file, `${XBPS_WRAPPERDIR}/meson/xbps_numpy.cross`, will be
+written to inform meson where common NumPy components may be found.
+
 - `python3` configures the cross-build environment to use Python libraries, header files, and
 interpreter configurations in the target root. The `python3` helper is added by default for
 packages that use the `python3-module` or `python3-pep517` build styles.
diff --git a/common/build-helper/numpy.sh b/common/build-helper/numpy.sh
index 74c6421165293..bf40390ad1cae 100644
--- a/common/build-helper/numpy.sh
+++ b/common/build-helper/numpy.sh
@@ -34,4 +34,14 @@ if [ "$CROSS_BUILD" ]; then
 		ln -sf "${_gfortran}" "${XBPS_WRAPPERDIR}/gfortran"
 	fi
 	unset _gfortran
+
+	# Write a secondary meson cross file for numpy configuration
+	if [[ "${build_helper}" = *meson* ]]; then
+		mkdir -p "${XBPS_WRAPPERDIR}/meson"
+		cat > "${XBPS_WRAPPERDIR}/meson/xbps_numpy.cross" <<-EOF
+			[properties]
+			numpy-include-dir = '${XBPS_CROSS_BASE}/${py3_sitelib}/numpy/core/include'
+			pythran-include-dir = '${XBPS_CROSS_BASE}/${py3_sitelib}/pythran'
+			EOF
+	fi
 fi

From 6806c2de034a5c39c6a6e69405820f049546b320 Mon Sep 17 00:00:00 2001
From: "Andrew J. Hesford" <ajh@sideband.org>
Date: Thu, 21 Sep 2023 11:01:24 -0400
Subject: [PATCH 2/4] common/build-style/python3-pep517.sh: use meson cross
 files by default

---
 common/build-style/python3-pep517.sh | 14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)

diff --git a/common/build-style/python3-pep517.sh b/common/build-style/python3-pep517.sh
index 2d5ff7caf34eb..5ec20dd117f8a 100644
--- a/common/build-style/python3-pep517.sh
+++ b/common/build-style/python3-pep517.sh
@@ -4,8 +4,18 @@
 
 do_build() {
 	: ${make_build_target:=.}
-	: ${make_build_args:=--no-isolation  --wheel}
-	python3 -m build ${make_build_args} ${make_build_target}
+
+	if [ "${CROSS_BUILD}" ] && [[ "${build_helper}" = *meson* ]]; then
+		local mcross="-Csetup-args=--cross-file=${XBPS_WRAPPERDIR}/meson"
+		make_build_args+=" ${mcross}/xbps_meson.cross"
+
+		if [[ "${build_helper}" = *numpy* ]]; then
+			make_build_args+=" ${mcross}/xbps_numpy.cross"
+		fi
+	fi
+
+	python3 -m build --no-isolation --wheel \
+		${make_build_args} ${make_build_target}
 }
 
 do_check() {

From 074abe24ade0bc2e4e9920ac5a0b5860b39eaa3a Mon Sep 17 00:00:00 2001
From: "Andrew J. Hesford" <ajh@sideband.org>
Date: Thu, 21 Sep 2023 11:19:38 -0400
Subject: [PATCH 3/4] python3-scipy: use meson-aware numpy helper and pep517
 style

---
 srcpkgs/python3-scipy/template | 23 ++++-------------------
 1 file changed, 4 insertions(+), 19 deletions(-)

diff --git a/srcpkgs/python3-scipy/template b/srcpkgs/python3-scipy/template
index e99919babf3bf..738da68ac0bc8 100644
--- a/srcpkgs/python3-scipy/template
+++ b/srcpkgs/python3-scipy/template
@@ -3,9 +3,10 @@ pkgname=python3-scipy
 version=1.11.2
 revision=1
 build_style=python3-pep517
-build_helper="meson"
-make_build_args="--no-isolation --wheel
- $(vopt_if openblas "" "-Csetup-args=-Dblas=blas -Csetup-args=-Dlapack=lapack")"
+build_helper="meson numpy"
+make_build_args="
+ $(vopt_if openblas "" "-Csetup-args=-Dblas=blas -Csetup-args=-Dlapack=lapack")
+"
 hostmakedepends="python3-build python3-installer python3-meson-python
  python3-wheel python3-Cython python3-pybind11 pythran python3-numpy
  gcc-fortran pkg-config"
@@ -23,11 +24,6 @@ make_check="no" # Tests need an installed copy to run and meson makes this tough
 build_options="openblas"
 
 if [ "$CROSS_BUILD" ]; then
-	make_build_args+="
-	 -Csetup-args=--cross-file=${XBPS_WRAPPERDIR}/meson/xbps_meson.cross
-	 -Csetup-args=--cross-file=${XBPS_WRAPPERDIR}/meson/python.cross
-	"
-
 	_pybind11_dir="${py3_sitelib}/pybind11"
 	export PKG_CONFIG_PATH="${XBPS_CROSS_BASE}/${_pybind11_dir}/share/pkgconfig"
 	# pybind11 uses a path relative to the pkgconfig file to set $prefix,
@@ -51,17 +47,6 @@ if [ "$build_option_openblas" ]; then
 	esac
 fi
 
-post_patch() {
-	if [ "$CROSS_BUILD" ]; then
-		local _xpy="${XBPS_CROSS_BASE}/${py3_sitelib}"
-		cat > "${XBPS_WRAPPERDIR}/meson/python.cross" <<-EOF
-			[properties]
-			numpy-include-dir = '${_xpy}/numpy/core/include'
-			pythran-include-dir = '${_xpy}/pythran'
-			EOF
-	fi
-}
-
 pre_build() {
 	if [ "$CROSS_BUILD" ]; then
 		# Meson can't tolerate $CC with arguments as set by the build helper

From 865bce1fca92c6c4282aaf768411097dd9e30d5f Mon Sep 17 00:00:00 2001
From: "Andrew J. Hesford" <ajh@sideband.org>
Date: Thu, 21 Sep 2023 11:29:51 -0400
Subject: [PATCH 4/4] python3-scikit-image: use meson-aware numpy helper and
 pep517 style

---
 srcpkgs/python3-scikit-image/template | 19 +------------------
 1 file changed, 1 insertion(+), 18 deletions(-)

diff --git a/srcpkgs/python3-scikit-image/template b/srcpkgs/python3-scikit-image/template
index b19743e50f37d..3fa9e9231e6cc 100644
--- a/srcpkgs/python3-scikit-image/template
+++ b/srcpkgs/python3-scikit-image/template
@@ -3,7 +3,7 @@ pkgname=python3-scikit-image
 version=0.21.0
 revision=1
 build_style=python3-pep517
-build_helper="meson"
+build_helper="meson numpy"
 hostmakedepends="python3-build python3-installer python3-meson-python
  python3-wheel python3-setuptools python3-packaging python3-Cython pythran
  python3-lazy_loader python3-numpy pkg-config"
@@ -20,23 +20,6 @@ checksum=53a82a9dbd3ed608d2ad3876269a271a7e922b12e228388eac996b508aadd652
 # Tests require data files and unpackaged dependencies
 make_check=no
 
-if [ "${CROSS_BUILD}" ]; then
-	make_build_args+=" --no-isolation --wheel
-	 -Csetup-args=--cross-file=${XBPS_WRAPPERDIR}/meson/xbps_meson.cross
-	 -Csetup-args=--cross-file=${XBPS_WRAPPERDIR}/meson/python.cross"
-fi
-
-post_patch() {
-	if [ "${CROSS_BUILD}" ]; then
-		local _xpy="${XBPS_CROSS_BASE}/${py3_sitelib}"
-		cat > "${XBPS_WRAPPERDIR}/meson/python.cross" <<-EOF
-		[properties]
-		numpy-include-dir = '${_xpy}/numpy/core/include'
-		pythran-include-dir = '${_xpy}/pythran'
-		EOF
-	fi
-}
-
 pre_build() {
 	if [ "${CROSS_BUILD}" ]; then
 		# Meson can't tolerate $CC with arguments as set by build helper

Re: Add meson awareness to `numpy` helper and `python3-pep517` style

[tornaria]

[-- Attachment #1: Type: text/plain, Size: 420 bytes --]

New comment by tornaria on void-packages repository

https://github.com/void-linux/void-packages/pull/46186#issuecomment-1732291617

Comment:
LGTM, do you want to cleanup `make_build_args` in pylint?

A (very) minor nitpick: maybe `[[ "${build_helper}" = *meson* ]]` should be `[[ " ${build_helper} " = *" meson "* ]]`, etc.

But the idiom is used all over the place and currently there is no conflict possible.

Re: Add meson awareness to `numpy` helper and `python3-pep517` style

[ahesford]

[-- Attachment #1: Type: text/plain, Size: 691 bytes --]

New comment by ahesford on void-packages repository

https://github.com/void-linux/void-packages/pull/46186#issuecomment-1732294929

Comment:
> A (very) minor nitpick: maybe `[[ "${build_helper}" = *meson* ]]` should be `[[ " ${build_helper} " = *" meson "* ]]`, etc.

Adding spaces might thwart a match when the variable embeds a newline, like

    build_helper="
     meson
     numpy
    "

which probably never ought to happen, but could cause hard-to-discover issues when it does. What we *really* need is to turn these space-separated variables into proper arrays so that we can test for membership the right way.

Thanks for reminding me about pylint; I'll add a fix here.

Re: Add meson awareness to `numpy` helper and `python3-pep517` style

[tornaria]

[-- Attachment #1: Type: text/plain, Size: 681 bytes --]

New comment by tornaria on void-packages repository

https://github.com/void-linux/void-packages/pull/46186#issuecomment-1732297940

Comment:
You may want to cherry-pick 0a2141b19ad from #46109, to further cleanup the scipy template (no need for the pybind11 thing).

I wonder about this:
```
	if [ "$CROSS_BUILD" ]; then
		# Meson can't tolerate $CC with arguments as set by the build helper
		CC="${XBPS_CROSS_TRIPLET}-gcc"
		# CXX needs to know where to find Python headers
		CXXFLAGS+=" -I${XBPS_CROSS_BASE}/${py3_inc}"
	fi
```

Is this necessary? I didn't add this to `python3-contourpy` and I had no trouble... This block is there both in scipy and scikit-image.

Re: Add meson awareness to `numpy` helper and `python3-pep517` style

[tornaria]

[-- Attachment #1: Type: text/plain, Size: 224 bytes --]

New comment by tornaria on void-packages repository

https://github.com/void-linux/void-packages/pull/46186#issuecomment-1732299153

Comment:
Aha, it probably refers to the `python3` build helper, which is no longer used...

Re: Add meson awareness to `numpy` helper and `python3-pep517` style

[ahesford]

[-- Attachment #1: Type: text/plain, Size: 319 bytes --]

New comment by ahesford on void-packages repository

https://github.com/void-linux/void-packages/pull/46186#issuecomment-1732327681

Comment:
The python3 build helper is automatically pulled in by the python3 build styles. I haven't tested whether it can be removed now, but it was definitely needed when it was added.

Re: Add meson awareness to `numpy` helper and `python3-pep517` style

[ahesford]

[-- Attachment #1: Type: text/plain, Size: 447 bytes --]

New comment by ahesford on void-packages repository

https://github.com/void-linux/void-packages/pull/46186#issuecomment-1732363348

Comment:
Closed in 0fcff106da5aa308221d38ec55b9e1d3774aa5fc with some additional changes:
- `pylint` cleanup
- Adopt `python3-pybind11` "system" installation, slightly differently than in #46109
- Massive cleanup of `python3-{scipy,scikit-image}`, including removal of obsolete `$CC` and `$CXXFLAGS` overrides.

Re: [PR PATCH] [Closed]: Add meson awareness to `numpy` helper and `python3-pep517` style

[ahesford]

[-- Attachment #1: Type: text/plain, Size: 596 bytes --]

There's a closed pull request on the void-packages repository

Add meson awareness to `numpy` helper and `python3-pep517` style
https://github.com/void-linux/void-packages/pull/46186

Description:
Now that a `meson` build helper exists, we can further remove redundancy in some numeric python packages. In addition to the two cleaned up here, I believe we have some pending PRs that would benefit from these changes.

While I'm touching the `numpy` helper, I might as well fix its missing documentation.

cc: @tornaria 

#### Testing the changes
- I tested the changes in this PR: **YES**