[PR PATCH] Python3 linkage

[PR PATCH] Python3 linkage

[sgn]

[-- Attachment #1: Type: text/plain, Size: 1246 bytes --]

There is a new pull request by sgn against master on the void-packages repository

https://github.com/sgn/void-packages python3-linkage
https://github.com/void-linux/void-packages/pull/40665

Python3 linkage
<!-- Uncomment relevant sections and delete options which are not applicable -->

#### Testing the changes
- I tested the changes in this PR: **YES**|**briefly**|**NO**

<!--
#### New package
- This new package conforms to the [package requirements](https://github.com/void-linux/void-packages/blob/master/CONTRIBUTING.md#package-requirements): **YES**|**NO**
-->

<!-- Note: If the build is likely to take more than 2 hours, please add ci skip tag as described in
https://github.com/void-linux/void-packages/blob/master/CONTRIBUTING.md#continuous-integration
and test at least one native build and, if supported, at least one cross build.
Ignore this section if this PR is not skipping CI.
-->
<!--
#### Local build testing
- I built this PR locally for my native architecture, (ARCH-LIBC)
- I built this PR locally for these architectures (if supported. mark crossbuilds):
  - aarch64-musl
  - armv7l
  - armv6l-musl
-->


A patch file from https://github.com/void-linux/void-packages/pull/40665.patch is attached

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: github-pr-python3-linkage-40665.patch --]
[-- Type: text/x-diff, Size: 7518 bytes --]

From 147d7da5d5261d09b61a31928ac1201a1bb7a200 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?=C4=90o=C3=A0n=20Tr=E1=BA=A7n=20C=C3=B4ng=20Danh?=
 <congdanhqx@gmail.com>
Date: Fri, 18 Nov 2022 15:10:18 +0700
Subject: [PATCH 1/4] mod_wsgi: link with python-3.11

---
 srcpkgs/mod_wsgi/patches/cross-python.patch | 2 +-
 srcpkgs/mod_wsgi/template                   | 3 +--
 2 files changed, 2 insertions(+), 3 deletions(-)

diff --git a/srcpkgs/mod_wsgi/patches/cross-python.patch b/srcpkgs/mod_wsgi/patches/cross-python.patch
index 6882cf9b135a..8157e1046de3 100644
--- a/srcpkgs/mod_wsgi/patches/cross-python.patch
+++ b/srcpkgs/mod_wsgi/patches/cross-python.patch
@@ -108,7 +108,7 @@
 -        import sysconfig; \
 -        stdout.write(sysconfig.get_config_var("LIBS"))'`
 -fi
-+PYTHONLIBDIR=$($PYTHON-config --ldflags)
++LDLIBS1=$($PYTHON-config --ldflags --embed)
  
  CFLAGS1=""
  for arg in ${CFLAGS}
diff --git a/srcpkgs/mod_wsgi/template b/srcpkgs/mod_wsgi/template
index 58fdf90ea18a..0e8092212941 100644
--- a/srcpkgs/mod_wsgi/template
+++ b/srcpkgs/mod_wsgi/template
@@ -1,7 +1,7 @@
 # Template file for 'mod_wsgi'
 pkgname=mod_wsgi
 version=4.9.4
-revision=1
+revision=2
 build_style=gnu-configure
 hostmakedepends="python3 perl automake libtool apache-devel"
 makedepends="apache-devel python3-devel apr-devel"
@@ -13,7 +13,6 @@ changelog="https://modwsgi.readthedocs.io/en/latest/release-notes/version-${vers
 distfiles="https://github.com/GrahamDumpleton/mod_wsgi/archive/${version}.tar.gz"
 checksum=ee926a3fd5675890b908ebc23db1f8f7f03dc3459241abdcf35d46c68e1be29b
 lib32disabled=yes
-LDFLAGS=-lpython3
 
 pre_configure() {
 	if [ "$CROSS_BUILD" ]; then

From 74e1ac1e7196d50809e0746a36b6f77d19263d11 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?=C4=90o=C3=A0n=20Tr=E1=BA=A7n=20C=C3=B4ng=20Danh?=
 <congdanhqx@gmail.com>
Date: Mon, 21 Nov 2022 10:14:41 +0700
Subject: [PATCH 2/4] boost-python: link with python

---
 srcpkgs/boost/template | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/srcpkgs/boost/template b/srcpkgs/boost/template
index fb01417cea1d..323a7d7d4c87 100644
--- a/srcpkgs/boost/template
+++ b/srcpkgs/boost/template
@@ -67,6 +67,8 @@ case "$XBPS_TARGET_MACHINE" in
 	ppc*)     _arch=power;    _abi=sysv  ;;
 esac
 
+LDFLAGS="-lpython3 -lpython${py3_ver}"
+
 do_build() {
 	# bootstrap.sh will reset CXX and CXXFLAGS (L229)
 	./bootstrap.sh --with-toolset=gcc \

From d258d2b41370e5881205f0169cd59b06b19e0946 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?=C4=90o=C3=A0n=20Tr=E1=BA=A7n=20C=C3=B4ng=20Danh?=
 <congdanhqx@gmail.com>
Date: Mon, 21 Nov 2022 11:10:51 +0700
Subject: [PATCH 3/4] hooks: introduce a link for Python linkage

There're a lot of packages that requires linkage to libpython3 to
be functional but they're under linkage. Thus, they become broken during
Python minor version update.

Since there're a lot of them, add linkage for them to check.
Disable the check with `noverifypython`.
---
 Manual.md                                     |  4 ++
 common/hooks/post-install-sourcepkg/.empty    |  0
 .../10-pkglint-python-linkage.sh              | 37 +++++++++++++++++++
 common/xbps-src/libexec/xbps-src-doinstall.sh |  1 +
 4 files changed, 42 insertions(+)
 create mode 100644 common/hooks/post-install-sourcepkg/.empty
 create mode 100755 common/hooks/post-install-sourcepkg/10-pkglint-python-linkage.sh

diff --git a/Manual.md b/Manual.md
index fa552f65ba5d..715558e8773c 100644
--- a/Manual.md
+++ b/Manual.md
@@ -634,6 +634,10 @@ sonames in shared libraries.
 their reverse dependencies. You need to specify all dependencies in the `depends` when you
 need to set this.
 
+- `noverifypython` Normally, `xbps-src` will verify linkage to Python3 if
+  `python3-devel` is pulled for build dependencies. If `noverifypython` is
+  set, `xbps-src` will skip the check.
+
 - `skiprdeps` White space separated list of filenames specified by their absolute path in
 the `$DESTDIR` which will not be scanned for runtime dependencies. This may be useful to
 skip files which are not meant to be run or loaded on the host but are to be sent to some
diff --git a/common/hooks/post-install-sourcepkg/.empty b/common/hooks/post-install-sourcepkg/.empty
new file mode 100644
index 000000000000..e69de29bb2d1
diff --git a/common/hooks/post-install-sourcepkg/10-pkglint-python-linkage.sh b/common/hooks/post-install-sourcepkg/10-pkglint-python-linkage.sh
new file mode 100755
index 000000000000..468c816f213b
--- /dev/null
+++ b/common/hooks/post-install-sourcepkg/10-pkglint-python-linkage.sh
@@ -0,0 +1,37 @@
+# vim: ft=bash
+# This hooks will warn if nothing linked with Python
+# but python3-devel has been installed
+
+hook() {
+	local _file _elf _rx
+	local _have_elf=
+	# Skip during bootstrapping
+	if [ ! "$CHROOT_READY" ]; then return 0; fi
+	if [ "$noverifyrdeps" ]; then return 0; fi
+	if [ "$noverifypython" ]; then return 0; fi
+	# We may want to $XBPS_QUERY_XCMD -p state python3-devel
+	# But it render many false positive
+	case "${makedepends}" in
+		*python3-devel*) ;;
+		*) return 0 ;;
+	esac
+	_rx="[[:space:]]*NEEDED[[:space:]]*libpython${py3_ver//./[.]}"
+	while read -r _file; do
+		if ! read -r -n4 _elf <"$_file"; then
+			:
+		elif [ "$_elf" = $'\177ELF' ]; then
+			_have_elf=yes
+			if $OBJDUMP -p "$_file" 2>/dev/null | grep -q "$_rx"
+			then
+				return 0
+			fi
+		fi
+	done < <(find "${DESTDIR}"/usr/bin \
+		"${DESTDIR}"/usr/lib \
+		"${DESTDIR}"/usr/libexec \
+		-type f 2>/dev/null || true)
+
+	if [ "$_have_elf" ]; then
+		msg_error "$pkgname: not linked to libpython${py3_ver} but python3-devel in \$makedepends\n"
+	fi
+}
diff --git a/common/xbps-src/libexec/xbps-src-doinstall.sh b/common/xbps-src/libexec/xbps-src-doinstall.sh
index fb5d35d1a59a..c962b4de4327 100755
--- a/common/xbps-src/libexec/xbps-src-doinstall.sh
+++ b/common/xbps-src/libexec/xbps-src-doinstall.sh
@@ -38,6 +38,7 @@ if [ "$SUBPKG_MODE"  = "no" ]; then
 
         touch -f $XBPS_INSTALL_DONE
     fi
+    run_pkg_hooks post-install-sourcepkg
     exit 0
 fi
 

From 8b35cc656b02a29f0d9e023aaaf363c6856a0c54 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?=C4=90o=C3=A0n=20Tr=E1=BA=A7n=20C=C3=B4ng=20Danh?=
 <congdanhqx@gmail.com>
Date: Mon, 21 Nov 2022 18:48:20 +0700
Subject: [PATCH 4/4] wrappers: wrap --ldflags for python3

---
 common/hooks/pre-configure/02-script-wrapper.sh | 16 +++++++++++++---
 1 file changed, 13 insertions(+), 3 deletions(-)

diff --git a/common/hooks/pre-configure/02-script-wrapper.sh b/common/hooks/pre-configure/02-script-wrapper.sh
index 641d7ab762c0..d225927108b6 100644
--- a/common/hooks/pre-configure/02-script-wrapper.sh
+++ b/common/hooks/pre-configure/02-script-wrapper.sh
@@ -65,17 +65,27 @@ _EOF
 }
 
 python_wrapper() {
-	local wrapper="$1" version="$2"
+	local wrapper="$1" version="$2" shlib="${3:-}"
 
 	[ -x ${XBPS_WRAPPERDIR}/${wrapper} ] && return 0
-	cat >>${XBPS_WRAPPERDIR}/${wrapper}<<_EOF
+	{
+		cat <<_EOF
 #!/bin/sh
 case "\$1" in
 --includes|--cflags)
 	echo "-I${XBPS_CROSS_BASE}/usr/include/python${version}" ;;
+_EOF
+	if [ "$shlib" ]; then
+		cat <<_EOF
+--ldflags)
+	echo "-lpython${shlib}" ;;
+_EOF
+	fi
+	cat <<_EOF
 esac
 exit 0
 _EOF
+	}>${XBPS_WRAPPERDIR}/${wrapper}
 	chmod 755 ${XBPS_WRAPPERDIR}/${wrapper}
 }
 
@@ -236,7 +246,7 @@ hook() {
 	generic_wrapper3 libetpan-config
 	generic_wrapper3 giblib-config
 	python_wrapper python-config 2.7
-	python_wrapper python3-config 3.11
+	python_wrapper python3-config ${py3_ver} "${py3_ver}${py3_abiver}"
 	apr_apu_wrapper apr-1-config
 	apr_apu_wrapper apu-1-config
 }

Re: [PR REVIEW] Python3 linkage

[ahesford]

[-- Attachment #1: Type: text/plain, Size: 221 bytes --]

New review comment by ahesford on void-packages repository

https://github.com/void-linux/void-packages/pull/40665#discussion_r1028181743

Comment:
I'm surprised we haven't needed to wrap more of `python-config` already!

Re: Python3 linkage

[sgn]

[-- Attachment #1: Type: text/plain, Size: 188 bytes --]

New comment by sgn on void-packages repository

https://github.com/void-linux/void-packages/pull/40665#issuecomment-1326602047

Comment:
Abandon this idea, too many things to think about.

Re: [PR PATCH] [Closed]: Python3 linkage

[sgn]

[-- Attachment #1: Type: text/plain, Size: 1095 bytes --]

There's a closed pull request on the void-packages repository

Python3 linkage
https://github.com/void-linux/void-packages/pull/40665

Description:
<!-- Uncomment relevant sections and delete options which are not applicable -->

#### Testing the changes
- I tested the changes in this PR: **YES**|**briefly**|**NO**

<!--
#### New package
- This new package conforms to the [package requirements](https://github.com/void-linux/void-packages/blob/master/CONTRIBUTING.md#package-requirements): **YES**|**NO**
-->

<!-- Note: If the build is likely to take more than 2 hours, please add ci skip tag as described in
https://github.com/void-linux/void-packages/blob/master/CONTRIBUTING.md#continuous-integration
and test at least one native build and, if supported, at least one cross build.
Ignore this section if this PR is not skipping CI.
-->
<!--
#### Local build testing
- I built this PR locally for my native architecture, (ARCH-LIBC)
- I built this PR locally for these architectures (if supported. mark crossbuilds):
  - aarch64-musl
  - armv7l
  - armv6l-musl
-->

Re: Python3 linkage

[eli-schwartz]

[-- Attachment #1: Type: text/plain, Size: 586 bytes --]

New comment by eli-schwartz on void-packages repository

https://github.com/void-linux/void-packages/pull/40665#issuecomment-1326611779

Comment:
Fixing mod_wsgi to link to libpython is correct, because upstream's configure script does so already. It was just an oversight that had it use python-config without `--embed`.

Generally linking to libpython for extension modules is probably wrong -- upstream Python specifically moved to requiring --embed for this, because anything that runs inside the python interpreter is expected to load whichever library the interpreter provides.