Re: [PR PATCH] [Updated] verify dependencies between subpkgs

[sgn <sgn@users.noreply.github.com>]

[-- Attachment #1: Type: text/plain, Size: 1727 bytes --]

There is an updated pull request by sgn against master on the void-packages repository

https://github.com/sgn/void-packages gen-rt-deps-subpkg
https://github.com/void-linux/void-packages/pull/45504

verify dependencies between subpkgs
<!-- Uncomment relevant sections and delete options which are not applicable -->

#### Testing the changes
- I tested the changes in this PR: **briefly**

With those threads on reddit about unable to use libreoffice to generate epub, then a note to revbump libreoffice when poppler is rebuilt.

This solution hopefully is not too stupid.

Expect problem to come:

- A package that would be build multiple time for multiple configuration. (Says `emacs` or `Cataclysm-DDA`), not sure how that would work. We may pick the first package that provides the shared library without SONAME. I don't know.

<!--
#### New package
- This new package conforms to the [package requirements](https://github.com/void-linux/void-packages/blob/master/CONTRIBUTING.md#package-requirements): **YES**|**NO**
-->

<!-- Note: If the build is likely to take more than 2 hours, please add ci skip tag as described in
https://github.com/void-linux/void-packages/blob/master/CONTRIBUTING.md#continuous-integration
and test at least one native build and, if supported, at least one cross build.
Ignore this section if this PR is not skipping CI.
-->
<!--
#### Local build testing
- I built this PR locally for my native architecture, (ARCH-LIBC)
- I built this PR locally for these architectures (if supported. mark crossbuilds):
  - aarch64-musl
  - armv7l
  - armv6l-musl
-->
[ci skip][skip ci]

A patch file from https://github.com/void-linux/void-packages/pull/45504.patch is attached

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: github-pr-gen-rt-deps-subpkg-45504.patch --]
[-- Type: text/x-diff, Size: 15937 bytes --]

From e303daf270c6335ff7a221526d59406a7ae210eb Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?=C4=90o=C3=A0n=20Tr=E1=BA=A7n=20C=C3=B4ng=20Danh?=
 <congdanhqx@gmail.com>
Date: Tue, 8 Aug 2023 13:44:43 +0700
Subject: [PATCH 1/6] hooks: move prepare-32bit and shlib-provides to
 post-install

In a later change, we will generate shlib-depends cross subpkgs during
pre-pkg stage.  Thus we need shlib-provides information of all subpkgs
ready before pre-pkg is run.

Those information can only be read in post-install stage at the
eariliest.

Let's move the shlib-provides to post-install.  This hook requires
prepare-32bit, so, let's move that hook, too.
---
 .../05-prepare-32bit.sh => post-install/80-prepare-32bit.sh}      | 0
 .../06-shlib-provides.sh => post-install/98-shlib-provides.sh}    | 0
 2 files changed, 0 insertions(+), 0 deletions(-)
 rename common/hooks/{pre-pkg/05-prepare-32bit.sh => post-install/80-prepare-32bit.sh} (100%)
 rename common/hooks/{pre-pkg/06-shlib-provides.sh => post-install/98-shlib-provides.sh} (100%)

diff --git a/common/hooks/pre-pkg/05-prepare-32bit.sh b/common/hooks/post-install/80-prepare-32bit.sh
similarity index 100%
rename from common/hooks/pre-pkg/05-prepare-32bit.sh
rename to common/hooks/post-install/80-prepare-32bit.sh
diff --git a/common/hooks/pre-pkg/06-shlib-provides.sh b/common/hooks/post-install/98-shlib-provides.sh
similarity index 100%
rename from common/hooks/pre-pkg/06-shlib-provides.sh
rename to common/hooks/post-install/98-shlib-provides.sh

From dfdd44bd7a9b862dcd50195308ca75f8a06c2a1c Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?=C4=90o=C3=A0n=20Tr=E1=BA=A7n=20C=C3=B4ng=20Danh?=
 <congdanhqx@gmail.com>
Date: Tue, 8 Aug 2023 22:08:27 +0700
Subject: [PATCH 2/6] hooks/shlib-provides: record shared libraries without
 SONAME

In a later change, we would like to generate runtime-deps between
sub-packages.

In order to do that, we can add everything into etc/shlibs or we can
look into other subpackages directly.  The former is cumbersome if such
package has lot of shared-objects.  The latter requires traversing and
checking a lot of files.  Furtunately, we can speed up the latter one by
storing all shared-objects' information in a centralised place.
---
 .../hooks/post-install/98-shlib-provides.sh   | 21 +++++++++++++++++--
 1 file changed, 19 insertions(+), 2 deletions(-)

diff --git a/common/hooks/post-install/98-shlib-provides.sh b/common/hooks/post-install/98-shlib-provides.sh
index a09eb6f5c3b33..e9d20e740ae9d 100644
--- a/common/hooks/post-install/98-shlib-provides.sh
+++ b/common/hooks/post-install/98-shlib-provides.sh
@@ -6,19 +6,25 @@ collect_sonames() {
 	local _pattern="^[[:alnum:]]+(.*)+\.so(\.[0-9]+)*$"
 	local _versioned_pattern="^[[:alnum:]]+(.*)+\.so(\.[0-9]+)+$"
 	local _tmpfile=$(mktemp) || exit 1
+	local _mainpkg="$2"
+	local _shlib_dir="${XBPS_STATEDIR}/shlib-provides"
+	local _no_soname=$(mktemp) || exit 1
 
+	mkdir -p "${_shlib_dir}" || exit 1
 	if [ ! -d ${_destdir} ]; then
 		rm -f ${_tmpfile}
+		rm -f ${_no_soname}
 		return 0
 	fi
 
+
 	# real pkg
 	find ${_destdir} -type f -name "*.so*" | while read f; do
 		_fname="${f##*/}"
 		case "$(file -bi "$f")" in
 		application/x-sharedlib*|application/x-pie-executable*)
 			# shared library
-			_soname=$(${OBJDUMP} -p "$f"|grep SONAME|awk '{print $2}')
+			_soname=$(${OBJDUMP} -p "$f"|awk '/SONAME/{print $2}')
 			# Register all versioned sonames, and
 			# unversioned sonames only when in libdir.
 			if [[ ${_soname} =~ ${_versioned_pattern} ]] ||
@@ -27,6 +33,9 @@ collect_sonames() {
 				  -e ${_destdir}/usr/lib32/${_fname} ) ]]; then
 				echo "${_soname}" >> ${_tmpfile}
 				echo "   SONAME ${_soname} from ${f##${_destdir}}"
+			else
+				# register all shared lib for rt-deps between sub-pkg
+				echo "${_fname}" >>${_no_soname}
 			fi
 			;;
 		esac
@@ -38,6 +47,14 @@ collect_sonames() {
 	if [ -s "${_tmpfile}" ]; then
 		tr '\n' ' ' < "${_tmpfile}" > ${_destdir}/shlib-provides
 		echo >> ${_destdir}/shlib-provides
+		if [ "$_mainpkg" ]; then
+			cp "${_tmpfile}" "${_shlib_dir}/${pkgname}.soname"
+		fi
+	fi
+	if [ "$_mainpkg" ] && [ -s "${_no_soname}" ]; then
+		mv "${_no_soname}" "${_shlib_dir}/${pkgname}.nosoname"
+	else
+		rm -f ${_no_soname}
 	fi
 	rm -f ${_tmpfile}
 }
@@ -50,7 +67,7 @@ hook() {
 	fi
 
 	# native pkg
-	collect_sonames ${PKGDESTDIR}
+	collect_sonames ${PKGDESTDIR} yes
 	# 32bit pkg
 	collect_sonames ${_destdir32}
 }

From 3c70f5840b8f02be656695a87345aec58455ddbe Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?=C4=90o=C3=A0n=20Tr=E1=BA=A7n=20C=C3=B4ng=20Danh?=
 <congdanhqx@gmail.com>
Date: Wed, 9 Aug 2023 13:19:01 +0700
Subject: [PATCH 3/6] hooks/gen-rt-deps: Look for rt-deps by subpkg first,
 common/shlibs later

---
 .../hooks/pre-pkg/04-generate-runtime-deps.sh | 80 +++++++------------
 1 file changed, 29 insertions(+), 51 deletions(-)

diff --git a/common/hooks/pre-pkg/04-generate-runtime-deps.sh b/common/hooks/pre-pkg/04-generate-runtime-deps.sh
index a291f2ccc8720..e2020ce05f79f 100644
--- a/common/hooks/pre-pkg/04-generate-runtime-deps.sh
+++ b/common/hooks/pre-pkg/04-generate-runtime-deps.sh
@@ -47,6 +47,7 @@ store_pkgdestdir_rundeps() {
 
 hook() {
     local depsftmp f lf j mapshlibs sorequires _curdep elfmagic broken_shlibs verify_deps
+    local _shlib_dir="${XBPS_STATEDIR}/shlib-provides"
 
     # Disable trap on ERR, xbps-uhelper cmd might return error... but not something
     # to be worried about because if there are broken shlibs this hook returns
@@ -95,65 +96,42 @@ hook() {
     # above, the mapping is done thru the common/shlibs file.
     #
     for f in ${verify_deps}; do
-        unset _f j rdep _rdep rdepcnt soname _pkgname _rdepver found
-        _f=$(echo "$f"|sed -E 's|\+|\\+|g')
-        rdep="$(grep -E "^${_f}[[:blank:]]+.*$" $mapshlibs|cut -d ' ' -f2)"
-        rdepcnt="$(grep -E "^${_f}[[:blank:]]+.*$" $mapshlibs|cut -d ' ' -f2|wc -l)"
-        if [ -z "$rdep" ]; then
+        unset _rdep _pkgname _rdepver
+
+        if [ "$(find ${PKGDESTDIR} -name "$f")" ]; then
             # Ignore libs by current pkg
-            soname=$(find ${PKGDESTDIR} -name "$f")
-            if [ -z "$soname" ]; then
+            echo "   SONAME: $f <-> $pkgname (ignored)"
+            continue
+        # If this library is provided by a subpkg of sourcepkg, use that subpkg
+        elif _pkgname="$(cd "$_shlib_dir" && grep -F -l -x "$f" *.soname 2>/dev/null)"; then
+            # If that library has SONAME, add it to shlibs-requires, too.
+            _pkgname=${_pkgname%.soname}
+            _sdep="${_pkgname}-${version}_${revision}"
+            sorequires+="${f} "
+        elif _pkgname="$(cd "$_shlib_dir" && grep -F -l -x "$f" *.nosoname 2>/dev/null)"; then
+            _pkgname=${_pkgname%.nosoname}
+            _sdep="${_pkgname}-${version}_${revision}"
+        else
+            _rdep="$(awk -v sl="$f" '$1 == sl { print $2; exit; }' "$mapshlibs")"
+
+            if [ -z "$_rdep" ]; then
                 msg_red_nochroot "   SONAME: $f <-> UNKNOWN PKG PLEASE FIX!\n"
                 broken_shlibs=1
-            else
-                echo "   SONAME: $f <-> $pkgname (ignored)"
-            fi
-            continue
-        elif [ "$rdepcnt" -gt 1 ]; then
-            unset j found
-            # Check if shlib is provided by multiple pkgs.
-            for j in ${rdep}; do
-                _pkgname=$($XBPS_UHELPER_CMD getpkgname "$j")
-                # if there's a SONAME matching pkgname, use it.
-                for x in ${pkgname} ${subpackages}; do
-                    [[ $_pkgname == $x ]] && found=1 && break
-                done
-                [[ $found ]] && _rdep=$j && break
-            done
-            if [ -z "${_rdep}" ]; then
-                # otherwise pick up the first one.
-                for j in ${rdep}; do
-                    [ -z "${_rdep}" ] && _rdep=$j
-                done
+                continue
             fi
-        else
-            _rdep=$rdep
-        fi
-        _pkgname=$($XBPS_UHELPER_CMD getpkgname "${_rdep}" 2>/dev/null)
-        _rdepver=$($XBPS_UHELPER_CMD getpkgversion "${_rdep}" 2>/dev/null)
-        if [ -z "${_pkgname}" -o -z "${_rdepver}" ]; then
-            msg_red_nochroot "   SONAME: $f <-> UNKNOWN PKG PLEASE FIX!\n"
-            broken_shlibs=1
-            continue
-        fi
-        # Check if pkg is a subpkg of sourcepkg; if true, ignore version
-        # in common/shlibs.
-        _sdep="${_pkgname}>=${_rdepver}"
-        for _subpkg in ${subpackages}; do
-            if [ "${_subpkg}" = "${_pkgname}" ]; then
-                _sdep="${_pkgname}-${version}_${revision}"
-                break
+            _pkgname=$($XBPS_UHELPER_CMD getpkgname "${_rdep}" 2>/dev/null)
+            _rdepver=$($XBPS_UHELPER_CMD getpkgversion "${_rdep}" 2>/dev/null)
+            if [ -z "${_pkgname}" -o -z "${_rdepver}" ]; then
+                msg_red_nochroot "   SONAME: $f <-> UNKNOWN PKG PLEASE FIX!\n"
+                broken_shlibs=1
+                continue
             fi
-        done
+            _sdep="${_pkgname}>=${_rdepver}"
 
-        if [ "${_pkgname}" != "${pkgname}" ]; then
-            echo "   SONAME: $f <-> ${_sdep}"
+            # By this point, SONAME can't be found in current pkg
             sorequires+="${f} "
-        else
-            # Ignore libs by current pkg
-            echo "   SONAME: $f <-> ${_rdep} (ignored)"
-            continue
         fi
+        echo "   SONAME: $f <-> ${_sdep}"
         add_rundep "${_sdep}"
     done
     #

From 37a1160ca407eb6616d77fc0397bb40058fa234b Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?=C4=90o=C3=A0n=20Tr=E1=BA=A7n=20C=C3=B4ng=20Danh?=
 <congdanhqx@gmail.com>
Date: Wed, 9 Aug 2023 19:50:06 +0700
Subject: [PATCH 4/6] libreoffice: verify dependencies

---
 srcpkgs/libreoffice-epub     |  1 -
 srcpkgs/libreoffice/template | 27 +++------------------------
 2 files changed, 3 insertions(+), 25 deletions(-)
 delete mode 120000 srcpkgs/libreoffice-epub

diff --git a/srcpkgs/libreoffice-epub b/srcpkgs/libreoffice-epub
deleted file mode 120000
index d5e3a56f35028..0000000000000
--- a/srcpkgs/libreoffice-epub
+++ /dev/null
@@ -1 +0,0 @@
-libreoffice
\ No newline at end of file
diff --git a/srcpkgs/libreoffice/template b/srcpkgs/libreoffice/template
index a5ade73cc1969..709ffe8d7161b 100644
--- a/srcpkgs/libreoffice/template
+++ b/srcpkgs/libreoffice/template
@@ -488,8 +488,9 @@ do_install() {
 
 libreoffice-common_package() {
 	short_desc+=" - Common files"
-	depends="hunspell hyphen mythes openldap libreoffice-i18n-en-US>=${version}_${revision}"
+	depends="libreoffice-i18n-en-US>=${version}_${revision}"
 	skiprdeps=/usr/lib/libreoffice/program/libofficebean.so
+	replaces="libreoffice-epub>=0"
 	pkg_install() {
 		_split common
 	}
@@ -498,7 +499,6 @@ libreoffice-common_package() {
 libreoffice-base_package() {
 	short_desc+=" - Database frontend"
 	depends="libreoffice-writer>=${version}_${revision}"
-	noverifyrdeps=yes
 	pkg_install() {
 		_split base
 	}
@@ -507,7 +507,6 @@ libreoffice-base_package() {
 libreoffice-calc_package() {
 	short_desc+=" - Spreadsheet"
 	depends="libreoffice-writer>=${version}_${revision}"
-	noverifyrdeps=yes
 	pkg_install() {
 		_split calc
 	}
@@ -516,7 +515,6 @@ libreoffice-calc_package() {
 libreoffice-draw_package() {
 	short_desc+=" - Drawing application"
 	depends="sane libreoffice-writer>=${version}_${revision}"
-	noverifyrdeps=yes
 	pkg_install() {
 		_split draw
 	}
@@ -556,8 +554,6 @@ libreoffice-fonts_package() {
 
 libreoffice-gnome_package() {
 	short_desc+=" - GNOME integration"
-	depends="libreoffice-common>=${version}_${revision}"
-	noverifyrdeps=yes
 	pkg_install() {
 		_split gnome
 	}
@@ -566,7 +562,6 @@ libreoffice-gnome_package() {
 libreoffice-impress_package() {
 	short_desc+=" - Presentation application"
 	depends="libreoffice-writer>=${version}_${revision}"
-	noverifyrdeps=yes
 	pkg_install() {
 		_split impress
 	}
@@ -574,8 +569,6 @@ libreoffice-impress_package() {
 
 libreoffice-kde_package() {
 	short_desc+=" - KDE integration"
-	depends="libreoffice-common>=${version}_${revision}"
-	noverifyrdeps=yes
 	pkg_install() {
 		cat > ${wrksrc}/file-lists/kde5_list.txt <<-EOF
 		%dir /usr/lib/libreoffice/program
@@ -593,8 +586,6 @@ libreoffice-kde_package() {
 
 libreoffice-qt6_package() {
 	short_desc+=" - Qt6 integration"
-	depends="libreoffice-common>=${version}_${revision}"
-	noverifyrdeps=yes
 	pkg_install() {
 		cat > ${wrksrc}/file-lists/qt6_list.txt <<-EOF
 		%dir /usr/lib/libreoffice/program
@@ -615,7 +606,6 @@ libreoffice-kit_package() {
 libreoffice-math_package() {
 	short_desc+=" - Equation editor"
 	depends="libreoffice-writer>=${version}_${revision}"
-	noverifyrdeps=yes
 	pkg_install() {
 		_split math
 	}
@@ -623,8 +613,7 @@ libreoffice-math_package() {
 
 libreoffice-postgresql_package() {
 	short_desc+=" - Connector for PostgreSQL"
-	depends="libreoffice-base>=${version}_${revision} libreoffice-common>=${version}_${revision}"
-	noverifyrdeps=yes
+	depends="libreoffice-base>=${version}_${revision}"
 	pkg_install() {
 		_split postgresql
 	}
@@ -632,25 +621,15 @@ libreoffice-postgresql_package() {
 
 libreoffice-writer_package() {
 	short_desc+=" - Word processor"
-	depends="libreoffice-common>=${version}_${revision}"
-	noverifyrdeps=yes
 	pkg_install() {
 		_split writer
 	}
 }
 
-libreoffice-epub_package() {
-	short_desc+=" - EPUB output"
-	build_style=meta
-	depends="libreoffice-common>=${version}_${revision} libepubgen libabw libe-book
-	 libetonyek libwps"
-}
-
 # Use a name which makes this catch-all subpackage the last one
 libreoffice-xtensions_package() {
 	short_desc+=" - Extensions"
 	depends="libreoffice-common>=${version}_${revision}"
-	noverifyrdeps=yes
 	pkg_install() {
 		# Remove empty files
 		find ${DESTDIR}/all -size 0 -delete

From 2177d82246ed84de3f3371b80537c8f47bd37b3d Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?=C4=90o=C3=A0n=20Tr=E1=BA=A7n=20C=C3=B4ng=20Danh?=
 <congdanhqx@gmail.com>
Date: Thu, 17 Aug 2023 07:05:45 +0700
Subject: [PATCH 5/6] CLion: verify runtime deps

---
 srcpkgs/CLion/template | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)

diff --git a/srcpkgs/CLion/template b/srcpkgs/CLion/template
index f302b7d6798e9..cc924d23fe49c 100644
--- a/srcpkgs/CLion/template
+++ b/srcpkgs/CLion/template
@@ -1,7 +1,7 @@
 # Template file for 'CLion'
 pkgname=CLion
 version=2021.3.4
-revision=1
+revision=2
 archs="x86_64 aarch64"
 depends="jetbrains-jdk-bin giflib libXtst"
 short_desc="Smart cross-platform IDE for C and C++"
@@ -13,8 +13,6 @@ checksum=f3b0b9e0dd0cd4aebef5d424e7a22868c732daad47d6c94f73630cef449ccf78
 repository=nonfree
 restricted=yes
 nopie=yes
-# JetBrains' tools are self-sufficient and while they include code that appears to be linked to libs from other packages, these libs are either included in the tool package, or the code works by looking for one of several supported libs.
-noverifyrdeps=yes
 python_version=3
 
 build_options="bundled_cmake bundled_gdb bundled_lldb"

From 8ab52f98e104c30ea95569921bef6057241efde6 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?=C4=90o=C3=A0n=20Tr=E1=BA=A7n=20C=C3=B4ng=20Danh?=
 <congdanhqx@gmail.com>
Date: Thu, 17 Aug 2023 07:07:40 +0700
Subject: [PATCH 6/6] PhpStorm: verify runtime-deps

---
 srcpkgs/PhpStorm/template | 2 --
 1 file changed, 2 deletions(-)

diff --git a/srcpkgs/PhpStorm/template b/srcpkgs/PhpStorm/template
index 306a6c77d0b36..cb84a08e13636 100644
--- a/srcpkgs/PhpStorm/template
+++ b/srcpkgs/PhpStorm/template
@@ -13,8 +13,6 @@ checksum=a23922f5f93bc8df1f2aabbb0f9969e27d28f706c09d18d66d4cc2d56c52ddc9
 repository=nonfree
 restricted=yes
 nopie=yes
-# JetBrains' tools are self-sufficient and while they include code that appears to be linked to libs from other packages, these libs are either included in the tool package, or the code works by looking for one of several supported libs.
-noverifyrdeps=yes
 python_version=3
 
 post_extract() {