[PR PATCH] zfs: divide under subpackages

[PR PATCH] zfs: divide under subpackages

[robertek]

[-- Attachment #1: Type: text/plain, Size: 2062 bytes --]

There is a new pull request by robertek against master on the void-packages repository

https://github.com/robertek/void-packages zfs
https://github.com/void-linux/void-packages/pull/42484

zfs: divide under subpackages
<!-- Uncomment relevant sections and delete options which are not applicable -->

The main intention of this PR is to separate the zfs tools from the kernel module. This change introduce several new subpackages for the `zfs` package. 
Namely:
-  zfs-tools
- zfs-dkms
- zfs-test
- zfs-zed

The original zfs package remain as a metapackage for the needed subpackages. This helps the transition (eg: for those having zfs installed, the system update will only introduce several new packages and will work as before). It is now possible to install purely the zfs-tools package if a non-distributed kernel is built with zfs.

I have outplaced the ztest to a new subpackage which is not as part of the zfs metapackage any more. Ztest is intended to be run only on testing systems.

I may provide additional information if needed. For example the contents of the packages, etc.

#### Testing the changes
- I tested the changes in this PR: **YES**
- Tested on native x86-64-glibc

<!--
#### New package
- This new package conforms to the [package requirements](https://github.com/void-linux/void-packages/blob/master/CONTRIBUTING.md#package-requirements): **YES**|**NO**
-->

<!-- Note: If the build is likely to take more than 2 hours, please add ci skip tag as described in
https://github.com/void-linux/void-packages/blob/master/CONTRIBUTING.md#continuous-integration
and test at least one native build and, if supported, at least one cross build.
Ignore this section if this PR is not skipping CI.
-->

#### Local build testing
- I built this PR locally for my native architecture, (x86-64-glibc)
- I built this PR locally for these architectures (if supported. mark crossbuilds):
  - aarch64 (cross)
  - x86-64-musl



A patch file from https://github.com/void-linux/void-packages/pull/42484.patch is attached

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: github-pr-zfs-42484.patch --]
[-- Type: text/x-diff, Size: 5656 bytes --]

From 0ba46483b4f4d89bb5d6e6ecded8b40a60a19056 Mon Sep 17 00:00:00 2001
From: Robert David <robert.david.public@gmail.com>
Date: Thu, 23 Feb 2023 19:35:42 +0100
Subject: [PATCH] zfs: divide under subpackages

---
 srcpkgs/zfs-dkms     |  1 +
 srcpkgs/zfs-test     |  1 +
 srcpkgs/zfs-tools    |  1 +
 srcpkgs/zfs-zed      |  1 +
 srcpkgs/zfs/template | 93 ++++++++++++++++++++++++++++++++++++--------
 5 files changed, 81 insertions(+), 16 deletions(-)
 create mode 120000 srcpkgs/zfs-dkms
 create mode 120000 srcpkgs/zfs-test
 create mode 120000 srcpkgs/zfs-tools
 create mode 120000 srcpkgs/zfs-zed

diff --git a/srcpkgs/zfs-dkms b/srcpkgs/zfs-dkms
new file mode 120000
index 000000000000..eb97035e1184
--- /dev/null
+++ b/srcpkgs/zfs-dkms
@@ -0,0 +1 @@
+zfs
\ No newline at end of file
diff --git a/srcpkgs/zfs-test b/srcpkgs/zfs-test
new file mode 120000
index 000000000000..eb97035e1184
--- /dev/null
+++ b/srcpkgs/zfs-test
@@ -0,0 +1 @@
+zfs
\ No newline at end of file
diff --git a/srcpkgs/zfs-tools b/srcpkgs/zfs-tools
new file mode 120000
index 000000000000..eb97035e1184
--- /dev/null
+++ b/srcpkgs/zfs-tools
@@ -0,0 +1 @@
+zfs
\ No newline at end of file
diff --git a/srcpkgs/zfs-zed b/srcpkgs/zfs-zed
new file mode 120000
index 000000000000..eb97035e1184
--- /dev/null
+++ b/srcpkgs/zfs-zed
@@ -0,0 +1 @@
+zfs
\ No newline at end of file
diff --git a/srcpkgs/zfs/template b/srcpkgs/zfs/template
index ae975caf9dd9..e1dda74134cb 100644
--- a/srcpkgs/zfs/template
+++ b/srcpkgs/zfs/template
@@ -1,27 +1,25 @@
 # Template file for 'zfs'
 pkgname=zfs
 version=2.1.9
-revision=1
+revision=2
 build_style=gnu-configure
 configure_args="--with-config=user --with-mounthelperdir=/usr/bin
  --with-udevdir=/usr/lib/udev --with-udevruledir=/usr/lib/udev/rules.d
  --with-dracutdir=/usr/lib/dracut --with-tirpc"
 hostmakedepends="dracut libtool pkg-config python3 automake"
-makedepends="pam-devel zlib-devel libuuid-devel libblkid-devel libtirpc-devel attr-devel"
-depends="dkms>=2.2.0.3_9 perl python3 xbps-triggers>=0.121_1
- libzfs>=${version}_${revision}"
-conf_files="/etc/zfs/zed.d/zed.rc"
-short_desc="Z File System -- userland, pyzfs, and kernel modules (using DKMS)"
+makedepends="pam-devel zlib-devel libuuid-devel libblkid-devel libtirpc-devel attr-devel eudev-libudev-devel"
+short_desc="Z File System --"
 maintainer="Toyam Cox <Vaelatern@voidlinux.org>"
 license="CDDL-1.0"
 homepage="https://openzfs.github.io/openzfs-docs/"
 distfiles="https://github.com/openzfs/zfs/releases/download/zfs-${version}/zfs-${version}.tar.gz"
 checksum=6b172cdf2eb54e17fcd68f900fab33c1430c5c59848fa46fab83614922fe50f6
-# dkms must be before initramfs-regenerate to build modules before images
-triggers="dkms initramfs-regenerate"
-dkms_modules="zfs ${version}"
 
-replaces="spl<=0.7.13_1" # Because SPL was merged into zfs in 0.8.0
+# the zfs-tools package needs to be done always the last!
+subpackages="libzfs zfs-devel zfs-dkms zfs-pam zfs-test zfs-zed zfs-tools"
+
+# SPL was merged into zfs in 0.8.0
+replaces="spl<=0.7.13_1"
 
 if [ "$XBPS_TARGET_NO_ATOMIC8" ]; then
 	makedepends+=" libatomic-devel"
@@ -37,8 +35,6 @@ post_install() {
 	vlicense LICENSE
 	vlicense NOTICE
 
-	vsv zed
-
 	make dist-gzip
 	vmkdir usr/src/
 	tar xf ${pkgname}-${version}.tar.gz -C ${DESTDIR}/usr/src/
@@ -61,16 +57,23 @@ post_install() {
 	rm -rf ${DESTDIR}/usr/share/zfs/zfs-tests
 }
 
+zfs_package() {
+	short_desc+=" metapackage"
+	build_style=meta
+	depends="zfs-tools>=${version}_${revision} zfs-zed>=${version}_${revision}
+	libzfs>=${version}_${revision} zfs-dkms>=${version}_${revision}"
+}
+
 libzfs_package() {
-	short_desc="Z File System - libraries"
+	short_desc+=" libraries"
 	pkg_install() {
 		vmove "usr/lib/*.so.*"
 	}
 }
 
 zfs-devel_package() {
+	short_desc+=" development files"
 	depends="libzfs>=${version}_${revision}"
-	short_desc="Z File System -- development files"
 	pkg_install() {
 		vmove usr/include
 		vmove "usr/lib/*.a"
@@ -79,14 +82,72 @@ zfs-devel_package() {
 	}
 }
 
+zfs-dkms_package() {
+	short_desc+=" kernel module (dkms)"
+	depends="dkms>=2.2.0.3_9 xbps-triggers>=0.121_1"
+
+	# dkms must be before initramfs-regenerate to build modules before images
+	triggers="dkms initramfs-regenerate"
+	dkms_modules="zfs ${version}"
+
+	pkg_install() {
+		vmove usr/src
+	}
+}
+
 zfs-pam_package() {
-	depends="${sourcepkg}>=${version}_${revision}"
-	short_desc="Z File System -- PAM integration"
+	short_desc+=" PAM integration"
+	depends="zfs-tools>=${version}_${revision}"
 	pkg_install() {
 		vmove usr/lib/security/pam_zfs_key.so
 	}
 }
 
+zfs-test_package() {
+	short_desc+=" tests"
+	depends="zfs-tools>=${version}_${revision}"
+	pkg_install() {
+		vmove usr/bin/raidz_test
+		vmove usr/bin/ztest
+		vmove usr/share/man/man1/raidz_test.1
+		vmove usr/share/man/man1/test-runner.1
+		vmove usr/share/man/man1/ztest.1
+		vmove usr/share/zfs/runfiles
+		vmove usr/share/zfs/test-runner
+		vmove usr/share/zfs/zfs-tests.sh
+		vmove usr/share/zfs/zimport.sh
+		vmove usr/share/zfs/zloop.sh
+	}
+}
+
+zfs-tools_package() {
+	short_desc+=" userland, pyzfs"
+	depends="libzfs>=${version}_${revision} python3"
+
+	pkg_install() {
+		vmove etc/zfs
+		vmove etc/sudoers.d
+		vmove usr/bin
+		vmove usr/lib
+		vmove usr/libexec
+		vmove usr/share
+	}
+}
+
+zfs-zed_package() {
+	short_desc+=" ZED"
+	depends="zfs-tools>=${version}_${revision}"
+	conf_files="/etc/zfs/zed.d/zed.rc"
+	pkg_install() {
+		vsv zed
+
+		vmove etc/zfs/zed.d
+		vmove usr/bin/zed
+		vmove usr/libexec/zfs/zed.d
+		vmove usr/share/man/man8/zed.8
+	}
+}
+
 # REMARKS:
 # Did not split out pyzfs, because it's distributed with zfs, and no reason to
 # send separately.

Re: [PR REVIEW] zfs: divide under subpackages

[ahesford]

[-- Attachment #1: Type: text/plain, Size: 281 bytes --]

New review comment by ahesford on void-packages repository

https://github.com/void-linux/void-packages/pull/42484#discussion_r1120024777

Comment:
This function will never run, see

https://github.com/void-linux/void-packages/actions/runs/4283529441/jobs/7459365515#step:7:7990

Re: [PR REVIEW] zfs: divide under subpackages

[ahesford]

[-- Attachment #1: Type: text/plain, Size: 242 bytes --]

New review comment by ahesford on void-packages repository

https://github.com/void-linux/void-packages/pull/42484#discussion_r1120019266

Comment:
This never gets overwritten in the main package and leaves an incomplete package description.

Re: [PR REVIEW] zfs: divide under subpackages

[ahesford]

[-- Attachment #1: Type: text/plain, Size: 243 bytes --]

New review comment by ahesford on void-packages repository

https://github.com/void-linux/void-packages/pull/42484#discussion_r1120020450

Comment:
Relying on specific ordering for subpackages is incredibly fragile and should not be required.

Re: [PR REVIEW] zfs: divide under subpackages

[robertek]

[-- Attachment #1: Type: text/plain, Size: 253 bytes --]

New review comment by robertek on void-packages repository

https://github.com/void-linux/void-packages/pull/42484#discussion_r1120090603

Comment:
Interesting. On local xbps-src it is built. I have tested this completely and it is working as expected.

Re: [PR REVIEW] zfs: divide under subpackages

[ahesford]

[-- Attachment #1: Type: text/plain, Size: 235 bytes --]

New review comment by ahesford on void-packages repository

https://github.com/void-linux/void-packages/pull/42484#discussion_r1120092206

Comment:
It didn't build locally, it is just shadowed by the main package that shares its name.

Re: [PR REVIEW] zfs: divide under subpackages

[robertek]

[-- Attachment #1: Type: text/plain, Size: 298 bytes --]

New review comment by robertek on void-packages repository

https://github.com/void-linux/void-packages/pull/42484#discussion_r1120093165

Comment:
There is no other way to force the zfs-tools be the last. Otherwise all the specific files would need to be defined, which is then much more fragile.

Re: [PR REVIEW] zfs: divide under subpackages

[robertek]

[-- Attachment #1: Type: text/plain, Size: 184 bytes --]

New review comment by robertek on void-packages repository

https://github.com/void-linux/void-packages/pull/42484#discussion_r1120095224

Comment:
ahh, this is valid. I will fix it. 

Re: [PR REVIEW] zfs: divide under subpackages

[ahesford]

[-- Attachment #1: Type: text/plain, Size: 352 bytes --]

New review comment by ahesford on void-packages repository

https://github.com/void-linux/void-packages/pull/42484#discussion_r1120097160

Comment:
It's not more fragile, just more explicit and tedious. In any case, the "catchall"  package should always be the primary package, which is what you have left after stripping out any specific subpackages.

Re: zfs: divide under subpackages

[robertek]

[-- Attachment #1: Type: text/plain, Size: 887 bytes --]

New comment by robertek on void-packages repository

https://github.com/void-linux/void-packages/pull/42484#issuecomment-1448235787

Comment:
The main point is to have a support for custom kernel with zfs built in. I came from Gentoo and have a long history of making my own kernel which then boots not only the Gentoo, but even others. It is easy to mask the linux package, but I'm not able to install zfs userland. Actually there are two things, kernel module and userland tools packed to one tar. And the userland tools does not need to be strictly 1:1 to the kernel ZFS version (to some point).

I was thinking about opting that out, but since all of this is can be packed to different packages, this seems to be much better to do it that way. 

I'm not insisting on this change and I'm free to keep it myself for myself. I tried to do that the way it can be shared with others.

Re: zfs: divide under subpackages

[ahesford]

[-- Attachment #1: Type: text/plain, Size: 951 bytes --]

New comment by ahesford on void-packages repository

https://github.com/void-linux/void-packages/pull/42484#issuecomment-1448281050

Comment:
I am strictly opposed to a package graph that facilitates these kinds of mismatches. Ubuntu has been bitten HARD by shipping different versions of the ZFS kmod and userspace utilities. Because we strive for a first-class ZFS experience in Void, the userspace tools  and DKMS package that we ship should definitely be part of a matched pair.

Just because our package installs the DKMS sources doesn't mean that you have to build them. Void kernel packages will build the kmod automatically, but your kernel workflow can skip the XBPS hooks. 

You can also `noextract` the bits you want. Yes, this gets a bit cumbersome, but our primary goal is to keep the package graph and maintainability simple for official packages, not complicate things for everybody else to support custom, unpackaged content. 

Re: zfs: divide under subpackages

[Vaelatern]

[-- Attachment #1: Type: text/plain, Size: 635 bytes --]

New comment by Vaelatern on void-packages repository

https://github.com/void-linux/void-packages/pull/42484#issuecomment-1448504761

Comment:
Thank you for your contribution! Unfortunately, there are some red flags here that can not be cleaned up. ZFS lives in that special place in our systems where, if it dies, we end up with massive cleanup efforts (and probably restoration from backups). As such, anything that threatens stability, even so innocuous as separating userspace commands and kernel modules installed (which is arguably tested... in one direction), risks data.

I'm afraid we won't be able to merge this into Void.

Re: [PR PATCH] [Closed]: zfs: divide under subpackages

[Vaelatern]

[-- Attachment #1: Type: text/plain, Size: 1913 bytes --]

There's a closed pull request on the void-packages repository

zfs: divide under subpackages
https://github.com/void-linux/void-packages/pull/42484

Description:
<!-- Uncomment relevant sections and delete options which are not applicable -->

The main intention of this PR is to separate the zfs tools from the kernel module. This change introduce several new subpackages for the `zfs` package. 
Namely:
-  zfs-tools
- zfs-dkms
- zfs-test
- zfs-zed

The original zfs package remain as a metapackage for the needed subpackages. This helps the transition (eg: for those having zfs installed, the system update will only introduce several new packages and will work as before). It is now possible to install purely the zfs-tools package if a non-distributed kernel is built with zfs.

I have outplaced the ztest to a new subpackage which is not as part of the zfs metapackage any more. Ztest is intended to be run only on testing systems.

I may provide additional information if needed. For example the contents of the packages, etc.

#### Testing the changes
- I tested the changes in this PR: **YES**
- Tested on native x86-64-glibc

<!--
#### New package
- This new package conforms to the [package requirements](https://github.com/void-linux/void-packages/blob/master/CONTRIBUTING.md#package-requirements): **YES**|**NO**
-->

<!-- Note: If the build is likely to take more than 2 hours, please add ci skip tag as described in
https://github.com/void-linux/void-packages/blob/master/CONTRIBUTING.md#continuous-integration
and test at least one native build and, if supported, at least one cross build.
Ignore this section if this PR is not skipping CI.
-->

#### Local build testing
- I built this PR locally for my native architecture, (x86-64-glibc)
- I built this PR locally for these architectures (if supported. mark crossbuilds):
  - aarch64 (cross)
  - x86-64-musl

Re: zfs: divide under subpackages

[robertek]

[-- Attachment #1: Type: text/plain, Size: 1365 bytes --]

New comment by robertek on void-packages repository

https://github.com/void-linux/void-packages/pull/42484#issuecomment-1448505954

Comment:
I understand the intention to keep this as simple as possible from the maintenance point of view. I have also refactored it the way it would not interfere the stock users and allow customization if wanted. I definitely do not want to brake something in spite of some custom users.

I understand the noextract option, but wouldn't be viable just create subpackages for stuff that is able to be "noextracted" with nearly no cost? Or if it is some non-written convention in Void not to create subpackages much (as for example Alpine do) I would understand that.

Wouldn't then be an option to keep the zfs-userland in zfs package and only create subpackages with dkms and zed? The zfs-test is something that should not be present on any semi-production/user system anyway (but this is true also for the zinject).

With the version incompatibility breakage I remember that form the time of 0.6 versions myself, but since 2.0 or 2.1 I have never see any commit in 2.x branch to break the interoperability between kernel-useralnd on the same branch. And it is not an intention of this refactor. DKMS users will still have the same version on both and the ones that build the zfs inside kernel just need to care themselves.

Re: zfs: divide under subpackages

[robertek]

[-- Attachment #1: Type: text/plain, Size: 163 bytes --]

New comment by robertek on void-packages repository

https://github.com/void-linux/void-packages/pull/42484#issuecomment-1448506593

Comment:
Ok I undestand that.