Re: [discuss] Illumos future and compatibility with Open-ZFS

["Joshua M. Clulow" <josh@sysmgr.org>]

On Fri, 6 Sept 2024 at 04:53, gea@napp-it.org <gea@napp-it.org> wrote:
> Open-ZFS is where the music plays with a lot of new features like ZSTD, Draid, Raid-Z Expansion or Fast Dedup and more to come. Lack of them means that you can no longer import a current Open-ZFS pool in Illumos and more important, these are killer features in some cases and therefor a criteria to use or not to use Illumos.

There is certainly much interesting work occurring in the OpenZFS
project!  There is in fact _so much_ work going on that it's frankly
difficult to be fully across all of it, and to ensure all of it has
received adequate review and testing.  At some point there is also the
fact that every change introduces risk, even if it is not obvious what
that risk would be.  We try to balance many different competing
concerns when we're importing code from the OpenZFS project, or any
other source base that we share with; e.g., FreeBSD.  Amongst those
concerns is that the OpenZFS development model centres around an
unstable master branch, with a stabilisation process to get to
periodic release branches.  This is quite different from our own model
here; critically, we want every commit that goes into our master
branch to be something you could ship to a customer.  If we make a
change to the ZFS pool format in a master branch, it's there to stay
forever.  In practice, no regressions on master is always something of
a stretch goal, but it's one we aspire to!

Importing code from OpenZFS without introducing regressions is a large
and time-consuming undertaking.  One must be absolutely sure to
understand not just the patch one wants to import, but all of the
many, many patches that fly in around that patch and touch other parts
of the code.  The impact of importing a patch without importing prior
commits that patch depends on, or without importing _subsequent_ fixes
that have been made after the patch landed, can be the difference
between uneventful success, or rampant and immediate data corruption
on user's machines.

It's the file system -- we must be cautious!

> 1. Open-ZFS Master (BSD, Linux), currently 2.2.6
> no longer a common roof for "Open" ZFS development but the place where development happens

To be clear, it's the place where development _on the OpenZFS project_
happens.  While our rate of change is demonstrably less frenzied, part
of that is because at a project level we generally place a higher
value on stability (in every sense of the word) over new features;
that's what it means to be dependable infrastructure software!

> I want to ask.
> When I look at the Illumos Issue tracker, I see mainly small fixes, hardly new features does not matter regarding Illumos services or Open-ZFS features. I suppose number of devs is limited.

What sort of features would you like to see?  Flicking back through
the last six months of commits, I've attempted to pick just based on
the synopses, things that are (as far as I recall) new feature working
going into the OS:

    16705 Want lifetime extensions for TCP_MD5SIG
    16708 SMBIOS 3.8 Support
    16675 want syncfs(3C)
    16654 nvme: expose internal counters via kstats
    16655 nvme: introduce kstats for admin commands
    16636 want ptsname_r
    16635 want SO_PROTOCOL socket option support
    14237 Want support for pthread_cond_clockwait() and friends
    16624 Want support for FD_CLOFORK and friends
    14736 zfs: Improve sorted scan memory accounting
    16627 Flesh out AMD Turin chip and Zen 5 uarch revisions.
    16076 pwdx should work on core files
    16475 loader.efi: replace comconsole with efiserialio
    13967 diskinfo(8) should show serial numbers in more cases
    14151 truss: truss should recognize VMM ioctl codes
    16547 savecore should report progress when saving compressed dump
    16524 driver for 38xx HBA in illumos
    16491 netcat should support setting the outgoing and minimum TTL
    16501 netcat should support setting IPV6_TCLASS for ipv6 sockets
    16455 want TCP_MD5SIG socket option
    16454 want IP_MINTTL socket option
    15977 smbadm option to show group member SIDs
    15976 List users connected via SMB
    16452 want strerrordesc_np and strerrorname_np
    16459 want emlxs to support Oracle branded LP adapters
    16408 AMD Zen 5 CPC support
    16056 want fmdump ability to AND event property filters
    16423 Import fletcher-4 algorithms from OpenZFS
    14919 tem: implement xenl
    16348 e1000g I219 V17 does not attach
    16347 e1000g LM+V24-27,29 support
    16327 wdc nvme assertion clearing support
    16326 Update NVMe error status codes for 2.x
    16325 nvmeadm: want ability to write log page to raw file
    16324 Want Micron 7300, 7400, 7450 log support

At the end of the day, we're an open source project with a finite set
of engineering resources just like any other project.  When feature
work happens, it's generally because somebody has a problem they need
to solve and they're motivated to do the work.  I work at Oxide, and
we have obviously a very strong motivation to keep working on illumos
and to ensure the OS continues to meet our goals for stability,
security, performance, and features.  The same is generally true of
all other contributors, whether it's a corporate or an unaffiliated
community member.

The bug tracker is also not the full picture.  We also have the
"illumos Project Discussion" (IPD) repository, in which we try to work
through longer term plans that might span many integrations into
master:

    https://github.com/illumos/ipd

Anybody is welcome to file and work on bugs, or to write and circulate
IPDs, if they have a project they'd like to spearhead or work they'd
like to do!

> What is the future idea of Illumos?
> I know a switch to Open-ZFS as upstream is not easy and can last and I am not even sure if it is really wanted or possible with current resources. But maybe this is the only option to make Illumos future proof?

I don't think there is currently a plan to abandon maintaining our own
ZFS and switch to importing OpenZFS each time they cut a release
branch.  As long as there is an illumos project -- and, after 14
years, I think it's safe to say that we're in it for the long haul! --
we'll be looking after it, and the rest of the code, as a cohesive
whole.  We will continue to import things from OpenZFS where we
believe they will not impact the stability of our own software, and
we'll continue to do work on our ZFS that may only be of interest to
us (e.g., we're the only OS with zones, another subsystem that has
integration with ZFS).

As with any truly open source project, you, and indeed anybody else,
are welcome to roll up your sleeves and work on the changes that would
benefit you and your use cases personally!  We depend on motivated
individuals scratching their own itches as much as anything else.  If
you can demonstrate genuine motivation and engagement, I know that
both the core team and the community at large will do our best to help
with the work you want to do.


Cheers!

-- 
Joshua M. Clulow
http://blog.sysmgr.org