Re: [discuss] Illumos future and compatibility with Open-ZFS

[Miles Nordin <carton@Ivy.NET>]

> the OpenZFS development model centres around an
> unstable master branch, with a stabilisation process to get to
> periodic release branches.

why not just sync to their release branches and ignore their development
branch?  Then this is not a problem.  I suspect other OpenZFS users (BSD?) 
do exactly that, or do they not?

This makes it less gratifying for illumos folk to work on ZFS since the
most efficient way will be to contribute to OpenZFS, not illumos, and see
the results in illumos much later.  But if you take the long term view
unfortunately this is the most efficient.

> Importing code from OpenZFS without introducing regressions is a large
> and time-consuming undertaking.  One must be absolutely sure to
> understand not just the patch one wants to import, but all of the
> many, many patches that fly in around that patch and touch other parts
> of the code.  The impact of importing a patch without importing prior
> commits that patch depends on, or without importing _subsequent_ fixes

why not sync, ie. take all the patches, and basically become an OpenZFS
user?  What is the justification for our fork?

Is there actually a record of our being more stable?  Maybe there is, but
I have not noticed it.

> It's the file system -- we must be cautious!

The divergence is also a problem.  I have both illumos and Linux machines,
to mitigate certain kinds of risk.  Differences in behaviour and in stream
compatibility are unwelcome.  This is a general thing in free software.
Branches cause wasted effort, compatibility problems, and generally add
cognitive load, so there's a relatively high bar to justify them.  Are you
sure it's met?

> While our rate of change is demonstrably less frenzied, part
> of that is because at a project level we generally place a higher
> value on stability (in every sense of the word) over new features;

Many of the features have been stability-related, such as resilvering
performance or resilvering-never-finishes bugs.  You would probably not
count that as "stability," but from ops perspective it is, 110%, because
it impacts durability which is usually more important than availability.

> What sort of features would you like to see?  Flicking back through
> the last six months of commits, I've attempted to pick just based on
> the synopses, things that are (as far as I recall) new feature working
> going into the OS:

the loss of the infiniband stack was a blow to me.  It's still there, but
it doesn't work any more.  It was a blow not to have it, and a blow to
spend so much time diagnosing the hangs through which the regression 
manifested itself.  but that's a different topic, and likely a 
losing argument since it didn't look well-used by mailing-list-search, 
it was forked from the mellanox framework, whatever that's called, and it 
was big.  I probably need to just use Linux if I need IB, now that Sun's
backing is gone.

You're right generally that avoiding regressions is really important, and
that trying to have fewer of them is a reason I use OmniOS rather than 
Linux.  It's not possible to get bandwidth back by automating when you are
drowning in bespoke regressions that have to be chased down and worked 
around.  All of my planning now is around making staying current with
patches cheaper.

> I don't think there is currently a plan to abandon maintaining our own
> ZFS and switch to importing OpenZFS each time they cut a release
> branch.  As long as there is an illumos project -- and, after 14
> years, I think it's safe to say that we're in it for the long haul! --
> we'll be looking after it, and the rest of the code, as a cohesive
> whole.

This is probably a mistake, IMHO.

I think what is really driving it is that it's fun to work on ZFS, and it
would be less fun to work on OpenZFS than illumos ZFS because of the rude 
and messy CADT culture.

But as subsystems die (ex infiniband) and large new subsystems become 
important (ex. TCP-BBR which in linux needs their queueing system, userspace 
networking and network cards with fancy clocks, new kinds of virtualization)
illumos and its forks will accumulate deal-breaker limitations for larger
numbers of sites.  I think survival depends on being extremely efficient,
and hobby forks are not efficient.  With anything open source, mechanical
efficiency and the joy of working on the project need to be balanced, and 
the balance ought to tilt further toward joy than seems sane to someone
accustomed to corporate environment.  But I worry here it's too costly, both
the direct cost in the forked ZFS implementations, and the indirect cost
in that there is not enough developer bandwidth available to keep illumos
relevant if any is wasted.

That said thank you for your balanced and well-reasoned defense of the 
status quo.