Re: zstd compression

[Ingo Schwarze <schwarze@usta.de>]

Hi Soeren,

Soeren Tempel wrote on Tue, Jan 28, 2025 at 08:10:53AM +0100:

> Compression schemes beyond gzip are currently being adopted by Linux
> distributions for man page compression. Case in point: Guix [1]

Yikes.  The goal at the very foundation of that software looks
misguided in the first place.  The purpose of privileged accounts
is to make sure that users cannot change the configuration of the
machine.  Installing software is a typical example where using a
priviledged account makes sense and is often even required because
it often involves creating daemon accounts or groups or directories
owned by specific users or groups or setting up jobs to run
periodically.  Also, it's good when system administrators have
an idea what people are running on a machine in order to keep it
properly secured, which also means that it's better when installation
is done by the admin.

Yes, i know users can always install their own software in their
home directory.  While trying to prevent that with technical means
is usually stupid because it doesn't work very well, people should
be discouraged from installing software they need themselves rather
than encouraged.

On a single-user machine, installing software with your normal
user account would be so stupid that it would leave me speechless.

So, what is the point?

Also, which system uses Guix?  I mean, a package manager normally
needs to be tightly integrated with the operating system, a stand-
alone package manager is an oxymoron (yes, i know the NetBSD thingy,
pksrc, wants to do precisely that, but that's mostly useful for
exotic systems coming with no packagge manager whatsoever, like AIX).

Are there any sane distros switching to zstd?

> has switched to zstd compression recently, which has sadly rendered the
> Guix mandoc package defunct [2].
> 
> For other downstream packagers impacted by this, I wanted to point out
> that the zstd project provides a wrapper library which is API-compatible
> with zlib and easily allows adapting software using zlib (e.g. mandoc)
> to zstd compression. A sample patchset for doing that is available in
> the Guix patch tracker [3].
> 
> Given mandoc's stance on man page compression [4], this is rather
> intended as an FYI for downstream packagers, but in case there is
> interested in proper upstream support for additional compression
> algorithm, feel free to let me know.

I feel serious tempted to rip out compression support altogether.
It's just so anachronistic.

By the way, your comment in the thread

  "Usually, its hard to convince them to add features that do not
   benefit OpenBSD."

is misleading.  I have implemented many features that do not benefit
OpenBSD in any way whatsoever, and some of these are even availabke
is the OpenBSD base system and not only in mandoc portable:

 * Style checks for manual pages specific to NetBSD.
 * The mdoc(7) .Lb macro, extensively used by FreeBSD and some others.
 * Very careful support for manual sections that are not pure
   numbers but have alphabetic suffixes, mostly for Illumos and
   Solaris, but also used by some Linux distributions.
 * ...

Some features are only in portable, not in OpenBSD:

 * Support for "make install" installing symlinks rather than
   hardlinks.
 * The READ_ALLOWED_PATH feature for Homwbrew and MacOSX
 * Support for installing libmandoc.a, mostly for NetBSD
 * A complete program, mandocd(8), mostly for Debian
 * and lots of other stuff, most of it visible in
   configure.local.example

But if somebody wants a feature and cannot explain what the benefit
is, i do not want such a feature to complicate the code.  Then again,
in this case, it might not complicate the code at all, in which
case i might be willing to accomodate even a foolish feature.

That said, let's see what cam be done for your case.
It appears you need four elements:

 * Five new *.o files.  I definitely do not want to add the
   source code for these five files to the mandoc repo.
   Can you instead install a system-wide shared object
   containing that code, such that mandoc can dynamically link,
   just like it is dynamically linking zlib right now?
 * The argument -lzstd on the ld(1) command line.
   That should be easy with an option in configure.local.example.
 * #include <zstd_zlibwrapper.h> in read.c.  Can easily be controlled
   with the same option.
 * Recognizing the .zst file extension.  Can easily be controlled
   with the same option.

By the way, is explicitly linking to both shared objects really
necessary?  Why does the libzstd_zlibwrapper.so (or however it is
called) not know that it depends on libz.so and let ld.so do
the work?  Not sure whether that's possible, if not, "-lz -lzstd"
is not out of the question, either.

Yours,
  Ingo
--
 To unsubscribe send an email to discuss+unsubscribe@mandoc.bsd.lv