[musl] musl -- FFS get your shit together, please

[musl] musl -- FFS get your shit together, please

[Dave Blanchard]

There's a lot to like about musl, but damn, there's some absolutely ridiculous aspects also:

1) How in the hell are you going to make a MAJOR change like changing #ifdefs from defined(_LARGEFILE64_SOURCE) || defined(_GNU_SOURCE) to just defined(_LARGEFILE64_SOURCE) in a PATCH level increment, from 1.2.3 to 1.2.4? What the hell is wrong with you? You just broke my entire build! Yet another patch had to be created on my end to UNDO this crazy change; the only alternative was patching half the packages on my system to fix their now-broken build! Do you know NOTHING about proper versioning???

2) Did it occur to anyone involved in this project to maybe actually organize and COMMENT the system header files, instead of just randomly throwing a random assortment of shit into an .H file and calling it good? The amount of duplicated, undocumented, assorted crap is pretty ridiculous for a project that's supposed to be a FROM SCRATCH libc implementation! How about getting it right from the beginning, with a clean and organized implementation, instead of starting off with a random pile of shit? Even glibc is better organized, for fuck's sake!

3) Why in the hell does musl duplicate/change(!) internal structures from Linux kernel headers instead of just #include'ing the damn Linux headers (and relevant structures) and be done with it? 

4) Would it kill you to add in various simple #defines and such in the headers which greatly improve compatibility with GNU code, without actually requiring any support in the libc library code??

Between the above, plus the 6-7 "musl addon" support packages required to be installed alongside to make my Linux system build with musl, at this point I have essentially FORKED musl!

Musl is clearly not designed with Linux workstation usage in mind!

Dave

Re: [musl] musl -- FFS get your shit together, please

[(GalaxyMaster)]

Dave,

> On 17 Jul 2023, at 16:17, Dave Blanchard <dave@killthe.net> wrote:
> 
> There's a lot to like about musl, but damn, there's some absolutely ridiculous aspects also:
> 
[skipped]
> 
> Musl is clearly not designed with Linux workstation usage in mind!

I understand that you've hit some frustration point whether it was triggered by a change in musl or not, but it does not help anyone to start a message in an offensive tone and basically call everyone in the community morons.  Nobody is forcing you to use the open source software (such as musl) and the majority of people are just grateful that there are people who are spending their spare time to improve a project in meaningful ways.

I would suggest to take a walk and calm down a bit when you are getting to agitated, then you can produce a message that contributes in a positive way and does not offend people.  Believe me, it is a win-win for everybody in this case.

-- 
(GM)

Re: [musl] musl -- FFS get your shit together, please

[Daniel Gutson]

[-- Attachment #1: Type: text/plain, Size: 2182 bytes --]

Dear Dave,

    Please file your comments in the issue tracker so they can be addressed
separately, properly assigned, and track their status and therefore be
planned for the next releases.

El lun, 17 de jul de 2023, 03:15, Dave Blanchard <dave@killthe.net>
escribió:

> There's a lot to like about musl, but damn, there's some absolutely
> ridiculous aspects also:
>
> 1) How in the hell are you going to make a MAJOR change like changing
> #ifdefs from defined(_LARGEFILE64_SOURCE) || defined(_GNU_SOURCE) to just
> defined(_LARGEFILE64_SOURCE) in a PATCH level increment, from 1.2.3 to
> 1.2.4? What the hell is wrong with you? You just broke my entire build! Yet
> another patch had to be created on my end to UNDO this crazy change; the
> only alternative was patching half the packages on my system to fix their
> now-broken build! Do you know NOTHING about proper versioning???
>
> 2) Did it occur to anyone involved in this project to maybe actually
> organize and COMMENT the system header files, instead of just randomly
> throwing a random assortment of shit into an .H file and calling it good?
> The amount of duplicated, undocumented, assorted crap is pretty ridiculous
> for a project that's supposed to be a FROM SCRATCH libc implementation! How
> about getting it right from the beginning, with a clean and organized
> implementation, instead of starting off with a random pile of shit? Even
> glibc is better organized, for fuck's sake!
>
> 3) Why in the hell does musl duplicate/change(!) internal structures from
> Linux kernel headers instead of just #include'ing the damn Linux headers
> (and relevant structures) and be done with it?
>
> 4) Would it kill you to add in various simple #defines and such in the
> headers which greatly improve compatibility with GNU code, without actually
> requiring any support in the libc library code??
>
> Between the above, plus the 6-7 "musl addon" support packages required to
> be installed alongside to make my Linux system build with musl, at this
> point I have essentially FORKED musl!
>
> Musl is clearly not designed with Linux workstation usage in mind!
>
> Dave
>

[-- Attachment #2: Type: text/html, Size: 2484 bytes --]

Re: [musl] musl -- FFS get your shit together, please

[Rich Felker]

On Mon, Jul 17, 2023 at 01:17:58AM -0500, Dave Blanchard wrote:
> There's a lot to like about musl, but damn, there's some absolutely ridiculous aspects also:
> 
> 1) How in the hell are you going to make a MAJOR change like
> changing #ifdefs from defined(_LARGEFILE64_SOURCE) ||
> defined(_GNU_SOURCE) to just defined(_LARGEFILE64_SOURCE) in a PATCH
> level increment, from 1.2.3 to 1.2.4? What the hell is wrong with
> you? You just broke my entire build! Yet another patch had to be
> created on my end to UNDO this crazy change; the only alternative
> was patching half the packages on my system to fix their now-broken
> build! Do you know NOTHING about proper versioning???

Our versioning system works like this: in x.y.z,

- increment of x, likely to never happen, would indicate a completely
  different ABI

- increment of y indicates a change whereby programs compiled for the
  new y, even without use of any new features added in new y, may not
  run with an older y. canonical example: time64.

- increment of z indicates a change whereby programs built for the new
  z should still run on older z (modulo any bugs that might be present
  in the older version) as long as they're not using new interfaces
  introduced in the new z.

All of these conditions are assuming the program used the public
interfaces and did not poke at unspecified internals, etc.; if it did,
all bets are off and any version change may be fully-breaking to the
program.

Note that all of these deal with ABI compatibility, not compile-time
compatibility.

> 2) Did it occur to anyone involved in this project to maybe actually
> organize and COMMENT the system header files, instead of just
> randomly throwing a random assortment of shit into an .H file and
> calling it good?

The header files intentionally do not contain nontrivial creative
content.

> 3) Why in the hell does musl duplicate/change(!) internal structures
> from Linux kernel headers instead of just #include'ing the damn
> Linux headers (and relevant structures) and be done with it?

Because we are defining the interface between an application and the
standard library functions not between libc and the kernel or the
application and the kernel.

Sometimes the types will be the same; sometimes they *cannot* (e.g.
when the kernel type is wrong or does not meet the specified
requirements).

Even when the types do match, the Linux uapi headers are generally not
namespace-safe.

> 4) Would it kill you to add in various simple #defines and such in
> the headers which greatly improve compatibility with GNU code,
> without actually requiring any support in the libc library code??

I'm not sure what this question even means. Are you talking about the
LFS64 stuff again or something else? The LFS64 macros were removed
(by default) because they were a repeated source of breakage that was
*our fault* compiling C++ programs (where GCC makes _GNU_SOURCE the
default), and thereby our responsibility to fix. The only reason they
were ever added to begin with was because of configure scripts wrongly
detecting LFS64 via link-only tests, resulting in failed builds or
(more often) broken binaries when the LFS64 symbols, which were only
there as ABI-compat for glibc-linked code, got used without any
declarations ("implicit function").

The way this was fixed, very little should have broken. From reports
so far, it seems to have been only a very small amount of
Linux-specific code or code that hard-coded "Linux implies LFS64",
most of which already has patches fixing it.

> Between the above, plus the 6-7 "musl addon" support packages
> required to be installed alongside to make my Linux system build
> with musl, at this point I have essentially FORKED musl!
> 
> Musl is clearly not designed with Linux workstation usage in mind!

*eyeroll*

[musl] ITT: Nothing but a bunch of excuses and no solutions

[Dave Blanchard]

On Mon, 17 Jul 2023 11:21:04 -0400
Rich Felker <dalias@libc.org> wrote:

> Our versioning system works like this: in x.y.z,
> 
> - increment of x, likely to never happen, would indicate a completely
>   different ABI
> 
> - increment of y indicates a change whereby programs compiled for the
>   new y, even without use of any new features added in new y, may not
>   run with an older y. canonical example: time64.
> 
> - increment of z indicates a change whereby programs built for the new
>   z should still run on older z (modulo any bugs that might be present
>   in the older version) as long as they're not using new interfaces
>   introduced in the new z.

Translated: you pretend like you are using the same semantic versioning that everyone else is using, except you actually aren't. You don't care about the suffering you create through your creative, self-serving reinterpretation of semantic versioning.

> > 2) Did it occur to anyone involved in this project to maybe actually
> > organize and COMMENT the system header files, instead of just
> > randomly throwing a random assortment of shit into an .H file and
> > calling it good?
> 
> The header files intentionally do not contain nontrivial creative
> content.

Translated: our code is an uncommented, disorganized mess and we love it.

> > 3) Why in the hell does musl duplicate/change(!) internal structures
> > from Linux kernel headers instead of just #include'ing the damn
> > Linux headers (and relevant structures) and be done with it?
> 
> Because we are defining the interface between an application and the
> standard library functions not between libc and the kernel or the
> application and the kernel.
> 
> Sometimes the types will be the same; sometimes they *cannot* (e.g.
> when the kernel type is wrong or does not meet the specified
> requirements).
> 
> Even when the types do match, the Linux uapi headers are generally not
> namespace-safe.

Translated: the end user will need to patch the musl headers to stop defining custom, bespoke, incompatible versions of kernel structures and just include the damn kernel header files so that their system will actually build properly.

> > 4) Would it kill you to add in various simple #defines and such in
> > the headers which greatly improve compatibility with GNU code,
> > without actually requiring any support in the libc library code??
> 
> I'm not sure what this question even means.

Obviously not!

> Are you talking about the LFS64 stuff again or something else? 

Maybe you should try building your own Linux distribution so you will get a clue!

> The LFS64 macros were removed (by default) because they were a repeated source of breakage that was
> *our fault* compiling C++ programs (where GCC makes _GNU_SOURCE the
> default), and thereby our responsibility to fix. The only reason they
> were ever added to begin with was because of configure scripts wrongly
> detecting LFS64 via link-only tests, resulting in failed builds or
> (more often) broken binaries when the LFS64 symbols, which were only
> there as ABI-compat for glibc-linked code, got used without any
> declarations ("implicit function").

Translation: the end user will now need to apply heavy patches to his/her system, and/or patch musl 1.2.4 to revert the old behavior, in order to get their damn system to actually build correctly.

> The way this was fixed, very little should have broken. From reports
> so far, it seems to have been only a very small amount of
> Linux-specific code or code that hard-coded "Linux implies LFS64",
> most of which already has patches fixing it.

Ahem, HELLO? How about THIS report that I'm making now where DOZENS OF PACKAGES on my system (starting with GCC!) now need new patches just to build, when they worked fine before? Didn't take me long to see the writing on the wall and nope the fuck out of your "upgrade"!

Oh, but those patches are already *in existence*, you see! Somewhere. All I have to do is spend hours combing the net to find them. My time is free, so that's no problem.

> > Between the above, plus the 6-7 "musl addon" support packages
> > required to be installed alongside to make my Linux system build
> > with musl, at this point I have essentially FORKED musl!
> > 
> > Musl is clearly not designed with Linux workstation usage in mind!
> 
> *eyeroll*

Once again: MUSL IS CLEARLY NOT DESIGNED WITH LINUX WORKSTATION USAGE IN MIND!

After further study, having discovered that musl 1.2.4 has not just fucked the header files but also removed weak links for the LFS64 functions from the damn library itself thus breaking various builds and requiring heavy patching to correct, I have permanently reverted to musl 1.2.3 where I will stay indefinitely. It appears this project's maintainers have their heads permanently and irretrievably installed inside their own asshole. I refuse to continue suffering constant headaches of undoing your neverending breaking changes with each PATCH LEVEL version "upgrade."

Dave

Re: [musl] ITT: Nothing but a bunch of excuses and no solutions

[Joakim Sindholt]

On Mon, 17 Jul 2023 10:43:59 -0500, Dave Blanchard <dave@killthe.net> wrote:
> 
> Once again: MUSL IS CLEARLY NOT DESIGNED WITH LINUX WORKSTATION USAGE IN MIND!

PEBKAC

Re: [musl] ITT: Nothing but a bunch of excuses and no solutions

[Rich Felker]

On Mon, Jul 17, 2023 at 10:43:59AM -0500, Dave Blanchard wrote:
> On Mon, 17 Jul 2023 11:21:04 -0400
> Rich Felker <dalias@libc.org> wrote:
> 
> > Our versioning system works like this: in x.y.z,
> > 
> > - increment of x, likely to never happen, would indicate a completely
> >   different ABI
> > 
> > - increment of y indicates a change whereby programs compiled for the
> >   new y, even without use of any new features added in new y, may not
> >   run with an older y. canonical example: time64.
> > 
> > - increment of z indicates a change whereby programs built for the new
> >   z should still run on older z (modulo any bugs that might be present
> >   in the older version) as long as they're not using new interfaces
> >   introduced in the new z.
> 
> Translated: you pretend like you are using the same semantic
> versioning that everyone else is using, except you actually aren't.
> You don't care about the suffering you create through your creative,
> self-serving reinterpretation of semantic versioning.

Basically anyone using "semantic versioning" is using their own
variant of it, because in order for it to be meaningful you have to
define the relevant interface boundaries, and those will differ for
different types of software. If you don't like ours, that a "you
problem" not an "us problem".

> > > 2) Did it occur to anyone involved in this project to maybe actually
> > > organize and COMMENT the system header files, instead of just
> > > randomly throwing a random assortment of shit into an .H file and
> > > calling it good?
> > 
> > The header files intentionally do not contain nontrivial creative
> > content.
> 
> Translated: our code is an uncommented, disorganized mess and we love it.

Apparently your translation function just randomly picks a new context
when you're wrong. Maybe it's wired up to ChatGPT or something?
*eyeroll*

If you have a complaint about whether/where code is commented, that's
utterly unrelated to whether/where public headers are commented.

> > > 3) Why in the hell does musl duplicate/change(!) internal structures
> > > from Linux kernel headers instead of just #include'ing the damn
> > > Linux headers (and relevant structures) and be done with it?
> > 
> > Because we are defining the interface between an application and the
> > standard library functions not between libc and the kernel or the
> > application and the kernel.
> > 
> > Sometimes the types will be the same; sometimes they *cannot* (e.g.
> > when the kernel type is wrong or does not meet the specified
> > requirements).
> > 
> > Even when the types do match, the Linux uapi headers are generally not
> > namespace-safe.
> 
> Translated: the end user will need to patch the musl headers to stop
> defining custom, bespoke, incompatible versions of kernel structures
> and just include the damn kernel header files so that their system
> will actually build properly.

Have fun breaking your system.

> > > 4) Would it kill you to add in various simple #defines and such in
> > > the headers which greatly improve compatibility with GNU code,
> > > without actually requiring any support in the libc library code??
> > 
> > I'm not sure what this question even means.
> 
> Obviously not!
> 
> > Are you talking about the LFS64 stuff again or something else? 
> 
> Maybe you should try building your own Linux distribution so you
> will get a clue!

LMAO this is galaxy-tier stuff.

> > The LFS64 macros were removed (by default) because they were a repeated source of breakage that was
> > *our fault* compiling C++ programs (where GCC makes _GNU_SOURCE the
> > default), and thereby our responsibility to fix. The only reason they
> > were ever added to begin with was because of configure scripts wrongly
> > detecting LFS64 via link-only tests, resulting in failed builds or
> > (more often) broken binaries when the LFS64 symbols, which were only
> > there as ABI-compat for glibc-linked code, got used without any
> > declarations ("implicit function").
> 
> Translation: the end user will now need to apply heavy patches to
> his/her system, and/or patch musl 1.2.4 to revert the old behavior,
> in order to get their damn system to actually build correctly.

There is no reason to want the old behavior except being unwilling to
fix something that was always wrong.

Any program that uses proper build-time detection of the (nonstandard)
LFS64 stuff works 100% before or after with no change; even the
resulting binaries should not change.

If a program is assuming those interfaces without testing, that's
broken, and breaking build of programs that make wrong/nonportable
assumptions about the standard headers is something musl has done *all
the time* since its inception. This does not harm end users in any
way, because the compiled binaries continue to work, as always.

If you have a build script building affected programs from source, it
may require new patches to fix whatever they're doing wrong. This has
always been the case, for lots of reasons other than LFS64. We
generally make attempts to assess impact and mitigate in cases where
it's expected that a single change/single release might put undue
burden on systems integrators. That's why the LFS64 macros were left
in place under _LARGEFILE64_SOURCE for now -- if you have an existing
integration and need it to continue working for the time being until
things are fixed on the application side, you can simply add
-D_LARGEFILE64_SOURCE to your global CFLAGS and everything works just
like before while you wait for someone else to do free labor and get
your other upstreams patched.

> > The way this was fixed, very little should have broken. From reports
> > so far, it seems to have been only a very small amount of
> > Linux-specific code or code that hard-coded "Linux implies LFS64",
> > most of which already has patches fixing it.
> 
> Ahem, HELLO? How about THIS report that I'm making now where DOZENS
> OF PACKAGES on my system (starting with GCC!) now need new patches
> just to build, when they worked fine before? Didn't take me long to
> see the writing on the wall and nope the fuck out of your "upgrade"!

As I remember it, the affected "GCC code" is not GCC code but the
library side of the sanitizers, which are generally based on poking at
implementation internals and may or may not work. There have been
improvements on them over the past few years, and they may roughly
work at this point, but have a lot of vestigial glibcisms and design
around poking at stuff that's probably not safe to assume.

I'm not sure of the extent to which distros were/are building and
testing them. I am not. So maybe we overlooked that "GCC" was in the
list. But I don't think it would have made a difference knowing this
ahead of time. Nothing was going to get fixed until there was pressure
from a release where the builds break.

> Oh, but those patches are already *in existence*, you see!
> Somewhere. All I have to do is spend hours combing the net to find
> them. My time is free, so that's no problem.

Apparently our time is free to listen and respond to getting berated
by users... *eyeroll*

Rich

Re: [musl] ITT: Nothing but a bunch of excuses and no solutions

[Laurent Bercot]

>Translated: our code is an uncommented, disorganized mess and we love it.

  Hmmm.

  Would you kindly give us a pointer to *your* code, so we can go study 
it
and learn from what is doubtlessly a paragon of clarity, cleanliness and
and well-commentedness?


>Translated: the end user will need to patch the musl headers to stop defining custom, bespoke, incompatible versions of kernel structures and just include the damn kernel header files so that their system will actually build properly.

  You do, of course, realize that the initial reason why some software
has trouble with mixing musl and linux headers is that Linux itself
generally does not favor including kernel headers in userspace, and the
whole uapi system is an ifdef forest that is still, as of today, full
of glibcisms, right?

  How long have you been building systems? Because doing wild stuff like
*building a Linux system with another libc* is a piece of cake today.
Back in 2002, it was downright impossible; the ecosystem has made a
lot of progress since.
  I would never dare suggest that the obstacles you seem to be bumping
into stem from your own inexperience - not in a million years. However,
now is really a good time to relax and enjoy the relative ease of
building systems, and it is a shame that you do not seem to be relaxing
and enjoying it.


>Maybe you should try building your own Linux distribution so you will get a clue!

  This is definitely strange, because I have been building my own systems
with musl for a while now, and I have not experienced half of the 
problems
you seem to be running into - and when I have run into obstacles, the
issues were usually with the rest of the software, not with musl.
I wonder why our experiences differ so much. It certainly cannot be a
skill or knowledge issue.


>Translation: the end user will now need to apply heavy patches to his/her system, and/or patch musl 1.2.4 to revert the old behavior, in order to get their damn system to actually build correctly.

  This end user disagrees, and wonders why you seem to have so much
difficulty using a computer. Have you tried turning it off and on again?


>Once again: MUSL IS CLEARLY NOT DESIGNED WITH LINUX WORKSTATION USAGE IN MIND!

  You know what? You're absolutely right - it is, at least, clearly not
designed with *your* Linux workstation usage in mind. The best path for
you is clearly to write down your musl usage attempt as a failure, and
give it up entirely. You should go back to using glibc, which will be
much easier, and you can make their mailing-list benefit from your
insightful, constructive, and heart-warming comments!

  There is no reason for you to keep interacting with a community that
does not understand your genius or address your so eloquently worded
concerns. We're not worthy of your presence, you should leave us to
our mediocrity.

--
  Laurent

Re: [musl] musl -- FFS get your shit together, please

[Jeffrey Walton]

On Mon, Jul 17, 2023 at 11:21 AM Rich Felker <dalias@libc.org> wrote:
>
> On Mon, Jul 17, 2023 at 01:17:58AM -0500, Dave Blanchard wrote:
> > There's a lot to like about musl, but damn, there's some absolutely ridiculous aspects also:
> >
> > 1) How in the hell are you going to make a MAJOR change like
> > changing #ifdefs from defined(_LARGEFILE64_SOURCE) ||
> > defined(_GNU_SOURCE) to just defined(_LARGEFILE64_SOURCE) in a PATCH
> > level increment, from 1.2.3 to 1.2.4? What the hell is wrong with
> > you? You just broke my entire build! Yet another patch had to be
> > created on my end to UNDO this crazy change; the only alternative
> > was patching half the packages on my system to fix their now-broken
> > build! Do you know NOTHING about proper versioning???
>
> Our versioning system works like this: in x.y.z,
>
> - increment of x, likely to never happen, would indicate a completely
>   different ABI
>
> - increment of y indicates a change whereby programs compiled for the
>   new y, even without use of any new features added in new y, may not
>   run with an older y. canonical example: time64.
>
> - increment of z indicates a change whereby programs built for the new
>   z should still run on older z (modulo any bugs that might be present
>   in the older version) as long as they're not using new interfaces
>   introduced in the new z.
>
> All of these conditions are assuming the program used the public
> interfaces and did not poke at unspecified internals, etc.; if it did,
> all bets are off and any version change may be fully-breaking to the
> program.
>
> Note that all of these deal with ABI compatibility, not compile-time
> compatibility.

To play devil's advocate... If a symbol in Musl disappears, then
shouldn't that be considered an ABI break? And then, shouldn't it
require a major or 'x' bump?

It seems like Musl signed that contract when it first published a
symbol under _LARGEFILE64_SOURCE or _GNU_SOURCE. When the symbol
disappeared using one or the other define, then the contract was
broken.

Jeff

Re: [musl] musl -- FFS get your shit together, please

[Joakim Sindholt]

On Mon, 17 Jul 2023 12:01:00 -0400, Jeffrey Walton <noloader@gmail.com> wrote:
> On Mon, Jul 17, 2023 at 11:21 AM Rich Felker <dalias@libc.org> wrote:
> >
> > On Mon, Jul 17, 2023 at 01:17:58AM -0500, Dave Blanchard wrote:
> > > There's a lot to like about musl, but damn, there's some absolutely ridiculous aspects also:
> > >
> > > 1) How in the hell are you going to make a MAJOR change like
> > > changing #ifdefs from defined(_LARGEFILE64_SOURCE) ||
> > > defined(_GNU_SOURCE) to just defined(_LARGEFILE64_SOURCE) in a PATCH
> > > level increment, from 1.2.3 to 1.2.4? What the hell is wrong with
> > > you? You just broke my entire build! Yet another patch had to be
> > > created on my end to UNDO this crazy change; the only alternative
> > > was patching half the packages on my system to fix their now-broken
> > > build! Do you know NOTHING about proper versioning???
> >
> > Our versioning system works like this: in x.y.z,
> >
> > - increment of x, likely to never happen, would indicate a completely
> >   different ABI
> >
> > - increment of y indicates a change whereby programs compiled for the
> >   new y, even without use of any new features added in new y, may not
> >   run with an older y. canonical example: time64.
> >
> > - increment of z indicates a change whereby programs built for the new
> >   z should still run on older z (modulo any bugs that might be present
> >   in the older version) as long as they're not using new interfaces
> >   introduced in the new z.
> >
> > All of these conditions are assuming the program used the public
> > interfaces and did not poke at unspecified internals, etc.; if it did,
> > all bets are off and any version change may be fully-breaking to the
> > program.
> >
> > Note that all of these deal with ABI compatibility, not compile-time
> > compatibility.
> 
> To play devil's advocate... If a symbol in Musl disappears, then
> shouldn't that be considered an ABI break? And then, shouldn't it
> require a major or 'x' bump?
> 
> It seems like Musl signed that contract when it first published a
> symbol under _LARGEFILE64_SOURCE or _GNU_SOURCE. When the symbol
> disappeared using one or the other define, then the contract was
> broken.

The symbols have only disappeared in the sense that you can't link new
binaries to them because they're not in the symbol table. The dynamic
linker resolves them at runtime as if the weak symbols still existed.
This way a link test will fail but an old binary will still load
properly.

Re: [musl] musl -- FFS get your shit together, please

[Rich Felker]

On Mon, Jul 17, 2023 at 12:01:00PM -0400, Jeffrey Walton wrote:
> On Mon, Jul 17, 2023 at 11:21 AM Rich Felker <dalias@libc.org> wrote:
> >
> > On Mon, Jul 17, 2023 at 01:17:58AM -0500, Dave Blanchard wrote:
> > > There's a lot to like about musl, but damn, there's some absolutely ridiculous aspects also:
> > >
> > > 1) How in the hell are you going to make a MAJOR change like
> > > changing #ifdefs from defined(_LARGEFILE64_SOURCE) ||
> > > defined(_GNU_SOURCE) to just defined(_LARGEFILE64_SOURCE) in a PATCH
> > > level increment, from 1.2.3 to 1.2.4? What the hell is wrong with
> > > you? You just broke my entire build! Yet another patch had to be
> > > created on my end to UNDO this crazy change; the only alternative
> > > was patching half the packages on my system to fix their now-broken
> > > build! Do you know NOTHING about proper versioning???
> >
> > Our versioning system works like this: in x.y.z,
> >
> > - increment of x, likely to never happen, would indicate a completely
> >   different ABI
> >
> > - increment of y indicates a change whereby programs compiled for the
> >   new y, even without use of any new features added in new y, may not
> >   run with an older y. canonical example: time64.
> >
> > - increment of z indicates a change whereby programs built for the new
> >   z should still run on older z (modulo any bugs that might be present
> >   in the older version) as long as they're not using new interfaces
> >   introduced in the new z.
> >
> > All of these conditions are assuming the program used the public
> > interfaces and did not poke at unspecified internals, etc.; if it did,
> > all bets are off and any version change may be fully-breaking to the
> > program.
> >
> > Note that all of these deal with ABI compatibility, not compile-time
> > compatibility.
> 
> To play devil's advocate... If a symbol in Musl disappears, then
> shouldn't that be considered an ABI break? And then, shouldn't it
> require a major or 'x' bump?

Only if that symbol is something that can be linked to as part of
using the published (via the headers) API. If for example you used
__clone, __dl_seterr, __funcs_on_exit, __map_file, __setxid, or any of
the other internal interfaces and one of these changed, it would not
be an "ABI break".

(Note: most of these are hidden nowadays for dynamic linking purposes,
but for the sake of this thought experiment you could think of static
linking your old .o files to a new libc.a and the same "ABI issues"
would apply, without the benefit of visibility protecting you from
being able to reference internal things.)

> It seems like Musl signed that contract when it first published a
> symbol under _LARGEFILE64_SOURCE or _GNU_SOURCE. When the symbol
> disappeared using one or the other define, then the contract was
> broken.

Our _LARGEFILE64_SOURCE explicitly did not/does not use any symbols.
It's purely macro-based at compile time. If you're going to jump in
and argue about something, at least take the time to check the basics
on something like this.

Rich

Re: [musl] musl -- FFS get your shit together, please

[Markus Wichmann]

I realize this is a flame, and I may be feeding the fire. I hope the OP
reads this when he has cooled off a bit.

Am Mon, Jul 17, 2023 at 01:17:58AM -0500 schrieb Dave Blanchard:
> 2) Did it occur to anyone involved in this project to maybe actually
> organize and COMMENT the system header files, instead of just randomly
> throwing a random assortment of shit into an .H file and calling it
> good? The amount of duplicated, undocumented, assorted crap is pretty
> ridiculous for a project that's supposed to be a FROM SCRATCH libc
> implementation! How about getting it right from the beginning, with a
> clean and organized implementation, instead of starting off with a
> random pile of shit? Even glibc is better organized, for fuck's sake!
>

Most documentation is in the commit comments. As for the organization
being bad, I would dispute that. Everything that ends up in libc is in
src/*/*.c or src/*/$ARCH/*.[csS], with the latter always included, and
overriding any of the former with the same name that may be present.
Dynlinker is in ldso/ and CRT is in crt/.

Meanwhile in glibc, you have the sysdeps directory, and if there is any
organization to it, I haven't found it.

> 3) Why in the hell does musl duplicate/change(!) internal structures
> from Linux kernel headers instead of just #include'ing the damn Linux
> headers (and relevant structures) and be done with it?
>

So musl is independent of the kernel version installed. This has many
advantages, such as being able to build musl without even having a
kernel installed at all. The kernel APIs (syscall/ioctl numbers and
structures) must all be stable anyway, or else violate the kernel's ABI
stability goals.

Some kernel interfaces also conflict with userspace interface. E.g.
struct msghdr is supposed to contain members of type "socklen_t" as per
POSIX, but Linux defines them to contain the length fields as "unsigned
long", which is a different type on 64-bit systems. So musl must define
padding fields to smoothe things over.

Generally, Linux header files are not compatible with userspace
programs. musl's approach is the one favored by most applications and
libraries that actually do anything with the kernel, such as libfuse and
libnl.

> 4) Would it kill you to add in various simple #defines and such in the
> headers which greatly improve compatibility with GNU code, without
> actually requiring any support in the libc library code??
>

Would probably not kill anyone, but being compatible with glibc is
simply not a musl goal. musl aims to be a POSIX implementation. Some
concessions to the real world have been made, beyond that, there's
gcompat. For everything else, there's patches.

> Between the above, plus the 6-7 "musl addon" support packages required
> to be installed alongside to make my Linux system build with musl, at
> this point I have essentially FORKED musl!
>

As is your right as per the license. Where's the problem?

Ciao,
Markus