[musl] Prototypes without implementations

[musl] Prototypes without implementations

[Alyssa Ross]

[-- Attachment #1: Type: text/plain, Size: 1415 bytes --]

<sys/io.h> includes prototypes for iopl() and ioperm(), but not all
architectures provide implementations, because the implementation is
conditionally compiled only if SYS_ioperm is defined.  This means that
on aarch64, musl is providing prototypes without implementations, which
is very surprising to me.

musl provides these prototypes unconditionally since commit
0004ea613ac310daaee30c167112d796db33fa70:

> fix breakage from introducing bits header for sys/io.h
> 
> apparently some other archs have sys/io.h and should not break just
> because they don't have the x86 port io functions. provide a blank
> bits/io.h everywhere for now.

Glibc only provides <sys/io.h> on alpha, ia64, i386, x86_64, of which
musl supports only the latter two.  It used to provide it on arm as
well, with stub implementations (ioperm() returning ENOSYS, inb
returning 0, …), but the header was dropped in Glibc 2.30.  Linux (as of
v6.11) has an ioperm syscall on x86, microblaze, mips, and powerpc, but
on everything but x86, it's just a stub that returns ENOSYS.

Some code in the wild I have found expects that it can use the existence
of <sys/io.h> as a proxy for being able to use inb/outb, etc.  Would it
make sense for musl to match the Glibc behaviour of only providing
sys/io.h on i386 and x86_64?  Regardless, I think that the presense of
unimplemented prototypes ought to be fixed somehow.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: [musl] Prototypes without implementations

[Rich Felker]

On Fri, Oct 25, 2024 at 10:01:37PM +0200, Alyssa Ross wrote:
> <sys/io.h> includes prototypes for iopl() and ioperm(), but not all
> architectures provide implementations, because the implementation is
> conditionally compiled only if SYS_ioperm is defined.  This means that
> on aarch64, musl is providing prototypes without implementations, which
> is very surprising to me.
> 
> musl provides these prototypes unconditionally since commit
> 0004ea613ac310daaee30c167112d796db33fa70:
> 
> > fix breakage from introducing bits header for sys/io.h
> > 
> > apparently some other archs have sys/io.h and should not break just
> > because they don't have the x86 port io functions. provide a blank
> > bits/io.h everywhere for now.
> 
> Glibc only provides <sys/io.h> on alpha, ia64, i386, x86_64, of which
> musl supports only the latter two.  It used to provide it on arm as
> well, with stub implementations (ioperm() returning ENOSYS, inb
> returning 0, …), but the header was dropped in Glibc 2.30.  Linux (as of
> v6.11) has an ioperm syscall on x86, microblaze, mips, and powerpc, but
> on everything but x86, it's just a stub that returns ENOSYS.
> 
> Some code in the wild I have found expects that it can use the existence
> of <sys/io.h> as a proxy for being able to use inb/outb, etc.  Would it
> make sense for musl to match the Glibc behaviour of only providing
> sys/io.h on i386 and x86_64?  Regardless, I think that the presense of
> unimplemented prototypes ought to be fixed somehow.

Generally we aim not to provide different interfaces for different
archs. That principle has only been partly followed here. I'm not sure
if the status quo is preferable, or if we should add iopl/ioperm
functions that just ENOSYS on archs without them, or if we should do
something like you suggest and suppress them on archs that don't
have/need them. But I don't really see a good motivation for the last
option except trying to make badly behaving applications happy..

From the application side, using sys/io.h as proxy for existence of
inb/outb is just wrong. If they want to know if inb/outb exist, they
can probe those specifically: including the header and attempting to
compile and link a test program that uses the interface.

Rich

Re: [musl] Prototypes without implementations

[Alyssa Ross]

[-- Attachment #1: Type: text/plain, Size: 2513 bytes --]

Rich Felker <dalias@libc.org> writes:

> On Fri, Oct 25, 2024 at 10:01:37PM +0200, Alyssa Ross wrote:
>> <sys/io.h> includes prototypes for iopl() and ioperm(), but not all
>> architectures provide implementations, because the implementation is
>> conditionally compiled only if SYS_ioperm is defined.  This means that
>> on aarch64, musl is providing prototypes without implementations, which
>> is very surprising to me.
>> 
>> musl provides these prototypes unconditionally since commit
>> 0004ea613ac310daaee30c167112d796db33fa70:
>> 
>> > fix breakage from introducing bits header for sys/io.h
>> > 
>> > apparently some other archs have sys/io.h and should not break just
>> > because they don't have the x86 port io functions. provide a blank
>> > bits/io.h everywhere for now.
>> 
>> Glibc only provides <sys/io.h> on alpha, ia64, i386, x86_64, of which
>> musl supports only the latter two.  It used to provide it on arm as
>> well, with stub implementations (ioperm() returning ENOSYS, inb
>> returning 0, …), but the header was dropped in Glibc 2.30.  Linux (as of
>> v6.11) has an ioperm syscall on x86, microblaze, mips, and powerpc, but
>> on everything but x86, it's just a stub that returns ENOSYS.
>> 
>> Some code in the wild I have found expects that it can use the existence
>> of <sys/io.h> as a proxy for being able to use inb/outb, etc.  Would it
>> make sense for musl to match the Glibc behaviour of only providing
>> sys/io.h on i386 and x86_64?  Regardless, I think that the presense of
>> unimplemented prototypes ought to be fixed somehow.
>
> Generally we aim not to provide different interfaces for different
> archs. That principle has only been partly followed here. I'm not sure
> if the status quo is preferable, or if we should add iopl/ioperm
> functions that just ENOSYS on archs without them, or if we should do
> something like you suggest and suppress them on archs that don't
> have/need them. But I don't really see a good motivation for the last
> option except trying to make badly behaving applications happy..

I think ENOSYS is probably the way to go, especially since (via the
kernel) that's already happening on some architectures.

> From the application side, using sys/io.h as proxy for existence of
> inb/outb is just wrong. If they want to know if inb/outb exist, they
> can probe those specifically: including the header and attempting to
> compile and link a test program that uses the interface.

Agreed.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re[2]: [musl] Prototypes without implementations

[Laurent Bercot]

>I think ENOSYS is probably the way to go, especially since (via the
>kernel) that's already happening on some architectures.

  Note that providing ENOSYS implementations makes cross-compiling 
harder.

  When the libc provides no implementation, a sysdep test can compile
and link a program using the API, and conclude that the functionality
doesn't exist when the link fails. This works when cross-compiling.

  When the libc provides an ENOSYS implementation, the link will succeed,
and a sysdep test needs to *run* a program to check that the
functionality works correctly. This is not possible when 
cross-compiling.

  I'd rather have libcs omit stub implementations entirely, so that
applications can test for functionality without having to run anything.
Stub implementations make tests and integration of replacement
implementations more difficult. (And not even only for cross builds.
Looking at you, musl's utmp functions.)

--
  Laurent

Re: [musl] Prototypes without implementations

[Thorsten Glaser]

On Sat, 26 Oct 2024, Laurent Bercot wrote:

> I'd rather have libcs omit stub implementations entirely, so that
> applications can test for functionality without having to run anything.
> Stub implementations make tests and integration of replacement

Fully agreed. It’s easy to make a compile+link test; my autoconf
replacement depends on it (and does that right) but runtime is a
whole other issue.

I don’t even mind about headers declaring functions that aren’t
there because only a fool tool would look at headers only… though
omitting them when it’s easy to know (e.g. via cpp predefs) it’s
not there is of course better.

bye,
//mirabilos
-- 
11:56⎜«liwakura:#!/bin/mksh» also, i wanted to add mksh to my own distro │
i was disappointed that there is no makefile │ but somehow the Build.sh is
the least painful built system i've ever seen │ honours CC, {CPP,C,LD}FLAGS
properly │ looks cleary like done by someone who knows what they are doing

Re: [musl] Prototypes without implementations

[Rich Felker]

On Sat, Oct 26, 2024 at 03:57:15AM +0200, Thorsten Glaser wrote:
> On Sat, 26 Oct 2024, Laurent Bercot wrote:
> 
> > I'd rather have libcs omit stub implementations entirely, so that
> > applications can test for functionality without having to run anything.
> > Stub implementations make tests and integration of replacement
> 
> Fully agreed. It’s easy to make a compile+link test; my autoconf
> replacement depends on it (and does that right) but runtime is a
> whole other issue.
> 
> I don’t even mind about headers declaring functions that aren’t
> there because only a fool tool would look at headers only… though
> omitting them when it’s easy to know (e.g. via cpp predefs) it’s
> not there is of course better.

My leaning would be not to do the ENOSYS thing, not only for this
reason, but because doing so might preclude having them with a
different signature on a different arch, if that were later needed.
I'm not sure how realistic that happening is, but if it's plausible,
it might be a good argument for moving the prototypes back to the
x86-specific bits headers like I think Alyssa was proposing. OTOH it
looks like some other archs already have these syscalls, and that was
the motivation for the original change.. so more investigation is
probably needed.

Rich

Re: [musl] Prototypes without implementations

[Markus Wichmann]

Am Sat, Oct 26, 2024 at 01:21:23AM +0000 schrieb Laurent Bercot:
>  When the libc provides no implementation, a sysdep test can compile
> and link a program using the API, and conclude that the functionality
> doesn't exist when the link fails. This works when cross-compiling.
>
>  When the libc provides an ENOSYS implementation, the link will succeed,
> and a sysdep test needs to *run* a program to check that the
> functionality works correctly. This is not possible when cross-compiling.
>

I don't understand the remainder of the thread as it is now, because
normally we consider the above behavior to be insanity. Just because a
function exists in the lib does not mean it will succeed at run-time.
This is already the case with functions like getrandom() or pselect().

Even if you could run run-time tests, just because it succeeds at
configure time does not mean it succeeds at any later date. And
conversely, just because it fails at configure time does not mean it
cannot succeed. Writing your software in the above manner is therefore
not sensible. Especially since the functions talked about here are
system calls that on some architectures have been stubbed out in the
kernel, so the run-time behavior depends on run-time kernel version.

But maybe the system calls talked about here are ultra-special in some
way.

Ciao,
Markus

Re[2]: [musl] Prototypes without implementations

[Laurent Bercot]

>I don't understand the remainder of the thread as it is now, because
>normally we consider the above behavior to be insanity.

  Who's "we"?


>  Just because a
>function exists in the lib does not mean it will succeed at run-time.
>This is already the case with functions like getrandom() or pselect().

  getrandom() is part of the problem, yes - a function that you cannot
test for at link time and that you have to make depend on a kernel
version in order to use. But because the problem already exists doesn't
mean it's a good idea to add to it.

  pselect() is posix, it has a specification. If your pselect() isn't
doing what it's supposed to, then your implementation is nonconformant.
Yes, nonconformant systems are ubiquitous too, that's a problem too,
and that's why we test stuff, but standards are still a good thing.
strlen() works everywhere and you don't need to test for it. (Yes,
it's a low bar. And it's not true for strnlen(). Don't get me started.)


>Even if you could run run-time tests, just because it succeeds at
>configure time does not mean it succeeds at any later date. And
>conversely, just because it fails at configure time does not mean it
>cannot succeed.

  The point isn't to test for "success" or "failure" (unless you've
found a solution to the halting problem, in which case I'm definitely
interested). The point is to have a decent heuristic for whether a
given function, identified by its name and signature, exists in your
system and has a reasonable chance of doing what you think it will do.
Is it perfect? No. Do we have a better way of writing portable software
that does not involve starting every single executable with a battery
of tests? Also no.


>  Writing your software in the above manner is therefore
>not sensible.

  Every single project using a configure script such as one created by
GNU autoconf, or a build system generator such as cmake or meson, is
therefore not sensible. That's fair, but maybe we should still try to
avoid breaking them?


>  Especially since the functions talked about here are
>system calls that on some architectures have been stubbed out in the
>kernel, so the run-time behavior depends on run-time kernel version.

  That's why, unfortunately, some functionality has to be documented as
depending on a run-time kernel version. It would be nice if we could
minimize the occurrences of having to document a dependency to a run-
time libc version, too.

--
  Laurent

Re: [musl] Prototypes without implementations

[Markus Wichmann]

Am Sat, Oct 26, 2024 at 10:28:38AM +0000 schrieb Laurent Bercot:
>  Who's "we"?

Me and my sockpuppet. I was under the impression that this is the
general consensus among regulars on this list, as it shone through on
past discussions, but maybe I'm mistaken.

> >  Just because a
> > function exists in the lib does not mean it will succeed at run-time.
> > This is already the case with functions like getrandom() or pselect().
>
>  getrandom() is part of the problem, yes - a function that you cannot
> test for at link time and that you have to make depend on a kernel
> version in order to use. But because the problem already exists doesn't
> mean it's a good idea to add to it.
>

You call the function, and if it fails, you fall back to another method
(which includes attempting to seed a PRNG with getauxval(AT_RANDOM) or
trying to open /dev/urandom or the like). Does it matter if it fails
because the kernel is too old or just having a bad day?

I would never check the kernel version before just calling the kernel
interface.

>  pselect() is posix, it has a specification. If your pselect() isn't
> doing what it's supposed to, then your implementation is nonconformant.

Failing with ENOSYS and no side effects and no wait time is conformant
as well. In the case of pselect(), it is probably the better behavior of
libc as well if the kernel doesn't provide the necessary syscall,
because the workaround if pselect() doesn't exist is so invasive.
Indeed, if you have an application that can deal with pselect() not
being available, you likely have an application that doesn't need
pselect() at all. (The workaround could be the self-pipe trick, which
obviates the need for pselect() completely).

> > Even if you could run run-time tests, just because it succeeds at
> > configure time does not mean it succeeds at any later date. And
> > conversely, just because it fails at configure time does not mean it
> > cannot succeed.
>
>  The point isn't to test for "success" or "failure" (unless you've
> found a solution to the halting problem, in which case I'm definitely
> interested).

The halting problem is precisely the reason why run-time tests are a bad
idea even not cross-compiling. If the function under test fails with
ENOSYS, you still have no idea if that is because of a stubbed out
implementation.

> The point is to have a decent heuristic for whether a
> given function, identified by its name and signature, exists in your
> system and has a reasonable chance of doing what you think it will do.

See above. Failing with ENOSYS and no side effects is a perfectly
conforming implementation of most POSIX functions.

> Is it perfect? No. Do we have a better way of writing portable software
> that does not involve starting every single executable with a battery
> of tests? Also no.
>

If your binary links against musl, it does do precisely those tests. Not
necessarily at the start of the runtime, but inside many functions, and
in many of those never saves the state about what is or is not possible
in the run-time kernel, so the tests are re-run everytime the function
is called. I am talking about things such as all of the socket functions
on socketcall architectures attempting to do the newer syscall directly
and falling back to the socketcall mechanism if that fails. Or
clock_gettime(), if it cannot be implemented in VDSO, trying first the
64-bit syscall, then the 32-bit one.

To me, not much difference exists between a function failing with ENOSYS
and it failing for any other reason. In the end, you still simply have a
function failure at run-time and need to cope with it. And it could
start working at any point in the future. Maybe not in the same process
lifetime, but maybe in another one.

> >  Writing your software in the above manner is therefore
> > not sensible.
>
>  Every single project using a configure script such as one created by
> GNU autoconf, or a build system generator such as cmake or meson, is
> therefore not sensible. That's fair, but maybe we should still try to
> avoid breaking them?
>

It is possible, though not effortless, to write sensible software using
any of the aforementioned build systems. I have done so with cmake at
least. It does require a shift in perspective, though, to not assuming a
function will be successful just because it exists.

Ciao,
Markus

Re: [musl] Prototypes without implementations

[Thorsten Glaser]

On Sat, 26 Oct 2024, Markus Wichmann wrote:

>See above. Failing with ENOSYS and no side effects is a perfectly
>conforming implementation of most POSIX functions.

Insisting on this ivory tower opinion will just make people
mark your software as buggy and move on.

For most functions, it’s reasonable to check for presence,
and if so, enable codepaths that do:

	if (pselect(…) == -1) {
		if (errno == EINTR) {
			// special handling
		}
		err(1, "pselect");
	}

The vast majority of software, OSS and not, works this way.
If a basic function in a standard is there it better function.

Linux’ getrandom() is an offensive exception to that and better
stay the only one.

bye,
//mirabilos
-- 
22:20⎜<asarch> The crazy that persists in his craziness becomes a master
22:21⎜<asarch> And the distance between the craziness and geniality is
only measured by the success 18:35⎜<asarch> "Psychotics are consistently
inconsistent. The essence of sanity is to be inconsistently inconsistent

Re: [musl] Prototypes without implementations

[Rich Felker]

On Sun, Oct 27, 2024 at 12:08:25AM +0200, Thorsten Glaser wrote:
> On Sat, 26 Oct 2024, Markus Wichmann wrote:
> 
> >See above. Failing with ENOSYS and no side effects is a perfectly
> >conforming implementation of most POSIX functions.
> 
> Insisting on this ivory tower opinion will just make people
> mark your software as buggy and move on.
> 
> For most functions, it’s reasonable to check for presence,
> and if so, enable codepaths that do:
> 
> 	if (pselect(…) == -1) {
> 		if (errno == EINTR) {
> 			// special handling
> 		}
> 		err(1, "pselect");
> 	}
> 
> The vast majority of software, OSS and not, works this way.
> If a basic function in a standard is there it better function.
> 
> Linux’ getrandom() is an offensive exception to that and better
> stay the only one.

This thread has veered really off-topic. I haven't read it in full,
but based on what it seems to be about, a few remarks:

- I don't buy that ENOSYS buys you a license to omit parts of POSIX
  you don't want to implement. Yes there's license to have additional
  errors not defined in the standard, but the only reasonable
  interpretation is that these should be for concrete conditions that
  can be remedied. At the very least, an interface always failing is
  bad QoI. As an example, I don't claim running on a nommu system,
  where fork() will necessarily ENOSYS, gives a POSIX-conforming
  environment; it's a POSIX subset. IIRC part of the rationale text
  addresses that and agrees with my interpretation here.

- pselect ENOSYS? Come on. This is 2024, almost 2025, not 1999.
  (Unless I'm misremembering it was added in POSIX 2001.)

- Back to extension interfaces (or very recent standards) which might
  or might not be available at runtime: Whether availability of a
  particular such interface is reasonable to defer to runtime really
  depends on what the interface is and what it's being used for.
  Trying to make broad sweeping generalizations does not seem helpful,
  and particularly, does not seem relevant to the ioperm/iopl topic.

- That said, yes, whether an interface is available at the library
  level, and whether it works on the machine you happen to be running
  on later, can differ, and this is something where "musl policy"
  (using the term very loosely, in the sense of what we recommend to
  projects when trying to get something fixed) is that, unless the
  standard says the interface and functionality is always present, you
  should both check for it at configure-time/build-time, and be
  prepared at runtime for the possibility that it might not be
  supported or might not support all features on the running system. I
  would expect software using iopl/ioperm already needs to be prepared
  for the possibility that they could fail for permissions reasons,
  and should avoid calling them anyway unless it knows it needs to
  perform userspace hardware IO. So I'm not sure how this matters or
  how an ENOSYS here would be any different from EPERM -- it's just
  another reason you can't do IO as a user process.

Can we use the remainder of this thread for figuring out the real
status of this (what's with non-x86 archs that have the syscalls? are
they used? what about archs that don't use them? do they have you mmap
a device instead?) and whether there's some action that should be
taken, rather than rehashing broad philosophy? :)

Rich

Re: [musl] Prototypes without implementations

[Alyssa Ross]

[-- Attachment #1: Type: text/plain, Size: 1691 bytes --]

Rich Felker <dalias@libc.org> writes:

> Can we use the remainder of this thread for figuring out the real
> status of this (what's with non-x86 archs that have the syscalls? are
> they used? what about archs that don't use them? do they have you mmap
> a device instead?) and whether there's some action that should be
> taken, rather than rehashing broad philosophy? :)

On all the other architectures where musl has SYS_ioperm defined, it's
mapped to sys_ni_syscall (ENOSYS), and has been at least since git was
introduced.

Glibc provides a fallback that uses /dev/mem on alpha, but no other
architecture.  In the past, it also provided an implementation for arm,
which was removed in the following commit:

	commit 6b33f373c7b9199e00ba5fbafd94ac9bfb4337b1
	Author: Florian Weimer <fweimer@redhat.com>
	Date:   Wed May 29 16:53:09 2019 +0200
	
	    arm: Remove ioperm/iopl/inb/inw/inl/outb/outw/outl support
	    
	    Linux only supports the required ISA sysctls on StrongARM devices,
	    which are armv4 and no longer tested during glibc development
	    and probably bit-rotted by this point.  (No reported test results,
	    and the last discussion of armv4 support was in the glibc 2.19
	    release notes.)

The functions remain on arm, but they are just ENOSYS stubs.

Does any of that make it any clearer what should be done in musl to
solve the prototypes without implementations problem?

I guess it does make sense for musl to define __SYS_ioperm on
microblaze, mips, powerpc and powerpc64 — it's in the kernel syscall
table for those architectures, even though as far as I can tell it has
never done anything other than return ENOSYS.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: [musl] Prototypes without implementations

[enh]

[-- Attachment #1: Type: text/plain, Size: 3305 bytes --]

fwiw, this came up for Android recently... we'd never had the header before
but the
https://blog.chromium.org/2024/06/building-faster-smarter-chromebook.html
work meant we needed new x86-only stuff for the first time in a decade.

we basically went with this:

#if defined(__NR_iopl)
static __inline int ioperm(unsigned long __from, unsigned long __n, int
__enabled) {
  return syscall(__NR_ioperm, __from, __n, __enabled);
}
#endif

(which answers the "some architectures" part, even though for Android
that's only x86/x86-64.)

here's my commit message with more rationale for why i made them static
inlines:

"""
Add <sys/io.h>.

This is unusual in being inline-only, but these are (a) trivial functions,
(b) ancient functions, and (c) unusable by apps. Plus they're arriving just
too late for API 35, which definitely influenced by "this is silly"
decision! They're also x86/x86-64 only, though that's a neutral argument
because it would mean almost no-one will be affected no matter what choice
we make here.

There is an argument for going the usual "callers should just have their
own header-only implementation that they `-include` on the compiler command
line" route, but the x86/x86-64 I/O port stuff isn't going away just yet,
and LTP was also already working around the absence of these.
"""

https://android-review.googlesource.com/c/platform/bionic/+/3135242

On Wed, Oct 30, 2024 at 5:29 AM Alyssa Ross <hi@alyssa.is> wrote:

> Rich Felker <dalias@libc.org> writes:
>
> > Can we use the remainder of this thread for figuring out the real
> > status of this (what's with non-x86 archs that have the syscalls? are
> > they used? what about archs that don't use them? do they have you mmap
> > a device instead?) and whether there's some action that should be
> > taken, rather than rehashing broad philosophy? :)
>
> On all the other architectures where musl has SYS_ioperm defined, it's
> mapped to sys_ni_syscall (ENOSYS), and has been at least since git was
> introduced.
>
> Glibc provides a fallback that uses /dev/mem on alpha, but no other
> architecture.  In the past, it also provided an implementation for arm,
> which was removed in the following commit:
>
>         commit 6b33f373c7b9199e00ba5fbafd94ac9bfb4337b1
>         Author: Florian Weimer <fweimer@redhat.com>
>         Date:   Wed May 29 16:53:09 2019 +0200
>
>             arm: Remove ioperm/iopl/inb/inw/inl/outb/outw/outl support
>
>             Linux only supports the required ISA sysctls on StrongARM
> devices,
>             which are armv4 and no longer tested during glibc development
>             and probably bit-rotted by this point.  (No reported test
> results,
>             and the last discussion of armv4 support was in the glibc 2.19
>             release notes.)
>
> The functions remain on arm, but they are just ENOSYS stubs.
>
> Does any of that make it any clearer what should be done in musl to
> solve the prototypes without implementations problem?
>
> I guess it does make sense for musl to define __SYS_ioperm on
> microblaze, mips, powerpc and powerpc64 — it's in the kernel syscall
> table for those architectures, even though as far as I can tell it has
> never done anything other than return ENOSYS.
>

[-- Attachment #2: Type: text/html, Size: 4281 bytes --]

Re[2]: [musl] Prototypes without implementations

[Laurent Bercot]

>You call the function, and if it fails, you fall back to another method
>(which includes attempting to seed a PRNG with getauxval(AT_RANDOM) or
>trying to open /dev/urandom or the like). Does it matter if it fails
>because the kernel is too old or just having a bad day?

  That's exactly what I'm doing. That's why I have a C programming 
library
that everything I write depends on: to wrap these excuses for interfaces
into something that an application can reasonably use. It was annoying 
to
write. And the part of the library that generates random numbers was
*especially* annoying, thanks in no small part to that getrandom()
behaviour. I needed 4 attempts to make it work in a satisfying way.

  And you're saying everyone should do that, for *every* interface 
exposed
by the system? That is putting very unreasonable expectations on
application writers, and making C even harder than people like to say
it is.


>Failing with ENOSYS and no side effects and no wait time is conformant
>as well.

  Yeah. You're right. And my point, shared with other people in this
thread, is that it's a bad thing, and we should avoid having these stub
functions wherever possible.


>  (The workaround could be the self-pipe trick, which
>obviates the need for pselect() completely).

  (Using a self-pipe is also better than pselect() for other reasons:
it allows you to handle signals as regular events and process them in
normal execution context rather than interruption context. So indeed
pselect() is never needed, and the self-pipe trick - also implementable
via signalfd() on Linux - should be the regular pattern, not a
workaround.)


>If the function under test fails with
>ENOSYS, you still have no idea if that is because of a stubbed out
>implementation.

  Exactly. So let's not add to the list of functions that do that.


>If your binary links against musl, it does do precisely those tests. Not
>necessarily at the start of the runtime, but inside many functions

  It is a very deliberate, very conscious choice in musl, and the
explicit goal is that binaries linked against musl should run on as many
kernel versions as possible, independently of the kernel version musl
was built with.

  It is a reasonable design goal for a libc to avoid tying userspace to
specific kernel features. If there is ever a place to test, live, 
whether
a system call is supported, and fall back to a different system call, or
sequence of, if it isn't, a libc would be that place; and I appreciate
that musl goes to these lengths to ensure independence from the kernel.

  It is not, however, a reasonable expectation that *applications* should
do so with the system they're running on. Applications should be able
to rely on the system - that's why specifications exist. The ecosystem
is already in bad enough shape that extensive build-time configuration
(that works badly, and that nobody likes) is needed for most software;
claiming that the sensible way of writing applications is run-time tests
for system independence is abandoning any pretense of practicality.


>It is possible, though not effortless, to write sensible software using
>any of the aforementioned build systems.

  Kudos to you for making it work. Now you should know as well as I do 
that
between writing sensible software with effort, and writing half-baked
software with less effort, most of the ecosystem, including incredibly
popular projects, will choose the latter. Let's not break them even more
than they already are, please.

--
  Laurent

Re: [musl] Prototypes without implementations

[Yao Zi]

On Sat, Oct 26, 2024 at 03:22:25PM +0200, Markus Wichmann wrote:
> See above. Failing with ENOSYS and no side effects is a perfectly
> conforming implementation of most POSIX functions.

Yes, failing with ENOSYS could be conformant. But ioperm() and iopl()
aren't really a POSIX thing. Why make more pitfalls instead of easing
everyone's life?

For stuff like ioctl(), I do agree with that standing with POSIX is
correct, but in this case we aren't getting anything valuable with a
stub implementation. And I think it is not a hard work to omit both the
implementation and declaration on archs without these syscalls.

Thanks,
Yao Zi

Re: [musl] Prototypes without implementations

[Robert Clausecker]

Am Sat, Oct 26, 2024 at 01:21:23AM +0000 schrieb Laurent Bercot:
> > I think ENOSYS is probably the way to go, especially since (via the
> > kernel) that's already happening on some architectures.
> 
>  Note that providing ENOSYS implementations makes cross-compiling harder.
> 
>  When the libc provides no implementation, a sysdep test can compile
> and link a program using the API, and conclude that the functionality
> doesn't exist when the link fails. This works when cross-compiling.
> 
>  When the libc provides an ENOSYS implementation, the link will succeed,
> and a sysdep test needs to *run* a program to check that the
> functionality works correctly. This is not possible when cross-compiling.
> 
>  I'd rather have libcs omit stub implementations entirely, so that
> applications can test for functionality without having to run anything.
> Stub implementations make tests and integration of replacement
> implementations more difficult. (And not even only for cross builds.
> Looking at you, musl's utmp functions.)

This entire debate happened multiple times with commercial UNIX vendors
some thirty years ago, who would declare all sorts of functions in their
headers and define all sort of syscalls in their libc, only to then not
actually implement them (they would return ENOSYS or ENOTSUPP) or only
implement them if you bought some extra option.

This was almost universally agreed to be a great annoyance to authors of
portable software and the consensus was to avoid this stuff as much as
possible.  They changed things so that e.g. if you didn't buy the inet
package, you just wouldn't have the header files for inet stuff installed,
making it so that software could very easily detect if the feature was
available or not at build time.

Please don't bring back the mistakes of the past.  If a function is not
present, don't declare it.  Failing that, don't define it.  But do not
define it and then just have it fail at runtime.  That's a really bad
move.

Yours,
Robert Clausecker

-- 
()  ascii ribbon campaign - for an encoding-agnostic world
/\  - against html email  - against proprietary attachments