mailing list of musl libc
 help / color / mirror / code / Atom feed
* [musl] Re: [PATCH] memmem.3: Added list of known systems where this is available
       [not found]     ` <CACKs7VAQsxDc2XrsAYSEbA9eqRnLHuXykVmNTit+tCFMyGLCwA@mail.gmail.com>
@ 2022-11-23 13:16       ` Alejandro Colomar
  2022-11-23 14:55         ` Jeffrey Walton
                           ` (2 more replies)
  0 siblings, 3 replies; 6+ messages in thread
From: Alejandro Colomar @ 2022-11-23 13:16 UTC (permalink / raw)
  To: Stefan Puiu, Guillem Jover, Andrew Clayton, linux-man,
	Michael Kerrisk, Alejandro Colomar, Brian Inglis, Rich Felker,
	musl


[-- Attachment #1.1: Type: text/plain, Size: 5547 bytes --]

Hi all!

On 11/23/22 13:52, Stefan Puiu wrote:
> Hi,
> 
> On Wed, Nov 23, 2022 at 1:39 AM Guillem Jover <guillem@hadrons.org> wrote:
>>
>> Hi!
>>
>> On Thu, 2022-11-10 at 12:36:47 +0100, Alejandro Colomar wrote:
>>> On 11/10/22 01:13, Andrew Clayton wrote:
>>>> While looking at which systems provide memmem(3) I have been able to
>>>> discern the following:
>>>>
>>>>     musl libc since v0.9.7
>>>>     bionic since Android 9
>>>>
>>>>     FreeBSD since 6.0
>>>>     OpenBSD since 5.4
>>>>     NetBSD
>>>>     macOS
>>>>     Illumos
>>
>>>> For macOS and Illumos I checked the memmem(3) man page on those systems.
>>>> For the rest there are links below to on-line man pages or commit logs.
>>
>>>> +FreeBSD 6.0, OpenBSD 5.4, NetBSD, macOS & Illumos.
>>
>>> For the commit message, it's interesting to note macOS and Bionic, for
>>> speleology purposes.  However, I'm opposed to adding them to the page itself
>>> because of the following:
>>>
>>> -  macOS is not free software.  I refuse to reference nonfree software on
>>> this project.
>>
>> I understand where you are coming from,

:-)

>> and from a full system PoV,
>> it's very true that macOS is non-free. But the lower layers of its
>> stack are free software (at least according to the FSF and OSI), such
>> as its kernel (Darwin), or its libc (where memmem() is implemented):
>>
>>    https://opensource.apple.com/source/Libc/Libc-1439.40.11/
>>
>> among others.

Hmmm.  Fair enough.

>>
>> Similar with Solaris and Illumos (but with the CDDL, MIT, BSD), AFAIUI.
>>
>>> -  Android is not a real Unix system, in that you can't even program in C in
>>> there, unless you're Google or have hacked your system.  It's not friendly
>>> to us programmers, so we don't need to be friendly to it.  I don't want to
>>> be cluttering the pages with information that is irrelevant to normal users.
>>
>> I'm assuming bionic is being used in some of the Android free
>> alternatives too, but then I'm not sure how usable for programming those
>> are either. And, well musl libc is not a real Unix system you can program
>> against either. :)

The difference is only that bionic is not in use in useful systems (AFAIK).  :)

So we have to do some decission here (and also about newlib, as reported by Brian).

>>
>> In any case I also find it useful to have this kind of portability
>> information when deciding what to use in code.

And I must admit it's also useful to me (this all started because Andrew and I 
had to use memmem(3) at a project where macOS compatibility is relevant --not 
critical, but relevant--).

>> But can understand the
>> reluctance to include such references, at least if thought out as the
>> entire system. Perhaps thinking about these merely at their kernel or
>> libc level would make including information about some of them more
>> palatable, given that standalone they are free software? So perhaps
>> an option is to instead refer to their specific components, say
>> "bionic libc X.Y", "Apple Libc M.N.O" or similar.

Yep, that's more palatable :)

I think I'd accept a patch stating Apple Libc version for memmem(3).

> 
> Not sure if it's the job of Linux man-pages to document when other
> OSes started supporting certain APIs. That info has to be maintained,
> updated etc. People can always read the man pages of other systems,
> right? Maybe it's worth only pointing out when an interface is
> Linux-only, or the Linux implementation diverges significantly.

The good thing is that in most cases, it's either something in POSIX (for which 
I gon't care at all if Apple Libc or another-weirdo-libc decide to not support 
it), or it's a Linux-only function.

So, there are few functions or syscalls that are generally available but are not 
in POSIX, and it might be interesting to document that they're effectively as 
portable as anything POSIX.  Maybe even POSIX editors read this and decide to 
add it.

In those cases, we still need to decide what we care about or not.

Musl libc is definitely a first-class citizen these days in Linux distributions. 
  I would start documenting them in the project at large if someone from musl 
provides patches (I discussed this some time ago, but can't remember with who).
Rich, if you would like to discuss about this, we can have some chat.

> 
> For musl and other libcs, I think the man pages already document some
> instances where their behavior diverges from glibc.

As said, for musl, I'd document them officially, if there's anyone interested 
enough to send patches.

For other libcs, we have to decide if they're important enough, and probably 
decide on a case-by-case basis.

Michael tried to have some decent coverage of non-GNU/Linux systems, both in the 
man-pages and in his TLPI book.  It's a useful thing.  So much that sometimes 
you don't even need to read other systems' pages at all to know how portable is 
a GNU/Linux function.

So, I'd like to get opinions on interest about documenting details about:

- newlib (I never heard about it before; is it a widespread thing? do you think 
it's useful?)
- Apple Libc (I still don't like it, but I must admit that it's relevant, and 
being open source, it's more palatable)
- bionic (does anyone know if that's useful at all for anyone at all?)

Support for those wouldn't go as far as musl or glibc.  For now it would only be 
for memmem(3).


Cheers,

Alex

-- 
<http://www.alejandro-colomar.es/>

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [musl] Re: [PATCH] memmem.3: Added list of known systems where this is available
  2022-11-23 13:16       ` [musl] Re: [PATCH] memmem.3: Added list of known systems where this is available Alejandro Colomar
@ 2022-11-23 14:55         ` Jeffrey Walton
  2022-11-23 15:11           ` Alejandro Colomar
  2022-11-23 15:33         ` [musl] " Shiz
  2022-11-24 18:57         ` [musl] " Stefan Puiu
  2 siblings, 1 reply; 6+ messages in thread
From: Jeffrey Walton @ 2022-11-23 14:55 UTC (permalink / raw)
  To: musl
  Cc: Stefan Puiu, Guillem Jover, Andrew Clayton, linux-man,
	Michael Kerrisk, Alejandro Colomar, Brian Inglis, Rich Felker

On Wed, Nov 23, 2022 at 9:29 AM Alejandro Colomar
<alx.manpages@gmail.com> wrote:
>  ...
> >> In any case I also find it useful to have this kind of portability
> >> information when deciding what to use in code.
>
> And I must admit it's also useful to me (this all started because Andrew and I
> had to use memmem(3) at a project where macOS compatibility is relevant --not
> critical, but relevant--).

If you are a die-hard free software person using GNU gear, then Gnulib
provides memmem. There's no need to worry about availability or
portability courtesy of Gnulib. See
https://www.gnu.org/software/gnulib/manual/html_node/memmem.html .

Jeff

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [musl] Re: [PATCH] memmem.3: Added list of known systems where this is available
  2022-11-23 14:55         ` Jeffrey Walton
@ 2022-11-23 15:11           ` Alejandro Colomar
  0 siblings, 0 replies; 6+ messages in thread
From: Alejandro Colomar @ 2022-11-23 15:11 UTC (permalink / raw)
  To: noloader, musl
  Cc: Stefan Puiu, Guillem Jover, Andrew Clayton, linux-man,
	Michael Kerrisk, Alejandro Colomar, Brian Inglis, Rich Felker


[-- Attachment #1.1: Type: text/plain, Size: 1778 bytes --]

Hi Jeffrey,

On 11/23/22 15:55, Jeffrey Walton wrote:
> On Wed, Nov 23, 2022 at 9:29 AM Alejandro Colomar
> <alx.manpages@gmail.com> wrote:
>>   ...
>>>> In any case I also find it useful to have this kind of portability
>>>> information when deciding what to use in code.
>>
>> And I must admit it's also useful to me (this all started because Andrew and I
>> had to use memmem(3) at a project where macOS compatibility is relevant --not
>> critical, but relevant--).
> 
> If you are a die-hard free software person using GNU gear, then Gnulib
> provides memmem. There's no need to worry about availability or
> portability courtesy of Gnulib. See
> https://www.gnu.org/software/gnulib/manual/html_node/memmem.html .

Thanks!

However, for this project I was talking about, it's not an option; such a 
dependency would not be accepted.

BTW, personally, I always found very confusing the usage of Gnulib compared to 
normal packaged libraries.  Maybe it's just me; don't know.  It also forces you 
to use GNU autotools, which I don't like at all.  I prefer the approach of 
libbsd, which just provides a couple of pc(5) files to allow using as a library 
or as an overlay over the system libc, and after that you're fine with whatever 
build system you prefer.  I know it has some issues, such as 
<https://gitlab.freedesktop.org/libbsd/libbsd/-/issues/5>, which may be the 
reason Gnulib works that way, I don't know.

Guillem, do you think that issue with libbsd and <queue.h> can be fixed?  Or is 
it an inherent issue of the way the overlay works?  Maybe it would be 
interesting to fix it, as a proof of concept that something like Gnulib could be 
implemented in that way.

Cheers,

Alex

-- 
<http://www.alejandro-colomar.es/>

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [musl] [PATCH] memmem.3: Added list of known systems where this is available
  2022-11-23 13:16       ` [musl] Re: [PATCH] memmem.3: Added list of known systems where this is available Alejandro Colomar
  2022-11-23 14:55         ` Jeffrey Walton
@ 2022-11-23 15:33         ` Shiz
  2022-11-30 19:39           ` enh
  2022-11-24 18:57         ` [musl] " Stefan Puiu
  2 siblings, 1 reply; 6+ messages in thread
From: Shiz @ 2022-11-23 15:33 UTC (permalink / raw)
  To: musl; +Cc: alx.manpages

Hi there,

> On 23 Nov Reiwa 4, at 14:16, Alejandro Colomar <alx.manpages@gmail.com> wrote:
> 
>>>> -  Android is not a real Unix system, in that you can't even program in C in
>>>> there, unless you're Google or have hacked your system.  It's not friendly
>>>> to us programmers, so we don't need to be friendly to it.  I don't want to
>>>> be cluttering the pages with information that is irrelevant to normal users.
>>> 
>>> I'm assuming bionic is being used in some of the Android free
>>> alternatives too, but then I'm not sure how usable for programming those
>>> are either. And, well musl libc is not a real Unix system you can program
>>> against either. :)
> 
> The difference is only that bionic is not in use in useful systems (AFAIK).  :)
> 
> So we have to do some decission here (and also about newlib, as reported by Brian).

I’d like to chime in here and note that, valuations of usefulness aside, it *is*
possible to develop in C against an Android device, without needing any hacks,
using the NDK[1]. As someone who has occasionally worked on big codebases written
in C and C++ for Android devices, I would argue there’s value to be had there. :-)

> Cheers,
> 
> Alex

- Shiz

[1]: https://developer.android.com/ndk

^ permalink raw reply	[flat|nested] 6+ messages in thread

* [musl] Re: [PATCH] memmem.3: Added list of known systems where this is available
  2022-11-23 13:16       ` [musl] Re: [PATCH] memmem.3: Added list of known systems where this is available Alejandro Colomar
  2022-11-23 14:55         ` Jeffrey Walton
  2022-11-23 15:33         ` [musl] " Shiz
@ 2022-11-24 18:57         ` Stefan Puiu
  2 siblings, 0 replies; 6+ messages in thread
From: Stefan Puiu @ 2022-11-24 18:57 UTC (permalink / raw)
  To: Alejandro Colomar
  Cc: Guillem Jover, Andrew Clayton, linux-man, Michael Kerrisk,
	Alejandro Colomar, Brian Inglis, Rich Felker, musl

Hi,

On Wed, Nov 23, 2022 at 3:16 PM Alejandro Colomar
<alx.manpages@gmail.com> wrote:
>
> Hi all!
>
[... snipped ...]
> >
> > Not sure if it's the job of Linux man-pages to document when other
> > OSes started supporting certain APIs. That info has to be maintained,
> > updated etc. People can always read the man pages of other systems,
> > right? Maybe it's worth only pointing out when an interface is
> > Linux-only, or the Linux implementation diverges significantly.
>
> The good thing is that in most cases, it's either something in POSIX (for which
> I gon't care at all if Apple Libc or another-weirdo-libc decide to not support
> it), or it's a Linux-only function.
>
> So, there are few functions or syscalls that are generally available but are not
> in POSIX, and it might be interesting to document that they're effectively as
> portable as anything POSIX.  Maybe even POSIX editors read this and decide to
> add it.
>
> In those cases, we still need to decide what we care about or not.

Aah OK, so memmem is non-standard, there is no standard to point to.
Do other OSes provide this kind of info? I don't see anything about
portability in the OpenBSD man page (https://man.openbsd.org/memmem),
only "memmem() is a GNU extension". The FreeBSD man page
(https://www.freebsd.org/cgi/man.cgi?query=memmem&manpath=FreeBSD+13.1-RELEASE+and+Ports)
does mention Linux, but only to mention that memmem was broken in old
versions of Linux libc (nothing about current Linux); it also has the
'GNU extension' mention. Interestingly, I don't see the mention of the
function being a GNU extension in the Linux version.

Have you checked, are there many such functions? Do you plan to add
this info for all of them?

>
> Musl libc is definitely a first-class citizen these days in Linux distributions.
>   I would start documenting them in the project at large if someone from musl
> provides patches (I discussed this some time ago, but can't remember with who).
> Rich, if you would like to discuss about this, we can have some chat.
>
> >
> > For musl and other libcs, I think the man pages already document some
> > instances where their behavior diverges from glibc.
>
> As said, for musl, I'd document them officially, if there's anyone interested
> enough to send patches.
>
> For other libcs, we have to decide if they're important enough, and probably
> decide on a case-by-case basis.
>
> Michael tried to have some decent coverage of non-GNU/Linux systems, both in the
> man-pages and in his TLPI book.  It's a useful thing.  So much that sometimes
> you don't even need to read other systems' pages at all to know how portable is
> a GNU/Linux function.

I know. But not sure how much Linux docs should cover about other
OSes, which are also constantly changing (and have their own fine
docs).

As always, just my 2c,
Stefan.

>
> So, I'd like to get opinions on interest about documenting details about:
>
> - newlib (I never heard about it before; is it a widespread thing? do you think
> it's useful?)
> - Apple Libc (I still don't like it, but I must admit that it's relevant, and
> being open source, it's more palatable)
> - bionic (does anyone know if that's useful at all for anyone at all?)
>
> Support for those wouldn't go as far as musl or glibc.  For now it would only be
> for memmem(3).
>
>
> Cheers,
>
> Alex
>
> --
> <http://www.alejandro-colomar.es/>

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [musl] [PATCH] memmem.3: Added list of known systems where this is available
  2022-11-23 15:33         ` [musl] " Shiz
@ 2022-11-30 19:39           ` enh
  0 siblings, 0 replies; 6+ messages in thread
From: enh @ 2022-11-30 19:39 UTC (permalink / raw)
  To: musl; +Cc: alx.manpages

[-- Attachment #1: Type: text/plain, Size: 4127 bytes --]

On Wed, Nov 23, 2022 at 7:33 AM Shiz <hi@shiz.me> wrote:

> Hi there,
>
> > On 23 Nov Reiwa 4, at 14:16, Alejandro Colomar <alx.manpages@gmail.com>
> wrote:
> >
> >>>> -  Android is not a real Unix system, in that you can't even program
> in C in
> >>>> there, unless you're Google or have hacked your system.  It's not
> friendly
> >>>> to us programmers, so we don't need to be friendly to it.  I don't
> want to
> >>>> be cluttering the pages with information that is irrelevant to normal
> users.
> >>>
> >>> I'm assuming bionic is being used in some of the Android free
> >>> alternatives too, but then I'm not sure how usable for programming
> those
> >>> are either. And, well musl libc is not a real Unix system you can
> program
> >>> against either. :)
> >
> > The difference is only that bionic is not in use in useful systems
> (AFAIK).  :)
> >
> > So we have to do some decission here (and also about newlib, as reported
> by Brian).
>
> I’d like to chime in here and note that, valuations of usefulness aside,
> it *is*
> possible to develop in C against an Android device, without needing any
> hacks,
> using the NDK[1]. As someone who has occasionally worked on big codebases
> written
> in C and C++ for Android devices, I would argue there’s value to be had
> there. :-)
>

plus i think bionic is the only libc that explicitly links to man7.org as
the canonical source for documentation in its header files :-)

funnily enough, this topic ("should the man pages talk about availability
in libcs other than glibc?") came up before, and specifically for bionic
mkerrisk seemed to think it was a reasonable idea ... but i (bionic
maintainer) was less convinced. although i think that, yes, if someone's
reading that part of the page, realistically they're either interested in
macOS/iOS or Android (sorry, AIX or IRIX fans!), but specifically for those
two systems the header files already contain machine-readable availability
information via __attribute__((__available__)).

fwiw, here's an example from bionic's <unistd.h> illustrating both of those
points:

```
/**
 * [copy_file_range(2)](
https://man7.org/linux/man-pages/man2/copy_file_range.2.html) copies
 * a range of data from one file descriptor to another.
 *
 * Returns the number of bytes copied on success, and returns -1 and sets
`errno`
 * on failure.
 *
 * Available since API level 34.
 */
ssize_t copy_file_range(int __fd_in, off64_t* __off_in, int __fd_out,
off64_t* __off_out, size_t __length, unsigned int __flags)
__INTRODUCED_IN(34);
```

(if there's anything android-specific to note, we also mention that, but
there are relatively few examples of those, and it's mostly just "SELinux
means that only system processes can enable/disable swap" or whatever.)

so it wasn't clear to me whether -- unless you automated it -- there was
much value to duplicating that in the man pages? if you try to compile,
you're going to get a clear compiler error saying "you need to be targeting
OS version $whatever" anyway.

i think the real question is "aye, but will they get that far?". for my
part, i think the most value would come from cases that talk about "GNU
extension" being a bit clearer whether that's "unique to glibc" (which a
lot of people assume is what it means) or "initially in glibc, but a good
enough idea that it's effectively ubiquitous now, even if it's not in POSIX
because Solaris will never add it". and maybe the best way to clarify stuff
like that is just to explicitly state "Available on macOS since 10.4.
Available on Android since API level 23."? dunno.

if anyone _does_ want to go that route, i'd be happy to help with the "raw
data" side. (and certainly if anyone does start adding that, you'll force
me to care about maintaining it :-) )

but my biggest problem right now is that man7.org isn't being updated, and
i can't get hold of mkerrisk to try to offer him money to do it :-( (i do
hope he's okay, wherever he is!)


> > Cheers,
> >
> > Alex
>
> - Shiz
>
> [1]: https://developer.android.com/ndk

[-- Attachment #2: Type: text/html, Size: 5417 bytes --]

^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2022-11-30 19:39 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
     [not found] <20221110001318.10696-1-andrew@digital-domain.net>
     [not found] ` <853fa349-8e78-8ce8-f76f-79b4b9353913@gmail.com>
     [not found]   ` <Y31XOPOsB932l0cd@thunder.hadrons.org>
     [not found]     ` <CACKs7VAQsxDc2XrsAYSEbA9eqRnLHuXykVmNTit+tCFMyGLCwA@mail.gmail.com>
2022-11-23 13:16       ` [musl] Re: [PATCH] memmem.3: Added list of known systems where this is available Alejandro Colomar
2022-11-23 14:55         ` Jeffrey Walton
2022-11-23 15:11           ` Alejandro Colomar
2022-11-23 15:33         ` [musl] " Shiz
2022-11-30 19:39           ` enh
2022-11-24 18:57         ` [musl] " Stefan Puiu

Code repositories for project(s) associated with this public inbox

	https://git.vuxu.org/mirror/musl/

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).