Re: Nonstandard functions with callbacks

[Rich Felker <dalias@libc.org>]

On Tue, Nov 11, 2014 at 09:22:53AM +0200, Timo Teras wrote:
> On Mon, 10 Nov 2014 23:50:46 -0500
> Rich Felker <dalias@libc.org> wrote:
> 
> > The topic of fopencookie has just come up on #musl again in regards to
> > Asterisk's dependence on it (or the similar funopen function), and in
> > response I'd like to offer a few ideas on this kind of function in
> > general.
> > 
> > Supporting nonstandard functions is always a potential pitfall. Unlike
> > functions with a rigorous or semi-rigorous specification in one or
> > more standards documents, they inevitably have all sorts of
> > underspecified or unspecified corner cases that some software ends up
> > depending on. And when they come from a single origin (e.g. glibc)
> > rather than various historical systems that all had their own quirks,
> > it's arguably reasonable for applications to expect the exact behavior
> > of the original (e.g. glibc) implementation.
> 
> Understand. Especially for musl which aims to be mostly ABI compatible.

IMO ABI is only a small part of the issue; the issue of source-level
compatibility when an apparently API-compatible function is provided
matters more.

> > My feeling is that "involves callbacks" should be an indication for
> > exclusion of nonstandard functions. In terms of what I've written
> > above, I think this follows from the existing principles of exclusion
> > based on cost of implementation complexity and high risk of
> > compatibility issues with applications.
> 
> As distribution, we want things to work. And we can limit support to
> certain applications that use the specific API. And as we compile
> everything against musl, we can also make additional ABI
> considerations. So I think we'll still consider doing fopencookie() or
> funopen() as distro-mainted patch. These are additional constraints we
> can live with - compared to musl upstream which is committed to
> maintain ABI between releases.
> 
> Though, we'll probably also file portability bugs against those apps
> that rely on this.

While of course I can't make the decision for you, I'd really
encourage distros not to add to or change the public API in musl in
ways that are not expected to ever make it upstream. The result is
application binaries that have an undocumented dependency on your
distro-specific version of musl and that won't work elsewhere, which
is not so much of an issue for your own distro binaries, but is a
potential gotcha for anyone compiling their own binaries on your dist
and expecting them to work on other musl-based systems. Up to now I've
been trying to reduce and hopefully eliminate the Alpine patches to
musl that affect public interfaces and I think we're almost there.

Have you looked at what other programs do as a fallback when there's
no fopencookie? I don't think it's possible to match the API 100% with
this approach (seeking won't work, for instance), but it seems
possible to do a fallback implementation based on pthread_create and
socketpair that doesn't depend on implementation internals of stdio,
so that it could go into the application or a shim library.

Rich