non-standard implementation of fflush()

non-standard implementation of fflush()

[y38h5z]

[-- Attachment #1: Type: text/plain, Size: 648 bytes --]

Hello everyone,

the implementation of fflush() in musl doesn't seem to conform to the opengroup standard:

https://pubs.opengroup.org/onlinepubs/009695399/functions/fflush.html

In addition to flushing unwritten data, which is expected, musl flushes the read buffer when calling fflush(). This leads to data loss in bidirectional communication uses. Other standard libraries don't do this.

As a reference compare musl to openbsd's libc:

https://git.musl-libc.org/cgit/musl/tree/src/stdio/fflush.c
https://github.com/openbsd/src/blob/master/lib/libc/stdio/fflush.c

I think this is unexpected behavior and should be changed.

Greetings,
  Michael

[-- Attachment #2: Type: text/html, Size: 1132 bytes --]

Re: non-standard implementation of fflush()

[Rich Felker]

On Thu, Nov 21, 2019 at 06:31:02PM +0000, y38h5z@protonmail.com wrote:
> Hello everyone,
> 
> the implementation of fflush() in musl doesn't seem to conform to
> the opengroup standard:
> 
> https://pubs.opengroup.org/onlinepubs/009695399/functions/fflush.html
> 
> In addition to flushing unwritten data, which is expected, musl
> flushes the read buffer when calling fflush(). This leads to data
> loss in bidirectional communication uses. Other standard libraries
> don't do this.
> 
> As a reference compare musl to openbsd's libc:
> 
> https://git.musl-libc.org/cgit/musl/tree/src/stdio/fflush.c
> https://github.com/openbsd/src/blob/master/lib/libc/stdio/fflush.c
> 
> I think this is unexpected behavior and should be changed.

ISO C leaves the behavior of fflush undefined unless "stream points to
an output stream or an update stream in which the most recent
operation was not input". POSIX further defines it for read, but only
if the underlying fd is seekable:

    "For a stream open for reading with an underlying file
    description, if the file is not already at EOF, and the file is
    one capable of seeking, the file offset of the underlying open
    file description shall be set to the file position of the stream,
    and any characters pushed back onto the stream by ungetc() or
    ungetwc() that have not subsequently been read from the stream
    shall be discarded (without further changing the file offset).

The case of reading from an unseekable stream is left undefined.
Correct programs should not be doing this at all. The current behavior
in musl is simply the default effect from not making any special
provisions to treat unseekable streams differently, since there is not
any particular behavior we're trying to achieve.

Rich

Re: non-standard implementation of fflush()

[Rich Felker]

On Thu, Nov 21, 2019 at 09:25:31PM +0000, y38h5z@protonmail.com wrote:
> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> Am Donnerstag, November 21, 2019 8:04 PM schrieb Rich Felker <dalias@libc.org>:
> 
> > On Thu, Nov 21, 2019 at 06:31:02PM +0000, y38h5z@protonmail.com wrote:
> >
> > > Hello everyone,
> > > the implementation of fflush() in musl doesn't seem to conform to
> > > the opengroup standard:
> > > https://pubs.opengroup.org/onlinepubs/009695399/functions/fflush.html
> > > In addition to flushing unwritten data, which is expected, musl
> > > flushes the read buffer when calling fflush(). This leads to data
> > > loss in bidirectional communication uses. Other standard libraries
> > > don't do this.
> > > As a reference compare musl to openbsd's libc:
> > > https://git.musl-libc.org/cgit/musl/tree/src/stdio/fflush.c
> > > https://github.com/openbsd/src/blob/master/lib/libc/stdio/fflush.c
> > > I think this is unexpected behavior and should be changed.
> >
> > ISO C leaves the behavior of fflush undefined unless "stream points to
> > an output stream or an update stream in which the most recent
> > operation was not input". POSIX further defines it for read, but only
> > if the underlying fd is seekable:
> >
> > "For a stream open for reading with an underlying file
> > description, if the file is not already at EOF, and the file is
> > one capable of seeking, the file offset of the underlying open
> > file description shall be set to the file position of the stream,
> > and any characters pushed back onto the stream by ungetc() or
> > ungetwc() that have not subsequently been read from the stream
> > shall be discarded (without further changing the file offset).
> >
> > The case of reading from an unseekable stream is left undefined.
> > Correct programs should not be doing this at all. The current behavior
> > in musl is simply the default effect from not making any special
> > provisions to treat unseekable streams differently, since there is not
> > any particular behavior we're trying to achieve.
> 
> In my case FILE * is a socket wrapped via fdopen(). According to
> https://linux.die.net/man/7/socket it does not support seek but
> clearly it makes sense to read from it and also flush written data.
> It does not make sense to flush the currently written data and at
> the same time flush data which may just have been received without
> the application having a chance to read at all.

It sounds like you're trying to use the same FILE for both reading and
writing to a socket. That simply does not work at all. The only time
it's allowed to switch between from reading to writing on a given FILE
is after a successful seek operation, which can never happen if the
underlying fd is not seekable. See 7.21.5.3 The fopen function, ¶7:

    "When a file is opened with update mode ('+' as the second or
    third character in the above list of mode argument values), both
    input and output may be performed on the associated stream.
    However, output shall not be directly followed by input without an
    intervening call to the fflush function or to a file positioning
    function (fseek, fsetpos, or rewind), and input shall not be
    directly followed by output without an intervening call to a file
    positioning function, unless the input operation encounters end-
    of-file."

Violation of such a "shall not" is undefined behavior.

For practical purposes, to use stdio with a socket via fdopen, you
must limit yourself to one direction through a given FILE. If you want
bidirectional access via stdio, you need to dup() the socket file
descriptor and fdopen both.

None of this is unique to musl. Theoretically an implementation could
split the buffer in half to support both read and write at the same
time, but there are lots of slippery corner cases involved in doing
this, and it penalizes everyone who's actually following the standard
and not trying to do undefined things (by giving them half the buffer
for the same amount of memory consumed), so it's probably not a good
idea to do this.

Rich

Re: non-standard implementation of fflush()

[y38h5z]

>
> For practical purposes, to use stdio with a socket via fdopen, you
> must limit yourself to one direction through a given FILE. If you want
> bidirectional access via stdio, you need to dup() the socket file
> descriptor and fdopen both.
>

Thanks for your help Rich. I've updated my code accordingly and got it to compile against different runtime libs now.

Greetings,
  Michael