Enforcing expected ordering of operations on stdout, stdin, and stderr

Enforcing expected ordering of operations on stdout, stdin, and stderr

[Max Ruttenberg]

[-- Attachment #1: Type: text/plain, Size: 310 bytes --]

All,

I guess my question is more easily asked through an example. If I have code
that makes a call to puts and then a call to getchar, what mechanism
enforces that stdout gets flushed before blocking for stdin? Is there a
such a mechanism? My gut says yes but I haven't been able to pinpoint it.

Thanks,
Max

[-- Attachment #2: Type: text/html, Size: 409 bytes --]

Re: Enforcing expected ordering of operations on stdout, stdin, and stderr

[Joakim Sindholt]

On Wed, 2016-02-10 at 16:49 -0500, Max Ruttenberg wrote:
> All,
> 
> 
> I guess my question is more easily asked through an example. If I have
> code that makes a call to puts and then a call to getchar, what
> mechanism enforces that stdout gets flushed before blocking for stdin?
> Is there a such a mechanism? My gut says yes but I haven't been able
> to pinpoint it. 
> 
> 
> Thanks,
> Max

fflush(stdout);

This is more of a basic C thing than a libc ml thing. You should
consider picking up a copy of The C Programming Language by Kernighan
and Ritchie. It will explain all of this.

Re: Enforcing expected ordering of operations on stdout, stdin, and stderr

[Max Ruttenberg]

[-- Attachment #1: Type: text/plain, Size: 2275 bytes --]

>
> fflush(stdout);
> This is more of a basic C thing than a libc ml thing. You should
> consider picking up a copy of The C Programming Language by Kernighan
> and Ritchie. It will explain all of this.


I know about fflush, thanks.

Consider this program:

int main()
{
   char buff[2];
   puts("enter a character");
   buff[0] = getchar();
   buff[1] = '\0';
   puts(buff);
   return 0;
}

If I compile that on linux-amd64, with or without musl, I will see "enter a
character" printed to my console and then be prompted for a character, as
opposed to the other way around. I don't know if this is formally
guaranteed by the C standard, but somehow that order seems to be
maintained.

But if I grep the source code in musl/src/stdio for "fflush" I don't see a
bunch of calls to fflush. I see a call to it in fclose and freopen... but
that's neither surprising nor helpful. If I do the same in
musl/src/internal I also don't get anything. I've even tried just greping
for "flush."

And yet somehow the order is maintained within those calls to puts and
getchar. So what I'm asking is: how? What part of the internal musl source
even attempts to enforce that ordering? I know the calling application can
do it with calls to fflush, but somehow that doesn't seem to be necessary
short of a signal interrupting the expected flow of execution. Am I just
getting lucky 100% of the time or is there some source in the stdio library
that's enforcing this?

On Wed, Feb 10, 2016 at 5:05 PM, Joakim Sindholt <opensource@zhasha.com>
wrote:

> On Wed, 2016-02-10 at 16:49 -0500, Max Ruttenberg wrote:
> > All,
> >
> >
> > I guess my question is more easily asked through an example. If I have
> > code that makes a call to puts and then a call to getchar, what
> > mechanism enforces that stdout gets flushed before blocking for stdin?
> > Is there a such a mechanism? My gut says yes but I haven't been able
> > to pinpoint it.
> >
> >
> > Thanks,
> > Max
>
> fflush(stdout);
>
> This is more of a basic C thing than a libc ml thing. You should
> consider picking up a copy of The C Programming Language by Kernighan
> and Ritchie. It will explain all of this.
>
>


-- 
Max Ruttenberg,
Member of the Technical Staff
Emu *Technology*
1400 E Angela Blvd, Unit 101
South Bend, IN 46617

[-- Attachment #2: Type: text/html, Size: 3963 bytes --]

Re: Enforcing expected ordering of operations on stdout, stdin, and stderr

[Szabolcs Nagy]

* Max Ruttenberg <mruttenberg@emutechnology.com> [2016-02-10 18:08:47 -0500]:
> Consider this program:
> 
> int main()
> {
>    char buff[2];
>    puts("enter a character");
>    buff[0] = getchar();
>    buff[1] = '\0';
>    puts(buff);
>    return 0;
> }
> 
> If I compile that on linux-amd64, with or without musl, I will see "enter a
> character" printed to my console and then be prompted for a character, as
> opposed to the other way around. I don't know if this is formally
> guaranteed by the C standard, but somehow that order seems to be
> maintained.
> 

that is line buffering of stdout on interactive terminals

if you pipe it to cat then there will be no line buffering.

Re: Enforcing expected ordering of operations on stdout, stdin, and stderr

[Rich Felker]

On Wed, Feb 10, 2016 at 06:08:47PM -0500, Max Ruttenberg wrote:
> >
> > fflush(stdout);
> > This is more of a basic C thing than a libc ml thing. You should
> > consider picking up a copy of The C Programming Language by Kernighan
> > and Ritchie. It will explain all of this.
> 
> 
> I know about fflush, thanks.
> 
> Consider this program:
> 
> int main()
> {
>    char buff[2];
>    puts("enter a character");
>    buff[0] = getchar();
>    buff[1] = '\0';
>    puts(buff);
>    return 0;
> }
> 
> If I compile that on linux-amd64, with or without musl, I will see "enter a
> character" printed to my console and then be prompted for a character, as
> opposed to the other way around. I don't know if this is formally
> guaranteed by the C standard, but somehow that order seems to be
> maintained.
> 
> But if I grep the source code in musl/src/stdio for "fflush" I don't see a
> bunch of calls to fflush. I see a call to it in fclose and freopen... but
> that's neither surprising nor helpful. If I do the same in
> musl/src/internal I also don't get anything. I've even tried just greping
> for "flush."
> 
> And yet somehow the order is maintained within those calls to puts and
> getchar. So what I'm asking is: how? What part of the internal musl source
> even attempts to enforce that ordering? I know the calling application can
> do it with calls to fflush, but somehow that doesn't seem to be necessary
> short of a signal interrupting the expected flow of execution. Am I just
> getting lucky 100% of the time or is there some source in the stdio library
> that's enforcing this?

When attached to a terminal ("interactive device" in the terminology
of the C standard), stdout is line-buffered by default. This means
that writing a newline, which puts() inherently does at the end of its
output, causes output to be flushed. The code path from puts.c is via
the macro form of putc_unlocked, which is defined in
src/internal/stdio_impl.h, and bypasses the fast write-to-buffer code
path when c==f->lbf (f->lbf is set to '\n' for line-buffered mode, -1
for other modes). Actually even if stdout were not a terminal you
would _happen_ to see this behavior on musl because whether stdout is
line-buffered is decided lazily at the first encounter of '\n', so the
"first line of output" is always line-buffered. But this is an
implementation detail and not something you should rely on. If you
want line-buffered behavior even for non-interactive stdout, you need
to call setvbuf (as the first action on the file).

Also, ISO C permitted and even encouraged (but made optional) a
behavior whereby attempting to read from a line-buffered input stream
causes all line-buffered output streams to be flushed. While this is
convenient for programmers who write prompt strings not ending in
newlines, and who don't want to be bothered with calling fflush, this
feature was conceived in an era where C did not have multi-threading,
and providing it when you have threads imposes a heavy synchronization
burden and can even lead to deadlock. Therefore musl does not do it.
So if you want to print prompt strings that don't end in a newline,
and have them appear before input is read, you have to use fflush
yourself.

Does this help?

Rich

Re: Enforcing expected ordering of operations on stdout, stdin, and stderr

[Max Ruttenberg]

[-- Attachment #1: Type: text/plain, Size: 3797 bytes --]

Rich,

Thank you, that last email was very helpful. I see now that if I swap
printf for that first puts and don't include a newline character I get
prompted for input before the message the appears.

Thanks so much,
Max

On Wed, Feb 10, 2016 at 6:39 PM, Rich Felker <dalias@libc.org> wrote:

> On Wed, Feb 10, 2016 at 06:08:47PM -0500, Max Ruttenberg wrote:
> > >
> > > fflush(stdout);
> > > This is more of a basic C thing than a libc ml thing. You should
> > > consider picking up a copy of The C Programming Language by Kernighan
> > > and Ritchie. It will explain all of this.
> >
> >
> > I know about fflush, thanks.
> >
> > Consider this program:
> >
> > int main()
> > {
> >    char buff[2];
> >    puts("enter a character");
> >    buff[0] = getchar();
> >    buff[1] = '\0';
> >    puts(buff);
> >    return 0;
> > }
> >
> > If I compile that on linux-amd64, with or without musl, I will see
> "enter a
> > character" printed to my console and then be prompted for a character, as
> > opposed to the other way around. I don't know if this is formally
> > guaranteed by the C standard, but somehow that order seems to be
> > maintained.
> >
> > But if I grep the source code in musl/src/stdio for "fflush" I don't see
> a
> > bunch of calls to fflush. I see a call to it in fclose and freopen... but
> > that's neither surprising nor helpful. If I do the same in
> > musl/src/internal I also don't get anything. I've even tried just greping
> > for "flush."
> >
> > And yet somehow the order is maintained within those calls to puts and
> > getchar. So what I'm asking is: how? What part of the internal musl
> source
> > even attempts to enforce that ordering? I know the calling application
> can
> > do it with calls to fflush, but somehow that doesn't seem to be necessary
> > short of a signal interrupting the expected flow of execution. Am I just
> > getting lucky 100% of the time or is there some source in the stdio
> library
> > that's enforcing this?
>
> When attached to a terminal ("interactive device" in the terminology
> of the C standard), stdout is line-buffered by default. This means
> that writing a newline, which puts() inherently does at the end of its
> output, causes output to be flushed. The code path from puts.c is via
> the macro form of putc_unlocked, which is defined in
> src/internal/stdio_impl.h, and bypasses the fast write-to-buffer code
> path when c==f->lbf (f->lbf is set to '\n' for line-buffered mode, -1
> for other modes). Actually even if stdout were not a terminal you
> would _happen_ to see this behavior on musl because whether stdout is
> line-buffered is decided lazily at the first encounter of '\n', so the
> "first line of output" is always line-buffered. But this is an
> implementation detail and not something you should rely on. If you
> want line-buffered behavior even for non-interactive stdout, you need
> to call setvbuf (as the first action on the file).
>
> Also, ISO C permitted and even encouraged (but made optional) a
> behavior whereby attempting to read from a line-buffered input stream
> causes all line-buffered output streams to be flushed. While this is
> convenient for programmers who write prompt strings not ending in
> newlines, and who don't want to be bothered with calling fflush, this
> feature was conceived in an era where C did not have multi-threading,
> and providing it when you have threads imposes a heavy synchronization
> burden and can even lead to deadlock. Therefore musl does not do it.
> So if you want to print prompt strings that don't end in a newline,
> and have them appear before input is read, you have to use fflush
> yourself.
>
> Does this help?
>
> Rich
>



-- 
Max Ruttenberg,
Member of the Technical Staff
Emu *Technology*
1400 E Angela Blvd, Unit 101
South Bend, IN 46617

[-- Attachment #2: Type: text/html, Size: 5312 bytes --]