From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.org/gmane.linux.lib.musl.general/9304 Path: news.gmane.org!not-for-mail From: Max Ruttenberg Newsgroups: gmane.linux.lib.musl.general Subject: Re: Enforcing expected ordering of operations on stdout, stdin, and stderr Date: Wed, 10 Feb 2016 20:17:12 -0500 Message-ID: References: <1455141901.26335.95.camel@zhasha.com> <20160210233900.GG9349@brightrain.aerifal.cx> Reply-To: musl@lists.openwall.com NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: multipart/alternative; boundary=e89a8f838ca973b00a052b744ffa X-Trace: ger.gmane.org 1455153448 13078 80.91.229.3 (11 Feb 2016 01:17:28 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Thu, 11 Feb 2016 01:17:28 +0000 (UTC) To: musl@lists.openwall.com Original-X-From: musl-return-9317-gllmg-musl=m.gmane.org@lists.openwall.com Thu Feb 11 02:17:28 2016 Return-path: Envelope-to: gllmg-musl@m.gmane.org Original-Received: from mother.openwall.net ([195.42.179.200]) by plane.gmane.org with smtp (Exim 4.69) (envelope-from ) id 1aTfst-0000Dm-R9 for gllmg-musl@m.gmane.org; Thu, 11 Feb 2016 02:17:28 +0100 Original-Received: (qmail 21649 invoked by uid 550); 11 Feb 2016 01:17:25 -0000 Mailing-List: contact musl-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Original-Received: (qmail 21615 invoked from network); 11 Feb 2016 01:17:24 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=emutechnology-com.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=YamYXIk03vBkjkTWOhPBQgC4u3wMw2Cwnk+6cx0abkc=; b=srPWvUGMsHPBKTg8E9M5MttqPmtZevktr70TmRVx9vx8dFRlTE2zbgwnoIeH4S6Jjp PUvBe0e9CJjH7RkRzbmpBB+jRs+EHM/VOcPBq9jD9aj97NDfajAz87yaHO6nNK0z8XcD cDBPdln0lnoOp/DSQLWmwwS0XPKSMnqgo8zIMlttgX1vKd/MqwmSEUP1+sgqunz5Buf+ xm3vRaBdXfhYl9QW4Jo2Goan+1GxLPug2h32okxMmIHotZdu+V2av5/iAwy69ORbMGzF ffX/r6JCuyFFLIWWriJaHijwEAdXzTkfEgz3WlcGigyB8QLVcqvfDH8JaeDVxULWXnji ljMA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:content-type; bh=YamYXIk03vBkjkTWOhPBQgC4u3wMw2Cwnk+6cx0abkc=; b=eMaasGPBSB9VfSubbWVOV4zJm2iOy0R7JG28D3i2jE3/djrAk4YvPmofJdqEagrtdv KoFqMVbIwAQO/06Mz1y36OR+bI0VSROH3rY+Y3gMOOhMA3iV2saNHr1GL5Ei1ozl85XD MlBgY9V3LvjK9hfhDChFt7JdWh9lcspsWsgzbdNUMl2yru+aZvItjPNIZ+3kisjq/JTD WqqcUi7OtTJzclwNFst2WRshE2o/uqi/8m9pLKyLWpV4OzYUzsnl+99JdfgFOy2ZPWnf oLfhaoYVwDUHnKXm3kG7n1cgwr+fJyy8qIfTa7X+x2OLuIjSKpDwQ9QXgpUStGwAgAQf +Awg== X-Gm-Message-State: AG10YOQdTQ8aCfbO/0PrBEg3SZN11zfkgZsFNn1xWHj+i0nD7pE3/htz48Ezf8TwQyW8nKWbFU5Uze8BmxKDqg== X-Received: by 10.182.63.42 with SMTP id d10mr2964696obs.65.1455153432728; Wed, 10 Feb 2016 17:17:12 -0800 (PST) In-Reply-To: <20160210233900.GG9349@brightrain.aerifal.cx> Xref: news.gmane.org gmane.linux.lib.musl.general:9304 Archived-At: --e89a8f838ca973b00a052b744ffa Content-Type: text/plain; charset=UTF-8 Rich, Thank you, that last email was very helpful. I see now that if I swap printf for that first puts and don't include a newline character I get prompted for input before the message the appears. Thanks so much, Max On Wed, Feb 10, 2016 at 6:39 PM, Rich Felker wrote: > On Wed, Feb 10, 2016 at 06:08:47PM -0500, Max Ruttenberg wrote: > > > > > > fflush(stdout); > > > This is more of a basic C thing than a libc ml thing. You should > > > consider picking up a copy of The C Programming Language by Kernighan > > > and Ritchie. It will explain all of this. > > > > > > I know about fflush, thanks. > > > > Consider this program: > > > > int main() > > { > > char buff[2]; > > puts("enter a character"); > > buff[0] = getchar(); > > buff[1] = '\0'; > > puts(buff); > > return 0; > > } > > > > If I compile that on linux-amd64, with or without musl, I will see > "enter a > > character" printed to my console and then be prompted for a character, as > > opposed to the other way around. I don't know if this is formally > > guaranteed by the C standard, but somehow that order seems to be > > maintained. > > > > But if I grep the source code in musl/src/stdio for "fflush" I don't see > a > > bunch of calls to fflush. I see a call to it in fclose and freopen... but > > that's neither surprising nor helpful. If I do the same in > > musl/src/internal I also don't get anything. I've even tried just greping > > for "flush." > > > > And yet somehow the order is maintained within those calls to puts and > > getchar. So what I'm asking is: how? What part of the internal musl > source > > even attempts to enforce that ordering? I know the calling application > can > > do it with calls to fflush, but somehow that doesn't seem to be necessary > > short of a signal interrupting the expected flow of execution. Am I just > > getting lucky 100% of the time or is there some source in the stdio > library > > that's enforcing this? > > When attached to a terminal ("interactive device" in the terminology > of the C standard), stdout is line-buffered by default. This means > that writing a newline, which puts() inherently does at the end of its > output, causes output to be flushed. The code path from puts.c is via > the macro form of putc_unlocked, which is defined in > src/internal/stdio_impl.h, and bypasses the fast write-to-buffer code > path when c==f->lbf (f->lbf is set to '\n' for line-buffered mode, -1 > for other modes). Actually even if stdout were not a terminal you > would _happen_ to see this behavior on musl because whether stdout is > line-buffered is decided lazily at the first encounter of '\n', so the > "first line of output" is always line-buffered. But this is an > implementation detail and not something you should rely on. If you > want line-buffered behavior even for non-interactive stdout, you need > to call setvbuf (as the first action on the file). > > Also, ISO C permitted and even encouraged (but made optional) a > behavior whereby attempting to read from a line-buffered input stream > causes all line-buffered output streams to be flushed. While this is > convenient for programmers who write prompt strings not ending in > newlines, and who don't want to be bothered with calling fflush, this > feature was conceived in an era where C did not have multi-threading, > and providing it when you have threads imposes a heavy synchronization > burden and can even lead to deadlock. Therefore musl does not do it. > So if you want to print prompt strings that don't end in a newline, > and have them appear before input is read, you have to use fflush > yourself. > > Does this help? > > Rich > -- Max Ruttenberg, Member of the Technical Staff Emu *Technology* 1400 E Angela Blvd, Unit 101 South Bend, IN 46617 --e89a8f838ca973b00a052b744ffa Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Rich,

Thank you, that last email was ve= ry helpful. I see now that if I swap printf for that first puts and don'= ;t include a newline character I get prompted for input before the message = the appears.

Thanks so much,
Max

On Wed, Feb 10, 201= 6 at 6:39 PM, Rich Felker <dalias@libc.org> wrote:
On Wed, Feb = 10, 2016 at 06:08:47PM -0500, Max Ruttenberg wrote:
> >
> > fflush(stdout);
> > This is more of a basic C thing than a libc ml thing. You should<= br> > > consider picking up a copy of The C Programming Language by Kerni= ghan
> > and Ritchie. It will explain all of this.
>
>
> I know about fflush, thanks.
>
> Consider this program:
>
> int main()
> {
>=C2=A0 =C2=A0 char buff[2];
>=C2=A0 =C2=A0 puts("enter a character");
>=C2=A0 =C2=A0 buff[0] =3D getchar();
>=C2=A0 =C2=A0 buff[1] =3D '\0';
>=C2=A0 =C2=A0 puts(buff);
>=C2=A0 =C2=A0 return 0;
> }
>
> If I compile that on linux-amd64, with or without musl, I will see &qu= ot;enter a
> character" printed to my console and then be prompted for a chara= cter, as
> opposed to the other way around. I don't know if this is formally<= br> > guaranteed by the C standard, but somehow that order seems to be
> maintained.
>
> But if I grep the source code in musl/src/stdio for "fflush"= I don't see a
> bunch of calls to fflush. I see a call to it in fclose and freopen... = but
> that's neither surprising nor helpful. If I do the same in
> musl/src/internal I also don't get anything. I've even tried j= ust greping
> for "flush."
>
> And yet somehow the order is maintained within those calls to puts and=
> getchar. So what I'm asking is: how? What part of the internal mus= l source
> even attempts to enforce that ordering? I know the calling application= can
> do it with calls to fflush, but somehow that doesn't seem to be ne= cessary
> short of a signal interrupting the expected flow of execution. Am I ju= st
> getting lucky 100% of the time or is there some source in the stdio li= brary
> that's enforcing this?

When attached to a terminal ("interactive device" in = the terminology
of the C standard), stdout is line-buffered by default. This means
that writing a newline, which puts() inherently does at the end of its
output, causes output to be flushed. The code path from puts.c is via
the macro form of putc_unlocked, which is defined in
src/internal/stdio_impl.h, and bypasses the fast write-to-buffer code
path when c=3D=3Df->lbf (f->lbf is set to '\n' for line-buffe= red mode, -1
for other modes). Actually even if stdout were not a terminal you
would _happen_ to see this behavior on musl because whether stdout is
line-buffered is decided lazily at the first encounter of '\n', so = the
"first line of output" is always line-buffered. But this is an implementation detail and not something you should rely on. If you
want line-buffered behavior even for non-interactive stdout, you need
to call setvbuf (as the first action on the file).

Also, ISO C permitted and even encouraged (but made optional) a
behavior whereby attempting to read from a line-buffered input stream
causes all line-buffered output streams to be flushed. While this is
convenient for programmers who write prompt strings not ending in
newlines, and who don't want to be bothered with calling fflush, this feature was conceived in an era where C did not have multi-threading,
and providing it when you have threads imposes a heavy synchronization
burden and can even lead to deadlock. Therefore musl does not do it.
So if you want to print prompt strings that don't end in a newline,
and have them appear before input is read, you have to use fflush
yourself.

Does this help?

Rich



--
Max Rutten= berg,
Member of the Technical Staff
Emu=C2=A0Technology
1400 E Angela Blvd, Unit 101
South Bend, IN 46617
=
--e89a8f838ca973b00a052b744ffa--