[9fans] i/o on a hangup channel asymmetry

[9fans] i/o on a hangup channel asymmetry

[Roman V. Shaposhnik]

In the "mom, why sky is blue" department, here's a silly question:
is there any good reason that read(2) on a hangup channel returns
an error, while write(2) on a hangup channel terminates an application
(by generating a note, of course, which can be ignored, but still)?

Thanks,
Roman.

P.S. And btw, is there any way to make UNIX (POSIX) behave like Plan9
read(1)? The EOF on a hangup channel is actually misleading, its not
that the channel can be ever resumed...

Re: [9fans] i/o on a hangup channel asymmetry

[erik quanstrom]

On Sat Jul 18 14:41:02 EDT 2009, rvs@sun.com wrote:
> In the "mom, why sky is blue" department, here's a silly question:
> is there any good reason that read(2) on a hangup channel returns
> an error, while write(2) on a hangup channel terminates an application
> (by generating a note, of course, which can be ignored, but still)?

hmm.  from a quick read of port/sysfile.c, i think the precise behavior
might depend on the underlying device.  if devtab[m->c->type]->bread/bwrite
are (ultimately) based on qbread and qbwrite, i read qio as saying that neither should
generate a note.

perhaps i've been asleep at the swtch, but i don't recall seing writes
on closed channels terminate programs with a note.

- erik

Re: [9fans] i/o on a hangup channel asymmetry

[Charles Forsyth]

>perhaps i've been asleep at the swtch, but i don't recall seing writes
>on closed channels terminate programs with a note.

sys: write on closed pipe

mainly to kill off a pipeline when the thing at the end has finished.
i think that might be the only instance where a note is used.

Re: [9fans] i/o on a hangup channel asymmetry

[Roman Shaposhnik]

On Jul 19, 2009, at 2:30 AM, Charles Forsyth wrote:
>> perhaps i've been asleep at the swtch, but i don't recall seing
>> writes
>> on closed channels terminate programs with a note.
>
> sys: write on closed pipe
>
> mainly to kill off a pipeline when the thing at the end has finished.
> i think that might be the only instance where a note is used.

I had considered this theory. And it definitely explains why such
a behavior would be beneficial for the pipelined jobs. What
makes me unhappy, though, is that for things that are *not*
connected to the pipe, but rather to a different kind of channel,
it makes things more difficult without any clear benefit.

Even for the pipelined case -- consider what happens when the
thing at the beginning exits: the next consumer has to be aware
of EOF condition first (and if it skips over EOF of the read returning
-1). The consumers are offered a chance to shutting down
gracefully, where the producers are not.

Why inequality?

Thanks,
Roman.

Re: [9fans] i/o on a hangup channel asymmetry

[Francisco J Ballesteros]

Bescause consumers produce pipeline results why
producers do not?

ls | wc > /tmp/nfiles

I want nfiles to be ok.

however

ls | date

should probably let ls die as soon as date completes

>
> Why inequality?
>
> Thanks,
> Roman.
>
>
>

Re: [9fans] i/o on a hangup channel asymmetry

[Roman Shaposhnik]

On Jul 19, 2009, at 2:01 PM, Francisco J Ballesteros wrote:
> Bescause consumers produce pipeline results why
> producers do not?
>
> ls | wc > /tmp/nfiles
>
> I want nfiles to be ok.
>
> however
>
> ls | date
>
> should probably let ls die as soon as date completes

True. However, you'd get the same result if write on a broken
pipe would return -1 (and set a error condition) and the
applications were paying attention (and I'm pretty sure all
applications on Plan 9 would do a reasonable thing when
presented with -1 from a write).

Thanks,
Roman.

Re: [9fans] i/o on a hangup channel asymmetry

[Charles Forsyth]

>pipe would return -1 (and set a error condition) and the
>applications were paying attention (and I'm pretty sure all
>applications on Plan 9 would do a reasonable thing when
>presented with -1 from a write).

they only notice when the write occurs. suppose that takes
several minutes of computation.

Re: [9fans] i/o on a hangup channel asymmetry

[Roman V Shaposhnik]

On Mon, 2009-07-20 at 10:53 +0100, Charles Forsyth wrote:
> >pipe would return -1 (and set a error condition) and the
> >applications were paying attention (and I'm pretty sure all
> >applications on Plan 9 would do a reasonable thing when
> >presented with -1 from a write).
>
> they only notice when the write occurs. suppose that takes
> several minutes of computation.

Just to correct the statement of fact: same as true for the
note -- it only gets sent when the write occurs. Thus -- there's
no difference, really.

Thanks,
Roman.

Re: [9fans] i/o on a hangup channel asymmetry

[Charles Forsyth]

[-- Attachment #1: Type: text/plain, Size: 42 bytes --]

you're right. i wasn't thinking correctly.

[-- Attachment #2: Type: message/rfc822, Size: 2948 bytes --]

From: Roman V Shaposhnik <rvs@sun.com>
To: Fans of the OS Plan 9 from Bell Labs <9fans@9fans.net>
Subject: Re: [9fans] i/o on a hangup channel asymmetry
Date: Mon, 20 Jul 2009 10:01:47 -0700
Message-ID: <1248109307.18344.10052.camel@work.SFBay.Sun.COM>

On Mon, 2009-07-20 at 10:53 +0100, Charles Forsyth wrote:
> >pipe would return -1 (and set a error condition) and the
> >applications were paying attention (and I'm pretty sure all
> >applications on Plan 9 would do a reasonable thing when
> >presented with -1 from a write).
>
> they only notice when the write occurs. suppose that takes
> several minutes of computation.

Just to correct the statement of fact: same as true for the
note -- it only gets sent when the write occurs. Thus -- there's
no difference, really.

Thanks,
Roman.

Re: [9fans] i/o on a hangup channel asymmetry

[Mechiel Lukkien]

On Sun, Jul 19, 2009 at 10:30:41AM +0100, Charles Forsyth wrote:
> >perhaps i've been asleep at the swtch, but i don't recall seing writes
> >on closed channels terminate programs with a note.
>
> sys: write on closed pipe
>
> mainly to kill off a pipeline when the thing at the end has finished.
> i think that might be the only instance where a note is used.

not for network connections?  that's what happens on unix at least, eg:

	A SIGPIPE signal is raised if a process sends on a broken stream;
	this causes naive processes, which do not handle the signal,
	to exit.

and because of that, on hosted inferno an exception is raised on sigpipe.
with the light use of threads, which are sometimes used to respond on a
limbo channel after completion of a job, that can be tricky (if you're
not aware of it;  i usually forget to consider the case).

mjl

Re: [9fans] i/o on a hangup channel asymmetry

[Charles Forsyth]

>not for network connections?

i think pipe is the only case, and even that is suppressed
for pipes that carry 9p, after mounting.

Re: [9fans] i/o on a hangup channel asymmetry

[Roman Shaposhnik]

On Jul 19, 2009, at 2:55 PM, Charles Forsyth wrote:
>> not for network connections?
>
> i think pipe is the only case, and even that is suppressed
> for pipes that carry 9p, after mounting.

one last kick of a dead horse: see that's exactly what I'm
talking about -- all these exceptions and for what? I'm
pretty sure if we change the devpipe today not to send
a note nobody would even notice...

Thanks,
Roman.

Re: [9fans] i/o on a hangup channel asymmetry

[erik quanstrom]

> one last kick of a dead horse: see that's exactly what I'm
> talking about -- all these exceptions and for what? I'm
> pretty sure if we change the devpipe today not to send
> a note nobody would even notice...

since you're confident that this exception is spurious,
why don't you remove it from your kernel?  your
argument would be much more convincing if you had
evidence that after a couple million pids, your cpu
server has experienced no funnies.

- erik

Re: [9fans] i/o on a hangup channel asymmetry

[Roman V Shaposhnik]

On Mon, 2009-07-20 at 01:21 -0400, erik quanstrom wrote:
> > one last kick of a dead horse: see that's exactly what I'm
> > talking about -- all these exceptions and for what? I'm
> > pretty sure if we change the devpipe today not to send
> > a note nobody would even notice...
>
> since you're confident that this exception is spurious,
> why don't you remove it from your kernel?  your
> argument would be much more convincing if you had
> evidence that after a couple million pids, your cpu
> server has experienced no funnies.

This does look like a curious experiment. Now, since I don't
run a native Plan9 kernel, I did modify 9vx (one can't thank
Russ enough for providing it!):

--- a/src/9vx/a/devpipe.c
+++ b/src/9vx/a/devpipe.c
@@ -313,8 +313,9 @@ pipewrite(Chan *c, void *va, long n, vlong offset)
                print("pipewrite hi %lux\n", getcallerpc(&c));
        if(waserror()) {
                /* avoid notes when pipe is a mounted queue */
-               if((c->flag & CMSG) == 0)
+               /* if((c->flag & CMSG) == 0)
                        postnote(up, 1, "sys: write on closed pipe", NUser);
+                */
                nexterror();
        }

@@ -345,8 +346,9 @@ pipebwrite(Chan *c, Block *bp, ulong offset)

        if(waserror()) {
                /* avoid notes when pipe is a mounted queue */
-               if((c->flag & CMSG) == 0)
+               /* if((c->flag & CMSG) == 0)
                        postnote(up, 1, "sys: write on closed pipe", NUser);
+                */
                nexterror();
        }

So far so good. The test2.c now returns -1 "i/o on a hangup channel" from write
and the rest of the system seems to be pretty stable.

The usual pipelines that I run ("ls | sort", "...|awk", "...|sed") all seem to
be doing quite well.

Unfortunately I don't think I can get to a couple of million pids in my
9vx as you've suggested. So I guess, in the end, it doesn't count,
right?

Thanks,
Roman.

Re: [9fans] i/o on a hangup channel asymmetry

[erik quanstrom]

i did some looking through the plan 9 source.
(it's great to have it all in one place.)

all of this group, to a quick scan, should work
if the note is removed.  many can lose their
note handlers, because the the pipe signal is
the only there to catch writes on closed pipes.

grep 'closed pipe' `{find . | grep '\.[chy]$'}
./9/port/devpipe.c: *  a write to a closed pipe causes a note to be sent to
./9/port/devpipe.c:			postnote(up, 1, "sys: write on closed pipe", NUser);
./9/port/devpipe.c:			postnote(up, 1, "sys: write on closed pipe", NUser);
./ape/lib/ap/plan9/signal.c:	{"sys: write on closed pipe",		SIGPIPE},
./cmd/con/con.c:	if(strstr(msg, "closed pipe")
./cmd/sam/plan9.c:	if(bpipeok && strcmp(s, "sys: write on closed pipe") == 0)
./cmd/srv.c:	if(strstr(c, "write on closed pipe") == 0){
./cmd/srv.c:		fprint(2, "write on closed pipe\n");
./cmd/unix/drawterm/kern/devpipe.c: *  a write to a closed pipe causes a note to be sent to
./cmd/unix/drawterm/kern/devpipe.c:			postnote(up, 1, "sys: write on closed pipe", NUser);
./cmd/unix/drawterm/kern/devpipe.c:			postnote(up, 1, "sys: write on closed pipe", NUser);
./cmd/upas/common/libsys.c: *  catch a write on a closed pipe
./cmd/upas/common/libsys.c:	static char *foo = "sys: write on closed pipe";
./cmd/upas/imap4d/imap4d.c:	if(strstr(msg, "closed pipe") != nil)
./cmd/upas/smtp/smtp.c:	if(strstr(msg, "closed pipe")){
./cmd/upas/smtp/smtp.c:			syslog(0, "smtp.fail", "%s closed pipe to %s", deliverytype(), farend);
./cmd/upas/smtp/smtp.c:		/* call _exits() to prevent Bio from trying to flush closed pipe */
./libauth/newns.c:static char *wocp = "sys: write on closed pipe";

needs fixing:
./cmd/sort.c:	if(strncmp(s, "sys: write on closed pipe", 25) == 0)

there's a write that is not checked for errors.  this is already a bug.
patch already submitted.

- erik

Re: [9fans] i/o on a hangup channel asymmetry

[Russ Cox]

The programs that know about the signal are
not the programs that need to be worried about.
I'm much more worried about making sure that
commands like

grep pattern /n/dump/slow/slow/sys/log/mail | sed 5q

stop as early as possible.  The note is nice
precisely because it doesn't involve editing the
source for every command.

Russ

Re: [9fans] i/o on a hangup channel asymmetry

[erik quanstrom]

On Mon Jul 20 22:05:50 EDT 2009, rsc@swtch.com wrote:
> The programs that know about the signal are
> not the programs that need to be worried about.
> I'm much more worried about making sure that
> commands like
>
> grep pattern /n/dump/slow/slow/sys/log/mail | sed 5q
>
> stop as early as possible.  The note is nice
> precisely because it doesn't involve editing the
> source for every command.

if the grep has very few matches, it could process
for quite a long time before needing to write at all.
is this really effective in saving cpu?

also, the only case that is a problem is
when the return status of writes is not checked.
(or am i missing something?)  and the problem
will resurface on a pipe accessed through the
mount driver.

- erik

Re: [9fans] i/o on a hangup channel asymmetry

[roger peppe]

one example of this behaviour that i use
all the time is:

  tar t < foo.tar | sed 10q

just to get some idea of the contents of foo.tar.
often the tar file is huge, and i won't get a command
prompt back until all the elements of the pipeline
have completed - whereas i want it to finish quickly.

it always annoys me when gnu tar prints
something like:

tar: Error in writing to standard output
tar: Error is not recoverable: exiting now

in this situation, where a silent exit is more appropriate.

> also, the only case that is a problem is
> when the return status of writes is not checked.

it's often much more of a hassle to check the
return status of all writes, as many programs will
use print(f) in many places, all of which would
have to be wrapped appropriately.

i think the fundamental asymmetricality
between read and write is one of knowledge:

- on a write, the program knows what it's producing,
and if the consumer isn't ready for it, that's
almost always an error. in a pure pipeline,
one where each component has no side effects,
if you can't write data, there's nothing more you
can do, so you might as well be dead.

- on a read, the program usually doesn't know exactly what
it's getting so it's in a good position to act appropriately
on an unexpected situation. most
programs will produce data into the pipeline
after all input data has been read, error or not,
so it's rarely appropriate for the program to die
after a read error.

mind you, i seem to remember that inferno's pipe
write (at least) generates an exception on the first
write to a defunct pipe. i think it should return -1
at least once before raising the exception, so
a program that checks the return code should never
encounter the exception.

that seems like a good compromise to me.

Re: [9fans] i/o on a hangup channel asymmetry

[Charles Forsyth]

[-- Attachment #1: Type: text/plain, Size: 665 bytes --]

you don't really want to get "write error" printed from every
process in a pipe line such as
	slowly-compute-big-file | dosomemore | andyetmore | sed -n '/something/;p;q'
or even
	slowly-compute-big-file | badusageofsomething
where the latter prints a "usage" message and quits.
you want to make sure the previous commands in the pipeline stop, right away.
(they might be in a script. they might even be costing money.)

actually, if i were looking for something exceptional to fix,
i'd get linux and the BSDs to stop thinking they still
need character and block "special files" as special
types of files in a physical file system, to represent devices.

[-- Attachment #2: Type: message/rfc822, Size: 2881 bytes --]

From: Roman Shaposhnik <rvs@sun.com>
To: Fans of the OS Plan 9 from Bell Labs <9fans@9fans.net>
Subject: Re: [9fans] i/o on a hangup channel asymmetry
Date: Sun, 19 Jul 2009 21:52:45 -0700
Message-ID: <0679CB58-FC3E-4FDD-9889-0FC28EB87C1B@sun.com>

On Jul 19, 2009, at 2:55 PM, Charles Forsyth wrote:
>> not for network connections?
>
> i think pipe is the only case, and even that is suppressed
> for pipes that carry 9p, after mounting.

one last kick of a dead horse: see that's exactly what I'm
talking about -- all these exceptions and for what? I'm
pretty sure if we change the devpipe today not to send
a note nobody would even notice...

Thanks,
Roman.

Re: [9fans] i/o on a hangup channel asymmetry

[Roman V Shaposhnik]

On Mon, 2009-07-20 at 10:52 +0100, Charles Forsyth wrote:
> you don't really want to get "write error" printed from every
> process in a pipe line such as

As much as I thought that could be an issue, I could NOT actually
make many of the commands I tried produce this message on a
modified 9vx:
   term% ls | date
   ....
Some did produce a message, but somehow it didn't look all that
problematic (helpful, in fact -- letting me know that somewhere
in my pipe there was a problem and not all of the data went through):
   term% awk 'BEGIN { printf("Hello\n"); }' | date
   awk: write error on stdout

> 	slowly-compute-big-file | dosomemore | andyetmore | sed -n '/something/;p;q'
> or even
> 	slowly-compute-big-file | badusageofsomething
> where the latter prints a "usage" message and quits.
> you want to make sure the previous commands in the pipeline stop, right away.

One again -- just to make things clear -- slowly-compute-big-file WILL
NOT stop before it attempts the first actual write on a closed pipe. It
will merrily compute away. A trivial:
   cpu% sleep 1000000 | sleep 1
would be enough to verify your hypothesis.

Thanks,
Roman.

Re: [9fans] i/o on a hangup channel asymmetry

[Roman Shaposhnik]

On Jul 18, 2009, at 6:06 PM, erik quanstrom wrote:
> On Sat Jul 18 14:41:02 EDT 2009, rvs@sun.com wrote:
>> In the "mom, why sky is blue" department, here's a silly question:
>> is there any good reason that read(2) on a hangup channel returns
>> an error, while write(2) on a hangup channel terminates an
>> application
>> (by generating a note, of course, which can be ignored, but still)?
>
> hmm.  from a quick read of port/sysfile.c, i think the precise
> behavior
> might depend on the underlying device.  if devtab[m->c->type]->bread/
> bwrite
> are (ultimately) based on qbread and qbwrite, i read qio as saying
> that neither should
> generate a note.
>
> perhaps i've been asleep at the swtch, but i don't recall seing writes
> on closed channels terminate programs with a note.

Observe:

cpu% cat test2.c
#include <u.h>
#include <libc.h>
#include <stdio.h>

void notary(void *v, char* s)
{
     fprintf(stderr, "NOTE: %s\n", s);
     noted(NCONT);
}

int main()
{
     notify(&notary);

     while (1) {
        fprintf(stderr, "%d\n", write(1, "roman", 5));
        sleep(5000);
     }
     return 0;
}

cpu% 8c test2.c
cpu% 8l test2.8
cpu% { ./8.out | cat  } & { sleep 10 ; slay cat | rc }
5
roman5
romanNOTE:  sys: write on closed pipe pc=0x00001525
-1
NOTE:  sys: write on closed pipe pc=0x00001525
-1
NOTE:  sys: write on closed pipe pc=0x00001525
-1

Thanks,
Roman.

Re: [9fans] i/o on a hangup channel asymmetry

[erik quanstrom]

> Observe:
>

also observe (as ron noted) this happens mostly on
pipes.  this would tend to cause pipes to shutdown
from right to left.

; 8c -FVTw roman.c && 8l -o roman roman.8
; {roman | dd >/dev/null} & sleep 7; slay dd|rc
write good
write good
; note: sys: write on closed pipe pc=0x0000270a
write: i/o on hungup channel

; roman > roman.out & sleep 7; rm roman.out
write good
write good
; write: phase error -- directory entry not allocated

---

#include <u.h>
#include <libc.h>

void
notary(void*, char *s)
{
	fprint(2, "note: %s\n", s);
	noted(NCONT);
}

void
main(void)
{
	notify(notary);
	for(;;){
		if(write(1, "roman", 5) != 5){
			fprint(2, "write: %r\n");
			exits("write");
		}
		fprint(2, "write good\n");
		sleep(5000);
	}
}