[9fans] pipe: bug or feature?

[9fans] pipe: bug or feature?

[arisawa]

Hello,

I have been playing with an experimental code on pipe.
the program read a file and write it to one end of pipe and then read it from another end of pipe.
the buffer for writing pipe is named buf0, and for reading pipe is named buf.
and I found the program does not finish unless sizeof(buf) > sizeof(buf0).
is this a bug or feature of pipe?

Kenji Arisawa

=== BEGIN a.c ===
#include <u.h>
#include <libc.h>

char *argv0;

void
usage(void)
{
	fprint(2,"usage: %s file\n",argv0);
	exits("usage");
}

void
main(int argc, char *argv[])
{
	int fd,pfd[2];
	char buf[256];
	char buf0[256];
	/* need to be sizeof(buf) > sizeof(buf0)
	 * but this condition is very curious to me */
	int n;
	char *file;
	argv0 = argv[0];
	argc--;argv++;
	USED(argc);
	if(argv[0] == nil)
		usage();
	file = argv[0];
	fd = open(file,OREAD);
	if(fd < 0)
		sysfatal("no such file");

	if(pipe(pfd) < 0)
		sysfatal("pipe error");
	print("pfd: %d %d\n",pfd[0],pfd[1]);

	while((n = read(fd,buf0,sizeof(buf0))) > 0){
		print("read: %d %s\n",n,file);
		n = write(pfd[1],buf0,n);
		print("write: %d\n",n);
	}
	close(pfd[1]);
	while((n = read(pfd[0],buf,sizeof(buf))) > 0){
		buf[n] = 0;
		print("%d %s\n",n,buf);
	}
	print("%d\n",n);
		
	exits(nil);
}
=== END a.c ===

Re: [9fans] pipe: bug or feature?

[Charles Forsyth]

[-- Attachment #1: Type: text/plain, Size: 739 bytes --]

On 28 March 2017 at 01:05, arisawa <karisawa@gmail.com> wrote:

> the program reads a file and writes it to one end of pipe and then reads
> it from another end of pipe.
> the buffer for writing pipe is named buf0, and for reading pipe is named
> buf.
>

The general problem with this is that a pipe is a full-duplex communication
channel, not a storage device.
The implementation has buffering, but it's not guaranteed to hold a file of
any particular size.
Obviously in practice you're fine if the file is "small enough", but how
small is that?
Having the same process write and read the pipe won't work in general.
The buffer will fill up and the writer will block, which is just right ...
if it's not also the reader.

[-- Attachment #2: Type: text/html, Size: 1247 bytes --]

Re: [9fans] pipe: bug or feature?

[Skip Tavakkolian]

[-- Attachment #1: Type: text/plain, Size: 2778 bytes --]

good catch! the behavior makes sense now.

On Sun, Apr 2, 2017 at 5:34 AM Antons Suspans <antox@ml.lv> wrote:

> This was writing a \0 just past end of buf[], into memory alloted for
> pfd[0].
> See the explanation by Alex in the 9front list:
> /n/9front.org/lists/9front/1490668028.00
>
> On Sun, Apr 02, 2017 at 07:34:52AM +0000, Skip Tavakkolian wrote:
> > this is either a bug in devpipe or the documentation is misleading. the
> man
> > page seems to say that the pair of fd's returned by pipe(2) are
> symmetrical,
> > but it matters which fd of the pair is used for writing.  if you switch
> around
> > pfd[0] and pfd[1], it works as you'd expect.
> >
> > On Fri, Mar 31, 2017 at 5:29 PM arisawa <karisawa@gmail.com> wrote:
> >
> >   Hello,
> >
> >   I was playing with an experimental code on pipe and met with a problem
> which
> >   I don’t understand.
> >
> >   the program reads a file and writes it to one end of pipe and then
> reads it
> >   from another end of pipe.
> >   the buffer for writing pipe is named buf0, and for reading pipe is
> named buf.
> >   and I found the program does not finish unless sizeof(buf) >
> sizeof(buf0).
> >   is this a bug or feature of pipe?
> >
> >   Kenji Arisawa
> >
> >   === BEGIN a.c ===
> >   #include <u.h>
> >   #include <libc.h>
> >
> >   char *argv0;
> >
> >   void
> >   usage(void)
> >   {
> >           fprint(2,"usage: %s file\n",argv0);
> >           exits("usage");
> >   }
> >
> >   void
> >   main(int argc, char *argv[])
> >   {
> >           int fd,pfd[2];
> >           char buf[256];
> >           char buf0[256];
> >           /* need to be sizeof(buf) > sizeof(buf0)
> >            * but this condition is very curious to me */
> >           int n;
> >           char *file;
> >           argv0 = argv[0];
> >           argc--;argv++;
> >           USED(argc);
> >           if(argv[0] == nil)
> >                   usage();
> >           file = argv[0];
> >           fd = open(file,OREAD);
> >           if(fd < 0)
> >                   sysfatal("no such file");
> >
> >           if(pipe(pfd) < 0)
> >                   sysfatal("pipe error");
> >           print("pfd: %d %d\n",pfd[0],pfd[1]);
> >
> >           while((n = read(fd,buf0,sizeof(buf0))) > 0){
> >                   print("read: %d %s\n",n,file);
> >                   n = write(pfd[1],buf0,n);
> >                   print("write: %d\n",n);
> >           }
> >           close(pfd[1]);
> >           while((n = read(pfd[0],buf,sizeof(buf))) > 0){
> >                   buf[n] = 0;
> >                   print("%d %s\n",n,buf);
> >           }
> >           print("%d\n",n);
> >
> >           exits(nil);
> >   }
> >   === END a.c ===
> >
>
>

[-- Attachment #2: Type: text/html, Size: 5375 bytes --]

Re: [9fans] pipe: bug or feature?

[Antons Suspans]

This was writing a \0 just past end of buf[], into memory alloted for pfd[0].
See the explanation by Alex in the 9front list:
/n/9front.org/lists/9front/1490668028.00

On Sun, Apr 02, 2017 at 07:34:52AM +0000, Skip Tavakkolian wrote:
> this is either a bug in devpipe or the documentation is misleading. the man
> page seems to say that the pair of fd's returned by pipe(2) are symmetrical,
> but it matters which fd of the pair is used for writing.  if you switch around
> pfd[0] and pfd[1], it works as you'd expect.
> 
> On Fri, Mar 31, 2017 at 5:29 PM arisawa <karisawa@gmail.com> wrote:
> 
>   Hello,
> 
>   I was playing with an experimental code on pipe and met with a problem which
>   I don’t understand.
> 
>   the program reads a file and writes it to one end of pipe and then reads it
>   from another end of pipe.
>   the buffer for writing pipe is named buf0, and for reading pipe is named buf.
>   and I found the program does not finish unless sizeof(buf) > sizeof(buf0).
>   is this a bug or feature of pipe?
> 
>   Kenji Arisawa
> 
>   === BEGIN a.c ===
>   #include <u.h>
>   #include <libc.h>
> 
>   char *argv0;
> 
>   void
>   usage(void)
>   {
>           fprint(2,"usage: %s file\n",argv0);
>           exits("usage");
>   }
> 
>   void
>   main(int argc, char *argv[])
>   {
>           int fd,pfd[2];
>           char buf[256];
>           char buf0[256];
>           /* need to be sizeof(buf) > sizeof(buf0)
>            * but this condition is very curious to me */
>           int n;
>           char *file;
>           argv0 = argv[0];
>           argc--;argv++;
>           USED(argc);
>           if(argv[0] == nil)
>                   usage();
>           file = argv[0];
>           fd = open(file,OREAD);
>           if(fd < 0)
>                   sysfatal("no such file");
> 
>           if(pipe(pfd) < 0)
>                   sysfatal("pipe error");
>           print("pfd: %d %d\n",pfd[0],pfd[1]);
> 
>           while((n = read(fd,buf0,sizeof(buf0))) > 0){
>                   print("read: %d %s\n",n,file);
>                   n = write(pfd[1],buf0,n);
>                   print("write: %d\n",n);
>           }
>           close(pfd[1]);
>           while((n = read(pfd[0],buf,sizeof(buf))) > 0){
>                   buf[n] = 0;
>                   print("%d %s\n",n,buf);
>           }
>           print("%d\n",n);
> 
>           exits(nil);
>   }
>   === END a.c ===
> 

Re: [9fans] pipe: bug or feature?

[arisawa]

thanks,

that is my fault.
it should be
	while((n = read(fd,buf0,sizeof(buf0)-1)) > 0){
and
	while((n = read(pfd[0],buf,sizeof(buf)-1)) > 0){

> 2017/04/02 16:34、Skip Tavakkolian <skip.tavakkolian@gmail.com> のメール：
> 
> this is either a bug in devpipe or the documentation is misleading. the man page seems to say that the pair of fd's returned by pipe(2) are symmetrical, but it matters which fd of the pair is used for writing.  if you switch around pfd[0] and pfd[1], it works as you'd expect.
> 
> On Fri, Mar 31, 2017 at 5:29 PM arisawa <karisawa@gmail.com> wrote:
> Hello,
> 
> I was playing with an experimental code on pipe and met with a problem which I don’t understand.
> 
> the program reads a file and writes it to one end of pipe and then reads it from another end of pipe.
> the buffer for writing pipe is named buf0, and for reading pipe is named buf.
> and I found the program does not finish unless sizeof(buf) > sizeof(buf0).
> is this a bug or feature of pipe?
> 
> Kenji Arisawa
> 
> === BEGIN a.c ===
> #include <u.h>
> #include <libc.h>
> 
> char *argv0;
> 
> void
> usage(void)
> {
>         fprint(2,"usage: %s file\n",argv0);
>         exits("usage");
> }
> 
> void
> main(int argc, char *argv[])
> {
>         int fd,pfd[2];
>         char buf[256];
>         char buf0[256];
>         /* need to be sizeof(buf) > sizeof(buf0)
>          * but this condition is very curious to me */
>         int n;
>         char *file;
>         argv0 = argv[0];
>         argc--;argv++;
>         USED(argc);
>         if(argv[0] == nil)
>                 usage();
>         file = argv[0];
>         fd = open(file,OREAD);
>         if(fd < 0)
>                 sysfatal("no such file");
> 
>         if(pipe(pfd) < 0)
>                 sysfatal("pipe error");
>         print("pfd: %d %d\n",pfd[0],pfd[1]);
> 
>         while((n = read(fd,buf0,sizeof(buf0))) > 0){
>                 print("read: %d %s\n",n,file);
>                 n = write(pfd[1],buf0,n);
>                 print("write: %d\n",n);
>         }
>         close(pfd[1]);
>         while((n = read(pfd[0],buf,sizeof(buf))) > 0){
>                 buf[n] = 0;
>                 print("%d %s\n",n,buf);
>         }
>         print("%d\n",n);
> 
>         exits(nil);
> }
> === END a.c ===

Re: [9fans] pipe: bug or feature?

[Skip Tavakkolian]

[-- Attachment #1: Type: text/plain, Size: 2120 bytes --]

this is either a bug in devpipe or the documentation is misleading. the man
page seems to say that the pair of fd's returned by pipe(2) are
symmetrical, but it matters which fd of the pair is used for writing.  if
you switch around pfd[0] and pfd[1], it works as you'd expect.

On Fri, Mar 31, 2017 at 5:29 PM arisawa <karisawa@gmail.com> wrote:

> Hello,
>
> I was playing with an experimental code on pipe and met with a problem
> which I don’t understand.
>
> the program reads a file and writes it to one end of pipe and then reads
> it from another end of pipe.
> the buffer for writing pipe is named buf0, and for reading pipe is named
> buf.
> and I found the program does not finish unless sizeof(buf) > sizeof(buf0).
> is this a bug or feature of pipe?
>
> Kenji Arisawa
>
> === BEGIN a.c ===
> #include <u.h>
> #include <libc.h>
>
> char *argv0;
>
> void
> usage(void)
> {
>         fprint(2,"usage: %s file\n",argv0);
>         exits("usage");
> }
>
> void
> main(int argc, char *argv[])
> {
>         int fd,pfd[2];
>         char buf[256];
>         char buf0[256];
>         /* need to be sizeof(buf) > sizeof(buf0)
>          * but this condition is very curious to me */
>         int n;
>         char *file;
>         argv0 = argv[0];
>         argc--;argv++;
>         USED(argc);
>         if(argv[0] == nil)
>                 usage();
>         file = argv[0];
>         fd = open(file,OREAD);
>         if(fd < 0)
>                 sysfatal("no such file");
>
>         if(pipe(pfd) < 0)
>                 sysfatal("pipe error");
>         print("pfd: %d %d\n",pfd[0],pfd[1]);
>
>         while((n = read(fd,buf0,sizeof(buf0))) > 0){
>                 print("read: %d %s\n",n,file);
>                 n = write(pfd[1],buf0,n);
>                 print("write: %d\n",n);
>         }
>         close(pfd[1]);
>         while((n = read(pfd[0],buf,sizeof(buf))) > 0){
>                 buf[n] = 0;
>                 print("%d %s\n",n,buf);
>         }
>         print("%d\n",n);
>
>         exits(nil);
> }
> === END a.c ===
>

[-- Attachment #2: Type: text/html, Size: 3909 bytes --]

Re: [9fans] pipe: bug or feature?

[Skip Tavakkolian]

[-- Attachment #1.1: Type: text/plain, Size: 1972 bytes --]

maybe it's a problem with eof detection.  I tried to reproduce it using a
threaded version (see attached), but it works correctly.

On Fri, Mar 31, 2017 at 5:29 PM arisawa <karisawa@gmail.com> wrote:

> Hello,
>
> I was playing with an experimental code on pipe and met with a problem
> which I don’t understand.
>
> the program reads a file and writes it to one end of pipe and then reads
> it from another end of pipe.
> the buffer for writing pipe is named buf0, and for reading pipe is named
> buf.
> and I found the program does not finish unless sizeof(buf) > sizeof(buf0).
> is this a bug or feature of pipe?
>
> Kenji Arisawa
>
> === BEGIN a.c ===
> #include <u.h>
> #include <libc.h>
>
> char *argv0;
>
> void
> usage(void)
> {
>         fprint(2,"usage: %s file\n",argv0);
>         exits("usage");
> }
>
> void
> main(int argc, char *argv[])
> {
>         int fd,pfd[2];
>         char buf[256];
>         char buf0[256];
>         /* need to be sizeof(buf) > sizeof(buf0)
>          * but this condition is very curious to me */
>         int n;
>         char *file;
>         argv0 = argv[0];
>         argc--;argv++;
>         USED(argc);
>         if(argv[0] == nil)
>                 usage();
>         file = argv[0];
>         fd = open(file,OREAD);
>         if(fd < 0)
>                 sysfatal("no such file");
>
>         if(pipe(pfd) < 0)
>                 sysfatal("pipe error");
>         print("pfd: %d %d\n",pfd[0],pfd[1]);
>
>         while((n = read(fd,buf0,sizeof(buf0))) > 0){
>                 print("read: %d %s\n",n,file);
>                 n = write(pfd[1],buf0,n);
>                 print("write: %d\n",n);
>         }
>         close(pfd[1]);
>         while((n = read(pfd[0],buf,sizeof(buf))) > 0){
>                 buf[n] = 0;
>                 print("%d %s\n",n,buf);
>         }
>         print("%d\n",n);
>
>         exits(nil);
> }
> === END a.c ===
>

[-- Attachment #1.2: Type: text/html, Size: 3761 bytes --]

[-- Attachment #2: pipeiotest.c --]
[-- Type: text/x-csrc, Size: 706 bytes --]

#include <u.h>
#include <libc.h>
#include <thread.h>

void
copyfunc(void *v)
{
	int *fd, n;
	char buf[256];
	int tot = 0;

	fd = v;
	while((n = read(fd[0], buf, sizeof buf)) > 0){
		if(write(fd[1], buf, n) != n)
			sysfatal("write: %r");
		tot += n;
		fprint(2, "copied %d byte chunk\n", n);
	}
	fprint(2, "total copied: %d\n", tot);
	close(fd[0]);
	close(fd[1]);
	threadexits(nil);
}

/* copy stdin to stdout through a pipe. run like ./pipetest </lib/constitution >/dev/null */
void
threadmain(int argc, char *argv[])
{
	int fd[4];
	int pfd[2];

	if(pipe(pfd) < 0)
		sysfatal("pipe error");

	fd[0] = 0;
	fd[1] = pfd[0];
	fd[2] = pfd[1];
	fd[3] = 1;

	proccreate(copyfunc, fd, 8192);
	copyfunc(fd+2);
}

[9fans] pipe: bug or feature?

[arisawa]

Hello,

I was playing with an experimental code on pipe and met with a problem which I don’t understand.

the program reads a file and writes it to one end of pipe and then reads it from another end of pipe.
the buffer for writing pipe is named buf0, and for reading pipe is named buf.
and I found the program does not finish unless sizeof(buf) > sizeof(buf0).
is this a bug or feature of pipe?

Kenji Arisawa

=== BEGIN a.c ===
#include <u.h>
#include <libc.h>

char *argv0;

void
usage(void)
{
	fprint(2,"usage: %s file\n",argv0);
	exits("usage");
}

void
main(int argc, char *argv[])
{
	int fd,pfd[2];
	char buf[256];
	char buf0[256];
	/* need to be sizeof(buf) > sizeof(buf0)
	 * but this condition is very curious to me */
	int n;
	char *file;
	argv0 = argv[0];
	argc--;argv++;
	USED(argc);
	if(argv[0] == nil)
		usage();
	file = argv[0];
	fd = open(file,OREAD);
	if(fd < 0)
		sysfatal("no such file");

	if(pipe(pfd) < 0)
		sysfatal("pipe error");
	print("pfd: %d %d\n",pfd[0],pfd[1]);

	while((n = read(fd,buf0,sizeof(buf0))) > 0){
		print("read: %d %s\n",n,file);
		n = write(pfd[1],buf0,n);
		print("write: %d\n",n);
	}
	close(pfd[1]);
	while((n = read(pfd[0],buf,sizeof(buf))) > 0){
		buf[n] = 0;
		print("%d %s\n",n,buf);
	}
	print("%d\n",n);
		
	exits(nil);
}
=== END a.c ===