* [9fans] Streaming 9P is out @ 2011-01-08 7:24 John Floren 2011-01-08 10:45 ` Seed ` (2 more replies) 0 siblings, 3 replies; 11+ messages in thread From: John Floren @ 2011-01-08 7:24 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs Ok, now that the thesis is signed, I feel ready to release my work on streaming for 9P, as outlined in my talk at IWP9; things have changed a lot since that talk, but the general idea remains the same. The repository at https://bitbucket.org/floren/tstream/ contains my code and the thesis documents. I used divergefs to keep my root clean, so that's what's distributed--use divergefs to mount the sys/ in my repo over the /sys on your fs, and you'll probably also want to set up a /386 divergefs mount too. I forgot to add my modified cp, and I'd add it but I'm on a bad wifi connection that seems to be giving hg a bad time. There is a modified exportfs in /sys/src/cmd/exportfs, and really it's not that hard to modify user programs like cp to do streaming, it's all explained in the thesis document. When I get a better connection I'll add the streaming cp to the repo anyway. Anyway, it's far from perfect, but any comments, patches, or suggestions are appreciated. John ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [9fans] Streaming 9P is out 2011-01-08 7:24 [9fans] Streaming 9P is out John Floren @ 2011-01-08 10:45 ` Seed 2011-01-08 13:11 ` Bruce Ellis 2011-01-08 15:16 ` David Leimbach 2011-01-09 0:31 ` cinap_lenrek 2 siblings, 1 reply; 11+ messages in thread From: Seed @ 2011-01-08 10:45 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs 11 at 6:24 PM, John Floren <slawmaster@gmail.com> wrote: > Anyway, it's far from perfect, but any comments, patches, or > suggestions are appreciated. Hi John, I just quickly went through your thesis, was nice to read. If the stream() function in you thesis is the last one it seems that there is an obvious memory leak when you return nil. Cool project for a master thesis though. -- ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [9fans] Streaming 9P is out 2011-01-08 10:45 ` Seed @ 2011-01-08 13:11 ` Bruce Ellis 0 siblings, 0 replies; 11+ messages in thread From: Bruce Ellis @ 2011-01-08 13:11 UTC (permalink / raw) To: seedrubbish, Fans of the OS Plan 9 from Bell Labs Cool. We spoke briefly in Seattle. It's perfect for CDM Midi and SPDIF. I haven't seen the video of you as the sacrificial lamb ... I need it. brucee On 1/8/11, Seed <seedrubbish@gmail.com> wrote: > 11 at 6:24 PM, John Floren <slawmaster@gmail.com> wrote: >> Anyway, it's far from perfect, but any comments, patches, or >> suggestions are appreciated. > > Hi John, > > I just quickly went through your thesis, was nice to read. > If the stream() function in you thesis is the last one it seems that > there is an obvious memory leak when you return nil. > Cool project for a master thesis though. > -- > > ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [9fans] Streaming 9P is out 2011-01-08 7:24 [9fans] Streaming 9P is out John Floren 2011-01-08 10:45 ` Seed @ 2011-01-08 15:16 ` David Leimbach 2011-01-09 0:31 ` cinap_lenrek 2 siblings, 0 replies; 11+ messages in thread From: David Leimbach @ 2011-01-08 15:16 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs [-- Attachment #1: Type: text/plain, Size: 1135 bytes --] Awesome! On Fri, Jan 7, 2011 at 11:24 PM, John Floren <slawmaster@gmail.com> wrote: > Ok, now that the thesis is signed, I feel ready to release my work on > streaming for 9P, as outlined in my talk at IWP9; things have changed > a lot since that talk, but the general idea remains the same. The > repository at https://bitbucket.org/floren/tstream/ contains my code > and the thesis documents. I used divergefs to keep my root clean, so > that's what's distributed--use divergefs to mount the sys/ in my repo > over the /sys on your fs, and you'll probably also want to set up a > /386 divergefs mount too. > > I forgot to add my modified cp, and I'd add it but I'm on a bad wifi > connection that seems to be giving hg a bad time. There is a modified > exportfs in /sys/src/cmd/exportfs, and really it's not that hard to > modify user programs like cp to do streaming, it's all explained in > the thesis document. When I get a better connection I'll add the > streaming cp to the repo anyway. > > Anyway, it's far from perfect, but any comments, patches, or > suggestions are appreciated. > > > John > > [-- Attachment #2: Type: text/html, Size: 1544 bytes --] ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [9fans] Streaming 9P is out 2011-01-08 7:24 [9fans] Streaming 9P is out John Floren 2011-01-08 10:45 ` Seed 2011-01-08 15:16 ` David Leimbach @ 2011-01-09 0:31 ` cinap_lenrek 2011-01-09 0:46 ` John Floren 2 siblings, 1 reply; 11+ messages in thread From: cinap_lenrek @ 2011-01-09 0:31 UTC (permalink / raw) To: 9fans [-- Attachment #1: Type: text/plain, Size: 1581 bytes --] the use of tcp to get flow control is an interesting idea. some remaks: the client only gets the dial string of a new tcp connection. an attacker could guess the port numbers and take over the stream. or the client might be delayed before it makes the dial() causing it to get the wrong stream or some other random tcp connection at the same port. the initial 9p channel might be encrypted (common when you use cpu), but the stream data excapes as cleartext... compromising security. on the implementation, the server might give the wrong ip address (because he is behind some nat gateway) causing the dial to fail. i think the stream() call should fallback to the original filedescriptor then. why do we pass the offset and mode in the pstream syscall? devmnt should know the current offset and mode from the Chan structure right? i think you could simplify the userspace interface a little bit further. for example, i see no use for the Stream structure and sread/swrite as they never fall back on the old filedescriptor and all the decisions are already made in the stream() call (except the isread check). what about this: newfd = stream(oldfd, ....); int stream(int fd, vlong offset, char isread) { for(;;){ char addr[Maxstring]; int r; if((r = pstream(fd, addr, offset, isread)) < 0) break; if(addr[0] == 0) break; if((r = dial(addr, 0, 0, 0)) < 0) break; return r; } /* server doesnt support Tstream */ if((fd = dup(fd, -1)) >= 0) seek(fd, offset, 0); return fd; } -- cinap [-- Attachment #2: Type: message/rfc822, Size: 3964 bytes --] From: John Floren <slawmaster@gmail.com> To: Fans of the OS Plan 9 from Bell Labs <9fans@9fans.net> Subject: [9fans] Streaming 9P is out Date: Fri, 7 Jan 2011 23:24:37 -0800 Message-ID: <AANLkTik4_kOt7UGvL1=wL=gEOAs6Dmn=ympJ_jiX5pV4@mail.gmail.com> Ok, now that the thesis is signed, I feel ready to release my work on streaming for 9P, as outlined in my talk at IWP9; things have changed a lot since that talk, but the general idea remains the same. The repository at https://bitbucket.org/floren/tstream/ contains my code and the thesis documents. I used divergefs to keep my root clean, so that's what's distributed--use divergefs to mount the sys/ in my repo over the /sys on your fs, and you'll probably also want to set up a /386 divergefs mount too. I forgot to add my modified cp, and I'd add it but I'm on a bad wifi connection that seems to be giving hg a bad time. There is a modified exportfs in /sys/src/cmd/exportfs, and really it's not that hard to modify user programs like cp to do streaming, it's all explained in the thesis document. When I get a better connection I'll add the streaming cp to the repo anyway. Anyway, it's far from perfect, but any comments, patches, or suggestions are appreciated. John ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [9fans] Streaming 9P is out 2011-01-09 0:31 ` cinap_lenrek @ 2011-01-09 0:46 ` John Floren 2011-01-09 1:05 ` erik quanstrom ` (2 more replies) 0 siblings, 3 replies; 11+ messages in thread From: John Floren @ 2011-01-09 0:46 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs On Sat, Jan 8, 2011 at 4:31 PM, <cinap_lenrek@gmx.de> wrote: > the use of tcp to get flow control is an interesting idea. > > some remaks: > > the client only gets the dial string of a new tcp connection. an > attacker could guess the port numbers and take over the stream. or > the client might be delayed before it makes the dial() causing it to > get the wrong stream or some other random tcp connection at the same > port. > > the initial 9p channel might be encrypted (common when you use > cpu), but the stream data excapes as cleartext... compromising security. > I had thought about the security problem. I acknowledge that it's possible for an attacker to guess the port, but he has to guess in that small time frame between when the server opens the port and the client connects. I haven't heard of FTP having problems with that, though. And, of course, if you encrypt the streaming data, it doesn't matter if an attacker guesses the port. > on the implementation, the server might give the wrong ip address > (because he is behind some nat gateway) causing the dial to fail. i think > the stream() call should fallback to the original filedescriptor then. I had considered this case (the server doesn't know which IP to give) but managed to forget to include the simple check "did dial fail?" > why do we pass the offset and mode in the pstream syscall? devmnt > should know the current offset and mode from the Chan structure right? > > i think you could simplify the userspace interface a little bit > further. for example, i see no use for the Stream structure and > sread/swrite as they never fall back on the old filedescriptor and all > the decisions are already made in the stream() call (except the > isread check). > > what about this: > > newfd = stream(oldfd, ....); > > int > stream(int fd, vlong offset, char isread) > { > for(;;){ > char addr[Maxstring]; > int r; > > if((r = pstream(fd, addr, offset, isread)) < 0) > break; > if(addr[0] == 0) > break; > if((r = dial(addr, 0, 0, 0)) < 0) > break; > > return r; > } > > /* server doesnt support Tstream */ > if((fd = dup(fd, -1)) >= 0) > seek(fd, offset, 0); > return fd; > } > This ought to work, but I wanted to establish a difference between regular fds, which you can read and write and seek and all that goodness, and streams, which are for reading OR writing sequentially. I also wanted to give an error if you tried to read from a write stream and vice versa; your stream() doesn't seem to do that. Like I said, it's not perfect, and there's probably a better way to do it. John ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [9fans] Streaming 9P is out 2011-01-09 0:46 ` John Floren @ 2011-01-09 1:05 ` erik quanstrom 2011-01-09 1:53 ` cinap_lenrek 2011-01-09 1:58 ` cinap_lenrek 2 siblings, 0 replies; 11+ messages in thread From: erik quanstrom @ 2011-01-09 1:05 UTC (permalink / raw) To: 9fans > This ought to work, but I wanted to establish a difference between > regular fds, which you can read and write and seek and all that > goodness, and streams, which are for reading OR writing sequentially. i'm not sure why one would want this. there are already fds on which must be read or written sequentially. they're called pipes. - erik ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [9fans] Streaming 9P is out 2011-01-09 0:46 ` John Floren 2011-01-09 1:05 ` erik quanstrom @ 2011-01-09 1:53 ` cinap_lenrek 2011-01-09 2:24 ` Lyndon Nerenberg (VE6BBM/VE7TFX) 2011-01-09 1:58 ` cinap_lenrek 2 siblings, 1 reply; 11+ messages in thread From: cinap_lenrek @ 2011-01-09 1:53 UTC (permalink / raw) To: 9fans [-- Attachment #1: Type: text/plain, Size: 355 bytes --] thinking about it... why not just let stream() fail and let the program decide if it makes sense to continue without it? way simpler than the struct Stream bookkeeping or returning a pipe and spawning a relay process in the background. fd = open(....); /* use streaming if possible */ if((sfd = stream(fd)) >= 0) dup(sfd, fd); -- cinap [-- Attachment #2: Type: message/rfc822, Size: 6773 bytes --] From: John Floren <slawmaster@gmail.com> To: Fans of the OS Plan 9 from Bell Labs <9fans@9fans.net> Subject: Re: [9fans] Streaming 9P is out Date: Sat, 8 Jan 2011 16:46:10 -0800 Message-ID: <AANLkTi=y4W7OdBSOxBnmnfuE4FOYr4NUv7n2dEXiAJgS@mail.gmail.com> On Sat, Jan 8, 2011 at 4:31 PM, <cinap_lenrek@gmx.de> wrote: > the use of tcp to get flow control is an interesting idea. > > some remaks: > > the client only gets the dial string of a new tcp connection. an > attacker could guess the port numbers and take over the stream. or > the client might be delayed before it makes the dial() causing it to > get the wrong stream or some other random tcp connection at the same > port. > > the initial 9p channel might be encrypted (common when you use > cpu), but the stream data excapes as cleartext... compromising security. > I had thought about the security problem. I acknowledge that it's possible for an attacker to guess the port, but he has to guess in that small time frame between when the server opens the port and the client connects. I haven't heard of FTP having problems with that, though. And, of course, if you encrypt the streaming data, it doesn't matter if an attacker guesses the port. > on the implementation, the server might give the wrong ip address > (because he is behind some nat gateway) causing the dial to fail. i think > the stream() call should fallback to the original filedescriptor then. I had considered this case (the server doesn't know which IP to give) but managed to forget to include the simple check "did dial fail?" > why do we pass the offset and mode in the pstream syscall? devmnt > should know the current offset and mode from the Chan structure right? > > i think you could simplify the userspace interface a little bit > further. for example, i see no use for the Stream structure and > sread/swrite as they never fall back on the old filedescriptor and all > the decisions are already made in the stream() call (except the > isread check). > > what about this: > > newfd = stream(oldfd, ....); > > int > stream(int fd, vlong offset, char isread) > { > for(;;){ > char addr[Maxstring]; > int r; > > if((r = pstream(fd, addr, offset, isread)) < 0) > break; > if(addr[0] == 0) > break; > if((r = dial(addr, 0, 0, 0)) < 0) > break; > > return r; > } > > /* server doesnt support Tstream */ > if((fd = dup(fd, -1)) >= 0) > seek(fd, offset, 0); > return fd; > } > This ought to work, but I wanted to establish a difference between regular fds, which you can read and write and seek and all that goodness, and streams, which are for reading OR writing sequentially. I also wanted to give an error if you tried to read from a write stream and vice versa; your stream() doesn't seem to do that. Like I said, it's not perfect, and there's probably a better way to do it. John ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [9fans] Streaming 9P is out 2011-01-09 1:53 ` cinap_lenrek @ 2011-01-09 2:24 ` Lyndon Nerenberg (VE6BBM/VE7TFX) 2011-01-09 2:42 ` ron minnich 0 siblings, 1 reply; 11+ messages in thread From: Lyndon Nerenberg (VE6BBM/VE7TFX) @ 2011-01-09 2:24 UTC (permalink / raw) To: 9fans > thinking about it... why not just let stream() fail and let the program > decide if it makes sense to continue without it? Exactly what I was thinking. If the program requires the semantics of stream(), it should be able to reliably discover when they aren't available. ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [9fans] Streaming 9P is out 2011-01-09 2:24 ` Lyndon Nerenberg (VE6BBM/VE7TFX) @ 2011-01-09 2:42 ` ron minnich 0 siblings, 0 replies; 11+ messages in thread From: ron minnich @ 2011-01-09 2:42 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs On Sat, Jan 8, 2011 at 6:24 PM, Lyndon Nerenberg (VE6BBM/VE7TFX) <lyndon@orthanc.ca> wrote: >> thinking about it... why not just let stream() fail and let the program >> decide if it makes sense to continue without it? > > Exactly what I was thinking. If the program requires the semantics of > stream(), it should be able to reliably discover when they aren't > available. I prefer John's approach. It's a trivial change and doesn't require most programmers to think too hard. Bear in mind that if you don't want to use John's functions, you don't have to: all those decisions are made at library level. Streaming is available whether or not you use his "client" functions. But rather than talk this to death, the best bet is to try it out and see. A good first cut is to pick a bunch of programs, mod them to use streams, and we can see which approach makes the most sense. Learn By Doing. ron ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [9fans] Streaming 9P is out 2011-01-09 0:46 ` John Floren 2011-01-09 1:05 ` erik quanstrom 2011-01-09 1:53 ` cinap_lenrek @ 2011-01-09 1:58 ` cinap_lenrek 2 siblings, 0 replies; 11+ messages in thread From: cinap_lenrek @ 2011-01-09 1:58 UTC (permalink / raw) To: 9fans [-- Attachment #1: Type: text/plain, Size: 103 bytes --] that, should have been: if((sfd = stream(fd)) >= 0){ dup(sfd, fd); close(sfd); } -- cinap [-- Attachment #2: Type: message/rfc822, Size: 6773 bytes --] From: John Floren <slawmaster@gmail.com> To: Fans of the OS Plan 9 from Bell Labs <9fans@9fans.net> Subject: Re: [9fans] Streaming 9P is out Date: Sat, 8 Jan 2011 16:46:10 -0800 Message-ID: <AANLkTi=y4W7OdBSOxBnmnfuE4FOYr4NUv7n2dEXiAJgS@mail.gmail.com> On Sat, Jan 8, 2011 at 4:31 PM, <cinap_lenrek@gmx.de> wrote: > the use of tcp to get flow control is an interesting idea. > > some remaks: > > the client only gets the dial string of a new tcp connection. an > attacker could guess the port numbers and take over the stream. or > the client might be delayed before it makes the dial() causing it to > get the wrong stream or some other random tcp connection at the same > port. > > the initial 9p channel might be encrypted (common when you use > cpu), but the stream data excapes as cleartext... compromising security. > I had thought about the security problem. I acknowledge that it's possible for an attacker to guess the port, but he has to guess in that small time frame between when the server opens the port and the client connects. I haven't heard of FTP having problems with that, though. And, of course, if you encrypt the streaming data, it doesn't matter if an attacker guesses the port. > on the implementation, the server might give the wrong ip address > (because he is behind some nat gateway) causing the dial to fail. i think > the stream() call should fallback to the original filedescriptor then. I had considered this case (the server doesn't know which IP to give) but managed to forget to include the simple check "did dial fail?" > why do we pass the offset and mode in the pstream syscall? devmnt > should know the current offset and mode from the Chan structure right? > > i think you could simplify the userspace interface a little bit > further. for example, i see no use for the Stream structure and > sread/swrite as they never fall back on the old filedescriptor and all > the decisions are already made in the stream() call (except the > isread check). > > what about this: > > newfd = stream(oldfd, ....); > > int > stream(int fd, vlong offset, char isread) > { > for(;;){ > char addr[Maxstring]; > int r; > > if((r = pstream(fd, addr, offset, isread)) < 0) > break; > if(addr[0] == 0) > break; > if((r = dial(addr, 0, 0, 0)) < 0) > break; > > return r; > } > > /* server doesnt support Tstream */ > if((fd = dup(fd, -1)) >= 0) > seek(fd, offset, 0); > return fd; > } > This ought to work, but I wanted to establish a difference between regular fds, which you can read and write and seek and all that goodness, and streams, which are for reading OR writing sequentially. I also wanted to give an error if you tried to read from a write stream and vice versa; your stream() doesn't seem to do that. Like I said, it's not perfect, and there's probably a better way to do it. John ^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2011-01-09 2:42 UTC | newest] Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2011-01-08 7:24 [9fans] Streaming 9P is out John Floren 2011-01-08 10:45 ` Seed 2011-01-08 13:11 ` Bruce Ellis 2011-01-08 15:16 ` David Leimbach 2011-01-09 0:31 ` cinap_lenrek 2011-01-09 0:46 ` John Floren 2011-01-09 1:05 ` erik quanstrom 2011-01-09 1:53 ` cinap_lenrek 2011-01-09 2:24 ` Lyndon Nerenberg (VE6BBM/VE7TFX) 2011-01-09 2:42 ` ron minnich 2011-01-09 1:58 ` cinap_lenrek
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).