From mboxrd@z Thu Jan 1 00:00:00 1970 Message-ID: Date: Sat, 16 Mar 2013 03:41:52 +0000 From: mycroftiv@sphericalharmony.com To: 9fans@9fans.net MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: 7bit Subject: [9fans] A personal history of pipe multiplexing Topicbox-Message-UUID: 2ba28512-ead8-11e9-9d60-3106f5b1d025 A short personal history of Plan 9 pipe muxing This post is purely personal, historical, technical, and makes zero comment on anything other than my own personal history of software development. In the summer of 2009, I'd been working with Plan 9 for slightly less than a year, and I was mostly working on learning how to program C, by working through k&r along with Nemo's "Intro to OS Abstractions" book. My personal work on pipes in Plan 9 started with the "window" script and section 12.9 of Nemo's book, on Rio. The "window" script and the demos showed by Nemo show how Rio is just a fileserver and the stuff printed on the windows comes from /dev/cons. I thought all of this was really cool, and I played with a lot, and tried to find new tricks to do. I found some things I thought were interesting and possibly useful and I made this blog post on the old 9gridchan site: http://www.9gridchan.org/blog/2009/06/17/1_blog_post/ This blog post is basically a tiny evolution of the window scripts and Nemo's demos with one VERY signifacnt difference. The "tee" command is inserted into the pipelines, like this: exec rc <{tee /dev/cons} |tee /dev/cons So this already "muxes the pipes" in a completely traditional way. It makes sense as something to do in Plan 9 because of how Rio works, you can make a new blank window and have the same shell write to 2 places. This seemed so easy and awesome to me, and I kept working, and the very next day you can see a blog post about doing "screen" style shell persistence with these tricks. http://www.9gridchan.org/blog/2009/06/18/1_blog_post/ Now this was fun but it was hardly robust and really usable. Maybe I could build on this and make a usable "screen" for Plan 9 with a little bit of C code to glue it together. http://www.9gridchan.org/older_stuff/alphasrvrc/ I don't know exactly when it was from - it was obviously before the "real" Iosrv was released, but you can see that this program is both a clear evolution of the earlier simple "tee" tricks and also moves the implementation to something a lot more sophisticated - but it was STILL basically a hack job, even though I felt like I had written a "real" c program now, not just a beginner's exercise. I continued working intensively with Nemo's book on os abstractions, and you can see very easily that iosrv's parallelized lock-based design was worked out by following along with Nemo's examples on locking and the "bounded buffer" problem in his work. I had heard that writing parallel code was hard, and it sure was - if you look at the final Iosrv, you can see an incredibly heavy debugging interface. I remember countless hours smashing data throught the pipes, watching everything come crashing to a halt as my locking implementation tangled itself into a heap, and then trying to figure out why Lock B wasn't released even though process C should have hit Lock A and... So anyway after working very hard to get things basically usable, I released Iosrv which is still up on sources. Iosrv is a weird way of implementing a 9pfs, because I didn't know how to write a 9pfs then! It makes pipes with the pipe device, creates buffers around them, and turns itself into a 9pfs just by wrapping the whole thing inside exportfs and telling exportfs: "You make me a 9p fs, I don't know how!" Iosrv also had this extraordinarily ugly command language. There was a lot of power in it, and things like starting/stopping all the pipes, individually killing attached processes, are all there - but it wasn't graceful to use, and it seemed like I was making a ridiculous amount of exportf processes for what the program was doing. So, I decided to learn to make a REAL 9p fs which implemented the exact same abstractions and ideas. I wanted to do it in the simplest way possible so I worked by modifying the tiny example ramfs.c in /sys/src/lib9p which is mostly a demo of how to use 9pfile. I worked to make hubfs pretty much a direct "port" of iosrv to being a real 9pfs, and I also worked very hard on making the usage model much more graceful and less awkward than Iosrv's per-pipe command language. So anyway, that was how I ended up writing hubfs. You can see even from my original post that I was very conscious that what I was doing was not a "new invention" - in fact, I was making a claim of UN-originality, because I was linking quotes from interviews showing that I believed hubfs was actually exactly what McIlroy had originally described, freeform pipe muxing. Even within the world of Plan 9, before I was doing pipe muxing, there has been aux/consolefs forever, and Inferno has drawmux which RSC mentioned as an idea for future Plan 9 work after I posted about Iosrv. I wanted to trace my own history in detail just to make it exactly explicit how I came to write Iosrv and Hubfs, and I also think it is an interesting story of how a beginner - me - was able to gradually develop skill in programming and using an operating system, and develop a piece of software that has some nice uses, because I still use hubfs in almost exactly the same form as it was initially released, and I think it is a nice simple tool. Thanks for your time! Peace and love to all Plan 9 users and developers, Ben Kidwell "mycroftiv"