* Re: [9fans] critique of sockets API [not found] <mailman.1007.1244590421.1513.9fans@9fans.net> @ 2009-06-10 22:50 ` Bhanu Nagendra Pisupati 2009-06-11 12:34 ` erik quanstrom 0 siblings, 1 reply; 24+ messages in thread From: Bhanu Nagendra Pisupati @ 2009-06-10 22:50 UTC (permalink / raw) To: 9fans > Did you do this on Plan 9 or bring some of the filesystem sanity of another OS? Actually my work was based on 9P based synthetic filesystems implemented using Npfs (see http://docs.google.com/Doc?id=dcb7zf48_503q8j84) These filesystems resided in resource constrained embedded devices which could not run Plan 9 in whole. ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [9fans] critique of sockets API 2009-06-10 22:50 ` [9fans] critique of sockets API Bhanu Nagendra Pisupati @ 2009-06-11 12:34 ` erik quanstrom 2009-06-11 14:38 ` [9fans] embedfs source? (was: Re: critique of sockets API) Nathaniel W Filardo 0 siblings, 1 reply; 24+ messages in thread From: erik quanstrom @ 2009-06-11 12:34 UTC (permalink / raw) To: 9fans On Wed Jun 10 18:52:19 EDT 2009, bpisupat@cs.indiana.edu wrote: > > > Did you do this on Plan 9 or bring some of the filesystem sanity of another OS? > > Actually my work was based on 9P based synthetic filesystems > implemented using Npfs (see > http://docs.google.com/Doc?id=dcb7zf48_503q8j84) > > These filesystems resided in resource constrained embedded devices which > could not run Plan 9 in whole. > > do you have a comparison to other similar projects like 1. styx on a brick http://www.vitanuova.com/inferno/rcx_paper.html 2. 9p for imbedded devices http://iwp9.inf.uth.gr/iwp9_proceedings08.pdf and or other information on your work? in particular, what hardware are you using? - erik ^ permalink raw reply [flat|nested] 24+ messages in thread
* [9fans] embedfs source? (was: Re: critique of sockets API) 2009-06-11 12:34 ` erik quanstrom @ 2009-06-11 14:38 ` Nathaniel W Filardo 0 siblings, 0 replies; 24+ messages in thread From: Nathaniel W Filardo @ 2009-06-11 14:38 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs [-- Attachment #1: Type: text/plain, Size: 273 bytes --] On Thu, Jun 11, 2009 at 08:34:57AM -0400, erik quanstrom wrote: > 2. 9p for imbedded devices > http://iwp9.inf.uth.gr/iwp9_proceedings08.pdf Is the source for embedfs available? Google doesn't seem to think so, but my googlefu might be lacking. Thanks. --nwf; [-- Attachment #2: Type: application/pgp-signature, Size: 204 bytes --] ^ permalink raw reply [flat|nested] 24+ messages in thread
[parent not found: <mailman.1.1244808001.26495.9fans@9fans.net>]
* Re: [9fans] critique of sockets API [not found] <mailman.1.1244808001.26495.9fans@9fans.net> @ 2009-06-13 1:39 ` Bhanu Nagendra Pisupati 0 siblings, 0 replies; 24+ messages in thread From: Bhanu Nagendra Pisupati @ 2009-06-13 1:39 UTC (permalink / raw) To: 9fans >do you have a comparison to other similar projects like >1. styx on a brick > http://www.vitanuova.com/inferno/rcx_paper.html >2. 9p for imbedded devices > http://iwp9.inf.uth.gr/iwp9_proceedings08.pdf > >and or other information on your work? in particular, >what hardware are you using? The "Styx on a Brick" paper was a definition inspiration for my work. I have acknowledged as much in my publications. I completed my PhD in '07 before the "embedFS" filesystem paper from IW9P came out last year, and I have not done a formal comparison per se. But reading through the text, I do see similarities in some of our ideas, such as having one outstanding messag, of read/read message size, using 9P (rather than 9p2000) etc. The IW9P paper does not give size numbers, but the basic 9P filesystem in my implementation was about 15KB in size when implemented on a 32-bit RISC processor named Nios (an embedded processor from Altera). Most of my work was done using the Nios embedded FPGA environment from Altera. For more information: http://www.cs.indiana.edu/cgi-bin/techreports/TRNNN.cgi?trnum=TR647 http://www.cs.indiana.edu/pub/techreports/TR647.html/embed.html ^ permalink raw reply [flat|nested] 24+ messages in thread
[parent not found: <b0a72b5826eb44300d3603f585859910@quanstro.net>]
* Re: [9fans] critique of sockets API [not found] <b0a72b5826eb44300d3603f585859910@quanstro.net> @ 2009-06-11 14:54 ` Eris Discordia 2009-06-11 18:24 ` erik quanstrom 0 siblings, 1 reply; 24+ messages in thread From: Eris Discordia @ 2009-06-11 14:54 UTC (permalink / raw) To: 9fans > the signal context is the original calling thread. unless > ms' diagram is incorrect, this is a single threaded operation; > only the i/o originator can process the event. so the plan 9 Of course it is single-threaded operation. That's the very idea behind using callbacks. Originally they were used to allow applications to do asynchronous I/O on earlier Windows incarnations that had no concept of threading. When threading became available it was correctly considered hazardous, as with any other form of concurrency, so it was and is avoided unless absolutely necessary. Callbacks survived the introduction of threading. There are actually three thinkable options here: read-and-wait (synchronous), subscribe-and-continue (asynchronous using callback), and continue-until-notified (asynchronous using message queue). My point was that as of some time ago, more than a decade I believe, Windows began offering all three options which may be used on their own or in tandem. The three options can be sorted in order of increasing initial cost: callback, message queue, thread (and/or "lightweight process"). Average cost reverses that ordering. The application determines which strategy is better. On the subject of who gets to process a certain event on Windows it should be noted that any process/thread that subscribes for an event will be notified, if an opportunity for subscription is provided in the first place. In case of a read other processes/threads have no notion that a specific read has been requested and they will never want to process the results of a read they haven't requested. Interface and other events that may concern more than one process/thread, on the other hand, can be processed by any worker that opts to process them. A chain of handlers with varying priorities is very easy to construct within this framework. > there's plenty of overlapped i/o in plan 9 — it's in the > disk and network device drivers. even stripping away > the register fiddling, it's not an i/o model that's attractive; > that's the reason all the details are hidden in the kernel. I might as well repeat myself: choice of strategy depends on the application. Given choice programmers can decide on which strategy or combination of strategies works best. Without choice, well, they will just live with what's available. It is common with Windows programmers to use a combination of threading, callbacks, and synchronous I/O that best represents the application's workings. GUI is often run in its own thread which is made to wait only on I/O that crucially affects future interactions, e.g. a user "Open File" request. Processing of sporadic network events, e.g. reports from a network resource, is done in auxiliary message queues. Light network operations are delegated to a single thread that uses callbacks. Only heavy network operations are aggressively threaded. Most notable here is the fact that allowing for various compatible options to grow within the same system can only enrich it while insisting on "The One Right Way" always leaves open the question of whether a different choice of strategy on the same platform, were a different choice available, would have yielded better results. > are you saying the author(s) had their windows blinders on > and might not have considered other options? I believe at this point it is clear that all thinkable options are available on the platform you refer to as (horse) "blinders" but that really isn't important. The author(s) had criticized a specific I/O model without mentioning, or probably even knowing, that alternatives existed and could be profiled for tangible, definitive results. I merely pointed that out. --On Thursday, June 11, 2009 08:16 -0400 erik quanstrom <quanstro@quanstro.net> wrote: > On Thu Jun 11 04:12:13 EDT 2009, eris.discordia@gmail.com wrote: >> > i don't think i understand what you're getting at. >> > it could be that the blog was getting at the fact that select >> > funnels a bunch of independent i/o down to one process. >> > it's an effective technique when (a) threads are not available >> > and (b) processing is very fast. >> >> This might help: what he is getting at is probably the question of why >> not make possible network applications that consist of a bunch of >> callbacks or a mix of callbacks and listener/worker threads. Windows >> implements both synchronous and asynchronous I/O. Threads are >> available. Callbacks, too, as well as message queues. > > are you saying the author(s) had their windows blinders on > and might not have considered other options? > > my windows-fu is very low. but according to microsoft > http://msdn.microsoft.com/en-us/library/aa365683(VS.85).aspx > windows "asynchronous" (overlapped) i/o signals the calling thread, so > the signal context is the original calling thread. unless > ms' diagram is incorrect, this is a single threaded operation; > only the i/o originator can process the event. so the plan 9 > model would seem to me to be better threaded and i think > the CSP-style of the plan 9 model makes it easier to > reason about. and has already been implemented under > unix without changing the kernel. see p9p. > >> Ideally, it is the programmer's informed choice >> based on their understanding of their application's priorities whether >> to use callbacks, listener/worker threads, message queues, or a >> combination. Someone may find it worth the effort to compare these >> approaches on a platform that provides both. (COM is notorious for >> implementing things through callback that get some wrapping of one's >> head around them before making sense.) > > there's plenty of overlapped i/o in plan 9 — it's in the > disk and network device drivers. even stripping away > the register fiddling, it's not an i/o model that's attractive; > that's the reason all the details are hidden in the kernel. > > - erik ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [9fans] critique of sockets API 2009-06-11 14:54 ` Eris Discordia @ 2009-06-11 18:24 ` erik quanstrom 2009-06-11 21:21 ` Eris Discordia [not found] ` <D9FE8C51EE2D568C05E555DD@192.168.1.2> 0 siblings, 2 replies; 24+ messages in thread From: erik quanstrom @ 2009-06-11 18:24 UTC (permalink / raw) To: 9fans > I might as well repeat myself: choice of strategy depends on the > application. Given choice programmers can decide on which strategy or > combination of strategies works best. Without choice, well, they will just > live with what's available. this is a very deep philosophical divide between windows and systems like plan 9, and research unix. the approach the labs took was to provide a minimal set of primatives from which one can build what's needed. compare just r?fork and exec with all the spawn variants windows has. i think you're trying to argue that — a priori — choice is good? but given that plan 9 is about having a system that's easy to understand and modify, i would think that it would be tough to demonstrate that asyncronous i/o or callbacks could make the system (or even applications) simplier. i doubt that they would make the system more efficient, either. do you have examples that demonstrate either? > One Right Way" always leaves open the question of whether a different > choice of strategy on the same platform, were a different choice available, > would have yielded better results. clearly if that position is accepted, computer science is a solved problem; we should all put our heads down and just code up the accepted wisdom. that's not the position i subscribe to. and since plan 9 is simple and easy to change, it makes an ideal system for someone who wants to try new things. - erik ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [9fans] critique of sockets API 2009-06-11 18:24 ` erik quanstrom @ 2009-06-11 21:21 ` Eris Discordia 2009-06-11 23:41 ` erik quanstrom [not found] ` <D9FE8C51EE2D568C05E555DD@192.168.1.2> 1 sibling, 1 reply; 24+ messages in thread From: Eris Discordia @ 2009-06-11 21:21 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs > but given that plan 9 is about having a system that's easy > to understand and modify, i would think that it would be > tough to demonstrate that asyncronous i/o or callbacks > could make the system (or even applications) simplier. > i doubt that they would make the system more efficient, > either. > > do you have examples that demonstrate either? I can't claim I have anything in code which would be necessary for an actual demonstration or for going beyond the "talk talk talk" stage. I can, however, present one simple case: in some applications asynchronous name resolving is a must and it can be realized by either of threads or callbacks. Crawlers and scanners come to mind. Spawning threads for DNS requests could be more costly than registering a set of callbacks within one thread and then harvesting the results within that same thread. > one can build what's needed. compare just r?fork and exec > with all the spawn variants windows has. The actual number of calls in CreateThread and CreateProcess family is rather small. Most calls are wrappers for more basic calls but we don't want to go into that. > i think you're trying to argue that — a priori — choice is good? I believe it is. How many of us are using strictly RISC machines on our desks today? Extending the set of available primitives and adding to the complexity of each primitive both are natural steps in the development of computer systems as they see more use. Limiting options doesn't seem to me to be an effective way of encouraging good programming practice. It can, however, successfully discourage potential middle users. > that's not the position i subscribe to. and since plan 9 > is simple and easy to change, it makes an ideal system > for someone who wants to try new things. And after the new things are tried out and, one may hope, proven advantageous? I think the next step would be incorporating them into the system. A process that in due time will make the system less simple and easy to change but more immediately useful. I see this more as a difference of intended audience than a difference of taste or philosophy. The real question is whom you imagine as your system's middle user: a wage-earning programmer or a researcher. Were I a programmer who worked for pay I'd very much appreciate being given every possible option that would let me do my job easier, faster, and, of course, more properly. --On Thursday, June 11, 2009 14:24 -0400 erik quanstrom <quanstro@quanstro.net> wrote: >> I might as well repeat myself: choice of strategy depends on the >> application. Given choice programmers can decide on which strategy or >> combination of strategies works best. Without choice, well, they will >> just live with what's available. > > this is a very deep philosophical divide between windows and > systems like plan 9, and research unix. the approach the labs > took was to provide a minimal set of primatives from which > one can build what's needed. compare just r?fork and exec > with all the spawn variants windows has. > > i think you're trying to argue that — a priori — choice is good? > > but given that plan 9 is about having a system that's easy > to understand and modify, i would think that it would be > tough to demonstrate that asyncronous i/o or callbacks > could make the system (or even applications) simplier. > i doubt that they would make the system more efficient, > either. > > do you have examples that demonstrate either? > >> One Right Way" always leaves open the question of whether a different >> choice of strategy on the same platform, were a different choice >> available, would have yielded better results. > > clearly if that position is accepted, computer science is a > solved problem; we should all put our heads down and just > code up the accepted wisdom. > > that's not the position i subscribe to. and since plan 9 > is simple and easy to change, it makes an ideal system > for someone who wants to try new things. > > - erik > ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [9fans] critique of sockets API 2009-06-11 21:21 ` Eris Discordia @ 2009-06-11 23:41 ` erik quanstrom 2009-06-12 4:32 ` Paul Lalonde 2009-06-12 7:19 ` Eris Discordia 0 siblings, 2 replies; 24+ messages in thread From: erik quanstrom @ 2009-06-11 23:41 UTC (permalink / raw) To: 9fans > > i think you're trying to argue that — a priori — choice is good? > > I believe it is. How many of us are using strictly RISC machines on our > desks today? Extending the set of available primitives and adding to the > complexity of each primitive both are natural steps in the development of > computer systems as they see more use. just to correct a basic fact, the size of the instruction set doesn't define RISC instruction set (http://en.wikipedia.org/wiki/RISC "instruction set size"). can you cite any references that claim that the size of intel's instruction set has contributed to its success? could it be that since transistors are very cheep, adding instructions is simply the cheapest go-faster trick? > Limiting options doesn't seem to me > to be an effective way of encouraging good programming practice. It can, > however, successfully discourage potential middle users. what is a middle user? and why would such a person be discouraged by having to learn fewer things? does the myriad of different ways to write an if statement in perl make it more useable? readable? > > that's not the position i subscribe to. and since plan 9 > > is simple and easy to change, it makes an ideal system > > for someone who wants to try new things. > > And after the new things are tried out and, one may hope, proven > advantageous? I think the next step would be incorporating them into the > system. A process that in due time will make the system less simple and > easy to change but more immediately useful. i don't see how progress == complicated. could you explain how they are one and the same? as a counter example, unix uses untyped files. not only does lack of typing make the system simplier, it makes it more expressive as well. > I see this more as a difference of intended audience than a difference of > taste or philosophy. The real question is whom (sic) you imagine as your system's > middle user: a wage-earning programmer or a researcher. Were I a programmer > who worked for pay I'd very much appreciate being given every possible > option that would let me do my job easier, faster, and, of course, more > properly. i don't see how a wage-earning programmer can't be a researcher as well. and being a wage-earning programmer, i appreciate simplicity and use it to advantage on a daily basis. several times, i've needed to get a particular bit of h/w working. given a choice, i go with the one with the thinner manual. - erik ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [9fans] critique of sockets API 2009-06-11 23:41 ` erik quanstrom @ 2009-06-12 4:32 ` Paul Lalonde 2009-06-12 7:19 ` Eris Discordia 1 sibling, 0 replies; 24+ messages in thread From: Paul Lalonde @ 2009-06-12 4:32 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Even more than transistors are so cheap that doing on-the-fly translation to a RISC instruction set in hardware is an essentially invisible cost. Plus you get to change the target microarchitecture to exploit new thinking in processor design without breaking existing software. That last reason, not choice, is how IA became dominant. Paul On Jun 11, 2009, at 4:41 PM, erik quanstrom wrote: > could it be that since transistors are very cheep, adding > instructions is > simply the cheapest go-faster trick? -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.3 (Darwin) iD8DBQFKMdpjpJeHo/Fbu1wRAsPTAKCmeg6YnYvCNBHpWQCd9hCAM5+Y/wCgw8// +MMUaZeL/qPutVO0W7sYTqM= =D+2E -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [9fans] critique of sockets API 2009-06-11 23:41 ` erik quanstrom 2009-06-12 4:32 ` Paul Lalonde @ 2009-06-12 7:19 ` Eris Discordia 1 sibling, 0 replies; 24+ messages in thread From: Eris Discordia @ 2009-06-12 7:19 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs > just to correct a basic fact, the size of the instruction set doesn't > define RISC instruction set (http://en.wikipedia.org/wiki/RISC > "instruction set size"). Heh. I did refer to exactly that article and consequently inserted "adding to the complexity of each primitive" as well as the modifier "strictly." > can you cite any references that claim that the size of intel's > instruction set has contributed to its success? Try the Wikipedia article on CISC which really isn't a source, I admit. The general impression is that not only the total number, and diversity, of the instruction set but how much is done using a single instruction from the viewpoint of a programmer/compiler have a role there. How a CISC instruction set is implemented is besides this point. Some translation may take place in the process but it doesn't matter as long as it is transparent to the programmer/compiler. > could it be that since transistors are very cheep, adding instructions is > simply the cheapest go-faster trick? Could be exactly that and further confirm my stance that added complexity, when it is carefully hidden and is exposed to the middle user only as a variety of orthogonal options, does not result in a bad system. In can actually result in a better system. I'm, of course, not claiming that the x86 instruction set represents an epitome of orthogonality or good design. > what is a middle user? and why would such a person be > discouraged by having to learn fewer things? does the myriad > of different ways to write an if statement in perl make it more > useable? readable? A middle user is someone who uses your product to create another product for an end user who very often doesn't create product on their own, although that may not be the case. A programmer who uses the language and compiler you designed, or the OS you created, or the API you developed. Having to learn fewer things is not discouraging, lack of expressiveness due to lack of options can be. In case of Perl its popularity establishes its merit for the area in which it is popular. Besides, we aren't talking about if redundancy makes a system more expressive (it could but we aren't talking about that), we are rather talking about addition of options that are orthogonal to existing ones: options that have no equivalent. A callback infrastructure can be created in user land using threading but that would have two disadvantages: the very point of lowering initial cost using callback strategy is made moot, and people will tend to disagree on how that extra infrastructure should look like so a variety of incompatible, highly redundant implementations will pop up. > i don't see how progress == complicated. could you explain > how they are one and the same? > > as a counter example, unix uses untyped files. not only does > lack of typing make the system simplier, it makes it more expressive > as well. I don't see how complex == complicated. You could have a very sophisticated system at hand that is rather easy to program. Increasing adaptivity through increased complexity is one well-known evolutionary path. The other well-known path is increasing resilience through decreased complexity. There is a point where resilience and adaptivity are just at the balance you desire for your audience. Making clear who your audience is should clear this problem as well. Regarding your example I believe you are mixing redundancy with choice. While I could even argue for some redundancy I'm not trying to do so. File typing, as seen on Apple systems, is not orthogonal to other features already present in those systems; it is redundant. Windows does not have any file typing similar to Apple's. UNIX-like systems do make some distinctions between files which have become rather blurry with time. I brought up the the subject in my awkward manner some time ago when caching 9P was being discussed on the list. It seemed to me that a form of typing that expressed at a reasonable level of detail the amenability of files to various caching strategies could have improved the situation of 9P caching. > i don't see how a wage-earning programmer can't be a researcher as well. > and being a wage-earning programmer, i appreciate simplicity and use > it to advantage on a daily basis. The vast majority of them aren't. That's a fact of life. You appreciate simplicity because you happen to work on a specific application for which your target system's API is exceptionally well-rounded. Try designing a complex UI for some CAD software and tell me how amenable your simple system is to that purpose. The platforms targeted for, say, creating the frontend to a CAD system are not chosen by luck really. They have been made suitable by designers to that and other purposes. Adding of orthogonal options, when done wisely, should not take away or negatively affect the core primitives you use and are content with. > several times, i've needed to get a particular bit of h/w working. > given a choice, i go with the one with the thinner manual. I bet a 8051's manual is thinner that a 80x86's. Does that mean the 8051 will be your platform of choice when it absolutely does not cut the task? I think it's somebody's motto: simple enough, but no(t) simpler. --On Thursday, June 11, 2009 19:41 -0400 erik quanstrom <quanstro@quanstro.net> wrote: >> > i think you're trying to argue that — a priori — choice is good? >> >> I believe it is. How many of us are using strictly RISC machines on our >> desks today? Extending the set of available primitives and adding to the >> complexity of each primitive both are natural steps in the development >> of computer systems as they see more use. > > just to correct a basic fact, the size of the instruction set doesn't > define RISC instruction set (http://en.wikipedia.org/wiki/RISC > "instruction set size"). > > can you cite any references that claim that the size of intel's > instruction set has contributed to its success? > > could it be that since transistors are very cheep, adding instructions is > simply the cheapest go-faster trick? > >> Limiting options doesn't seem to me >> to be an effective way of encouraging good programming practice. It can, >> however, successfully discourage potential middle users. > > what is a middle user? and why would such a person be > discouraged by having to learn fewer things? does the myriad > of different ways to write an if statement in perl make it more > useable? readable? > >> > that's not the position i subscribe to. and since plan 9 >> > is simple and easy to change, it makes an ideal system >> > for someone who wants to try new things. >> >> And after the new things are tried out and, one may hope, proven >> advantageous? I think the next step would be incorporating them into the >> system. A process that in due time will make the system less simple and >> easy to change but more immediately useful. > > i don't see how progress == complicated. could you explain > how they are one and the same? > > as a counter example, unix uses untyped files. not only does > lack of typing make the system simplier, it makes it more expressive > as well. > >> I see this more as a difference of intended audience than a difference >> of taste or philosophy. The real question is whom (sic) you imagine as >> your system's middle user: a wage-earning programmer or a researcher. >> Were I a programmer who worked for pay I'd very much appreciate being >> given every possible option that would let me do my job easier, faster, >> and, of course, more properly. > > i don't see how a wage-earning programmer can't be a researcher as well. > and being a wage-earning programmer, i appreciate simplicity and use > it to advantage on a daily basis. > > several times, i've needed to get a particular bit of h/w working. > given a choice, i go with the one with the thinner manual. > > - erik > ^ permalink raw reply [flat|nested] 24+ messages in thread
[parent not found: <D9FE8C51EE2D568C05E555DD@192.168.1.2>]
* Re: [9fans] critique of sockets API [not found] ` <D9FE8C51EE2D568C05E555DD@192.168.1.2> @ 2009-06-11 23:34 ` Devon H. O'Dell 2009-06-12 7:21 ` Eris Discordia 0 siblings, 1 reply; 24+ messages in thread From: Devon H. O'Dell @ 2009-06-11 23:34 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs 2009/6/11 Eris Discordia <eris.discordia@gmail.com>: >> but given that plan 9 is about having a system that's easy >> to understand and modify, i would think that it would be >> tough to demonstrate that asyncronous i/o or callbacks >> could make the system (or even applications) simplier. >> i doubt that they would make the system more efficient, >> either. >> >> do you have examples that demonstrate either? > > I can't claim I have anything in code which would be necessary for an actual > demonstration or for going beyond the "talk talk talk" stage. I can, > however, present one simple case: in some applications asynchronous name > resolving is a must and it can be realized by either of threads or > callbacks. Crawlers and scanners come to mind. Spawning threads for DNS > requests could be more costly than registering a set of callbacks within one > thread and then harvesting the results within that same thread. s/could be/is/ >From real world product experience across multiple operating systems and architectures. ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [9fans] critique of sockets API 2009-06-11 23:34 ` Devon H. O'Dell @ 2009-06-12 7:21 ` Eris Discordia 0 siblings, 0 replies; 24+ messages in thread From: Eris Discordia @ 2009-06-12 7:21 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs > s/could be/is/ > > From real world product experience across multiple operating systems > and architectures. So there is at least one example to support the case for callbacks. I am pretty convinced there are many more examples. --On Thursday, June 11, 2009 19:34 -0400 "Devon H. O'Dell" <devon.odell@gmail.com> wrote: > 2009/6/11 Eris Discordia <eris.discordia@gmail.com>: >>> but given that plan 9 is about having a system that's easy >>> to understand and modify, i would think that it would be >>> tough to demonstrate that asyncronous i/o or callbacks >>> could make the system (or even applications) simplier. >>> i doubt that they would make the system more efficient, >>> either. >>> >>> do you have examples that demonstrate either? >> >> I can't claim I have anything in code which would be necessary for an >> actual demonstration or for going beyond the "talk talk talk" stage. I >> can, however, present one simple case: in some applications asynchronous >> name resolving is a must and it can be realized by either of threads or >> callbacks. Crawlers and scanners come to mind. Spawning threads for DNS >> requests could be more costly than registering a set of callbacks within >> one thread and then harvesting the results within that same thread. > > s/could be/is/ > > From real world product experience across multiple operating systems > and architectures. > ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [9fans] critique of sockets API @ 2009-06-11 12:16 erik quanstrom 0 siblings, 0 replies; 24+ messages in thread From: erik quanstrom @ 2009-06-11 12:16 UTC (permalink / raw) To: eris.discordia, 9fans On Thu Jun 11 04:12:13 EDT 2009, eris.discordia@gmail.com wrote: > > i don't think i understand what you're getting at. > > it could be that the blog was getting at the fact that select > > funnels a bunch of independent i/o down to one process. > > it's an effective technique when (a) threads are not available > > and (b) processing is very fast. > > This might help: what he is getting at is probably the question of why not > make possible network applications that consist of a bunch of callbacks or > a mix of callbacks and listener/worker threads. Windows implements both > synchronous and asynchronous I/O. Threads are available. Callbacks, too, as > well as message queues. are you saying the author(s) had their windows blinders on and might not have considered other options? my windows-fu is very low. but according to microsoft http://msdn.microsoft.com/en-us/library/aa365683(VS.85).aspx windows "asynchronous" (overlapped) i/o signals the calling thread, so the signal context is the original calling thread. unless ms' diagram is incorrect, this is a single threaded operation; only the i/o originator can process the event. so the plan 9 model would seem to me to be better threaded and i think the CSP-style of the plan 9 model makes it easier to reason about. and has already been implemented under unix without changing the kernel. see p9p. > Ideally, it is the programmer's informed choice > based on their understanding of their application's priorities whether to > use callbacks, listener/worker threads, message queues, or a combination. > Someone may find it worth the effort to compare these approaches on a > platform that provides both. (COM is notorious for implementing things > through callback that get some wrapping of one's head around them before > making sense.) there's plenty of overlapped i/o in plan 9 — it's in the disk and network device drivers. even stripping away the register fiddling, it's not an i/o model that's attractive; that's the reason all the details are hidden in the kernel. - erik ^ permalink raw reply [flat|nested] 24+ messages in thread
[parent not found: <mailman.1.1244635201.19660.9fans@9fans.net>]
* Re: [9fans] critique of sockets API [not found] <mailman.1.1244635201.19660.9fans@9fans.net> @ 2009-06-10 23:46 ` Bhanu Nagendra Pisupati 0 siblings, 0 replies; 24+ messages in thread From: Bhanu Nagendra Pisupati @ 2009-06-10 23:46 UTC (permalink / raw) To: 9fans > perhaps you think this is doging the question, but the cannonical > plan 9 approach to this is to note that it's easy (trivial) to have a n > reader threads and m worker threads s.t. i/o threads don't block I'll agree. With multi threading the network read/write operations could be pipelined to minimize the overhead. ^ permalink raw reply [flat|nested] 24+ messages in thread
[parent not found: <mailman.998.1244574121.1513.9fans@9fans.net>]
* Re: [9fans] critique of sockets API [not found] <mailman.998.1244574121.1513.9fans@9fans.net> @ 2009-06-09 23:20 ` Bhanu Nagendra Pisupati 2009-06-09 23:24 ` J.R. Mauro ` (2 more replies) 0 siblings, 3 replies; 24+ messages in thread From: Bhanu Nagendra Pisupati @ 2009-06-09 23:20 UTC (permalink / raw) To: 9fans First off, I really am a big fan of filesystem interfaces as used in Plan 9 - after all my PhD work was based on the model :) My objective here is to debate and understand how the points made in the paper relate to the Plan9 networking model. >> * performance overhead: app requesting data from a socket typically >> needs to perform 2 system calls (select/read or alt/read) > > alt ? which is not required ? is not a system call. only a read or write is > required. Well, select() or alt might or might not be required depending on whether you want your thread to wait till the read operation waiting for data from the network completes. You may argue that since threads are "cheap" in Plan9 you can afford to have a thread wait on the read operation. But that to me is a different question... >> * lack of an "kernel up-call API": which allows the kernel to inform an >> app each time network data is available > > plan 9 uses synchronous i/o, so this statement doesn't make sense > in plan 9. you can use threads to do i/o asynch w.r.t. your application, > but the i/o itself is still synchronous w.r.t. the kernel. Whether the IO is synchronous or not, there is this read()->process()->read()... alternating sequence of operations that is required, wherein the application has to explicitly go fetch data from the network using the read operation. To borrow text from the paper: <snip> The API does not provide the programmer a way in which to say, "Whenever there is data for me, call me to process it directly." </snip> >> * hard to implement "multi homing" with support for multiple network >> interfaces > > i have no idea how this relates to the use of a fs in implementing the > network stack. why would using a filsystem (or not) make any difference > in the ability to multihome? > > by the way, plan 9 beats the pants off anything else i've used for multiple > network / interface support. it's support for mulitple ip stacks is quite > neat. The question was meant to ask as to how easy it is to programmatically use the filesystem interface in a multi home network. But I agree that support for multiple network interfaces in Plan9 is way superior. ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [9fans] critique of sockets API 2009-06-09 23:20 ` Bhanu Nagendra Pisupati @ 2009-06-09 23:24 ` J.R. Mauro 2009-06-09 23:33 ` Devon H. O'Dell 2009-06-10 0:07 ` erik quanstrom 2 siblings, 0 replies; 24+ messages in thread From: J.R. Mauro @ 2009-06-09 23:24 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs On Tue, Jun 9, 2009 at 7:20 PM, Bhanu Nagendra Pisupati<bpisupat@cs.indiana.edu> wrote: > First off, I really am a big fan of filesystem interfaces as used in Plan 9 > - after all my PhD work was based on the model :) Did you do this on Plan 9 or bring some of the filesystem sanity of another OS? > My objective here is to debate and understand how the points made in the > paper relate to the Plan9 networking model. > >>> * performance overhead: app requesting data from a socket typically needs >>> to perform 2 system calls (select/read or alt/read) >> >> alt ? which is not required ? is not a system call. only a read or write >> is >> required. > > Well, select() or alt might or might not be required depending on whether > you want your thread to wait till the read operation waiting for data from > the network completes. You may argue that since threads are "cheap" in Plan9 > you can afford to have a thread wait on the read operation. But that to me > is a different question... > >>> * lack of an "kernel up-call API": which allows the kernel to inform an >>> app each time network data is available >> >> plan 9 uses synchronous i/o, so this statement doesn't make sense >> in plan 9. you can use threads to do i/o asynch w.r.t. your application, >> but the i/o itself is still synchronous w.r.t. the kernel. > > Whether the IO is synchronous or not, there is this > read()->process()->read()... alternating sequence of operations that is > required, wherein the application has to explicitly go fetch data from the > network using the read operation. To borrow text from the paper: > <snip> > The API does not provide the programmer a way in which to say, "Whenever > there is data for me, call me to process it directly." > </snip> > > >>> * hard to implement "multi homing" with support for multiple network >>> interfaces >> >> i have no idea how this relates to the use of a fs in implementing the >> network stack. why would using a filsystem (or not) make any difference >> in the ability to multihome? >> >> by the way, plan 9 beats the pants off anything else i've used for >> multiple >> network / interface support. it's support for mulitple ip stacks is quite >> neat. > > The question was meant to ask as to how easy it is to programmatically use > the filesystem interface in a multi home network. But I agree that support > for multiple network interfaces in Plan9 is way superior. > > ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [9fans] critique of sockets API 2009-06-09 23:20 ` Bhanu Nagendra Pisupati 2009-06-09 23:24 ` J.R. Mauro @ 2009-06-09 23:33 ` Devon H. O'Dell 2009-06-10 3:33 ` Gary Wright 2009-06-10 0:07 ` erik quanstrom 2 siblings, 1 reply; 24+ messages in thread From: Devon H. O'Dell @ 2009-06-09 23:33 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs >> i have no idea how this relates to the use of a fs in implementing the >> network stack. why would using a filsystem (or not) make any difference >> in the ability to multihome? >> >> by the way, plan 9 beats the pants off anything else i've used for >> multiple >> network / interface support. it's support for mulitple ip stacks is quite >> neat. > > The question was meant to ask as to how easy it is to programmatically use > the filesystem interface in a multi home network. But I agree that support > for multiple network interfaces in Plan9 is way superior. I think you mean `networks with multiple gateways'? I don't see how Plan 9's model would affect this idea one way or the other, but perhaps I (or all of us) misunderstand this part of your questions. --me ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [9fans] critique of sockets API 2009-06-09 23:33 ` Devon H. O'Dell @ 2009-06-10 3:33 ` Gary Wright 0 siblings, 0 replies; 24+ messages in thread From: Gary Wright @ 2009-06-10 3:33 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs On Jun 9, 2009, at 7:33 PM, Devon H. O'Dell wrote: > I think you mean `networks with multiple gateways'? I don't see how > Plan 9's model would affect this idea one way or the other, but > perhaps I (or all of us) misunderstand this part of your questions. Based on scanning the original article, I think the question was how Plan 9 could support protocols such as: SCTP (Stream Control Transport Protocol. In SCTP a communication endpoint is represented by more than one IP address. Gary Wright ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [9fans] critique of sockets API 2009-06-09 23:20 ` Bhanu Nagendra Pisupati 2009-06-09 23:24 ` J.R. Mauro 2009-06-09 23:33 ` Devon H. O'Dell @ 2009-06-10 0:07 ` erik quanstrom 2009-06-10 0:34 ` erik quanstrom 2009-06-11 7:07 ` Eris Discordia 2 siblings, 2 replies; 24+ messages in thread From: erik quanstrom @ 2009-06-10 0:07 UTC (permalink / raw) To: 9fans On Tue Jun 9 19:22:39 EDT 2009, bpisupat@cs.indiana.edu wrote: > Well, select() or alt might or might not be required depending on whether > you want your thread to wait till the read operation waiting > for data from the network completes. your thread will always wait until any system call completes; they're all synchronous all the time. if you want your application to do something else at the same time, you're going to need two threads and a synch device (like a lock + shared memory or a channel). > read()->process()->read()... alternating sequence of operations that is > required, wherein the application has to explicitly go fetch data from the network > using the read operation. To borrow text from the paper: > <snip> > The API does not provide the programmer a way in which to say, "Whenever > there is data for me, call me to process it directly." > </snip> i don't think i understand what you're getting at. it could be that the blog was getting at the fact that select funnels a bunch of independent i/o down to one process. it's an effective technique when (a) threads are not available and (b) processing is very fast. perhaps you think this is doging the question, but the cannonical plan 9 approach to this is to note that it's easy (trivial) to have a n reader threads and m worker threads s.t. i/o threads don't block unless (a) there's nothing to i/o, or (b) the workers aren't draining the queue fast enough; and s.t. worker threads don't block unless (a) the i/o threads can't keep up. in this case, there is no work to do anyway. consider these two types of threads; let mb be a pointer to a message buffer. thread type 1 for(;;) mb <- freechan read(fd, mb->wp, BUFSIZE); mb -> fullchan thread type 2; for(;;) mb <- fullchan do stuff mb -> freechan if your server issues responses, it's easy to add thread type 3. as you can see, this is a simple generalization of your case. (if we have a queue depth of 0 and one thread 1 and one thread 2, we will get your loop.) yet there should be no waiting. > The question was meant to ask as to how easy it is to programmatically > use the filesystem interface in a multi home network. But I agree that support for > multiple network interfaces in Plan9 is way superior. i think the answer to your question is that programs don't care. programs that take a cannonical network address are just as happy to accept /net/tcp!www.example.com!http as they are to accept /net.alt/tcp!www.example.com!http. for a short while i ran a 10gbit network on /net.10g. - erik ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [9fans] critique of sockets API 2009-06-10 0:07 ` erik quanstrom @ 2009-06-10 0:34 ` erik quanstrom 2009-06-11 7:07 ` Eris Discordia 1 sibling, 0 replies; 24+ messages in thread From: erik quanstrom @ 2009-06-10 0:34 UTC (permalink / raw) To: 9fans > i don't think i understand what you're getting at. > it could be that the blog was getting at the fact that select > funnels a bunch of independent i/o down to one process. > it's an effective technique when (a) threads are not available > and (b) processing is very fast. ack. but, you can select on the same fd in many processes. select usage is so idiomatic that this use is easy to forget, though i've written programs that take advantage of this myself. - erik ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [9fans] critique of sockets API 2009-06-10 0:07 ` erik quanstrom 2009-06-10 0:34 ` erik quanstrom @ 2009-06-11 7:07 ` Eris Discordia 1 sibling, 0 replies; 24+ messages in thread From: Eris Discordia @ 2009-06-11 7:07 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs > i don't think i understand what you're getting at. > it could be that the blog was getting at the fact that select > funnels a bunch of independent i/o down to one process. > it's an effective technique when (a) threads are not available > and (b) processing is very fast. This might help: what he is getting at is probably the question of why not make possible network applications that consist of a bunch of callbacks or a mix of callbacks and listener/worker threads. Windows implements both synchronous and asynchronous I/O. Threads are available. Callbacks, too, as well as message queues. Ideally, it is the programmer's informed choice based on their understanding of their application's priorities whether to use callbacks, listener/worker threads, message queues, or a combination. Someone may find it worth the effort to compare these approaches on a platform that provides both. (COM is notorious for implementing things through callback that get some wrapping of one's head around them before making sense.) --On Tuesday, June 09, 2009 20:07 -0400 erik quanstrom <quanstro@quanstro.net> wrote: > On Tue Jun 9 19:22:39 EDT 2009, bpisupat@cs.indiana.edu wrote: >> Well, select() or alt might or might not be required depending on >> whether you want your thread to wait till the read operation waiting >> for data from the network completes. > > your thread will always wait until any system call completes; > they're all synchronous all the time. if you want your application > to do something else at the same time, you're going to need two > threads and a synch device (like a lock + shared memory or a channel). > >> read()->process()->read()... alternating sequence of operations that is >> required, wherein the application has to explicitly go fetch data from >> the network using the read operation. To borrow text from the paper: >> <snip> >> The API does not provide the programmer a way in which to say, "Whenever >> there is data for me, call me to process it directly." >> </snip> > > i don't think i understand what you're getting at. > it could be that the blog was getting at the fact that select > funnels a bunch of independent i/o down to one process. > it's an effective technique when (a) threads are not available > and (b) processing is very fast. > > perhaps you think this is doging the question, but the cannonical > plan 9 approach to this is to note that it's easy (trivial) to have a n > reader threads and m worker threads s.t. i/o threads don't block > unless (a) there's nothing to i/o, or (b) the workers aren't draining > the queue fast enough; and s.t. worker threads don't block > unless (a) the i/o threads can't keep up. in this case, there is no > work to do anyway. consider these two types of threads; > let mb be a pointer to a message buffer. > > thread type 1 > for(;;) > mb <- freechan > read(fd, mb->wp, BUFSIZE); > mb -> fullchan > thread type 2; > for(;;) > mb <- fullchan > do stuff > mb -> freechan > > if your server issues responses, it's easy to add thread type 3. > > as you can see, this is a simple generalization of your case. (if we > have a queue depth of 0 and one thread 1 and one thread 2, > we will get your loop.) yet there should be no waiting. > >> The question was meant to ask as to how easy it is to programmatically >> use the filesystem interface in a multi home network. But I agree that >> support for multiple network interfaces in Plan9 is way superior. > > i think the answer to your question is that programs don't care. > programs that take a cannonical network address are just as > happy to accept /net/tcp!www.example.com!http as they are to > accept /net.alt/tcp!www.example.com!http. for a short while > i ran a 10gbit network on /net.10g. > > - erik > ^ permalink raw reply [flat|nested] 24+ messages in thread
* [9fans] critique of sockets API @ 2009-06-09 18:49 Bhanu Nagendra Pisupati 2009-06-09 18:59 ` erik quanstrom 2009-06-09 22:11 ` Russ Cox 0 siblings, 2 replies; 24+ messages in thread From: Bhanu Nagendra Pisupati @ 2009-06-09 18:49 UTC (permalink / raw) To: 9fans Interesting read: http://cacm.acm.org/magazines/2009/6/28495-whither-sockets/fulltext If I am right, the filesystem based networking interface offered by Plan 9 has the three limitations discussed here: * performance overhead: app requesting data from a socket typically needs to perform 2 system calls (select/read or alt/read) * lack of an "kernel up-call API": which allows the kernel to inform an app each time network data is available * hard to implement "multi homing" with support for multiple network interfaces Any thoughts/comments? Thanks, -Bhanu ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [9fans] critique of sockets API 2009-06-09 18:49 Bhanu Nagendra Pisupati @ 2009-06-09 18:59 ` erik quanstrom 2009-06-09 22:11 ` Russ Cox 1 sibling, 0 replies; 24+ messages in thread From: erik quanstrom @ 2009-06-09 18:59 UTC (permalink / raw) To: 9fans On Tue Jun 9 14:50:58 EDT 2009, bpisupat@cs.indiana.edu wrote: > Interesting read: > http://cacm.acm.org/magazines/2009/6/28495-whither-sockets/fulltext > > If I am right, the filesystem based networking interface offered by Plan 9 > has the three limitations discussed here: > * performance overhead: app requesting data from a socket typically needs > to perform 2 system calls (select/read or alt/read) alt — which is not required — is not a system call. only a read or write is required. > * lack of an "kernel up-call API": which allows the kernel to inform an > app each time network data is available plan 9 uses synchronous i/o, so this statement doesn't make sense in plan 9. you can use threads to do i/o asynch w.r.t. your application, but the i/o itself is still synchronous w.r.t. the kernel. > * hard to implement "multi homing" with support for multiple network > interfaces i have no idea how this relates to the use of a fs in implementing the network stack. why would using a filsystem (or not) make any difference in the ability to multihome? by the way, plan 9 beats the pants off anything else i've used for multiple network / interface support. it's support for mulitple ip stacks is quite neat. - erik ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [9fans] critique of sockets API 2009-06-09 18:49 Bhanu Nagendra Pisupati 2009-06-09 18:59 ` erik quanstrom @ 2009-06-09 22:11 ` Russ Cox 1 sibling, 0 replies; 24+ messages in thread From: Russ Cox @ 2009-06-09 22:11 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs > * performance overhead: app requesting data from a socket typically needs to > perform 2 system calls (select/read or alt/read) * lack of an "kernel > up-call API": which allows the kernel to inform an app each time network > data is available there is a mechanism. user programs call read(2). when data is available, the read finishes. ^ permalink raw reply [flat|nested] 24+ messages in thread
end of thread, other threads:[~2009-06-13 1:39 UTC | newest] Thread overview: 24+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- [not found] <mailman.1007.1244590421.1513.9fans@9fans.net> 2009-06-10 22:50 ` [9fans] critique of sockets API Bhanu Nagendra Pisupati 2009-06-11 12:34 ` erik quanstrom 2009-06-11 14:38 ` [9fans] embedfs source? (was: Re: critique of sockets API) Nathaniel W Filardo [not found] <mailman.1.1244808001.26495.9fans@9fans.net> 2009-06-13 1:39 ` [9fans] critique of sockets API Bhanu Nagendra Pisupati [not found] <b0a72b5826eb44300d3603f585859910@quanstro.net> 2009-06-11 14:54 ` Eris Discordia 2009-06-11 18:24 ` erik quanstrom 2009-06-11 21:21 ` Eris Discordia 2009-06-11 23:41 ` erik quanstrom 2009-06-12 4:32 ` Paul Lalonde 2009-06-12 7:19 ` Eris Discordia [not found] ` <D9FE8C51EE2D568C05E555DD@192.168.1.2> 2009-06-11 23:34 ` Devon H. O'Dell 2009-06-12 7:21 ` Eris Discordia 2009-06-11 12:16 erik quanstrom [not found] <mailman.1.1244635201.19660.9fans@9fans.net> 2009-06-10 23:46 ` Bhanu Nagendra Pisupati [not found] <mailman.998.1244574121.1513.9fans@9fans.net> 2009-06-09 23:20 ` Bhanu Nagendra Pisupati 2009-06-09 23:24 ` J.R. Mauro 2009-06-09 23:33 ` Devon H. O'Dell 2009-06-10 3:33 ` Gary Wright 2009-06-10 0:07 ` erik quanstrom 2009-06-10 0:34 ` erik quanstrom 2009-06-11 7:07 ` Eris Discordia -- strict thread matches above, loose matches on Subject: below -- 2009-06-09 18:49 Bhanu Nagendra Pisupati 2009-06-09 18:59 ` erik quanstrom 2009-06-09 22:11 ` Russ Cox
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).