* [9fans] About 9P ... @ 2007-06-19 20:11 Philippe Anel 2007-06-20 0:37 ` Russ Cox 2007-06-22 0:32 ` Derek Fawcus 0 siblings, 2 replies; 52+ messages in thread From: Philippe Anel @ 2007-06-19 20:11 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs Hello 9fans, Regarding 9P design, I wonder why Fids are chosen by the client : I tried to find an answer in the docs, manuals or in 9fans archive but I have not found an answer (please tell me where I can find an answer if I missed something). Can someone explain me the reason why the Fids are not returned by the server (along with the Qids by example) ? Because Fids are chosen by the client, the server must search in a table (hash table in lib9p) the associated file structure, Whereas if Fids are chosen by the server, a simple array is enough. I don't know however, if the performance improvement is really important. In fact I'm quite sure it is not. I also think this is not because of a security issue even if there may be multiple clients sharing the same connection. Indeed, because the transactions are only between the server and the kernel or an remote (tcp/il) connection, it seems a process cannot steal the fid of another process (or another namespace). Regarding the transactions with the kernel, a process has no access to fids. And regarding the transaction with a remote connection, the connection point can be used as a kind of fid space. Maybe I'm wrong about the security issue, I only want to know why 9P has been designed with such policy. Thank you, Phil; ^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [9fans] About 9P ... 2007-06-19 20:11 [9fans] About 9P Philippe Anel @ 2007-06-20 0:37 ` Russ Cox 2007-06-20 2:06 ` Roman Shaposhnick 2007-06-22 0:32 ` Derek Fawcus 1 sibling, 1 reply; 52+ messages in thread From: Russ Cox @ 2007-06-20 0:37 UTC (permalink / raw) To: 9fans > Because Fids are chosen by the client, the server must search in > a table (hash table in lib9p) the associated file structure, Whereas > if Fids are chosen by the server, a simple array is enough. I don't > know however, if the performance improvement is really important. > In fact I'm quite sure it is not. If Fids were chosen by the server you would have written the opposite. One side can have a simple array lookup and one side can't. It doesn't matter either way. You just pick one. Or you could do what SSH did and pick two: there the equivalent of a "fid" is actually a pair of integers, one chosen by the client and one chosen by the server, so that both can use an array lookup. Because after all the encryption and public key operations, it's really looking up the fids that is eating all the cpu. Russ ^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [9fans] About 9P ... 2007-06-20 0:37 ` Russ Cox @ 2007-06-20 2:06 ` Roman Shaposhnick 2007-06-20 11:29 ` Anthony Sorace 0 siblings, 1 reply; 52+ messages in thread From: Roman Shaposhnick @ 2007-06-20 2:06 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs On Tue, 2007-06-19 at 20:37 -0400, Russ Cox wrote: > Because after > all the encryption and public key operations, it's really > looking up the fids that is eating all the cpu. Now this is an interesting point, because it sort of argues in favor of doing the lookups on the client and conserving CPU cycles of the server where the cycles can be more precious. Thanks, Roman. P.S. But then again, in Plan 9: the server is a client is a server ;-) ^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [9fans] About 9P ... 2007-06-20 2:06 ` Roman Shaposhnick @ 2007-06-20 11:29 ` Anthony Sorace 0 siblings, 0 replies; 52+ messages in thread From: Anthony Sorace @ 2007-06-20 11:29 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs perhaps my sense of humor's tweaked, but i understood russ' comment to be sarcastic, since the added complexity of making fid lookup slightly faster on both ends seems hard to justify given how vastly outweighed fid lookup is bound to be by the encryption-related operations. ^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [9fans] About 9P ... 2007-06-19 20:11 [9fans] About 9P Philippe Anel 2007-06-20 0:37 ` Russ Cox @ 2007-06-22 0:32 ` Derek Fawcus 2007-06-22 6:58 ` Philippe Anel 1 sibling, 1 reply; 52+ messages in thread From: Derek Fawcus @ 2007-06-22 0:32 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs On Tue, Jun 19, 2007 at 10:11:31PM +0200, Philippe Anel wrote: > > Can someone explain me the reason why the Fids are not returned > by the server (along with the Qids by example) ? I always assumed it was to enable pipelined operations. DF ^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [9fans] About 9P ... 2007-06-22 0:32 ` Derek Fawcus @ 2007-06-22 6:58 ` Philippe Anel 2007-06-22 15:57 ` Derek Fawcus 0 siblings, 1 reply; 52+ messages in thread From: Philippe Anel @ 2007-06-22 6:58 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs What do you mean ? Derek Fawcus a écrit : > On Tue, Jun 19, 2007 at 10:11:31PM +0200, Philippe Anel wrote: > >> Can someone explain me the reason why the Fids are not returned >> by the server (along with the Qids by example) ? >> > > I always assumed it was to enable pipelined operations. > > DF > > > ^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [9fans] About 9P ... 2007-06-22 6:58 ` Philippe Anel @ 2007-06-22 15:57 ` Derek Fawcus 2007-06-22 20:32 ` Philippe Anel 0 siblings, 1 reply; 52+ messages in thread From: Derek Fawcus @ 2007-06-22 15:57 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs On Fri, Jun 22, 2007 at 08:58:04AM +0200, Philippe Anel wrote: > What do you mean ? Well, I've not read the protocol details for a while. But from memory I thought it allowed this type of operation: send: open,fid,file send: read,fid,args send: read,fid,args (wait one rtt) recv: open success/fail recv: read result / read error due to unknown fid recv: read result / read error due to unknown fid DF > > Derek Fawcus a écrit : > > On Tue, Jun 19, 2007 at 10:11:31PM +0200, Philippe Anel wrote: > > > >> Can someone explain me the reason why the Fids are not returned > >> by the server (along with the Qids by example) ? > >> > > > > I always assumed it was to enable pipelined operations. > > > > DF > > > > > > ^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [9fans] About 9P ... 2007-06-22 15:57 ` Derek Fawcus @ 2007-06-22 20:32 ` Philippe Anel 2007-06-22 21:10 ` David Leimbach 2007-06-22 21:16 ` Skip Tavakkolian 0 siblings, 2 replies; 52+ messages in thread From: Philippe Anel @ 2007-06-22 20:32 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs Well even if it can work, I don't think 9P was designed to allow such type of operation. Indeed, 9P basis is the transaction : ie a Request followed by a Reply. So I think you should not pipeline the requests. Can a 9P specialist can confirm this ? If so ... I now understand why Fids are (or must be) choosen by the client. Phil; > Well, I've not read the protocol details for a while. > But from memory I thought it allowed this type of operation: > > send: open,fid,file > send: read,fid,args > send: read,fid,args > > (wait one rtt) > > recv: open success/fail > recv: read result / read error due to unknown fid > recv: read result / read error due to unknown fid > > DF > > ^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [9fans] About 9P ... 2007-06-22 20:32 ` Philippe Anel @ 2007-06-22 21:10 ` David Leimbach 2007-06-22 21:35 ` Philippe Anel 2007-06-22 21:16 ` Skip Tavakkolian 1 sibling, 1 reply; 52+ messages in thread From: David Leimbach @ 2007-06-22 21:10 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs [-- Attachment #1: Type: text/plain, Size: 1126 bytes --] Transactions are tagged right? So you can, in fact, have many in flight at once. Perhaps I'm missing what you meant by pipelined. On 6/22/07, Philippe Anel <xigh@free.fr> wrote: > > Well even if it can work, I don't think 9P was designed to allow such > type of > operation. Indeed, 9P basis is the transaction : ie a Request followed > by a Reply. > So I think you should not pipeline the requests. > Can a 9P specialist can confirm this ? If so ... I now understand why > Fids are > (or must be) choosen by the client. > > Phil; > > Well, I've not read the protocol details for a while. > > But from memory I thought it allowed this type of operation: > > > > send: open,fid,file > > send: read,fid,args > > send: read,fid,args > > > > (wait one rtt) > > > > recv: open success/fail > > recv: read result / read error due to unknown fid > > recv: read result / read error due to unknown fid > > > > DF > > > > > > -- - Passage Matthew 5:37: But let your communication be, Yea, yea; Nay, nay: for whatsoever is more than these cometh of evil. [-- Attachment #2: Type: text/html, Size: 1710 bytes --] ^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [9fans] About 9P ... 2007-06-22 21:10 ` David Leimbach @ 2007-06-22 21:35 ` Philippe Anel 2007-06-22 21:46 ` David Leimbach 0 siblings, 1 reply; 52+ messages in thread From: Philippe Anel @ 2007-06-22 21:35 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs David Leimbach a écrit : > Transactions are tagged right? So you can, in fact, have many in > flight at once. > > Perhaps I'm missing what you meant by pipelined. I meant sending a couple of requests on one fid before receiving replies. Yes transaction are tagged, this doesn't mean that requests are serialized (processed in order). See the example sent by Derek : 1 send: open,fid,file 2 send: read,fid,args 3 send: read,fid,args (wait one rtt) 4 recv: open success/fail 5 recv: read result / read error due to unknown fid 7 recv: read result / read error due to unknown fid I see no reason for the server to not reply the 2nd request before the 1st one : by example the server can receive all the requests and distpach those reqs to different tasks. Because all transactions are tagged ... this wouldn't break 9P. However, this doesn't work as expected. That's why I think 9P was not designed to allow this. But maybe I'm wrong. Phil; ^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [9fans] About 9P ... 2007-06-22 21:35 ` Philippe Anel @ 2007-06-22 21:46 ` David Leimbach 2007-06-22 21:55 ` Philippe Anel 2007-06-22 23:20 ` [9fans] About 9P Skip Tavakkolian 0 siblings, 2 replies; 52+ messages in thread From: David Leimbach @ 2007-06-22 21:46 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs [-- Attachment #1: Type: text/plain, Size: 2310 bytes --] On 6/22/07, Philippe Anel <xigh@free.fr> wrote: > > David Leimbach a écrit : > > Transactions are tagged right? So you can, in fact, have many in > > flight at once. > > > > Perhaps I'm missing what you meant by pipelined. > I meant sending a couple of requests on one fid before receiving > replies. That I believe is correct, there's no guarantee that a server has to process your requests on a single fid in the order the requests are sent/received. It will eventually reply to your requests in some order, and you'll be able to tell what order they occurred since I believe 9p requires a transport guaranteeing the sequence sent will be the sequence received. (I'm a 9p amateur though) Yes transaction are tagged, this doesn't mean that requests are > serialized (processed in order). > > See the example sent by Derek : > > 1 send: open,fid,file > 2 send: read,fid,args > 3 send: read,fid,args > > (wait one rtt) > > 4 recv: open success/fail > 5 recv: read result / read error due to unknown fid > 7 recv: read result / read error due to unknown fid > > I see no reason for the server to not reply the 2nd request > before the 1st one : by example the server can receive all > the requests and distpach those reqs to different tasks. Well you may have a point, but why not 1. send: open, fid, file 2. wait for reply to 1. 3. send read, fid, args 4. send read, fid, args 5. wait for either... 6. wait for remaining? At that point you're still pipelining, and since you're reading presumably into separate buffers, or different locations in the same buffer, who cares about the order? > Because all transactions are tagged ... this wouldn't break > 9P. However, this doesn't work as expected. Depends on what's expected :-) That's why I think 9P was not designed to allow this. But > maybe I'm wrong. It's probably designed to allow what I just said, you can sequence some operations, but then things that don't need to be sequenced could be pipelined. Different servers may behave differently, 9P makes no guarantee AFAIK. Dave Phil; > > > -- - Passage Matthew 5:37: But let your communication be, Yea, yea; Nay, nay: for whatsoever is more than these cometh of evil. [-- Attachment #2: Type: text/html, Size: 3993 bytes --] ^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [9fans] About 9P ... 2007-06-22 21:46 ` David Leimbach @ 2007-06-22 21:55 ` Philippe Anel 2007-06-22 22:25 ` Philippe Anel 2007-06-22 23:20 ` [9fans] About 9P Skip Tavakkolian 1 sibling, 1 reply; 52+ messages in thread From: Philippe Anel @ 2007-06-22 21:55 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs Ok I agree ... as long as you don't expect the replies to be returned in the same order as the requests, requests can be pipelined. Therefore, now it makes sense to have FIDs chosen by the client. Thank you. > > Well you may have a point, but why not > > 1. send: open, fid, file > 2. wait for reply to 1. > 3. send read, fid, args > 4. send read, fid, args > 5. wait for either... > 6. wait for remaining? > > At that point you're still pipelining, and since you're reading > presumably into separate buffers, or different locations in the same > buffer, who cares about the order? > > > > Because all transactions are tagged ... this wouldn't break > 9P. However, this doesn't work as expected. > > > > Depends on what's expected :-) > > > That's why I think 9P was not designed to allow this. But > maybe I'm wrong. > > > > It's probably designed to allow what I just said, you can sequence > some operations, but then things that don't need to be sequenced could > be pipelined. > > Different servers may behave differently, 9P makes no guarantee AFAIK. > > Dave > > > Phil; > > > > > > -- > - Passage Matthew 5:37: > But let your communication be, Yea, yea; Nay, nay: for whatsoever > is more than these cometh of evil. ^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [9fans] About 9P ... 2007-06-22 21:55 ` Philippe Anel @ 2007-06-22 22:25 ` Philippe Anel 2007-06-22 22:57 ` David Leimbach 2007-06-23 4:30 ` Russ Cox 0 siblings, 2 replies; 52+ messages in thread From: Philippe Anel @ 2007-06-22 22:25 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs Sometimes, I whish I could think faster. The fact request can be pipelined like this -I mean, by taking care of the fact replies are not necessarly returned in requests order- doesn't not mean that clients have to choose the FIDs. This would also work if FIDs are choosen by the server (and even better because the client would not have to take care of the order). So unless Russ is right and that the fact FIDS are choosen by the client has been decided arbitrarily ... I still have no answer to my question. Phil; Philippe Anel a écrit : > Ok I agree ... as long as you don't expect the replies to be returned > in the same order as the requests, requests can be pipelined. > > Therefore, now it makes sense to have FIDs chosen by the client. > > Thank you. > >> >> Well you may have a point, but why not >> >> 1. send: open, fid, file >> 2. wait for reply to 1. >> 3. send read, fid, args >> 4. send read, fid, args >> 5. wait for either... >> 6. wait for remaining? >> >> At that point you're still pipelining, and since you're reading >> presumably into separate buffers, or different locations in the same >> buffer, who cares about the order? >> >> >> >> Because all transactions are tagged ... this wouldn't break >> 9P. However, this doesn't work as expected. >> >> >> Depends on what's expected :-) >> >> >> That's why I think 9P was not designed to allow this. But >> maybe I'm wrong. >> >> >> >> It's probably designed to allow what I just said, you can sequence >> some operations, but then things that don't need to be sequenced >> could be pipelined. >> >> Different servers may behave differently, 9P makes no guarantee AFAIK. >> >> Dave >> >> >> Phil; >> >> >> >> >> >> -- >> - Passage Matthew 5:37: >> But let your communication be, Yea, yea; Nay, nay: for whatsoever >> is more than these cometh of evil. > > > ^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [9fans] About 9P ... 2007-06-22 22:25 ` Philippe Anel @ 2007-06-22 22:57 ` David Leimbach 2007-06-22 23:01 ` Anant Narayanan 2007-06-23 4:30 ` Russ Cox 1 sibling, 1 reply; 52+ messages in thread From: David Leimbach @ 2007-06-22 22:57 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs [-- Attachment #1: Type: text/plain, Size: 2573 bytes --] On 6/22/07, Philippe Anel <xigh@free.fr> wrote: > > > Sometimes, I whish I could think faster. > > The fact request can be pipelined like this -I mean, by taking care of the > fact replies are not necessarly returned in requests order- doesn't not > mean that clients have to choose the FIDs. This would also work if FIDs > are choosen by the server (and even better because the client would not > have to take care of the order). > > So unless Russ is right and that the fact FIDS are choosen by the client > has > been decided arbitrarily ... I still have no answer to my question. I think servers often serve multiple clients, and therefore it's probably best to leave some things client side. Perhaps that's why google's not doing full SOAP anymore and just returning XML that you have to parse yourself on the client side. It may not really be arbitrary after all :-) Phil; > > Philippe Anel a écrit : > > Ok I agree ... as long as you don't expect the replies to be returned > > in the same order as the requests, requests can be pipelined. > > > > Therefore, now it makes sense to have FIDs chosen by the client. > > > > Thank you. > > > >> > >> Well you may have a point, but why not > >> > >> 1. send: open, fid, file > >> 2. wait for reply to 1. > >> 3. send read, fid, args > >> 4. send read, fid, args > >> 5. wait for either... > >> 6. wait for remaining? > >> > >> At that point you're still pipelining, and since you're reading > >> presumably into separate buffers, or different locations in the same > >> buffer, who cares about the order? > >> > >> > >> > >> Because all transactions are tagged ... this wouldn't break > >> 9P. However, this doesn't work as expected. > >> > >> > >> Depends on what's expected :-) > >> > >> > >> That's why I think 9P was not designed to allow this. But > >> maybe I'm wrong. > >> > >> > >> > >> It's probably designed to allow what I just said, you can sequence > >> some operations, but then things that don't need to be sequenced > >> could be pipelined. > >> > >> Different servers may behave differently, 9P makes no guarantee AFAIK. > >> > >> Dave > >> > >> > >> Phil; > >> > >> > >> > >> > >> > >> -- > >> - Passage Matthew 5:37: > >> But let your communication be, Yea, yea; Nay, nay: for whatsoever > >> is more than these cometh of evil. > > > > > > > > -- - Passage Matthew 5:37: But let your communication be, Yea, yea; Nay, nay: for whatsoever is more than these cometh of evil. [-- Attachment #2: Type: text/html, Size: 3710 bytes --] ^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [9fans] About 9P ... 2007-06-22 22:57 ` David Leimbach @ 2007-06-22 23:01 ` Anant Narayanan 0 siblings, 0 replies; 52+ messages in thread From: Anant Narayanan @ 2007-06-22 23:01 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs David Leimbach wrote: <snip> > Perhaps that's why google's not doing full SOAP anymore and just > returning XML that you have to parse yourself on the client side. If they wanted to "leave" things to the client side, they wouldn't use XML at all. -- Anant ^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [9fans] About 9P ... 2007-06-22 22:25 ` Philippe Anel 2007-06-22 22:57 ` David Leimbach @ 2007-06-23 4:30 ` Russ Cox 2007-06-23 5:10 ` Skip Tavakkolian ` (3 more replies) 1 sibling, 4 replies; 52+ messages in thread From: Russ Cox @ 2007-06-23 4:30 UTC (permalink / raw) To: 9fans > So unless Russ is right and that the > fact FIDS are choosen by the client has > been decided arbitrarily ... I still have > no answer to my question. In the current version of 9P, there is no reason why the client has to choose the fids (file ids). You don't have to believe me, but it's true. The client must choose the tags (request ids) since it sends the first request message without any input from the server. The tags are what enable having multiple messages outstanding. There is an historical reason why fids are chosen by the client, but like most historical reasons, it no longer applies. In very early versions of 9P (before the first edition release), there were no tags. Replies were matched based on fid and type, so fids had to be chosen by the client, just as tags do now. Tags were introduced on November 21, 1990: http://swtch.com/go/fid2tag Russ ^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [9fans] About 9P ... 2007-06-23 4:30 ` Russ Cox @ 2007-06-23 5:10 ` Skip Tavakkolian 2007-06-23 9:25 ` Francisco J Ballesteros 2007-06-23 6:24 ` Philippe Anel ` (2 subsequent siblings) 3 siblings, 1 reply; 52+ messages in thread From: Skip Tavakkolian @ 2007-06-23 5:10 UTC (permalink / raw) To: 9fans > Tags were introduced on November 21, 1990: > http://swtch.com/go/fid2tag good question and great historical perspective. is software archaeology a formal discipline yet? ^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [9fans] About 9P ... 2007-06-23 5:10 ` Skip Tavakkolian @ 2007-06-23 9:25 ` Francisco J Ballesteros 2007-06-23 15:50 ` Martin Atkins 0 siblings, 1 reply; 52+ messages in thread From: Francisco J Ballesteros @ 2007-06-23 9:25 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs Don't know, but we have strong group on that on the other corridor of the building. They count lines, and count, and count... They use Linux mostly, and have a huge ammount of lines to count. On 6/23/07, Skip Tavakkolian <9nut@9netics.com> wrote: > > Tags were introduced on November 21, 1990: > > http://swtch.com/go/fid2tag > > good question and great historical perspective. > > is software archaeology a formal discipline yet? > > ^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [9fans] About 9P ... 2007-06-23 9:25 ` Francisco J Ballesteros @ 2007-06-23 15:50 ` Martin Atkins 0 siblings, 0 replies; 52+ messages in thread From: Martin Atkins @ 2007-06-23 15:50 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs Francisco J Ballesteros wrote: > Don't know, but we have strong group on that on the other corridor of > the building. > They count lines, and count, and count... > They use Linux mostly, and have a huge ammount of lines to count. Of course :-) But then not many people are "fortunate enough" to see the lines in other systems, in order to be able to count them.... ^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [9fans] About 9P ... 2007-06-23 4:30 ` Russ Cox 2007-06-23 5:10 ` Skip Tavakkolian @ 2007-06-23 6:24 ` Philippe Anel 2007-06-23 15:00 ` Rob Pike 2007-06-24 0:52 ` [9fans] 9P optimization Lyndon Nerenberg 3 siblings, 0 replies; 52+ messages in thread From: Philippe Anel @ 2007-06-23 6:24 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs Thank you Russ. Russ Cox a écrit : >> So unless Russ is right and that the >> fact FIDS are choosen by the client has >> been decided arbitrarily ... I still have >> no answer to my question. >> > > In the current version of 9P, there is no reason > why the client has to choose the fids (file ids). > You don't have to believe me, but it's true. > > The client must choose the tags (request ids) > since it sends the first request message without > any input from the server. The tags are what > enable having multiple messages outstanding. > > There is an historical reason why fids are > chosen by the client, but like most historical > reasons, it no longer applies. In very early > versions of 9P (before the first edition release), > there were no tags. Replies were matched > based on fid and type, so fids had to be chosen > by the client, just as tags do now. > > Tags were introduced on November 21, 1990: > http://swtch.com/go/fid2tag > > Russ > > > > ^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [9fans] About 9P ... 2007-06-23 4:30 ` Russ Cox 2007-06-23 5:10 ` Skip Tavakkolian 2007-06-23 6:24 ` Philippe Anel @ 2007-06-23 15:00 ` Rob Pike 2007-06-24 0:52 ` [9fans] 9P optimization Lyndon Nerenberg 3 siblings, 0 replies; 52+ messages in thread From: Rob Pike @ 2007-06-23 15:00 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs > > Tags were introduced on November 21, 1990: > http://swtch.com/go/fid2tag Thanksgiving weekend. We kicked everyone off for the weekend and redid 9P to fix all the problems we'd figured out so far. -rob ^ permalink raw reply [flat|nested] 52+ messages in thread
* [9fans] 9P optimization 2007-06-23 4:30 ` Russ Cox ` (2 preceding siblings ...) 2007-06-23 15:00 ` Rob Pike @ 2007-06-24 0:52 ` Lyndon Nerenberg 2007-06-24 3:04 ` Francisco J Ballesteros 3 siblings, 1 reply; 52+ messages in thread From: Lyndon Nerenberg @ 2007-06-24 0:52 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs Before we think about turning 9P into NFSv4, it would be useful to examine the current use of cfs, and collect some stats on how effective it is. ^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [9fans] 9P optimization 2007-06-24 0:52 ` [9fans] 9P optimization Lyndon Nerenberg @ 2007-06-24 3:04 ` Francisco J Ballesteros 2007-06-24 3:15 ` Lyndon Nerenberg 0 siblings, 1 reply; 52+ messages in thread From: Francisco J Ballesteros @ 2007-06-24 3:04 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs The problem with cfs is that you still suffer the RPC to check for validity. With 100ms of RTT, count 5 and you have a second. Also, for synthesized files, the cache does not save you from doing a walk, open, read. That sayd, I don't think a change in 9p is needed. Translating to another protocol for low links suffices. On 6/24/07, Lyndon Nerenberg <lyndon@orthanc.ca> wrote: > Before we think about turning 9P into NFSv4, it would be useful to examine > the current use of cfs, and collect some stats on how effective it is. > ^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [9fans] 9P optimization 2007-06-24 3:04 ` Francisco J Ballesteros @ 2007-06-24 3:15 ` Lyndon Nerenberg 2007-06-24 4:08 ` ron minnich 2007-06-24 4:16 ` Bruce Ellis 0 siblings, 2 replies; 52+ messages in thread From: Lyndon Nerenberg @ 2007-06-24 3:15 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs > The problem with cfs is that you still suffer the RPC to check for validity. > With 100ms of RTT, count 5 and you have a second. Also, for synthesized > files, > the cache does not save you from doing a walk, open, read. I'll be accused of being a heretic, but let me claim that much of the design of IMAP is geared towards avoiding those RTTs for that very reason. A good IMAP client wins by doing intelligent cacheing, and it gets there by using the subtle aspects of the protocol that accomodate that. Much as people slag the protocol, it does work well over slow (high latency) links, given a client that understands the protocol (sadly, most don't). I'm very curious about the efficiency of cfs, but my P9 environment doesn't have a remote terminal. It would be interesting to instrument cfs and collect some stats on the protocol behaviour behind the cache ... --lyndon ^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [9fans] 9P optimization 2007-06-24 3:15 ` Lyndon Nerenberg @ 2007-06-24 4:08 ` ron minnich 2007-06-24 9:34 ` Steve Simon 2007-06-24 12:14 ` erik quanstrom 2007-06-24 4:16 ` Bruce Ellis 1 sibling, 2 replies; 52+ messages in thread From: ron minnich @ 2007-06-24 4:08 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs I wrote (a long time ago) a caching file system, for root file systems, described here: http://mbgokhale.org/rminnich/job/autocacher/autocache.ps, that did the simple thing: once you have the file, don't bother checking with the server until time + DeltaT, This might not work well for you, but in 1989, it worked wonderfully well as a root file system because: 1. most files in most root file systems are never even looked at (particularly man pages!) 2. They don't change much anyway -- think about it -- how often do people like to upgrade :-) So it is not always the case that you need to stat the cached files on each access. In fact, we watched NFS traffic on our backbone at the SRC for several months, and, as many other people have found, most files are mostly read. This type of optimization is fairly trivial for CFS, and would end up covering most files (see above); the few read-write ones left would not really hurt you too much. You could do much better in 9p because it is stateful. I have a new implementation of the autocacher (it's only for NFS) which I wrote ca. 1999, before I got to LANL; I had lost it but found it on my last day of work at LANL. I'll try to get it on my web page. It's a very simple program. thanks ron ^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [9fans] 9P optimization 2007-06-24 4:08 ` ron minnich @ 2007-06-24 9:34 ` Steve Simon 2007-06-24 10:06 ` Lyndon Nerenberg 2007-06-24 12:14 ` erik quanstrom 1 sibling, 1 reply; 52+ messages in thread From: Steve Simon @ 2007-06-24 9:34 UTC (permalink / raw) To: 9fans > http://mbgokhale.org/rminnich/job/autocacher/autocache.ps Interesting idea, I must read up on what else has been published on cacheing to avoid high latency links. It would be fun to modify cfs to have a couple of cache coherency algorithms, the timeout you describe could be one, Boyds ftp cache rule is also interesting - the timeout value is proportional to how long ago the file was last written, so old files tend to be stable. The cache could be dynamic on the device type on which the file lives, fossil/kfs/kenfs get timeouts, others are assumed to be synthetic files and so should have their verson numbers checked every time as cfs does now. We could even implement a /dev/fossilchg file which contains the dirstat(2) info of all modified files as they change. This would allow persistant (or at least long lived) connections to attempt to keep a near consistant view of the file. That is any file which has had an update since the local cfs has been started would have a known state (be it consistant or inconsisitant); similarly any file who had to have its version checked would have a known state also. This is of course a seperate issue to the coelessing of multiple p9 operations in a single packet as, though as previous posters pointed out it would help cfs too. my 2¢ -Steve ^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [9fans] 9P optimization 2007-06-24 9:34 ` Steve Simon @ 2007-06-24 10:06 ` Lyndon Nerenberg 2007-06-24 11:38 ` Steve Simon 2007-06-25 1:02 ` Dave Eckhardt 0 siblings, 2 replies; 52+ messages in thread From: Lyndon Nerenberg @ 2007-06-24 10:06 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs > It would be fun to modify cfs to have a couple of cache coherency algorithms, > the timeout you describe could be one, Boyds ftp cache rule is also interesting - > the timeout value is proportional to how long ago the file was last written, so > old files tend to be stable. I loath the idea of pushing state into 9P, but I wonder if a combination of client side cfs with a kqueue-like 'file is modified' server event might not solve most of the problem. Layered into aan perhaps? --lyndon What is this talk of 'release'? Klingons do not make software 'releases'. Our software 'escapes' leaving a bloody trail of designers and quality assurance people in its wake. ^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [9fans] 9P optimization 2007-06-24 10:06 ` Lyndon Nerenberg @ 2007-06-24 11:38 ` Steve Simon 2007-06-24 13:34 ` Lyndon Nerenberg 2007-06-25 1:02 ` Dave Eckhardt 1 sibling, 1 reply; 52+ messages in thread From: Steve Simon @ 2007-06-24 11:38 UTC (permalink / raw) To: 9fans > I loath the idea of pushing state into 9P, but I wonder if a combination > of client side cfs with a kqueue-like 'file is modified' server event > might not solve most of the problem. Layered into aan perhaps? Not sure if I missunderstand you but I am _not_ suggesting pushing any state into p9. My thoughts where modifying cfs heuristics to reduce the probability of waiting for a coherence check (which could be quite a few RTTs). In the limit I also suggested providing a synthetic file which contains dirstat(1) info on modified files as they change - which I think is what you mean by a "kqueue-like 'file is modified' server". The problem with implementing this as a server program on the fileserver machine is that _all_ fossil/kfs/kenfs file accesses would have to pass through this server to be sure that a when a file is changed by somone else you are informed. As fossil (what I use) already knows this info I thougt it would be as well for it to provide this file itself. hope this clarifies. -Steve ^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [9fans] 9P optimization 2007-06-24 11:38 ` Steve Simon @ 2007-06-24 13:34 ` Lyndon Nerenberg 2007-06-25 0:34 ` ron minnich 0 siblings, 1 reply; 52+ messages in thread From: Lyndon Nerenberg @ 2007-06-24 13:34 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs > Not sure if I missunderstand you but I am _not_ suggesting pushing any > state into p9. > My thoughts where modifying cfs heuristics to reduce the probability of > waiting for a coherence check (which could be quite a few RTTs). We're on the same track. This stuff would fit in cfs just as easily. What I'm curious about is if a 'the file was modified' callback would help -- basically, just a notification to flush the cache, which would let the client be more aggressive about holding blocks in the cache without having to ping the server on every read. 9P wouldn't need any mods -- this would work easily as a filter pushed on the 9fs connection. But it would need a callback facility to notify the server side of the cache filter that the file had changed. --lyndon ^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [9fans] 9P optimization 2007-06-24 13:34 ` Lyndon Nerenberg @ 2007-06-25 0:34 ` ron minnich 2007-06-25 17:53 ` Lyndon Nerenberg 0 siblings, 1 reply; 52+ messages in thread From: ron minnich @ 2007-06-25 0:34 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs On 6/24/07, Lyndon Nerenberg <lyndon@orthanc.ca> wrote: > 9P wouldn't need any mods -- this would work easily as a filter pushed on > the 9fs connection. But it would need a callback facility to notify the > server side of the cache filter that the file had changed. callback? Couldn't you just post a read on a file in the server and have a thread hang on that? ron ^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [9fans] 9P optimization 2007-06-25 0:34 ` ron minnich @ 2007-06-25 17:53 ` Lyndon Nerenberg 0 siblings, 0 replies; 52+ messages in thread From: Lyndon Nerenberg @ 2007-06-25 17:53 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs > callback? Couldn't you just post a read on a file in the server and > have a thread hang on that? I was thinking more along the lines of a notify server where you would register pathnames of interest and post a read on a file that would receive the event notifications. --lyndon Never hit a man with glasses. Hit him with a baseball bat. ^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [9fans] 9P optimization 2007-06-24 10:06 ` Lyndon Nerenberg 2007-06-24 11:38 ` Steve Simon @ 2007-06-25 1:02 ` Dave Eckhardt 1 sibling, 0 replies; 52+ messages in thread From: Dave Eckhardt @ 2007-06-25 1:02 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs There is some literature on the topic, e.g., AFS callbacks (later called "leases" in the NFS world) and Coda's variable- granularity callbacks... the problems aren't exactly the same, but it seems like there might well be an opportunity to learn from existing experience. This describes experience gained from a system with actual users (i.e., isn't just a simulation study): Exploiting Weak Connectivity for Mobile File Access http://www.cs.cmu.edu/~coda/docdir/s15.pdf Dave Eckhardt ^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [9fans] 9P optimization 2007-06-24 4:08 ` ron minnich 2007-06-24 9:34 ` Steve Simon @ 2007-06-24 12:14 ` erik quanstrom 2007-06-24 13:36 ` Lyndon Nerenberg 1 sibling, 1 reply; 52+ messages in thread From: erik quanstrom @ 2007-06-24 12:14 UTC (permalink / raw) To: 9fans rather than go to these extremes, why not use a local root? run replica in a cron job if necessary. a paired-down root can fit in 32MB more-or-less. i use import so i can run acme locally, and cpu in acme windows to access machines at work. this way only command stdin, stdout, stderr and the files i edit cross the wire. - erik ^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [9fans] 9P optimization 2007-06-24 12:14 ` erik quanstrom @ 2007-06-24 13:36 ` Lyndon Nerenberg 2007-06-24 21:47 ` erik quanstrom 0 siblings, 1 reply; 52+ messages in thread From: Lyndon Nerenberg @ 2007-06-24 13:36 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs > rather than go to these extremes, why not use a local root? Even with a local root, importing stuff over (say) wifi can be painful. --lyndon NT as a file server is faster than a dead bat carrying Post-It notes underwater. But not by much. ^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [9fans] 9P optimization 2007-06-24 13:36 ` Lyndon Nerenberg @ 2007-06-24 21:47 ` erik quanstrom 2007-06-25 17:48 ` Lyndon Nerenberg 0 siblings, 1 reply; 52+ messages in thread From: erik quanstrom @ 2007-06-24 21:47 UTC (permalink / raw) To: 9fans > Even with a local root, importing stuff over (say) wifi can be painful. perhaps there's something about your setup that's different than mine. even mounting root over a wireless connection is not a problem. my desktop machine is connected a shared (with a dozen other computers) wireless bridge. the ping time of 3-4ms is not noticable unless one starts compiling the kernel locally. but cpu solves that problem quite nicely. datakit terminals had similar latency and worse bandwidth. at home, i have a 40ms ping time (surely your wireless connection is faster) to work servers. i sometimes notice just a slight delay saving files over an encrypted, imported connection. but since i compile over a cpu session, it's just not an issue. - erik ^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [9fans] 9P optimization 2007-06-24 21:47 ` erik quanstrom @ 2007-06-25 17:48 ` Lyndon Nerenberg 2007-06-25 18:46 ` erik quanstrom 0 siblings, 1 reply; 52+ messages in thread From: Lyndon Nerenberg @ 2007-06-25 17:48 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs > perhaps there's something about your setup that's different than > mine. There is. When I tried it it was over a 11 Mb/s wireless link with much worse latency than you have. (We were seeing 200+ms RTTs to anywhere that mattered, with 15-20% packet loss.) --lyndon I love to sing, and I love to drink scotch. Most people would rather hear me drink scotch. -- George Burns ^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [9fans] 9P optimization 2007-06-25 17:48 ` Lyndon Nerenberg @ 2007-06-25 18:46 ` erik quanstrom 0 siblings, 0 replies; 52+ messages in thread From: erik quanstrom @ 2007-06-25 18:46 UTC (permalink / raw) To: 9fans that's pretty massive packet loss. tcp is likely to suffer mightily. i used dialup until last year. the rtt time to the gatway was 150ms and the provider was proud enough of their network to push the default mtu to 1500 bytes. i never measured any packet loss. perhaps you would have better luck with dialup? - erik On Mon Jun 25 13:48:55 EDT 2007, lyndon@orthanc.ca wrote: > There is. When I tried it it was over a 11 Mb/s wireless link with much > worse latency than you have. (We were seeing 200+ms RTTs to anywhere that > mattered, with 15-20% packet loss.) > > --lyndon ^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [9fans] 9P optimization 2007-06-24 3:15 ` Lyndon Nerenberg 2007-06-24 4:08 ` ron minnich @ 2007-06-24 4:16 ` Bruce Ellis 2007-06-24 18:05 ` Skip Tavakkolian 1 sibling, 1 reply; 52+ messages in thread From: Bruce Ellis @ 2007-06-24 4:16 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs rangbooming has lead to extreme optimiazations. i'll let skip comment on it (he's asleep i guess). but i couldn't have much worse RTT (sydney-seattle), and the traffic seems a lot lower now, and the latency has improved extraordinarily. brucee On 6/24/07, Lyndon Nerenberg <lyndon@orthanc.ca> wrote: > > The problem with cfs is that you still suffer the RPC to check for validity. > > With 100ms of RTT, count 5 and you have a second. Also, for synthesized > > files, > > the cache does not save you from doing a walk, open, read. > > I'll be accused of being a heretic, but let me claim that much of the > design of IMAP is geared towards avoiding those RTTs for that very reason. > A good IMAP client wins by doing intelligent cacheing, and it gets there > by using the subtle aspects of the protocol that accomodate that. > > Much as people slag the protocol, it does work well over slow (high > latency) links, given a client that understands the protocol (sadly, most > don't). > > I'm very curious about the efficiency of cfs, but my P9 environment > doesn't have a remote terminal. It would be interesting to instrument cfs > and collect some stats on the protocol behaviour behind the cache ... > > --lyndon > ^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [9fans] 9P optimization 2007-06-24 4:16 ` Bruce Ellis @ 2007-06-24 18:05 ` Skip Tavakkolian 0 siblings, 0 replies; 52+ messages in thread From: Skip Tavakkolian @ 2007-06-24 18:05 UTC (permalink / raw) To: 9fans > rangbooming has lead to extreme optimiazations. i'll let skip > comment on it (he's asleep i guess). but i couldn't have much > worse RTT (sydney-seattle), and the traffic seems a lot lower now, > and the latency has improved extraordinarily. behavior of windows exploder (rer) and network latency really magnify this problem. we cache the walks, assisted by staling and usage tracking. staling >1×rtt will satiate incessant walk requests from cache. it is useful for what we're doing. brucee has an idea on how it could be applied to something like exportfs, for general use. regarding locks or cache expiry notification that others suggested, the proposal in snspaper was to address that, but in the end we all agreed that if you need it, a control file between the fileserver and the importing side would be sufficient. ^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [9fans] About 9P ... 2007-06-22 21:46 ` David Leimbach 2007-06-22 21:55 ` Philippe Anel @ 2007-06-22 23:20 ` Skip Tavakkolian 2007-06-22 23:48 ` Francisco J Ballesteros 1 sibling, 1 reply; 52+ messages in thread From: Skip Tavakkolian @ 2007-06-22 23:20 UTC (permalink / raw) To: 9fans there were discussions about aysnc syscalls. /sys/src/cmd/fcp.c is a good example of why they're not needed. streaming and long delay networks can be handled this way too, as was pointed out (by Russ i think) at iwp9. ^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [9fans] About 9P ... 2007-06-22 23:20 ` [9fans] About 9P Skip Tavakkolian @ 2007-06-22 23:48 ` Francisco J Ballesteros 2007-06-23 0:27 ` Roman Shaposhnick 2007-06-23 12:08 ` erik quanstrom 0 siblings, 2 replies; 52+ messages in thread From: Francisco J Ballesteros @ 2007-06-22 23:48 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs Skip wrote: :there were discussions about aysnc syscalls. /sys/src/cmd/fcp.c is a :good example of why they're not needed. streaming and long delay :networks can be handled this way too, as was pointed out (by Russ i :think) at iwp9. But there is one problem. Consider "lc". Usually you see walk clunk walk clunk walk open write clunk and also walk stat clunk walk open read clunk The problem is, how to know which RPCs to pack? I mean, to send without waiting for reply. Note that several different fids might be used in these dialogs, even though they come in a small time window from the same process. I tried to imagine how to bundle them, for using 9P over high latency links, and couldn't find a good way of doing it (after iwp9). In the end, I had to implement a couple of processes to bridge islands with good internal latency, connected by poor links. They try to refrain from sending requests by using cached directory entries (and data) to try to make such runs in one or two RPCs. The hard part is making it work well both for regular files and for synthesized files. We have a version that works reasonable well for both, (only connections multiplexed over the same file, eg. clone opens, are not handled properly). Our current version of the protocol has "file descriptors" that are fids in that they "mean" a path for a file, and are not fids in that more than one fid may map to the same descriptor, and also in that paths are sent along with the descriptor to make the server stateless (or to make clients think so). 9P is just great for use when latency is reasonable (or not too bad, with cfs), but to go further away and still be comfortable using remote files, I'd say we need another protocol. I'd love to be proved wrong :) ^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [9fans] About 9P ... 2007-06-22 23:48 ` Francisco J Ballesteros @ 2007-06-23 0:27 ` Roman Shaposhnick 2007-06-23 0:27 ` Francisco J Ballesteros 2007-06-23 12:08 ` erik quanstrom 1 sibling, 1 reply; 52+ messages in thread From: Roman Shaposhnick @ 2007-06-23 0:27 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs On Sat, 2007-06-23 at 01:48 +0200, Francisco J Ballesteros wrote: > Skip wrote: > :there were discussions about aysnc syscalls. /sys/src/cmd/fcp.c is a > :good example of why they're not needed. streaming and long delay > :networks can be handled this way too, as was pointed out (by Russ i > :think) at iwp9. > > But there is one problem. Consider "lc". > > Usually you see > walk > clunk > walk > clunk > walk > open > write > clunk > > and also > walk > stat clunk > walk > open > read > clunk > > The problem is, how to know which RPCs to pack? Isn't it the very same problem that compilers have with instruction pipelining? http://en.wikipedia.org/wiki/Instruction_pipelining With an additional complication that you can't actually look ahead very often? Thanks, Roman. ^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [9fans] About 9P ... 2007-06-23 0:27 ` Roman Shaposhnick @ 2007-06-23 0:27 ` Francisco J Ballesteros 0 siblings, 0 replies; 52+ messages in thread From: Francisco J Ballesteros @ 2007-06-23 0:27 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs I don't think so. Because we try to use results from past, and not to look ahead. On 6/23/07, Roman Shaposhnick <rvs@sun.com> wrote: > On Sat, 2007-06-23 at 01:48 +0200, Francisco J Ballesteros wrote: > > Skip wrote: > > :there were discussions about aysnc syscalls. /sys/src/cmd/fcp.c is a > > :good example of why they're not needed. streaming and long delay > > :networks can be handled this way too, as was pointed out (by Russ i > > :think) at iwp9. > > > > But there is one problem. Consider "lc". > > > > Usually you see > > walk > > clunk > > walk > > clunk > > walk > > open > > write > > clunk > > > > and also > > walk > > stat clunk > > walk > > open > > read > > clunk > > > > The problem is, how to know which RPCs to pack? > > Isn't it the very same problem that compilers have with > instruction pipelining? > http://en.wikipedia.org/wiki/Instruction_pipelining > > With an additional complication that you can't actually > look ahead very often? > > Thanks, > Roman. > > ^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [9fans] About 9P ... 2007-06-22 23:48 ` Francisco J Ballesteros 2007-06-23 0:27 ` Roman Shaposhnick @ 2007-06-23 12:08 ` erik quanstrom 2007-06-23 12:27 ` Francisco J Ballesteros ` (2 more replies) 1 sibling, 3 replies; 52+ messages in thread From: erik quanstrom @ 2007-06-23 12:08 UTC (permalink / raw) To: 9fans > 9P is just great for use when latency is reasonable (or not too bad, > with cfs), but to go further > away and still be comfortable using remote files, I'd say we need > another protocol. > I'd love to be proved wrong :) are there any protocols that deal well with latency? the only way i know to deal with latency is to do some sort of tagged queueing. (perhaps i don't know enough computer history.) unfortunately, this doesn't work if part of the data you need depend on some prior part; a conversation means ping-pong communication. the great news for 9p is that latency is decreasing. in /sys/doc/net/net.ps, the IL/ether latency is listed as having a latency of 1420µs. my home fileserver has a 57µs latency to my cpu server. (the fileserver is a pIII with a $19 rtl8169.) even custom Cyclone interface between the file server and the cpu server is listed as having a latency of 375µs. let's hope the same holds true for wireless networking. - erik ^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [9fans] About 9P ... 2007-06-23 12:08 ` erik quanstrom @ 2007-06-23 12:27 ` Francisco J Ballesteros 2007-06-23 12:57 ` Lucio De Re 2007-06-23 13:01 ` Uriel 2 siblings, 0 replies; 52+ messages in thread From: Francisco J Ballesteros @ 2007-06-23 12:27 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs Yes, but latency from Madrid to Greece is 150ms (rtt, icmp) and that I'd say is going to stay that way for a long time. To the US (east coast) it's 120ms. I still like to use my system while in the move. Regarding protocols, lbfs is nice, but focuses on bandwidth and so, it requires multiple RPCs for things that, for example, NFSv4, does in one or two. On 6/23/07, erik quanstrom <quanstro@coraid.com> wrote: > > 9P is just great for use when latency is reasonable (or not too bad, > > with cfs), but to go further > > away and still be comfortable using remote files, I'd say we need > > another protocol. > > I'd love to be proved wrong :) > > are there any protocols that deal well with latency? > > the only way i know to deal with latency is to do some > sort of tagged queueing. (perhaps i don't know enough > computer history.) unfortunately, this doesn't > work if part of the data you need depend on some > prior part; a conversation means ping-pong communication. > > the great news for 9p is that latency is decreasing. in > /sys/doc/net/net.ps, the IL/ether latency is listed as > having a latency of 1420µs. my home fileserver > has a 57µs latency to my cpu server. (the fileserver is a > pIII with a $19 rtl8169.) > > even custom Cyclone interface between the file server and > the cpu server is listed as having a latency of 375µs. > > let's hope the same holds true for wireless networking. > > - erik > ^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [9fans] About 9P ... 2007-06-23 12:08 ` erik quanstrom 2007-06-23 12:27 ` Francisco J Ballesteros @ 2007-06-23 12:57 ` Lucio De Re 2007-06-23 13:01 ` Uriel 2 siblings, 0 replies; 52+ messages in thread From: Lucio De Re @ 2007-06-23 12:57 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs On Sat, 2007-06-23 at 08:08 -0400, erik quanstrom wrote: > are there any protocols that deal well with latency? > IP does by design. The correct approach to the problem is the sliding-window technique and I can only presume that applications that suffer from high latency are written to conflict with the underlying protocols. I mean, we've had X-modem since the ark. I've tried to avoid becoming too familiar with 9P to be able to comment because my mind isn't as sharp as it was when I produced end-to-end protocols to run on 300 bps lines, but maybe somebody can explain to me whether 9P is one such ill-mannered application. From the immediately preceding discussion, it doesn't seem so, but I do not speak from a position of knowledge. ++L ^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [9fans] About 9P ... 2007-06-23 12:08 ` erik quanstrom 2007-06-23 12:27 ` Francisco J Ballesteros 2007-06-23 12:57 ` Lucio De Re @ 2007-06-23 13:01 ` Uriel 2007-06-23 13:36 ` Francisco J Ballesteros 2 siblings, 1 reply; 52+ messages in thread From: Uriel @ 2007-06-23 13:01 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs I thought russ (and charles and sape?) proposed a good solutions to the latency issues in 9P at IWP9. Actually as far as I can remember it was the only change to 9P that was accepted as justified. My memory is bad, probably got the details wrong but I think there were two approaches: one by allowing multiple requests with the same tag to be grouped, and another with an extra flag filed that indicates if more messages in a group are on their way. I still don't understand what could be wrong with either approach (but again, I don't even remember the details of either) and why nemo had to invent yet another protocol. Maybe someone can enlighten my usual ignorance. uriel On 6/23/07, erik quanstrom <quanstro@coraid.com> wrote: > > 9P is just great for use when latency is reasonable (or not too bad, > > with cfs), but to go further > > away and still be comfortable using remote files, I'd say we need > > another protocol. > > I'd love to be proved wrong :) > > are there any protocols that deal well with latency? > > the only way i know to deal with latency is to do some > sort of tagged queueing. (perhaps i don't know enough > computer history.) unfortunately, this doesn't > work if part of the data you need depend on some > prior part; a conversation means ping-pong communication. > > the great news for 9p is that latency is decreasing. in > /sys/doc/net/net.ps, the IL/ether latency is listed as > having a latency of 1420µs. my home fileserver > has a 57µs latency to my cpu server. (the fileserver is a > pIII with a $19 rtl8169.) > > even custom Cyclone interface between the file server and > the cpu server is listed as having a latency of 375µs. > > let's hope the same holds true for wireless networking. > > - erik > ^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [9fans] About 9P ... 2007-06-23 13:01 ` Uriel @ 2007-06-23 13:36 ` Francisco J Ballesteros 2007-06-23 14:27 ` Lucio De Re 0 siblings, 1 reply; 52+ messages in thread From: Francisco J Ballesteros @ 2007-06-23 13:36 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs It would be fine to just use the same tag as suggested by Russ ans Sape if you just happen to have several 9P requests that you know that have to be issued to serve a particular call. For example, say you add a library function (or syscall) get(), you could send Twalk tag3 nfid F Topen fid F, Tread fid F, Tread fid F, .... all with the same tag, without waiting for replies, and the server, knowning they have the same tag, would sever Topen, Tread, ... only if Twalk worked. That would batch all the requests, as intended. Now, say you have a program that does: if (stat(afile)){ open(afile) read(afile) close(afile) } By the time of the stat you could have read the initial part of the file along with its metadata (maybe the entire file!, if it's small). Also, some other program might be using the same file, and you'd send the RPCs on its behalf. If instead of changing the protocol, you use intermediate processes that do this "caching", this is what happens: by the time of the stat, the process does a get (even if it has a cache entry for the file, to check for cache validity). That get, retrieves initial data for the file. Because the entry is known valid, any call to read bytes (and any stat done by any other process in the mean time) would be served from the local process, no further RPCs. In particular, the open() above would use the entry in the cache and would not walk to the file again. This does not hurt (much) coherency. Suppose the file was removed after open, you cannot tell if what happen is that it was removed after read (going to the server to do the read) or what happen is that it was removed after open (and data was served from the the data just read during open, or during stat). As this caching thing changes the semantics quite a bit, it seemed ok to use the simplest protocol that we could imagine that would implement such behaviour, instead of batching 9p requests just to have 9p on the wire. Note that having two programs <file calls> <-> op and op <-> styx means that nobody but these two programs has to be aware of the new protocol. It's buried deep in the bad latency communications link, and everyone else thinks that styx is the protocol spoken everywhere. Again, if anyone sees anything wrong if this approach, even though it's already implemented and seems to be working fine, I'd love to find out a better way. On 6/23/07, Uriel <uriel99@gmail.com> wrote: > I thought russ (and charles and sape?) proposed a good solutions to > the latency issues in 9P at IWP9. Actually as far as I can remember it > was the only change to 9P that was accepted as justified. > > My memory is bad, probably got the details wrong but I think there > were two approaches: one by allowing multiple requests with the same > tag to be grouped, and another with an extra flag filed that indicates > if more messages in a group are on their way. > > I still don't understand what could be wrong with either approach (but > again, I don't even remember the details of either) and why nemo had > to invent yet another protocol. Maybe someone can enlighten my usual > ignorance. > > uriel > > On 6/23/07, erik quanstrom <quanstro@coraid.com> wrote: > > > 9P is just great for use when latency is reasonable (or not too bad, > > > with cfs), but to go further > > > away and still be comfortable using remote files, I'd say we need > > > another protocol. > > > I'd love to be proved wrong :) > > > > are there any protocols that deal well with latency? > > > > the only way i know to deal with latency is to do some > > sort of tagged queueing. (perhaps i don't know enough > > computer history.) unfortunately, this doesn't > > work if part of the data you need depend on some > > prior part; a conversation means ping-pong communication. > > > > the great news for 9p is that latency is decreasing. in > > /sys/doc/net/net.ps, the IL/ether latency is listed as > > having a latency of 1420µs. my home fileserver > > has a 57µs latency to my cpu server. (the fileserver is a > > pIII with a $19 rtl8169.) > > > > even custom Cyclone interface between the file server and > > the cpu server is listed as having a latency of 375µs. > > > > let's hope the same holds true for wireless networking. > > > > - erik > > > ^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [9fans] About 9P ... 2007-06-23 13:36 ` Francisco J Ballesteros @ 2007-06-23 14:27 ` Lucio De Re 2007-06-23 14:45 ` Francisco J Ballesteros 0 siblings, 1 reply; 52+ messages in thread From: Lucio De Re @ 2007-06-23 14:27 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs On Sat, 2007-06-23 at 15:36 +0200, Francisco J Ballesteros wrote: > > Now, say you have a program that does: > if (stat(afile)){ > open(afile) > read(afile) > close(afile) > } > So those are 9P's bad manners. How many such idioms can one subsume into a "caching" process? Because in my opinion it makes more sense, as I understand to have been Russ and Sape's proposal, to provide a sliding-window version of the above that sends all of them at once: if (stat(afile), open(afile), read(afile), close(afile)) { ... } else { perror (""); } Somehow, we may need to differentiate the error return and it sure looks like programmers need to be aware of this implementation technique, but is it preferable to construct a specialised cache that knows to generate a get() whenever a stat() is performed? The latter doesn't really scale to other potential scenarios and may add unwanted overheads. ++L ^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [9fans] About 9P ... 2007-06-23 14:27 ` Lucio De Re @ 2007-06-23 14:45 ` Francisco J Ballesteros 0 siblings, 0 replies; 52+ messages in thread From: Francisco J Ballesteros @ 2007-06-23 14:45 UTC (permalink / raw) To: lucio, Fans of the OS Plan 9 from Bell Labs Note that if (stat()){ open read close } is not new code. It's code in ls, 8c, mk, acme, etc. etc. etc. The point is how to make that code work nicely across bad latency links. But you do not want to change that code. Also, there's no overhead if you don't use our stuff while "within the building". On 6/23/07, Lucio De Re <lucio@proxima.alt.za> wrote: > On Sat, 2007-06-23 at 15:36 +0200, Francisco J Ballesteros wrote: > > > > > Now, say you have a program that does: > > if (stat(afile)){ > > open(afile) > > read(afile) > > close(afile) > > } > > > So those are 9P's bad manners. How many such idioms can one subsume > into a "caching" process? Because in my opinion it makes more sense, as > I understand to have been Russ and Sape's proposal, to provide a > sliding-window version of the above that sends all of them at once: > > if (stat(afile), open(afile), read(afile), close(afile)) { > ... > } else { > perror (""); > } > > Somehow, we may need to differentiate the error return and it sure looks > like programmers need to be aware of this implementation technique, but > is it preferable to construct a specialised cache that knows to generate > a get() whenever a stat() is performed? The latter doesn't really scale > to other potential scenarios and may add unwanted overheads. > > ++L > > > ^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [9fans] About 9P ... 2007-06-22 20:32 ` Philippe Anel 2007-06-22 21:10 ` David Leimbach @ 2007-06-22 21:16 ` Skip Tavakkolian 2007-06-22 21:37 ` Philippe Anel 1 sibling, 1 reply; 52+ messages in thread From: Skip Tavakkolian @ 2007-06-22 21:16 UTC (permalink / raw) To: 9fans > So I think you should not pipeline the requests. > Can a 9P specialist can confirm this ? not true. ^ permalink raw reply [flat|nested] 52+ messages in thread
* Re: [9fans] About 9P ... 2007-06-22 21:16 ` Skip Tavakkolian @ 2007-06-22 21:37 ` Philippe Anel 0 siblings, 0 replies; 52+ messages in thread From: Philippe Anel @ 2007-06-22 21:37 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs My bad, I'm not a good english writter ... I meant replies to request are not supposed to be returned in request order. Skip Tavakkolian a écrit : >> So I think you should not pipeline the requests. >> Can a 9P specialist can confirm this ? >> > > not true. > > > > ^ permalink raw reply [flat|nested] 52+ messages in thread
end of thread, other threads:[~2007-06-25 18:46 UTC | newest] Thread overview: 52+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2007-06-19 20:11 [9fans] About 9P Philippe Anel 2007-06-20 0:37 ` Russ Cox 2007-06-20 2:06 ` Roman Shaposhnick 2007-06-20 11:29 ` Anthony Sorace 2007-06-22 0:32 ` Derek Fawcus 2007-06-22 6:58 ` Philippe Anel 2007-06-22 15:57 ` Derek Fawcus 2007-06-22 20:32 ` Philippe Anel 2007-06-22 21:10 ` David Leimbach 2007-06-22 21:35 ` Philippe Anel 2007-06-22 21:46 ` David Leimbach 2007-06-22 21:55 ` Philippe Anel 2007-06-22 22:25 ` Philippe Anel 2007-06-22 22:57 ` David Leimbach 2007-06-22 23:01 ` Anant Narayanan 2007-06-23 4:30 ` Russ Cox 2007-06-23 5:10 ` Skip Tavakkolian 2007-06-23 9:25 ` Francisco J Ballesteros 2007-06-23 15:50 ` Martin Atkins 2007-06-23 6:24 ` Philippe Anel 2007-06-23 15:00 ` Rob Pike 2007-06-24 0:52 ` [9fans] 9P optimization Lyndon Nerenberg 2007-06-24 3:04 ` Francisco J Ballesteros 2007-06-24 3:15 ` Lyndon Nerenberg 2007-06-24 4:08 ` ron minnich 2007-06-24 9:34 ` Steve Simon 2007-06-24 10:06 ` Lyndon Nerenberg 2007-06-24 11:38 ` Steve Simon 2007-06-24 13:34 ` Lyndon Nerenberg 2007-06-25 0:34 ` ron minnich 2007-06-25 17:53 ` Lyndon Nerenberg 2007-06-25 1:02 ` Dave Eckhardt 2007-06-24 12:14 ` erik quanstrom 2007-06-24 13:36 ` Lyndon Nerenberg 2007-06-24 21:47 ` erik quanstrom 2007-06-25 17:48 ` Lyndon Nerenberg 2007-06-25 18:46 ` erik quanstrom 2007-06-24 4:16 ` Bruce Ellis 2007-06-24 18:05 ` Skip Tavakkolian 2007-06-22 23:20 ` [9fans] About 9P Skip Tavakkolian 2007-06-22 23:48 ` Francisco J Ballesteros 2007-06-23 0:27 ` Roman Shaposhnick 2007-06-23 0:27 ` Francisco J Ballesteros 2007-06-23 12:08 ` erik quanstrom 2007-06-23 12:27 ` Francisco J Ballesteros 2007-06-23 12:57 ` Lucio De Re 2007-06-23 13:01 ` Uriel 2007-06-23 13:36 ` Francisco J Ballesteros 2007-06-23 14:27 ` Lucio De Re 2007-06-23 14:45 ` Francisco J Ballesteros 2007-06-22 21:16 ` Skip Tavakkolian 2007-06-22 21:37 ` Philippe Anel
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).