* [9fans] synthetic filesystems and changing data @ 2007-08-10 1:47 John Marshall 2007-08-10 1:57 ` erik quanstrom 2007-08-10 6:49 ` matt 0 siblings, 2 replies; 8+ messages in thread From: John Marshall @ 2007-08-10 1:47 UTC (permalink / raw) To: 9fans Hi, I am trying to understand the general practice, under Plan 9, for synthetic filesystems for serving up a "file" which: 1) returns data that changes (quickly), and 2) returns data whose size is larger than the message size agreed upon at *version interchange. Practically, what I am wondering is: 1) what to do when the "file" content changes between multiple *read operations (assuming that the reads are done quickly)? Is it just tough luck for the client? 2) whether or not synthetic filesystems generally handle (or not) the offset parameter? I imagine to do this requires relatively static content, is difficult, or there is an assumption that the size of the content _must_ fit in the message. Thanks, John ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [9fans] synthetic filesystems and changing data 2007-08-10 1:47 [9fans] synthetic filesystems and changing data John Marshall @ 2007-08-10 1:57 ` erik quanstrom 2007-08-10 8:13 ` Charles Forsyth 2007-08-10 6:49 ` matt 1 sibling, 1 reply; 8+ messages in thread From: erik quanstrom @ 2007-08-10 1:57 UTC (permalink / raw) To: 9fans why not just tie the fid to a "file" version. then it would be easy to handle offsets. - erik On Thu Aug 9 21:47:02 EDT 2007, John.Marshall@ec.gc.ca wrote: > Hi, > > I am trying to understand the general practice, under > Plan 9, for synthetic filesystems for serving up a "file" > which: > 1) returns data that changes (quickly), and > 2) returns data whose size is larger than the message > size agreed upon at *version interchange. > > Practically, what I am wondering is: > 1) what to do when the "file" content changes between > multiple *read operations (assuming that the reads > are done quickly)? Is it just tough luck for the client? > 2) whether or not synthetic filesystems generally handle > (or not) the offset parameter? I imagine to do this > requires relatively static content, is difficult, or > there is an assumption that the size of the content > _must_ fit in the message. > > Thanks, > John > ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [9fans] synthetic filesystems and changing data 2007-08-10 1:57 ` erik quanstrom @ 2007-08-10 8:13 ` Charles Forsyth 2007-08-10 9:40 ` maht 0 siblings, 1 reply; 8+ messages in thread From: Charles Forsyth @ 2007-08-10 8:13 UTC (permalink / raw) To: 9fans > why not just tie the fid to a "file" version. then it would be easy > to handle offsets. that's effectively what i do in similar cases. so does ndb/cs, amongst others. ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [9fans] synthetic filesystems and changing data 2007-08-10 8:13 ` Charles Forsyth @ 2007-08-10 9:40 ` maht 0 siblings, 0 replies; 8+ messages in thread From: maht @ 2007-08-10 9:40 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs And just to elucidate further one frees the data on a clunk. It could be the source of a DoS if your 9p is public facing. >> why not just tie the fid to a "file" version. then it would be easy >> to handle offsets. >> > > that's effectively what i do in similar cases. so does ndb/cs, amongst others. > > > ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [9fans] synthetic filesystems and changing data 2007-08-10 1:47 [9fans] synthetic filesystems and changing data John Marshall 2007-08-10 1:57 ` erik quanstrom @ 2007-08-10 6:49 ` matt 2007-08-10 12:32 ` John Marshall 1 sibling, 1 reply; 8+ messages in thread From: matt @ 2007-08-10 6:49 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs The coder for the file system makes the choice whether to keep a snapshot of the data to be read per FID, this way the FID reader will always get the file they asked for, which, after the first read, isn't necessarily the current version or to just return whatever each Tread asks for regardless of whether the underlying data has changed. It is akin to an SQL cursor. When I make an SQL request the rows returned are a snapshot of the data when the query was made. If I read them at one row per second there's a chance that the actual data in the datastore changes between reads. Either way there's a "tough luck" aspect, if you have a solution I'm sure everyone would be delighted to hear it :) It's a perinnial problem. When opening a file for writing some programs (acme for instance) will warn you if the file has changed since you opened it. That's a solution to the same problem. Matt ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [9fans] synthetic filesystems and changing data 2007-08-10 6:49 ` matt @ 2007-08-10 12:32 ` John Marshall 2007-08-10 15:13 ` Kris Maglione 0 siblings, 1 reply; 8+ messages in thread From: John Marshall @ 2007-08-10 12:32 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs matt wrote: > The coder for the file system makes the choice whether to keep a > snapshot of the data to be read per FID, this way the FID reader will > always get the file they asked for, which, after the first read, isn't > necessarily the current version or to just return whatever each Tread > asks for regardless of whether the underlying data has changed. > > It is akin to an SQL cursor. When I make an SQL request the rows > returned are a snapshot of the data when the query was made. If I read > them at one row per second there's a chance that the actual data in the > datastore changes between reads. Hi, Although I was not immediately convinced that a versioning approach was all that useful, I can now see that it roughly matches what I've always done for servicing web requests for dynamically generated data: each connection keeps a version, which may or may not be out of date, which problem is left up to the client to deal with. I guess the difference was that buffering each version in the web context has always been so natural. All the "state" (connection, user, object being referenced, cached data) was kept separately for the entire lifetime of the connection/transaction and I did not have to worry about anything. I could even timeout the transaction easily. In the versioning case I would have to 1) store the different versions somewhere (local fs or memory), 2) track their association with the related fids, 3) and remove them on clunk. And I would have to keep the version around until the clunk which might happen in 1s, 10 minutes, or never. > Either way there's a "tough luck" aspect, if you have a solution I'm > sure everyone would be delighted to hear it :) > > It's a perinnial problem. When opening a file for writing some programs > (acme for instance) will warn you if the file has changed since you > opened it. That's a solution to the same problem. From people's experience, do the following seem like reasonable alternatives, if the situation allows for it? 1) return an error message if the message size is too small to return the content in one response? The application would simply trap for the error condition and retry. 2) if, over the course of returning the individual messages for a read, an error message was returned which indicated a changed file (the read operation would no longer return data)? Again, the application would simply trap for the error condition and retry. This is something like what editors do (acme, as you mention), except that the read operation is affected rather than trying to prevent an overwrite of possibly newer data. I guess I am trying to avoid versioning because there are too many aspects that I am concerned I could not reasonably deal with. I'd like to be able to support something akin to an atomic transaction (although it would involve multiple reads/messages). Any comments on a third option? 3) if versioning is used, reads would only be supported on a cached version for a limited period of time. Afterwards, the cached version would be dropped and an error message would be returned indicating something like "File version has expired." I realize that this none of these may be general solutions but for specific kinds of data (e.g., status information) they may be. And, the client accessing the files would have to be prepared to deal with the different kinds of error messages. The goal is to get correct/consistent data. Thanks for everyone's input, John ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [9fans] synthetic filesystems and changing data 2007-08-10 12:32 ` John Marshall @ 2007-08-10 15:13 ` Kris Maglione 2007-08-10 15:47 ` nemo 0 siblings, 1 reply; 8+ messages in thread From: Kris Maglione @ 2007-08-10 15:13 UTC (permalink / raw) To: 9fans On Fri, Aug 10, 2007 at 08:32:18AM -0400, John Marshall wrote: > Although I was not immediately convinced that a versioning approach was > all that useful, I can now see that it roughly matches what I've always > done for servicing web requests for dynamically generated data: each > connection keeps a version... Using lib9p, each open Fid gives you a void* pointer to store whatever fid-specific data you need. Whether that's just a copy of the file's contents, or you implement some copy-on-write scheme, it's not so different. You can even timeout connections fairly easily, if you want to. Just keep a list of open Fids, with pointers to structs containing the time they were opened. Periodically check the list for timed out Fids, free the resources they occupy, and return Rerror when the next read comes in. -- Kris Maglione Any given program will expand to fill all available memory. ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [9fans] synthetic filesystems and changing data 2007-08-10 15:13 ` Kris Maglione @ 2007-08-10 15:47 ` nemo 0 siblings, 0 replies; 8+ messages in thread From: nemo @ 2007-08-10 15:47 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs In cases where multiple clients might map to the same fid you can also freeze in a buffer file contents when the offset if zero, and serve from there for further reads. Using 9p (and no cache in the client) means that you can use the fid to mux the clients, and assign also a buffer per client, and release at clunk. ^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2007-08-10 15:47 UTC | newest] Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2007-08-10 1:47 [9fans] synthetic filesystems and changing data John Marshall 2007-08-10 1:57 ` erik quanstrom 2007-08-10 8:13 ` Charles Forsyth 2007-08-10 9:40 ` maht 2007-08-10 6:49 ` matt 2007-08-10 12:32 ` John Marshall 2007-08-10 15:13 ` Kris Maglione 2007-08-10 15:47 ` nemo
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).