* Re: [9fans] venti workalike in Python @ 2003-12-10 12:11 Geoff Collyer 0 siblings, 0 replies; 8+ messages in thread From: Geoff Collyer @ 2003-12-10 12:11 UTC (permalink / raw) To: 9fans I've had a request, so I've added libdbm.tar and libmdbm.tar to my Plan 9 software page, http://www.collyer.net/~geoff/9. ^ permalink raw reply [flat|nested] 8+ messages in thread
* [9fans] venti workalike in Python @ 2003-12-10 2:00 John E. Barham 2003-12-10 2:31 ` Russ Cox ` (3 more replies) 0 siblings, 4 replies; 8+ messages in thread From: John E. Barham @ 2003-12-10 2:00 UTC (permalink / raw) To: 9fans I'm attempting to decipher the venti and vac code and write a workalike in Python. (packet.c had me bogged down for a while...) At this point I don't care that much about compatibility, and can't really test it as I don't have a running instance of Plan 9. But as has been noted here before, the venti procol is relatively trivial--hello, ping, read, write, goodbye, error, although the two auth messages appear to be unused for the moment?--so making it compatible at some point shouldn't be too much of a problem. Having said that, is there any documentation on the venti protocol apart from the source? I'm not planning on re-implementing venti's arena code but thought that it would be quick and easy for testing purposes to implement the back-end storage using something like dbm or, mea culpa, a relational database. (I added binary type support to the Python extension for SQLite with this in mind.) Thoughts? John ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [9fans] venti workalike in Python 2003-12-10 2:00 John E. Barham @ 2003-12-10 2:31 ` Russ Cox 2003-12-10 2:51 ` Geoff Collyer 2003-12-10 22:52 ` William Josephson ` (2 subsequent siblings) 3 siblings, 1 reply; 8+ messages in thread From: Russ Cox @ 2003-12-10 2:31 UTC (permalink / raw) To: 9fans > I'm attempting to decipher the venti and vac code and write a workalike in > Python. (packet.c had me bogged down for a while...) At this point I don't The discussion of the Vac file system format in /sys/doc/fossil.ps may be helpful. > care that much about compatibility, and can't really test it as I don't have > a running instance of Plan 9. But as has been noted here before, the venti > procol is relatively trivial--hello, ping, read, write, goodbye, error, > although the two auth messages appear to be unused for the moment?--so > making it compatible at some point shouldn't be too much of a problem. > Having said that, is there any documentation on the venti protocol apart > from the source? Not really, but here's a summary. Venti uses a simple RPC protocol with only a few messages: size[2] RError tag[1] error[s] size[2] QPing tag[1] size[2] RPing tag[1] size[2] QHello tag[1] version[s] uid[s] strength[1] ncrypto[1] crypto[ncrypto] ncodec[1] codec[ncodec] size[2] RHello tag[1] sid[s] crypto[1] codec[1] size[2] QGoodbye tag[1] size[2] QRead tag[1] score[20] type[1] 0[1] count[2] size[2] RRead tag[1] data[] size[2] QWrite tag[1] type[1] 0[3] data[] size[2] RWrite score[20] size[2] QSync tag[1] size[2] RSync tag[1] In the notation, field[s] indicates a string: a two-byte length followed by that many bytes. The actual message types are one-byte values: enum { VtRError = 1, VtQPing, VtRPing, VtQHello, VtRHello, VtQGoodbye, VtRGoodbye, /* not used */ VtQAuth0, /* not used */ VtRAuth0, /* not used */ VtQAuth1, /* not used */ VtRAuth1, /* not used */ VtQRead, VtRRead, VtQWrite, VtRWrite, VtQSync, VtRSync, VtMaxOp }; Transactions and tags are like in 9P, except that tags are one byte and the request messages have names beginning with Q instead of T. The strength, crypto, and codec stuff was never fleshed out and is essentially unused. > I'm not planning on re-implementing venti's arena code but thought that it > would be quick and easy for testing purposes to implement the back-end > storage using something like dbm or, mea culpa, a relational database. (I > added binary type support to the Python extension for SQLite with this in > mind.) Thoughts? We use Berkeley DB at MIT for another project to store 8kB blocks indexed by SHA1 hash. It's adequate. Russ ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [9fans] venti workalike in Python 2003-12-10 2:31 ` Russ Cox @ 2003-12-10 2:51 ` Geoff Collyer 0 siblings, 0 replies; 8+ messages in thread From: Geoff Collyer @ 2003-12-10 2:51 UTC (permalink / raw) To: 9fans Berkeley DB has grown a lot over the years. I recently ported Ken's dbm library from V7 to native Plan 9, and modified it to store all integers on disk in big-endian order, so that dbm databases can be shared across architectures (there are still locking problems to be solved for concurrent access, though). I did the same for Chris Torek's extended dbm, mdbm, which adds multiple keys per value and the ability to have multiple databases open simultaneously. My mdbm port will also run on Unix, so one can share my mdbm databases across architectures there too. All of this has been only lightly exercised, but if you'd like a copy, let me know. ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [9fans] venti workalike in Python 2003-12-10 2:00 John E. Barham 2003-12-10 2:31 ` Russ Cox @ 2003-12-10 22:52 ` William Josephson 2003-12-11 7:20 ` John E. Barham 2003-12-11 9:46 ` John E. Barham 3 siblings, 0 replies; 8+ messages in thread From: William Josephson @ 2003-12-10 22:52 UTC (permalink / raw) To: 9fans On Tue, Dec 09, 2003 at 06:00:41PM -0800, John E. Barham wrote: > I'm not planning on re-implementing venti's arena code but thought that it > would be quick and easy for testing purposes to implement the back-end > storage using something like dbm or, mea culpa, a relational database. (I > added binary type support to the Python extension for SQLite with this in > mind.) Thoughts? I'd use Berkeley DB. In fact I have a small prototype system that does just that using 4K data blocks. For small amounts of data it works fine and I'd expect it to be OK up to at least a hundred gigabytes. ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [9fans] venti workalike in Python 2003-12-10 2:00 John E. Barham 2003-12-10 2:31 ` Russ Cox 2003-12-10 22:52 ` William Josephson @ 2003-12-11 7:20 ` John E. Barham 2003-12-11 9:46 ` John E. Barham 3 siblings, 0 replies; 8+ messages in thread From: John E. Barham @ 2003-12-11 7:20 UTC (permalink / raw) To: 9fans > > Having said that, is there any documentation on the venti protocol apart > > from the source? > > Not really, but here's a summary. > > Venti uses a simple RPC protocol with only a few messages: > ... Many thanks. > size[2] QSync tag[1] > size[2] RSync tag[1] These aren't in the source I have. Presumably they were added relatively recently. I assume they mean save all transactions to disk? John ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [9fans] venti workalike in Python 2003-12-10 2:00 John E. Barham ` (2 preceding siblings ...) 2003-12-11 7:20 ` John E. Barham @ 2003-12-11 9:46 ` John E. Barham 2003-12-11 16:23 ` Russ Cox 3 siblings, 1 reply; 8+ messages in thread From: John E. Barham @ 2003-12-11 9:46 UTC (permalink / raw) To: 9fans > > Having said that, is there any documentation on the venti protocol apart > > from the source? > > Not really, but here's a summary. > > Venti uses a simple RPC protocol with only a few messages: > ... Many thanks. > size[2] QSync tag[1] > size[2] RSync tag[1] These aren't in the source I have. Presumably they were added relatively recently. I assume they mean save all transactions to disk? John ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [9fans] venti workalike in Python 2003-12-11 9:46 ` John E. Barham @ 2003-12-11 16:23 ` Russ Cox 0 siblings, 0 replies; 8+ messages in thread From: Russ Cox @ 2003-12-11 16:23 UTC (permalink / raw) To: 9fans > > > Having said that, is there any documentation on the venti protocol apart > > > from the source? > > > > Not really, but here's a summary. > > > > Venti uses a simple RPC protocol with only a few messages: > > ... > > Many thanks. > > > size[2] QSync tag[1] > > size[2] RSync tag[1] > > These aren't in the source I have. Presumably they were added relatively > recently. I assume they mean save all transactions to disk? I added them about a year ago. Sync means "don't return until everything I've sent up to this point is committed to disk". Russ ^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2003-12-11 16:23 UTC | newest] Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2003-12-10 12:11 [9fans] venti workalike in Python Geoff Collyer -- strict thread matches above, loose matches on Subject: below -- 2003-12-10 2:00 John E. Barham 2003-12-10 2:31 ` Russ Cox 2003-12-10 2:51 ` Geoff Collyer 2003-12-10 22:52 ` William Josephson 2003-12-11 7:20 ` John E. Barham 2003-12-11 9:46 ` John E. Barham 2003-12-11 16:23 ` Russ Cox
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).