From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.io/gmane.emacs.gnus.general/81468 Path: news.gmane.org!not-for-mail From: Ted Zlatanov Newsgroups: gmane.emacs.gnus.general Subject: Re: I can haz cloud idea Date: Tue, 21 Feb 2012 16:23:29 -0500 Organization: =?utf-8?B?0KLQtdC+0LTQvtGAINCX0LvQsNGC0LDQvdC+0LI=?= @ Cienfuegos Message-ID: <87aa4banji.fsf@lifelogs.com> References: <87fwebnzd8.fsf@gnus.org> <1swr7nl0tf.fsf@voll.uninett.no> <87mx8elugg.fsf@gnus.org> Reply-To: ding@gnus.org NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain X-Trace: dough.gmane.org 1329859455 1790 80.91.229.3 (21 Feb 2012 21:24:15 GMT) X-Complaints-To: usenet@dough.gmane.org NNTP-Posting-Date: Tue, 21 Feb 2012 21:24:15 +0000 (UTC) To: ding@gnus.org Original-X-From: ding-owner+M29748@lists.math.uh.edu Tue Feb 21 22:24:11 2012 Return-path: Envelope-to: ding-account@gmane.org Original-Received: from util0.math.uh.edu ([129.7.128.18]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1RzxBy-00089P-Fn for ding-account@gmane.org; Tue, 21 Feb 2012 22:24:10 +0100 Original-Received: from localhost ([127.0.0.1] helo=lists.math.uh.edu) by util0.math.uh.edu with smtp (Exim 4.63) (envelope-from ) id 1RzxBj-0000y2-7G; Tue, 21 Feb 2012 15:23:55 -0600 Original-Received: from mx1.math.uh.edu ([129.7.128.32]) by util0.math.uh.edu with esmtps (TLSv1:AES256-SHA:256) (Exim 4.63) (envelope-from ) id 1RzxBh-0000xt-VZ for ding@lists.math.uh.edu; Tue, 21 Feb 2012 15:23:53 -0600 Original-Received: from quimby.gnus.org ([80.91.231.51]) by mx1.math.uh.edu with esmtps (TLSv1:AES256-SHA:256) (Exim 4.76) (envelope-from ) id 1RzxBc-0002co-8v for ding@lists.math.uh.edu; Tue, 21 Feb 2012 15:23:53 -0600 Original-Received: from plane.gmane.org ([80.91.229.3]) by quimby.gnus.org with esmtp (Exim 4.72) (envelope-from ) id 1RzxBa-0002yP-GQ for ding@gnus.org; Tue, 21 Feb 2012 22:23:46 +0100 Original-Received: from list by plane.gmane.org with local (Exim 4.69) (envelope-from ) id 1RzxBX-0007wU-GR for ding@gnus.org; Tue, 21 Feb 2012 22:23:43 +0100 Original-Received: from c-76-28-40-19.hsd1.vt.comcast.net ([76.28.40.19]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Tue, 21 Feb 2012 22:23:43 +0100 Original-Received: from tzz by c-76-28-40-19.hsd1.vt.comcast.net with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Tue, 21 Feb 2012 22:23:43 +0100 X-Injected-Via-Gmane: http://gmane.org/ Mail-Followup-To: ding@gnus.org Original-Lines: 91 Original-X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: c-76-28-40-19.hsd1.vt.comcast.net X-Face: bd.DQ~'29fIs`T_%O%C\g%6jW)yi[zuz6;d4V0`@y-~$#3P_Ng{@m+e4o<4P'#(_GJQ%TT= D}[Ep*b!\e,fBZ'j_+#"Ps?s2!4H2-Y"sx" Mail-Copies-To: never User-Agent: Gnus/5.130004 (Ma Gnus v0.4) Emacs/24.0.93 (gnu/linux) Cancel-Lock: sha1:U+uDHtDzoJdbOlodxNF+VXEvhA8= X-Spam-Score: -1.9 (-) List-ID: Precedence: bulk Xref: news.gmane.org gmane.emacs.gnus.general:81468 Archived-At: On Sun, 19 Feb 2012 16:21:51 +0100 Lars Ingebrigtsen wrote: LI> So the original idea would be to have a bundle of files distributed via LI> IMAP, plus diffs to avoid down/uploading so much stuff. When stuff LI> changes in the bundle, Gnus updates the appropriate files. The actual amount of data for a group, as a JSON object in LeSync, is tiny. I would avoid deltas if possible. They are fragile, hard to debug, and unshareable with other MUAs. Put all the data a group needs in a single place (a message in the mailbox, in this case, or a CouchDB document in LeSync) and rewrite the whole document atomically when it changes. Yes, you have a versioning problem, if someone else writes before you. So use a revision number. Please look at how CouchDB does it: only save the document if the latest document's version matches the latest version you know. Otherwise, reconcile with the latest saved. LI> Why rely on other files for the significant stuff? We control the LI> vertical and the horizontal, so why not just rethink this? LI> The score files, for instance, would more naturally be a structure LI> stored in .newsrc.eld. Finding out what score files that match what LI> group is pretty gross code-wise. Why not just make that into a nice LI> alist and put it in the .newsrc.eld file? LI> The same goes for other obvious stuff like ~/.signature. LI> Just put it all in .newsrc.eld, and then distribute that via IMAP. I would have the gnus-sync backend store user profiles, like the posting styles in Gnus. The profile would point to or inline the signature file, score files, SMTP server, and everything else the user needs, as customizations. Then you pick a profile on each machine you use. I would make each profile, with all the customizations, a single document that users can edit and understand. Don't spread things out just because you can. LI> Ted Zlatanov writes: >> Why not add a gnus-sync backend to do this? LI> Because I want it all to be asynchronous, opportunistic and streaming. That's fine. The gnus-sync backend should support that when you call `gnus-sync-save'. You're taking the mechanics, the way code will operate, and using that as an argument against the architecture, the way data and user-customizable variables are planned. With CouchDB you can stream by opening a persistent connection to the server. url.el can be asynchronous, and so can you against CouchDB. I'm not sure what "opportunistic" means exactly, but if you mean "save when possible" then I think that would work. Just realize that by setting the gnus-sync backend to files, you can be backwards compatible with the current Gnus. That way you don't have to have a special "cloud mode" but simply "sync to local filesystem." Your way is a special case all the way from data layout to mechanics. LI> When exiting a group, we'll be squirting off an APPEND command to the LI> IMAP server (with the current delta), but we don't wait for it to LI> return finish. We queue them up, and if we get three group exits before LI> the first APPEND has finished, we just skip the second one. LI> Conversely, using IDLE on the mail box will keep the other instances LI> updated asynchronously without anybody having to issue any commands. LI> It'll be like using the IMAP synchronisation methodology, only for all LI> backends and stuff. OK. But that's nearly impossible to share with other MUAs, unlike my LeSync proposal, and I hope you don't use deltas as I said above. >> The CouchDB gnus-sync backend has built-in document versioning, which I >> think is really nice. LI> I don't think having CouchDB as a prerequisite is totally reasonable. LI> It should just work without having to install anything. You're basing your side on a working, always-reachable IMAP server, which is less reasonable in the context of the Internet and requires significant infrastructure. Port 80 is much more likely to be open than other ports, especially in a corporate environment. I didn't decide on CouchDB because it's trendy (it's SO 2009!), but because it's a schemaless database+web server hybrid with custom views (e.g. I can put together a XML view of the marks that NewsTap can import directly). You can get a free instance at http://iriscouch.com or https://cloudant.com and get running in minutes, or set your own up even quicker, or I can set one up for you at WHATEVER.lesync.info. Ted