From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.io/gmane.emacs.gnus.general/81418 Path: news.gmane.org!not-for-mail From: Ted Zlatanov Newsgroups: gmane.emacs.gnus.general Subject: Re: I can haz cloud idea Date: Thu, 16 Feb 2012 08:07:46 -0500 Organization: =?utf-8?B?0KLQtdC+0LTQvtGAINCX0LvQsNGC0LDQvdC+0LI=?= @ Cienfuegos Message-ID: <8739absz8d.fsf@lifelogs.com> References: <87fwebnzd8.fsf@gnus.org> Reply-To: ding@gnus.org NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain X-Trace: dough.gmane.org 1329397736 3397 80.91.229.3 (16 Feb 2012 13:08:56 GMT) X-Complaints-To: usenet@dough.gmane.org NNTP-Posting-Date: Thu, 16 Feb 2012 13:08:56 +0000 (UTC) To: ding@gnus.org Original-X-From: ding-owner+M29698@lists.math.uh.edu Thu Feb 16 14:08:54 2012 Return-path: Envelope-to: ding-account@gmane.org Original-Received: from util0.math.uh.edu ([129.7.128.18]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1Ry14u-0003NT-61 for ding-account@gmane.org; Thu, 16 Feb 2012 14:08:52 +0100 Original-Received: from localhost ([127.0.0.1] helo=lists.math.uh.edu) by util0.math.uh.edu with smtp (Exim 4.63) (envelope-from ) id 1Ry14H-00085p-Ck; Thu, 16 Feb 2012 07:08:13 -0600 Original-Received: from mx2.math.uh.edu ([129.7.128.33]) by util0.math.uh.edu with esmtps (TLSv1:AES256-SHA:256) (Exim 4.63) (envelope-from ) id 1Ry14G-00085a-2z for ding@lists.math.uh.edu; Thu, 16 Feb 2012 07:08:12 -0600 Original-Received: from quimby.gnus.org ([80.91.231.51]) by mx2.math.uh.edu with esmtps (TLSv1:AES256-SHA:256) (Exim 4.76) (envelope-from ) id 1Ry14B-0005ZG-1V for ding@lists.math.uh.edu; Thu, 16 Feb 2012 07:08:12 -0600 Original-Received: from plane.gmane.org ([80.91.229.3]) by quimby.gnus.org with esmtp (Exim 4.72) (envelope-from ) id 1Ry148-00051N-P3 for ding@gnus.org; Thu, 16 Feb 2012 14:08:04 +0100 Original-Received: from list by plane.gmane.org with local (Exim 4.69) (envelope-from ) id 1Ry145-0002hD-5x for ding@gnus.org; Thu, 16 Feb 2012 14:08:01 +0100 Original-Received: from c-76-28-40-19.hsd1.vt.comcast.net ([76.28.40.19]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 16 Feb 2012 14:08:00 +0100 Original-Received: from tzz by c-76-28-40-19.hsd1.vt.comcast.net with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 16 Feb 2012 14:08:00 +0100 X-Injected-Via-Gmane: http://gmane.org/ Mail-Followup-To: ding@gnus.org Original-Lines: 97 Original-X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: c-76-28-40-19.hsd1.vt.comcast.net User-Agent: Gnus/5.130004 (Ma Gnus v0.4) Emacs/24.0.93 (gnu/linux) X-Face: bd.DQ~'29fIs`T_%O%C\g%6jW)yi[zuz6;d4V0`@y-~$#3P_Ng{@m+e4o<4P'#(_GJQ%TT= D}[Ep*b!\e,fBZ'j_+#"Ps?s2!4H2-Y"sx" Mail-Copies-To: never Cancel-Lock: sha1:EDp370r6Wb55jyfL3rOl4WJr+uU= X-Spam-Score: -1.9 (-) List-ID: Precedence: bulk Xref: news.gmane.org gmane.emacs.gnus.general:81418 Archived-At: On Thu, 16 Feb 2012 06:03:47 +0100 Lars Ingebrigtsen wrote: LI> The main problems with going all cloudy is that you have to have yet LI> another server out there somewhere. Whether you use Dropbox/sshfs/a yet LI> to be determined Gnus-specific server, that's not very sexy. LI> But how about if we just cheat and put all the information on the IMAP LI> server? LI> That is: Gnus would create and maintain a special group on one of your LI> IMAP servers that would contain all the data Gnus needs. LI> You'd set this up by setting, er, something, like: LI> (setq gnus-cloud-server '(nnimap "imap.gmail.com")) Why not add a gnus-sync backend to do this? LI> This would make Gnus save all the data on that server whenever you exit LI> Gnus. (Or on certain intervals.) It could gzip and encrypt the data LI> before uploading, I guess. LI> So when you go to your other machine, you'd say `M-x gnus-cloud', and it LI> would prompt you for the server and password(s), download the data, and LI> then start up more or less as normal. LI> And it would upload the data on exit, as well, so you'd be all LI> cloudy. This is what `gnus-sync-read' and `gnus-sync-save' do already, for some of the newsrc data. There's no problem with storing more, my only issue was to figure out how to synchronize it all cleanly. I also want to use gnus-sync to store the gnus-registry and even BBDB data. So it makes sense to make this a general facility. LI> What would be stored on the server? Well, the .newsrc.eld file, for LI> one. And the SCORE files, I guess. And... well, whatever else you LI> want. It could be a simple archive "file" type, and Gnus could just LI> decrypt, uncompress and unpack the files. Some sanity check about LI> overwriting newer files with older files, I guess. The CouchDB gnus-sync backend has built-in document versioning, which I think is really nice. Also it's portable through JSON and HTTP (what I call the LeSync protocol), so other newsreaders can support it too. The developer of NewsTap, a iOS app, is on board with this and told me it's on top of his feature list. LI> If this is to work smoothly (i.e., at all), then every time you hit `g' LI> in a Gnus, it'll have to know whether something else updated the state LI> group. This also means that every time you exit a group or does LI> something else that changes state, it'll have to be uploaded pretty much LI> immediately. This also means that there would be no explicit "upload" LI> commands. So basically a `gnus-sync-save' on group exit. I would amortize it so it's not done so frequently (currently, gnus-sync saves when the newsrc is saved) because network transactions are expensive. LI> Here's how I see this working: LI> When you first start the thing, it does a total upload. Then, every LI> time you (say) exit a group, it only uploads the delta between the LI> previous total upload and the current state. So after you've read a LI> couple of groups, the delta file will look like LI> ("gwene.com.feedburner.cheezburger" (add (read (56 . 59))) (del (tick 58))) LI> or something. Gnus will continue to upload the entire delta file to the LI> server on each group exit (and delete any previous deltas). Until the LI> delta gets so big that Gnus just does a total upload again, and removes LI> all previous data. LI> It sucks that `g' has to query the state group, though. But it really LI> doesn't. If we maintain a separate cloud connection, then we can just LI> IDLE on the state group. Then every time a different Gnus somewhere LI> does something, then the other Gnus instances can just download the LI> state change, and update itself automatically. Totally transparently LI> and in the background. LI> Unless the IMAP server has dropped the connection, of course. Then `g' LI> will have to reconnect and do the update while you wait. With CouchDB, at least, you can do a differential poll: tell me what has changed since change X (integer numeric sequence). You can also open a persistent connection and get changes live when they are made. So this could be, again, a gnus-sync feature per backend, and the gnus-sync LeSync and IMAP backends would support live updates while the file backend wouldn't. CouchDB's document model is interesting: you have to read the current revision to write a new one back. So you can't accidentally overwrite a document, you have to read it first (which gives you a chance to merge). So at least for CouchDB/LeSync you don't need to do delta files. Ted