From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.io/gmane.emacs.gnus.general/76524 Path: news.gmane.org!not-for-mail From: Lars Ingebrigtsen Newsgroups: gmane.emacs.gnus.general Subject: Gnus Cloud Date: Wed, 09 Feb 2011 11:24:37 -0800 Organization: Programmerer Ingebrigtsen Message-ID: <87pqr1huyy.fsf@gnus.org> NNTP-Posting-Host: lo.gmane.org Mime-Version: 1.0 Content-Type: text/plain X-Trace: dough.gmane.org 1297279525 7258 80.91.229.12 (9 Feb 2011 19:25:25 GMT) X-Complaints-To: usenet@dough.gmane.org NNTP-Posting-Date: Wed, 9 Feb 2011 19:25:25 +0000 (UTC) To: ding@gnus.org Original-X-From: ding-owner+M24871@lists.math.uh.edu Wed Feb 09 20:25:20 2011 Return-path: Envelope-to: ding-account@gmane.org Original-Received: from util0.math.uh.edu ([129.7.128.18]) by lo.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1PnFfD-000521-Ja for ding-account@gmane.org; Wed, 09 Feb 2011 20:25:19 +0100 Original-Received: from localhost ([127.0.0.1] helo=lists.math.uh.edu) by util0.math.uh.edu with smtp (Exim 4.63) (envelope-from ) id 1PnFf0-0004gV-Qo; Wed, 09 Feb 2011 13:25:06 -0600 Original-Received: from mx1.math.uh.edu ([129.7.128.32]) by util0.math.uh.edu with esmtps (TLSv1:AES256-SHA:256) (Exim 4.63) (envelope-from ) id 1PnFez-0004gI-D0 for ding@lists.math.uh.edu; Wed, 09 Feb 2011 13:25:05 -0600 Original-Received: from quimby.gnus.org ([80.91.231.51]) by mx1.math.uh.edu with esmtp (Exim 4.72) (envelope-from ) id 1PnFeu-0000mJ-6t for ding@lists.math.uh.edu; Wed, 09 Feb 2011 13:25:05 -0600 Original-Received: from lo.gmane.org ([80.91.229.12]) by quimby.gnus.org with esmtp (Exim 4.72) (envelope-from ) id 1PnFes-0001vD-13 for ding@gnus.org; Wed, 09 Feb 2011 20:24:58 +0100 Original-Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1PnFer-0004mt-QI for ding@gnus.org; Wed, 09 Feb 2011 20:24:57 +0100 Original-Received: from cpe-76-83-194-6.dc.rr.com ([76.83.194.6]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Wed, 09 Feb 2011 20:24:57 +0100 Original-Received: from larsi by cpe-76-83-194-6.dc.rr.com with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Wed, 09 Feb 2011 20:24:57 +0100 X-Injected-Via-Gmane: http://gmane.org/ Mail-Followup-To: ding@gnus.org Original-Lines: 87 Original-X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: cpe-76-83-194-6.dc.rr.com Mail-Copies-To: never User-Agent: Gnus/5.110011 (No Gnus v0.11) Emacs/24.0.50 (gnu/linux) Cancel-Lock: sha1:NDmWVbD6PypQNckootFwVot8wU0= X-Spam-Score: -1.9 (-) List-ID: Precedence: bulk Xref: news.gmane.org gmane.emacs.gnus.general:76524 Archived-At: So, Paso Robles (well, the wines as Justin) inspired me to do some more thinking about Gnus syncing and stuff. I'm juggling two laptops, as well as my home account when reading news/mail now. And I think these sort of syncing problems are going to get to be a bigger deal this summer, when the new MeeGo phones are being released. They're real Linux machines, so you have X and Emacs running on your phone. So you (i.e., I) need to have a way to keep stuff in sync. And since the networks are mainly going to be 3G, we need to keep the data being synced as small as possible. So here's my current thought: The Gnus Cloud thing needs to be as non-general as possible. We need to sync only the things that need syncing, and nothing more. This is how it would work. You cloud-enable the servers/groups you're interested in having be device-independent. For me, this would be Gmane/Gwene groups, as well as my home IMAP server. Gnus will then ping the new RESTful cloud.gnus.org server with updates on new groups and marks. From the Gnus side, this will look pretty much like what's Gnus is doing now with IMAP -- we tell the server what we've read, and what groups we're (un)subscribing. Signing up for the server would be a zero-click process. I.e., you just say what your cloud user name and password is, and it'll be created if it doesn't exist, and if it does, Gnus will keep the account updated. Then if you say `M-x gnus-on-the-go' on your cell phone, and it's a new installation, it'll just prompt you for user name/password, and download pretty much the entire .newsrc.eld file (well, the cloud-enabled portion, at least). Then subsequent updates will be on the form "send me all deltas since timestamp YYYYMMDDTHHMMS". The server will figure out the diff between that timestamp and the current state, create a nice data package, gzip it up, and squirt it out. So the idea is that the cloud server will be a sort of journaled file system, where it knows the current state, and knows all the actions taken to get back to an arbitrary point in time, so it can generate group/marks deltas that Gnus can read to get everything updated. I think this can be implemented pretty efficiently, and Gnus can slurp in the new data fast. Keeping the server updated would be, as I said, mostly a matter of hooking into the backend marks updating subsystem. But a nice wrinkle here is that these are basically ordered transactions, so you can queue them up (in a file, for instance), and then squirt them over asynchronously at Gnus's leisure. For instance, you might be using Gnus offline, and Gnus would squirt stuff over when you go online. And if cloud.gnus.org is down, Gnus can just queue stuff and send it when it's back up. This stuff can be implemented in a way that's fairly robust to transient failures. I think the things that should be kept updated are: * server definitions * groups * marks * topic topology * score files * some group parameters We'd need to be somewhat careful about what we're putting in the cloud. For instance, some group parameters are eval-ed, and it would not be nice to provide a way to hack all Gnus users remotely by hacking cloud.gnus.org. One sticky issue is whether to sync passwords. .authinfo stuff is necessary to contact some servers, but passwords are way more secret than the rest of the data. Even if cloud.gnus.org is https only, it would still be a juicy hacking target if we just stored passwords (in clear text) there. Put perhaps we could keep .authinfo.gpg stuff synced? I think that area needs more thought, but the rest seems fairly straightforward to me. And one nice thing about this setup is that there really would be no administration or data loss importance (for me). Since cloud.gnus.org is basically just a cache of data you have elsewhere, if you lose your cloud password, you can just create a new account. If the service loses your data -- the same thing. Oops. I think somebody wants to do something touristy... gotta run... -- (domestic pets only, the antidote for overdose, milk.) larsi@gnus.org * Lars Magne Ingebrigtsen