From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.io/gmane.emacs.gnus.general/75291 Path: news.gmane.org!not-for-mail From: Steinar Bang Newsgroups: gmane.emacs.gnus.general Subject: Re: gnus-sync ideas Date: Sun, 19 Dec 2010 21:08:21 +0100 Organization: Probably a good idea Message-ID: <87fwttv83u.fsf@dod.no> References: <87zks9ecf7.fsf@dod.no> <87r5djxahm.fsf_-_@lifelogs.com> NNTP-Posting-Host: lo.gmane.org Mime-Version: 1.0 Content-Type: text/plain X-Trace: dough.gmane.org 1292789346 6302 80.91.229.12 (19 Dec 2010 20:09:06 GMT) X-Complaints-To: usenet@dough.gmane.org NNTP-Posting-Date: Sun, 19 Dec 2010 20:09:06 +0000 (UTC) To: ding@gnus.org Original-X-From: ding-owner+M23643@lists.math.uh.edu Sun Dec 19 21:09:02 2010 Return-path: Envelope-to: ding-account@gmane.org Original-Received: from util0.math.uh.edu ([129.7.128.18]) by lo.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1PUPYz-0000IX-Rt for ding-account@gmane.org; Sun, 19 Dec 2010 21:09:02 +0100 Original-Received: from localhost ([127.0.0.1] helo=lists.math.uh.edu) by util0.math.uh.edu with smtp (Exim 4.63) (envelope-from ) id 1PUPYn-000804-QC; Sun, 19 Dec 2010 14:08:49 -0600 Original-Received: from mx2.math.uh.edu ([129.7.128.33]) by util0.math.uh.edu with esmtps (TLSv1:AES256-SHA:256) (Exim 4.63) (envelope-from ) id 1PUPYk-0007zl-Ke for ding@lists.math.uh.edu; Sun, 19 Dec 2010 14:08:46 -0600 Original-Received: from quimby.gnus.org ([80.91.231.51]) by mx2.math.uh.edu with esmtp (Exim 4.72) (envelope-from ) id 1PUPYi-0001tY-GJ for ding@lists.math.uh.edu; Sun, 19 Dec 2010 14:08:46 -0600 Original-Received: from lo.gmane.org ([80.91.229.12]) by quimby.gnus.org with esmtp (Exim 4.72) (envelope-from ) id 1PUPYf-0000jF-CR for ding@gnus.org; Sun, 19 Dec 2010 21:08:41 +0100 Original-Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1PUPYe-00007d-Ld for ding@gnus.org; Sun, 19 Dec 2010 21:08:40 +0100 Original-Received: from cm-84.208.226.192.getinternet.no ([84.208.226.192]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Sun, 19 Dec 2010 21:08:40 +0100 Original-Received: from sb by cm-84.208.226.192.getinternet.no with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Sun, 19 Dec 2010 21:08:40 +0100 X-Injected-Via-Gmane: http://gmane.org/ Mail-Followup-To: ding@gnus.org Original-Lines: 126 Original-X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: cm-84.208.226.192.getinternet.no Mail-Copies-To: never User-Agent: Gnus/5.110011 (No Gnus v0.11) Emacs/22.2 (gnu/linux) Cancel-Lock: sha1:8kPkCWMmclMM0dMxU7GvcLaIAho= X-Spam-Score: -1.9 (-) List-ID: Precedence: bulk Xref: news.gmane.org gmane.emacs.gnus.general:75291 Archived-At: >>>>> Ted Zlatanov : [snip!] > First of all, how should it operate in Gnus? IMO it should be > immediate and not triggered as it is now; a sync should happen every > time you enter or exit a group (and can also be triggered explicitly). Definitely. The way it is today, where I have to remember to stop gnusen every time I leave them, or minimum remember to save, is a nuisance. [snip!] > Second, how should the data be stored? IMO it should be stored in a > free-form key-value format Um... do you mean that the values should be blobs in elisp or somesuch format? (if so, what is the keys? Not obvious to me) Or do you mean a fine-grained "everything as a key/value pair" storage? (hm.. neither the keys, nor the values are obvious to me here either... :-) ) > but the storage backend should be able to understand the rtree format; > the ELisp list, vector, and plist formats; and maybe JSON and merge > such data (with parameters that control how the merge will happen). Hm... why so many formats? We're not going for interoperability with anything else (ie. why use JSON?). And what's the difference between elisp lists and the rtree format? (FWIW I think the format should be s-expressions/elisp lists, because it is simple and low cost on the emacs side) > I think a custom network server would actually be the best storage > backend solution here; imap-hash.el could also work with some love; > the file storage backend is also sensible to duplicate the current > newsrc.eld functionality; and, of course, people will come up with a > Git-based and RDBMS-based storage backend Because They Can. Heh... the server could be ssh-ing to a central computer and starting an emacs remotely to interpret the sync data, and consolidate with the lisp list data stored on the server... "emacs in the cloud" :-) (not serious: relying on ssh is too limiting, because it is blocked by too many corporate firewalls) Actually I think there only needs to be a single protocol http/https with PUT or POST for updates (which could also return updated sync data uploaded by other clients), and GET for initial sync on start, or explicit syncs. Wrt. round trip times: GET can be pipelined. PUT and POST can't. But if you don't care about piggybacking updated sync info on the response (only read upstream sync data with GET) you could just fire them off in the background with the thread thing Lars talked about in a different thread. > Third, what data are we storing? I think marks should only be synced > for news-p backends. But we should generally be able to save, in order > of importance IMHO: > - marks for news-p servers (medium difficulty) That I definitely want. +1 > - splitting rules (trivial but needs a view/edit interface) +0 (I don't use it, but don't let me stop you...:-) ) > - BBDB (hard but very important, and useful beyond Gnus) Currently version controlled. As you've pointed out, merge is not really possible on the file level. +1 > - topic hierarchy (medium) 0 (neutral, for the same reason as the next one) > - subscriptions (but the user can pick subsets as the "work" and "home" > setup for instance) (hard) -1 I like to set up those manually. But I would like a newly subscribed group to get its read state from the sync server, if it is read in a different gnus > - the Gnus registry (medium) 0 (neutral, because I don't use it) - Score rules +1 (because I use it) > So because this is such diverse data I think the storage backend > should just be a key-value store. The rtree/ELisp data merge support > is really the only thing special about it. And maybe it's not a > Gnus-specific thing but could be useful for Emacs in general, in which > case it should be called data-sync.el and the Gnus glue is > gnus-sync.el. I'm sure the org-mode users, for instance, would like > this kind of functionality too, and maybe they already have it and we > can just steal it :) Hm... I still don't have a clear idea how this maps to a key value store. These are nested constructs, aren't they? Will you construct a key by combining identificators at different level? > Fourth, for general accessibility, data-sync.el will have a simple > view/edit interface to load a specific key in a buffer, edit it in > Emacs Lisp, rtree, or JSON mode, and commit it back if it validates > (no merge, just overwrite). This functionality could also connect > with revisions if the storage backend supports them, so you could look > at older versions of your data. Why involve JSON? Why not just lisp list structures? > Fifth, the network server storage backend should be easy to set up and > run. Maybe a HTTP-backed implementation would make the most sense > because those are so easy to set up and are not firewalled. Yep. It's what I think as well. See comments above. > Finally, what other parts of Emacs besides BBDB and org-mode could > benefit from a general data-sync facility? I'm sure others will think of something once it is in place. The diary, perhaps? Though that has other sync possibilities (iCalendar).