From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.io/gmane.emacs.gnus.general/79470 Path: news.gmane.org!not-for-mail From: Steinar Bang Newsgroups: gmane.emacs.gnus.general Subject: Re: gnus sync Date: Thu, 14 Jul 2011 09:39:59 +0200 Organization: Probably a good idea Message-ID: <87bowxgvjk.fsf@dod.no> References: <9uhb77o73s.fsf@news.eternal-september.org> <87wrg3v0yi.fsf@lifelogs.com> <87boxebh2q.fsf@dod.no> <871uyajgxw.fsf@lifelogs.com> <87zkkmkk5i.fsf@dod.no> NNTP-Posting-Host: lo.gmane.org Mime-Version: 1.0 Content-Type: text/plain X-Trace: dough.gmane.org 1310629273 26314 80.91.229.12 (14 Jul 2011 07:41:13 GMT) X-Complaints-To: usenet@dough.gmane.org NNTP-Posting-Date: Thu, 14 Jul 2011 07:41:13 +0000 (UTC) To: ding@gnus.org Original-X-From: ding-owner+M27766@lists.math.uh.edu Thu Jul 14 09:41:07 2011 Return-path: Envelope-to: ding-account@gmane.org Original-Received: from util0.math.uh.edu ([129.7.128.18]) by lo.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1QhGXj-0005Iz-6J for ding-account@gmane.org; Thu, 14 Jul 2011 09:41:07 +0200 Original-Received: from localhost ([127.0.0.1] helo=lists.math.uh.edu) by util0.math.uh.edu with smtp (Exim 4.63) (envelope-from ) id 1QhGXA-0003Z3-4G; Thu, 14 Jul 2011 02:40:32 -0500 Original-Received: from mx1.math.uh.edu ([129.7.128.32]) by util0.math.uh.edu with esmtps (TLSv1:AES256-SHA:256) (Exim 4.63) (envelope-from ) id 1QhGX7-0003Yj-2S for ding@lists.math.uh.edu; Thu, 14 Jul 2011 02:40:29 -0500 Original-Received: from quimby.gnus.org ([80.91.231.51]) by mx1.math.uh.edu with esmtps (TLSv1:AES256-SHA:256) (Exim 4.76) (envelope-from ) id 1QhGWx-0008W5-Tl for ding@lists.math.uh.edu; Thu, 14 Jul 2011 02:40:24 -0500 Original-Received: from lo.gmane.org ([80.91.229.12]) by quimby.gnus.org with esmtp (Exim 4.72) (envelope-from ) id 1QhGWv-0006UK-2C for ding@gnus.org; Thu, 14 Jul 2011 09:40:17 +0200 Original-Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1QhGWu-0004u9-Ss for ding@gnus.org; Thu, 14 Jul 2011 09:40:16 +0200 Original-Received: from 178.16.69.64 ([178.16.69.64]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 14 Jul 2011 09:40:16 +0200 Original-Received: from sb by 178.16.69.64 with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 14 Jul 2011 09:40:16 +0200 X-Injected-Via-Gmane: http://gmane.org/ Mail-Followup-To: ding@gnus.org Original-Lines: 90 Original-X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: 178.16.69.64 Mail-Copies-To: never User-Agent: Gnus/5.110018 (No Gnus v0.18) Emacs/23.1 (gnu/linux) Cancel-Lock: sha1:FDiaZoVNvI5gnMHlg5tTK5JUNf4= X-Spam-Score: -3.7 (---) List-ID: Precedence: bulk Xref: news.gmane.org gmane.emacs.gnus.general:79470 Archived-At: >>>>> Steinar Bang : > I've had some thoughts about the implementation myself. I've been > thinking of a RESTful approach. Some thoughts on gnus behaviour: I think that on startup it would be best to build up a complete ".newsrc.eld" in memory, that is the combination of the ~/.newsrc.eld on disk, and what is received from the server. If the server can't be contacted (ie. if using the agent), then only the ~/.newsrc.eld comes into play. This means that when saving the .newsrc.eld, one is saving the combination of what has been done locally and what has been received from the server. This means that merging of server and client information all the way down to the tick marks, have been done, before starting to read news. There should be a "last synchronized" time stamp in the .newsrc.eld, and when polling the server it should relate everything against that (it might be that there needs to be more fine-grained sync timestamping in the .newsrc.eld, but I suggest just going with the single timestamp initially). When getting the backend list, use conditional GET against the time stamp, and if you get a 304 not modified, then .newsrc.eld is the master in conflicts, ie. differences in the number of backends. Adding backends is easy and conflictless, but DELETE of a backend feels pretty drastic... it would delete all of the backend's servers, and their groups. I should definitely be possible to DELETE a backend, but maybe this should be excluded from the regular sync...? There is a reason rsync has --delete separate from --archive, it's so... destructive and final. The Gnus client should PUT any new backends immediately. When you get the server list for a backend, use conditional GET against the .newsrc "last synchronized" timestamp, and if you get a 304 not modified, then .newsrc.eld is the master in conflicts. I see the same concerns about DELETE of servers, as for delete of backends: ie. it should be possible, but maybe not part of the regular sync. The gnus client should PUT any new servers it has, as compared to the gnus server, immediately. I have no clear thoughts on GET of server parameters, and how they should behave in case on conflicts. When you get the group list for a server, use conditional GET against the .newsrc "last synchronized" timestamp, and if you get a 304 not modified, then .newsrc.eld is the master in conflicts. Here I think it's OK to do a full sync, including DELETE of groups. Worst case is that you lose the tick marks of a group, and that has happened to me many times. The gnus client should PUT and DELETE groups from detected differences immediately. I have no clear thoughts on GET of group parameters, and how they should be merged. When doing a conditional GET of a group's marks, you will get a 304 not modified if there are no marks newer than the timestamp. If there are marks newer than the timestamp, you will get a lisp or JSON list containing their article numbers (possibly with the actual mark values inlined). The gnus client should PUT all marks it has that has been set between the "last synchronized" timestamp, and now, immediately. This means that the .newsrc.eld needs to hold some more timestamping. Having a timestamp on the individual tickmarks feels a bit heavy, but maybe it should be tried...? (I think the server needs to hold individual mark timestamps, though) When actually setting new tick marks, while online, the Gnus client should just fire of PUTs for the tick marks as they are set. When subscribing to new groups when online, the Gnus client should just PUT them on the server. Ditto for backends and servers (though they would be rare). That's all I have thought about, so far.