From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.io/gmane.emacs.gnus.general/82033 Path: news.gmane.org!not-for-mail From: Matt Ford Newsgroups: gmane.emacs.gnus.general Subject: gnus-sync.el patch and notes Date: Mon, 16 Jul 2012 23:18:51 +0100 Organization: Dancingfrog Message-ID: <87d33vtkms.fsf@dancingfrog.co.uk> NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain X-Trace: dough.gmane.org 1342477225 5118 80.91.229.3 (16 Jul 2012 22:20:25 GMT) X-Complaints-To: usenet@dough.gmane.org NNTP-Posting-Date: Mon, 16 Jul 2012 22:20:25 +0000 (UTC) To: ding@gnus.org Original-X-From: ding-owner+M30303@lists.math.uh.edu Tue Jul 17 00:20:25 2012 Return-path: Envelope-to: ding-account@gmane.org Original-Received: from util0.math.uh.edu ([129.7.128.18]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1SqteQ-0004nb-EP for ding-account@gmane.org; Tue, 17 Jul 2012 00:20:22 +0200 Original-Received: from localhost ([127.0.0.1] helo=lists.math.uh.edu) by util0.math.uh.edu with smtp (Exim 4.63) (envelope-from ) id 1SqtdG-0004Ap-34; Mon, 16 Jul 2012 17:19:10 -0500 Original-Received: from mx1.math.uh.edu ([129.7.128.32]) by util0.math.uh.edu with esmtps (TLSv1:AES256-SHA:256) (Exim 4.63) (envelope-from ) id 1SqtdE-0004Ah-PT for ding@lists.math.uh.edu; Mon, 16 Jul 2012 17:19:08 -0500 Original-Received: from quimby.gnus.org ([80.91.231.51]) by mx1.math.uh.edu with esmtps (TLSv1:AES256-SHA:256) (Exim 4.76) (envelope-from ) id 1SqtdC-0002oQ-MY for ding@lists.math.uh.edu; Mon, 16 Jul 2012 17:19:08 -0500 Original-Received: from plane.gmane.org ([80.91.229.3]) by quimby.gnus.org with esmtp (Exim 4.72) (envelope-from ) id 1SqtdA-0001pR-B0 for ding@gnus.org; Tue, 17 Jul 2012 00:19:04 +0200 Original-Received: from list by plane.gmane.org with local (Exim 4.69) (envelope-from ) id 1Sqtd9-0003Zl-D3 for ding@gnus.org; Tue, 17 Jul 2012 00:19:03 +0200 Original-Received: from rain.gmane.org ([80.91.229.7]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Tue, 17 Jul 2012 00:19:03 +0200 Original-Received: from matt by rain.gmane.org with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Tue, 17 Jul 2012 00:19:03 +0200 X-Injected-Via-Gmane: http://gmane.org/ Original-Lines: 127 Original-X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: rain.gmane.org User-Agent: Gnus/5.130006 (Ma Gnus v0.6) Emacs/24.1 (gnu/linux) Cancel-Lock: sha1:Bwflp451nUfWOlAQ9s+HKka4gHs= X-Spam-Score: -1.9 (-) List-ID: Precedence: bulk Xref: news.gmane.org gmane.emacs.gnus.general:82033 Archived-At: Hi, I've been trying to use the gnus-sync.el to, well, keep things in sync. And after much to and fro I've got it working and I've even made a partial patch for the bit I'm interested in (the file based sync). Firstly, there seems to be a bug in the both the couchdb and the file base gnus-sync-read code. Both methods of syncing assume that a group node in the back-end will match the structure of a group node in the active gnus-newsrc-alist. Not true! If you have no marks or ticks in the active group node then you have a structure like this ("gwene.net.debian.planet" 3 ((1 . 5491))) and you could easily have a group node looking like this in the sync backend ("gwene.net.debian.planet" 3 ((1 . 5491)) ((seen (5287 . 5491)) (tick 5460))) The (setf (nth X ...) type merging won't work as the tail element doesn't exist in the active groups gnus-newsrc-alist entry. You get sync failures along the lines of invalid type listp or consp. In the end, I decided the best bet was to remove the entry if it exists from the existing gnus-newsrc-alist and push the backend entry into gnus-newsrc-alist. i.e., just replace it. Whilst I was at it I decided that invalid groups were actually missing groups and I pop those into the hash also. > As an aside the merge code for the file based syncing looks wrong to > me even if the structures are assumed to be the same. The loop over > the store needs an integer counter to use as the index for the > `(nth...)' function not `(car store)'. Other things I've learned: (setq gnus-sync-backend "/ssh:home:lesync/gnus-sync" gnus-sync-global-vars '(gnus-newsrc-last-checked-date gnus-topic-alist gnus-topic-topology gnus-category-alist ) gnus-sync-newsrc-groups '("gwene" "gmane")) - The global variables to sync as listed above are all pretty safe. It is certainly not safe to use (gnus-newsrc-alist) as I think was previously suggested in the comment docs (especially if you are using offine imap and slave imap servers). What's more it clashes with the gnus-sync-newsrc-groups functionality as both operate on the same data. - The actual functionality is simple enough the backend file is read in as a lisp buffer and is essentially executed as is. Global-var are over-written and the gnus-sync-newsrc-groups are merged and added. - Let IMAP manage IMAP syncing don't use this stuff for that. Obvious in hindsight. Hope this helps some people. Cheers! --8<---------------cut here---------------start------------->8--- diff --git a/site-elisp/gnus/gnus-sync.el b/site-elisp/gnus/gnus-sync.el index 7e13b88..eb11676 100644 --- a/site-elisp/gnus/gnus-sync.el +++ b/site-elisp/gnus/gnus-sync.el @@ -843,31 +843,33 @@ With a prefix, SUBSCRIBE-ALL is set and unknown groups will be subscribed." (load gnus-sync-backend nil t) (error (error "Error in %s: %s" gnus-sync-backend (cadr var))))) - (let ((valid-count 0) - invalid-groups) + (let ((count 0) + missing-groups) (dolist (node gnus-sync-newsrc-loader) (if (gnus-gethash (car node) gnus-newsrc-hashtb) (progn - (incf valid-count) - (loop for store in (cdr node) - do (setf (nth (car store) - (assoc (car node) gnus-newsrc-alist)) - (cdr store)))) - (push (car node) invalid-groups))) + (incf count) + (setq gnus-newsrc-alist + (delq (assoc (car node) gnus-newsrc-alist) + gnus-newsrc-alist)) + (push node gnus-newsrc-alist)) + (progn + (push (car node) missing-groups) + (push node gnus-newsrc-alist)))) (gnus-message 7 - "gnus-sync-read: loaded %d groups (out of %d) from %s" - valid-count (length gnus-sync-newsrc-loader) + "gnus-sync-read: loaded %d present groups (out of %d) from %s" + count (length gnus-sync-newsrc-loader) gnus-sync-backend) - (when invalid-groups + (when missing-groups (gnus-message 7 - "gnus-sync-read: skipped %d groups (out of %d) from %s" - (length invalid-groups) + "gnus-sync-read: added missing %d groups (out of %d) from %s" + (length missing-groups) (length gnus-sync-newsrc-loader) gnus-sync-backend) - (gnus-message 9 "gnus-sync-read: skipped groups: %s" - (mapconcat 'identity invalid-groups ", "))))) + (gnus-message 9 "gnus-sync-read: missing groups: %s" + (mapconcat 'identity missing-groups ", "))))) (nil)) (gnus-message 9 "gnus-sync-read: remaking the newsrc hashtable") --8<---------------cut here---------------end--------------->8--- -- Matt p.s., Gnus and it's developers rock - a big thank-you for all your effort. :-)