gnus-sync.el patch and notes

[Matt Ford <matt@dancingfrog.co.uk>]

Hi,

I've been trying to use the gnus-sync.el to, well, keep things in sync.
And after much to and fro I've got it working and I've even made a
partial patch for the bit I'm interested in (the file based sync).

Firstly, there seems to be a bug in the both the couchdb and the file
base gnus-sync-read code.  Both methods of syncing assume that a group
node in the back-end will match the structure of a group node in the
active gnus-newsrc-alist.  Not true!  If you have no marks or ticks in
the active group node then you have a structure like this

("gwene.net.debian.planet" 3 ((1 . 5491)))

and you could easily have a group node looking like this in the sync
backend

("gwene.net.debian.planet" 3
 ((1 . 5491))
 ((seen
   (5287 . 5491))
  (tick 5460)))

The (setf (nth X ...) type merging won't work as the tail element
doesn't exist in the active groups gnus-newsrc-alist entry.  You get
sync failures along the lines of invalid type listp or consp.

In the end, I decided the best bet was to remove the entry if it exists
from the existing gnus-newsrc-alist and push the backend entry into
gnus-newsrc-alist.  i.e., just replace it.  Whilst I was at it I decided
that invalid groups were actually missing groups and I pop those into
the hash also.

> As an aside the merge code for the file based syncing looks wrong to
> me even if the structures are assumed to be the same.  The loop over
> the store needs an integer counter to use as the index for the
> `(nth...)'  function not `(car store)'.

Other things I've learned:

 (setq gnus-sync-backend "/ssh:home:lesync/gnus-sync" 
            gnus-sync-global-vars '(gnus-newsrc-last-checked-date
                              gnus-topic-alist
                              gnus-topic-topology
                              gnus-category-alist
                              )
      gnus-sync-newsrc-groups '("gwene" "gmane"))

- The global variables to sync as listed above are all pretty safe.  It is
  certainly not safe to use (gnus-newsrc-alist) as I think was previously
  suggested in the comment docs (especially if you are using offine imap
  and slave imap servers).  What's more it clashes with the
  gnus-sync-newsrc-groups functionality as both operate on the same data.

- The actual functionality is simple enough the backend file is read in
  as a lisp buffer and is essentially executed as is.  Global-var are
  over-written and the gnus-sync-newsrc-groups are merged and added.

- Let IMAP manage IMAP syncing don't use this stuff for that.  Obvious
  in hindsight.

Hope this helps some people.

Cheers!


--8<---------------cut here---------------start------------->8---
diff --git a/site-elisp/gnus/gnus-sync.el b/site-elisp/gnus/gnus-sync.el
index 7e13b88..eb11676 100644
--- a/site-elisp/gnus/gnus-sync.el
+++ b/site-elisp/gnus/gnus-sync.el
@@ -843,31 +843,33 @@ With a prefix, SUBSCRIBE-ALL is set and unknown groups will be subscribed."
             (load gnus-sync-backend nil t)
           (error
            (error "Error in %s: %s" gnus-sync-backend (cadr var)))))
-      (let ((valid-count 0)
-            invalid-groups)
+      (let ((count 0)
+            missing-groups)
         (dolist (node gnus-sync-newsrc-loader)
           (if (gnus-gethash (car node) gnus-newsrc-hashtb)
               (progn
-                (incf valid-count)
-                (loop for store in (cdr node)
-                      do (setf (nth (car store)
-                                    (assoc (car node) gnus-newsrc-alist))
-                               (cdr store))))
-            (push (car node) invalid-groups)))
+                (incf count)
+                (setq gnus-newsrc-alist
+                      (delq (assoc (car node) gnus-newsrc-alist)
+                            gnus-newsrc-alist))
+                (push node gnus-newsrc-alist))  
+            (progn
+              (push (car node) missing-groups)
+              (push node gnus-newsrc-alist))))
         (gnus-message
          7
-         "gnus-sync-read: loaded %d groups (out of %d) from %s"
-         valid-count (length gnus-sync-newsrc-loader)
+         "gnus-sync-read: loaded %d present groups (out of %d) from %s"
+         count (length gnus-sync-newsrc-loader)
          gnus-sync-backend)
-        (when invalid-groups
+        (when missing-groups
           (gnus-message
            7
-           "gnus-sync-read: skipped %d groups (out of %d) from %s"
-           (length invalid-groups)
+           "gnus-sync-read: added missing %d groups (out of %d) from %s"
+           (length missing-groups)
            (length gnus-sync-newsrc-loader)
            gnus-sync-backend)
-          (gnus-message 9 "gnus-sync-read: skipped groups: %s"
-                        (mapconcat 'identity invalid-groups ", ")))))
+          (gnus-message 9 "gnus-sync-read: missing groups: %s"
+                        (mapconcat 'identity missing-groups ", ")))))
      (nil))
 
     (gnus-message 9 "gnus-sync-read: remaking the newsrc hashtable") 
--8<---------------cut here---------------end--------------->8---

-- 
Matt

p.s., Gnus and it's developers rock - a big thank-you for all your
effort. :-)