Re: gnus sync

[Ted Zlatanov <tzz@lifelogs.com>]

On Fri, 01 Jul 2011 09:22:53 +0200 Steinar Bang <sb@dod.no> wrote: 

>>>>>> Ted Zlatanov <tzz@lifelogs.com>:
>> On Thu, 30 Jun 2011 14:07:35 +0200 Richard Riley <rileyrg@googlemail.com> wrote: 
RR> I'm looking for a working sync solution for gnus amongst multple
RR> machiens : specifically with the local nnml7archive backend. The problem
RR> might be be that .newsrc.eld controls everything pretty much and yet I
RR> would like "local" stuff to remain local. Stuff like imap and nntp are
RR> not an issue - the .newsrc.eld marks are valid across machiens which I
RR> currently sync with rsync. Any comments or pointers appreciated.

>> It's on my TODO list for this release of Gnus.

SB> In the meanwhile your first cut kinda works for me, as long as I accept
SB> its limitations (ie. only read the file on gnus startup, stop each gnus
SB> session when finished with it, and always being online when reading mail
SB> and news (agent doesn't work anymore, so...)).

SB> I've reenabled it and been running with it for some months now.

Yeah, same here.  It's definitely useful to have the functionality.

My plan is the same as before: server-centric, accepts a few formats,
does merge/intersection/reset, and outputs into JSON and Lisp
expressions.  It will be simple enough that other MUAs will be able to
use it, which is where the *real* fun will begin.  For example, NewsTap
(one of my favorite applications on the iPad) could synchronize with
this sync server.  It already lets you synchronize the newsrc file
directly.

I've cc-ed Alexander Clauss, the NewsTap author, to see if he's
interested in developing a standard for synchronization between NewsTap
and Gnus and other newsreaders.  I think the Gnus mailing list is the
right place to discuss this (and hope Alexander doesn't mind the
intrusion), but if there's a better forum I'll go there.  I'd also love
to get other newsreader authors involved.

Alexander, if you need some background on this thread, please let me
know and I'll fill you in.  I'd love to make NewsTap work with Gnus to
synchronize newsgroup subscriptions and article marks.  This is purely
selfish, since I use both, but I think it would benefit many users to
develop an open standard for this that's better than the newsrc format.

Also, if anyone knows of an existing standard for newsreader
synchronization, please let me know.  So far I've only seen solutions
that synchronize the newsrc or use a proprietary format.  My current
idea for the structure is:

subscription-set (just a name, only one per storage instance)
  params-list
  subscription-topic (e.g. "Programming" or "All")
    params-list
    group-name (e.g. "nntp servername.com groupname")
      params-list
      timestamp (floating-point unix epoch)
      "unread" unread-articles
      "read" read-articles
      "ticked" ticked-articles (favorites)
      mark1-name mark1-articles
      mark2-name mark2-articles
      ...

So in Lisp this might look like this:

(("Ted's sync" . 
   (() ;; no parameters
    ("Programming" . 
      (((created . 123.456)) ;; example parameter
       ("nntp gmane.org gmane.programming.articles" .
         (((level . 3)) ;; example parameter
          (timestamp . 123.456) ;; millisecond-precision
          (unread (10 200)) ;; seen but not read
          (read (1 9))
          (ticked (3 4) 5)
          (specialmark1 (8)))))))))

The equivalent JSON and XML representations would be trivial, though I
think just JSON is enough.

Let me know what you think.  I hope to get this done by August,
including the server side, and put up a free server that offers this
service (with the source code open, so others can do the same).

Thanks!
Ted