Re: I can haz cloud idea

[Dan Christensen <jdc@uwo.ca>]

Andy Moreton <andrewjmoreton@gmail.com> writes:

> Please, please do *not* add more stuff to .newsrc.eld. 

I agree.

> That file is already over-complex, and trying to do too many
> things. Keep things simple and unixy - a single responsibility for
> each file.

Moreover, as soon as you have multiple clients sharing data, you are
bound to get into a situation where some merging is required.  (E.g.
I go to work and learn that my home machine hasn't synced;  I've got
to get work done, so I've got to make changes at work as well.)
Merging is generally easier when things are stored in logically
separate containers.

In addition, updating single "records" should be fast, and if everything
is stored in the .newsrc.eld on an abstract filesystem, that's hard, and
you have to resort to storing diffs, etc, to be efficient.

Even more, it's doubtful that I'm going to want to sync the full 
.newsrc.eld, since on some machines I'll access different servers.
Sync/cloud should be configured separately for each backend, if
only for efficiency.

I'm also not sure that a DCVS is a magic bullet here, since we're
going to need problem specific merging and updating.  For example,
Gnus should remember the last state it saw on the server, and 
propagate changes back and forth, rather than copying over the
new data (in case it has also been changed by another client).

I haven't looked into it closely, but doesn't Ted's gnus-sync framework
seem like a good basis for continued development?  This really seems to
me to fit into the database point of view.

I also have to say that I always liked the idea of having the backends
store marks.  It allows naive methods of syncing and backup to just
work, and it makes the backends behave more like nnimap which is now
becoming one of the most used backends.

I feels to me like the syncing gnus already has to do between it's
knowledge of the marks in an nnimap group and the remote server's
knowledge is really similar to what needs to happen for other backends
in a cloud set-up.  This is also closely related to the
plugged/unplugged/agent feature.  Can all of this be merged into one
concept?

0) Gnus doesn't store any marks in the .newsrc.eld.
1) On startup, Gnus asks the backend for the marks, and displays the group.
2) As the user changes things, Gnus keeps track of the changes made
   to marks.
3) As various intervals, Gnus syncs the changes to the backend.
   Being "unplugged" which just mean deferring any syncs.

The user could configure the backend to be nnimap, which already
handles storing the marks on a server.

Or the user could configure the backend to be nnml, with marks
just stored locally.

Or the user could configure the backend to be nnml, with marks
stored on a server (e.g. Ted's LeSync idea, or some other remote
storage).

It really seems like getting all the marks out of .newsrc.eld
and putting them in the backends would unify many ideas in Gnus.
(And by default, servers could just store marks in a file
like .newsrc.server-name.eld, using nnoo common code, so there
wouldn't need to be code duplication in the backends.  Even
the default syncing code could be in common, but overridden
by servers like nnimap that have a custom way to store marks.)

Dan