Gnus Cloud

[Lars Ingebrigtsen <larsi@gnus.org>]

So, Paso Robles (well, the wines as Justin) inspired me to do some more
thinking about Gnus syncing and stuff.

I'm juggling two laptops, as well as my home account when reading
news/mail now.  And I think these sort of syncing problems are going to
get to be a bigger deal this summer, when the new MeeGo phones are being
released.  They're real Linux machines, so you have X and Emacs running
on your phone.  So you (i.e., I) need to have a way to keep stuff in
sync.  And since the networks are mainly going to be 3G, we need to keep
the data being synced as small as possible.

So here's my current thought:  The Gnus Cloud thing needs to be as
non-general as possible.  We need to sync only the things that need
syncing, and nothing more.

This is how it would work.

You cloud-enable the servers/groups you're interested in having be
device-independent.  For me, this would be Gmane/Gwene groups, as well
as my home IMAP server.

Gnus will then ping the new RESTful cloud.gnus.org server with updates
on new groups and marks.  From the Gnus side, this will look pretty much
like what's Gnus is doing now with IMAP -- we tell the server what we've
read, and what groups we're (un)subscribing.  Signing up for the server
would be a zero-click process.  I.e., you just say what your cloud user
name and password is, and it'll be created if it doesn't exist, and if
it does, Gnus will keep the account updated.

Then if you say `M-x gnus-on-the-go' on your cell phone, and it's a new
installation, it'll just prompt you for user name/password, and download
pretty much the entire .newsrc.eld file (well, the cloud-enabled
portion, at least).  Then subsequent updates will be on the form "send
me all deltas since timestamp YYYYMMDDTHHMMS".  The server will figure
out the diff between that timestamp and the current state, create a nice
data package, gzip it up, and squirt it out.

So the idea is that the cloud server will be a sort of journaled file
system, where it knows the current state, and knows all the actions
taken to get back to an arbitrary point in time, so it can generate
group/marks deltas that Gnus can read to get everything updated.  I
think this can be implemented pretty efficiently, and Gnus can slurp in
the new data fast.

Keeping the server updated would be, as I said, mostly a matter of
hooking into the backend marks updating subsystem.  But a nice wrinkle
here is that these are basically ordered transactions, so you can queue
them up (in a file, for instance), and then squirt them over
asynchronously at Gnus's leisure.  For instance, you might be using Gnus
offline, and Gnus would squirt stuff over when you go online.  And if
cloud.gnus.org is down, Gnus can just queue stuff and send it when it's
back up.  This stuff can be implemented in a way that's fairly robust to
transient failures.

I think the things that should be kept updated are:

* server definitions
* groups
* marks
* topic topology
* score files
* some group parameters

We'd need to be somewhat careful about what we're putting in the cloud.
For instance, some group parameters are eval-ed, and it would not be
nice to provide a way to hack all Gnus users remotely by hacking
cloud.gnus.org.

One sticky issue is whether to sync passwords.  .authinfo stuff is
necessary to contact some servers, but passwords are way more secret
than the rest of the data.  Even if cloud.gnus.org is https only, it
would still be a juicy hacking target if we just stored passwords (in
clear text) there.  Put perhaps we could keep .authinfo.gpg stuff
synced?  I think that area needs more thought, but the rest seems fairly
straightforward to me.

And one nice thing about this setup is that there really would be no
administration or data loss importance (for me).  Since cloud.gnus.org
is basically just a cache of data you have elsewhere, if you lose your
cloud password, you can just create a new account.  If the service loses
your data -- the same thing.

Oops.  I think somebody wants to do something touristy...  gotta run...

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen