Gnus and rsync

Gnus and rsync

[Nuno J. Silva]

So I've now hit the issue where I have two computers on which I want to
keep gnus stuff on-sync.

I've tried the simple solution: rsyncing the whole bunch of files (news,
mail, .gnus.el, and some other gnus files.

Unfortunately, this doesn't work as expected: in some mail folders the
unread messages are months old, perhaps from the time the mail files
were on the computer I want to sync to (or even before that - in a
nutshell: my Gnus files have flied through several machines several
times (but I had no trouble with that)). 

I plan to look at this deeply (like, seeing if the messages are actually
there, if something differs from a plain tar+scp+untar, and some other
forensics), but I'd like to know if anyone has already experienced
anything like this.

The backend possibly matters: it's nnmaildir.

-- 
Nuno J. Silva
gopher://sdf-eu.org/1/users/njsg

Re: Gnus and rsync

[Richard Riley]

nunojsilva@invalid.invalid (Nuno J. Silva) writes:

> Eric S Fraga <ucecesf@ucl.ac.uk> writes:
>
>> nunojsilva@invalid.invalid (Nuno J. Silva) writes:
>>
>>> So I've now hit the issue where I have two computers on which I want to
>>> keep gnus stuff on-sync.
>>>
>>> I've tried the simple solution: rsyncing the whole bunch of files (news,
>>> mail, .gnus.el, and some other gnus files.
>>
>> maybe have a look at unison?  works very well for keeping systems in
>> sync.
>
> I will, but AFAICT, from the debugging I made today, the issue is not
> rsync, so I'll delay looking unison for a while.
>
>>> Unfortunately, this doesn't work as expected: in some mail folders the
>>> unread messages are months old,[...] 
>
> As promised, I tested diffing the rsync files with a plain tarball sync,
> and it's the same.
>
> What happens is that, after starting gnus, a lot of NOV files get
> changed.
>
> Removing every nov folder (*/.nnmaildir/nov, relative to nnmaildir root)
> before starting gnus forces gnus to regenerate these, what causes a
> slightly long delay but works (that is, now the unread messages are the
> same as in the original gnus setup).
>
> Next step seems to be understanding which changes are made to NOV files
> so I can know why is this happening...
>
> (Same Gnus and Emacs version on both computers.)

I tried a plain old sync for a while with gnus and  more often that not
things went wrong.

The best way I found is to use offlineimap to sync a local dovecot
server. Since gnus uses the server flags all is dandy.

Re: Gnus and rsync

[Nuno J. Silva]

Richard Riley <rileyrg@googlemail.com> writes:

> nunojsilva@invalid.invalid (Nuno J. Silva) writes:
>
>> Eric S Fraga <ucecesf@ucl.ac.uk> writes:
>>
>>> nunojsilva@invalid.invalid (Nuno J. Silva) writes:
>>>
>>>> So I've now hit the issue where I have two computers on which I want to
>>>> keep gnus stuff on-sync.
>>>>
>>>> I've tried the simple solution: rsyncing the whole bunch of files (news,
>>>> mail, .gnus.el, and some other gnus files.
[...]
>>>> Unfortunately, this doesn't work as expected: in some mail folders the
>>>> unread messages are months old,[...] 
>>
>> What happens is that, after starting gnus, a lot of NOV files get
>> changed.
>>
>> Removing every nov folder (*/.nnmaildir/nov, relative to nnmaildir root)
>> before starting gnus forces gnus to regenerate these, what causes a
>> slightly long delay but works (that is, now the unread messages are the
>> same as in the original gnus setup).
[...]
>
> I tried a plain old sync for a while with gnus and  more often that not
> things went wrong.

It's sad, I'd really like if it were as simple as a naïve rsync. But at
least now I know it's not something weird in my config.

> The best way I found is to use offlineimap to sync a local dovecot
> server. Since gnus uses the server flags all is dandy.

Can you give me some more hints of what would that setup look like? 

I suppose it requires running an IMAP server (dovecot) on each machine I
want to use, and running offlineimap to sync both IMAP servers
(equivalent to rsync in the "plain sync" idea).

I'm currently using Gnus to fetch e-mails. As offlineimap was designed
exactly to sync newly created files, I suppose it will work - I just
need to change the mail folders from nnmaildir to dovecot and save the
new messages there.

What about performance? I suppose it's pretty okay, as the IMAP server
is running locally; and I expect offlineimap to take some more time.


Many thanks for your reply - it is pointing me towards a *working*
solution, and that's what I was looking for :-)

-- 
Nuno J. Silva
gopher://sdf-eu.org/1/users/njsg

Re: Gnus and rsync

[Eric S Fraga]

nunojsilva@invalid.invalid (Nuno J. Silva) writes:

> Eric S Fraga <ucecesf@ucl.ac.uk> writes:
>
>> nunojsilva@invalid.invalid (Nuno J. Silva) writes:
>>
>>> So I've now hit the issue where I have two computers on which I want to
>>> keep gnus stuff on-sync.
>>>
>>> I've tried the simple solution: rsyncing the whole bunch of files (news,
>>> mail, .gnus.el, and some other gnus files.
>>
>> maybe have a look at unison?  works very well for keeping systems in
>> sync.
>
> I will, but AFAICT, from the debugging I made today, the issue is not
> rsync, so I'll delay looking unison for a while.
>
> What happens is that, after starting gnus, a lot of NOV files get
> changed.

I found that I had to include .newsrc.eld in the syncing operation
(whether via rsync or unison) but that, for some reason, I also find
that all the nov files get updated and need re-syncing every time.  Very
annoying but so far no problems (other than slowness in syncing).

> Next step seems to be understanding which changes are made to NOV files
> so I can know why is this happening...

Please do let us know.  I would like to avoid this overhead...

-- 
: Eric S Fraga (GnuPG: 0xC89193D8FFFCF67D) in Emacs 23.2.1

Re: Gnus and rsync

[Nuno J. Silva]

Eric S Fraga <ucecesf@ucl.ac.uk> writes:

> nunojsilva@invalid.invalid (Nuno J. Silva) writes:
>
>>> nunojsilva@invalid.invalid (Nuno J. Silva) writes:
>>>
>>>> So I've now hit the issue where I have two computers on which I want to
>>>> keep gnus stuff on-sync.
>>>>
>>>> I've tried the simple solution: rsyncing the whole bunch of files (news,
>>>> mail, .gnus.el, and some other gnus files.
[...]
>> What happens is that, after starting gnus, a lot of NOV files get
>> changed.
[...]
>> Next step seems to be understanding which changes are made to NOV files
>> so I can know why is this happening...
>
> Please do let us know.  I would like to avoid this overhead...

I'm now using the IMAP workaround, so that I can have a working syncable
mail configuration even while this remains unsolved.

But I'm adding this to the TO-DOS file -- if I ever fix this I'll let
you know.

-- 
Nuno J. Silva
gopher://sdf-eu.org/1/users/njsg