Announcements and discussions for Gnus, the GNU Emacs Usenet newsreader
 help / color / mirror / Atom feed
* synchronizing gnus sessions on multiple computers
@ 2021-09-09 23:21 Roland Winkler
  2021-09-10  4:46 ` Pankaj Jangid
  2021-09-12  4:47 ` James Thomas
  0 siblings, 2 replies; 15+ messages in thread
From: Roland Winkler @ 2021-09-09 23:21 UTC (permalink / raw)
  To: info-gnus-english

In my work, I am alternating between two computers where the user
directories are complete mirror images (updated via rsync whenever I
need to switch from one to the other computer).  Somehow I am having
difficulties to get this to work with gnus:

- How can I tell gnus (in a running emacs session) that I've rsync'ed
  all local user directories and gnus should update itself accordingly
  when, for example, articles are marked as read or expired.

- Where does gnus store the information that I have read or marked for
  expiration some news or mail articles in a group?

  (Somehow, right now when switching to the other computer, gnus appears
  to be becoming aware of new articles that arrived on the other
  computer and were subsequently rsync'ed to the local machine; only the
  marks regarding the status of these articles are lost when switching
  to the other computer, as if these marks were stored in a file that is
  ignored by rsync.)

I hope that such a work flow is not running into fundamental
difficulties.

Roland



^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: synchronizing gnus sessions on multiple computers
  2021-09-09 23:21 synchronizing gnus sessions on multiple computers Roland Winkler
@ 2021-09-10  4:46 ` Pankaj Jangid
  2021-09-10 11:06   ` Leo Butler
  2021-09-12  4:47 ` James Thomas
  1 sibling, 1 reply; 15+ messages in thread
From: Pankaj Jangid @ 2021-09-10  4:46 UTC (permalink / raw)
  To: info-gnus-english

"Roland Winkler" <winkler@gnu.org> writes:

> - How can I tell gnus (in a running emacs session) that I've rsync'ed
>   all local user directories and gnus should update itself accordingly
>   when, for example, articles are marked as read or expired.
>
> - Where does gnus store the information that I have read or marked for
>   expiration some news or mail articles in a group?

When you are rsync’ing between two machines, you need to do it for the
following files/directories:

~/News/
~/Mail/
~/.gnus.el
~/emacs.d/ ; (this is required for syncing other Emacs settings)
~/.newsrc  ; (there are multiple formats newsrc.eld, newsrc.el
           ; or just .newsrc, This is what you are looking for.)
~/.authinfo.gpg ; (.newsrc, .authinfo etc.)

Others, please add if I am missing some files/dirs.



^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: synchronizing gnus sessions on multiple computers
  2021-09-10  4:46 ` Pankaj Jangid
@ 2021-09-10 11:06   ` Leo Butler
  2021-09-10 19:45     ` Bob Newell
  0 siblings, 1 reply; 15+ messages in thread
From: Leo Butler @ 2021-09-10 11:06 UTC (permalink / raw)
  To: Pankaj Jangid; +Cc: info-gnus-english

Pankaj Jangid <pankaj@codeisgreat.org> writes:

> "Roland Winkler" <winkler@gnu.org> writes:
>
>> - How can I tell gnus (in a running emacs session) that I've rsync'ed
>>   all local user directories and gnus should update itself accordingly
>>   when, for example, articles are marked as read or expired.
>>
>> - Where does gnus store the information that I have read or marked for
>>   expiration some news or mail articles in a group?
>
> When you are rsync’ing between two machines, you need to do it for the
> following files/directories:
>
> ~/News/
> ~/Mail/
> ~/.gnus.el
> ~/emacs.d/ ; (this is required for syncing other Emacs settings)
> ~/.newsrc  ; (there are multiple formats newsrc.eld, newsrc.el
>            ; or just .newsrc, This is what you are looking for.)
> ~/.authinfo.gpg ; (.newsrc, .authinfo etc.)
>
> Others, please add if I am missing some files/dirs.


I think one needs to save the dribble file, and copy it, too:

~/.newsrc-dribble

Leo


^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: synchronizing gnus sessions on multiple computers
  2021-09-10 11:06   ` Leo Butler
@ 2021-09-10 19:45     ` Bob Newell
  2021-09-10 21:15       ` Roland Winkler
  0 siblings, 1 reply; 15+ messages in thread
From: Bob Newell @ 2021-09-10 19:45 UTC (permalink / raw)
  To: Leo Butler; +Cc: info-gnus-english, Pankaj Jangid

I have rsync'd gnus for years with success. In addition to the files
already listed I sync .newsrc.eld, but that is updated infrequently.
Not strictly gnus entries, but related closely enough, I also sync
.authinfo and .msmtprc (as I use msmtp) and also .bbdb.

I also have a humongously long startup-gnus.el which is called from
.emacs and with which I tinker more often than I should.

Bob Newell
Honolulu, Hawai`i

On Fri, Sep 10, 2021 at 1:12 AM Leo Butler <leo.butler@umanitoba.ca> wrote:
>
> Pankaj Jangid <pankaj@codeisgreat.org> writes:
>
> > "Roland Winkler" <winkler@gnu.org> writes:
> >
> >> - How can I tell gnus (in a running emacs session) that I've rsync'ed
> >>   all local user directories and gnus should update itself accordingly
> >>   when, for example, articles are marked as read or expired.
> >>
> >> - Where does gnus store the information that I have read or marked for
> >>   expiration some news or mail articles in a group?
> >
> > When you are rsync’ing between two machines, you need to do it for the
> > following files/directories:
> >
> > ~/News/
> > ~/Mail/
> > ~/.gnus.el
> > ~/emacs.d/ ; (this is required for syncing other Emacs settings)
> > ~/.newsrc  ; (there are multiple formats newsrc.eld, newsrc.el
> >            ; or just .newsrc, This is what you are looking for.)
> > ~/.authinfo.gpg ; (.newsrc, .authinfo etc.)
> >
> > Others, please add if I am missing some files/dirs.
>
>
> I think one needs to save the dribble file, and copy it, too:
>
> ~/.newsrc-dribble
>
> Leo
>


-- 
Bob Newell
Honolulu, Hawai`i

Via Linux/Emacs/Gnus/BBDB.


^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: synchronizing gnus sessions on multiple computers
  2021-09-10 19:45     ` Bob Newell
@ 2021-09-10 21:15       ` Roland Winkler
  2021-09-10 21:50         ` Eric Abrahamsen
  2021-09-11 16:07         ` Greg Farough
  0 siblings, 2 replies; 15+ messages in thread
From: Roland Winkler @ 2021-09-10 21:15 UTC (permalink / raw)
  To: info-gnus-english

On Fri, Sep 10 2021, Bob Newell wrote:
> I have rsync'd gnus for years with success. In addition to the files
> already listed I sync .newsrc.eld, but that is updated infrequently.
> Not strictly gnus entries, but related closely enough, I also sync
> .authinfo and .msmtprc (as I use msmtp) and also .bbdb.

Thanks to everyone who replied to this thread.

When you are rsyncing gnus, are you doing this while emacs is running on
the different machines?  Or are you starting a fresh emacs session after
all the files got updated on a machine?

Digging through things I realized that .newsrc.eld seems to store
article marks.  For example it sets gnus-newsrc-alist that holds an
assoc list of read articles.  However, in a running emacs session the
same information is stored also in gnus-newsrc-hashtb.  So the question
becomes: What steps are needed to fully restart gnus while emacs is
running so that internal variables such as gnus-newsrc-hashtb get
re-initialized, too.

-- I realized that testing all this does not require two different
machines.  But two instances of emacs running on one computer
effectively give the same situation to play with.  It seems that a
running gnus session does not warn the user that another emacs instance
has fiddled with files such as .newsrc.eld.  Is this true?



^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: synchronizing gnus sessions on multiple computers
  2021-09-10 21:15       ` Roland Winkler
@ 2021-09-10 21:50         ` Eric Abrahamsen
  2021-09-10 22:39           ` Bob Newell
                             ` (2 more replies)
  2021-09-11 16:07         ` Greg Farough
  1 sibling, 3 replies; 15+ messages in thread
From: Eric Abrahamsen @ 2021-09-10 21:50 UTC (permalink / raw)
  To: Roland Winkler; +Cc: info-gnus-english

"Roland Winkler" <winkler@gnu.org> writes:

> On Fri, Sep 10 2021, Bob Newell wrote:
>> I have rsync'd gnus for years with success. In addition to the files
>> already listed I sync .newsrc.eld, but that is updated infrequently.
>> Not strictly gnus entries, but related closely enough, I also sync
>> .authinfo and .msmtprc (as I use msmtp) and also .bbdb.
>
> Thanks to everyone who replied to this thread.
>
> When you are rsyncing gnus, are you doing this while emacs is running on
> the different machines?  Or are you starting a fresh emacs session after
> all the files got updated on a machine?
>
> Digging through things I realized that .newsrc.eld seems to store
> article marks.  For example it sets gnus-newsrc-alist that holds an
> assoc list of read articles.  However, in a running emacs session the
> same information is stored also in gnus-newsrc-hashtb.  So the question
> becomes: What steps are needed to fully restart gnus while emacs is
> running so that internal variables such as gnus-newsrc-hashtb get
> re-initialized, too.
>
> -- I realized that testing all this does not require two different
> machines.  But two instances of emacs running on one computer
> effectively give the same situation to play with.  It seems that a
> running gnus session does not warn the user that another emacs instance
> has fiddled with files such as .newsrc.eld.  Is this true?

Yes, the .newsrc.eld file only gets written when you save Gnus ("s" in
the *Group* buffer), and I would absolutely not try to update files for
a Gnus that's still running, it will not handle that gracefully. It
should, however, be enough to shut Gnus down first (you can see the
`gnus-clear-system' function to see what happens then), rsync, then
restart Gnus.


^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: synchronizing gnus sessions on multiple computers
  2021-09-10 21:50         ` Eric Abrahamsen
@ 2021-09-10 22:39           ` Bob Newell
  2021-09-11  4:04           ` Roland Winkler
  2021-10-06  4:28           ` Roland Winkler
  2 siblings, 0 replies; 15+ messages in thread
From: Bob Newell @ 2021-09-10 22:39 UTC (permalink / raw)
  To: info-gnus-english

>> On Fri, Sep 10 2021, Bob Newell wrote:
>>> I have rsync'd gnus for years with success. In addition to the files
>>> already listed I sync .newsrc.eld, but that is updated infrequently.
>>> Not strictly gnus entries, but related closely enough, I also sync
>>> .authinfo and .msmtprc (as I use msmtp) and also .bbdb.

>> When you are rsyncing gnus, are you doing this while emacs is running on
>> the different machines?  Or are you starting a fresh emacs session after
>> all the files got updated on a machine?

I should have been clearer about how I myself do the
rsyncing. I avoid concurrency problems (among what has grown
to be nine devices) by only using one device at a time. I
upsync typically just before shutting down a machine, and
downsync just after starting one up.

I only use one instance of emacs at a time, globally. That may
be too limiting for some use cases, but it avoids a host of
issues. It doesn't limit me very much except that I need to be
careful not to be running emacs under termux on an Android
phone at the same time as I'm running on a laptop or desktop.

I never, ever downsync while emacs/gnus etc. are running. I
may, however upsync, but I still need to upsync again just
prior to shutting down and after exiting emacs and other
applications. As noted elsewhere in this thread, emacs updates
some files only on exit.


-- 
Bob Newell
Honolulu, Hawai`i

- Via GNU/Linux/Emacs/Gnus/BBDB


^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: synchronizing gnus sessions on multiple computers
  2021-09-10 21:50         ` Eric Abrahamsen
  2021-09-10 22:39           ` Bob Newell
@ 2021-09-11  4:04           ` Roland Winkler
  2021-10-06  4:28           ` Roland Winkler
  2 siblings, 0 replies; 15+ messages in thread
From: Roland Winkler @ 2021-09-11  4:04 UTC (permalink / raw)
  To: info-gnus-english

On Fri, Sep 10 2021, Eric Abrahamsen wrote:
> "Roland Winkler" <winkler@gnu.org> writes:
>> -- I realized that testing all this does not require two different
>> machines.  But two instances of emacs running on one computer
>> effectively give the same situation to play with.  It seems that a
>> running gnus session does not warn the user that another emacs instance
>> has fiddled with files such as .newsrc.eld.  Is this true?
>
> Yes, the .newsrc.eld file only gets written when you save Gnus ("s" in
> the *Group* buffer), and I would absolutely not try to update files for
> a Gnus that's still running, it will not handle that gracefully. It
> should, however, be enough to shut Gnus down first (you can see the
> `gnus-clear-system' function to see what happens then), rsync, then
> restart Gnus.

...It seems that the command gnus-group-exit (bound to q in the group
buffer) is really doing the complete job.  This command also calls
gnus-clear-system, besides other things.

The gnus info pages note repeatedly

  Gnus gives you all the opportunity you could possibly want for
  shooting yourself in the foot.

So I'll have to set up things such that a careless rsync cannot do too
much damage.  I guess when switching from machine A to machine B it is
always machine A (i.e., the source for rsync) where I should run
gnus-group-exit which will also flush everything to disk that needs to
be rsync'ed to machine B.  When rsync has done its job (A -> B), I can
start working on B.  Then, when I am done working on B, I can run
gnus-group-exit on B, rsync B -> A, and continue working on A...

Sounds like a plan.



^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: synchronizing gnus sessions on multiple computers
  2021-09-10 21:15       ` Roland Winkler
  2021-09-10 21:50         ` Eric Abrahamsen
@ 2021-09-11 16:07         ` Greg Farough
  2021-09-12 18:30           ` Bob Newell
  1 sibling, 1 reply; 15+ messages in thread
From: Greg Farough @ 2021-09-11 16:07 UTC (permalink / raw)
  To: info-gnus-english

On Fri, Sep 10 2021, "Roland Winkler" <winkler@gnu.org> wrote:

> On Fri, Sep 10 2021, Bob Newell wrote:
>> I have rsync'd gnus for years with success. In addition to the files
>> already listed I sync .newsrc.eld, but that is updated infrequently.
>> Not strictly gnus entries, but related closely enough, I also sync
>> .authinfo and .msmtprc (as I use msmtp) and also .bbdb.
>
> Thanks to everyone who replied to this thread.
>
> When you are rsyncing gnus, are you doing this while emacs is running on
> the different machines?  Or are you starting a fresh emacs session after
> all the files got updated on a machine?

Just chipping into say that I sync all the files mentioned in this
thread through Syncthing, between multiple Emacs sessions that run
24/7 on different machines. Since Syncthing watches for inode changes
and is pretty speedy, I've never run into discrepancy problems between
the various newsrc.eld files and so on.

To be safe, though, I usually do quit Gnus and load it again when I'm
sitting down at another computer.

-g

-- 
"We carry a new world here, in our hearts. That world is growing in
this minute."



^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: synchronizing gnus sessions on multiple computers
  2021-09-09 23:21 synchronizing gnus sessions on multiple computers Roland Winkler
  2021-09-10  4:46 ` Pankaj Jangid
@ 2021-09-12  4:47 ` James Thomas
  2021-10-06  4:29   ` Roland Winkler
  1 sibling, 1 reply; 15+ messages in thread
From: James Thomas @ 2021-09-12  4:47 UTC (permalink / raw)
  To: Roland Winkler; +Cc: info-gnus-english

Roland Winkler wrote:

> - How can I tell gnus (in a running emacs session) that I've rsync'ed
>   all local user directories and gnus should update itself accordingly
>   when, for example, articles are marked as read or expired.

Perhaps you can look into the Gnus Cloud code. I haven't ever used that
feature myself, though.


^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: synchronizing gnus sessions on multiple computers
  2021-09-11 16:07         ` Greg Farough
@ 2021-09-12 18:30           ` Bob Newell
  0 siblings, 0 replies; 15+ messages in thread
From: Bob Newell @ 2021-09-12 18:30 UTC (permalink / raw)
  To: info-gnus-english


Greg Farough <gregf@gnu.org> writes:
>
> Just chipping into say that I sync all the files mentioned in this
> thread through Syncthing, between multiple Emacs sessions that run
> 24/7 on different machines.

First --- I should have added .gnus.registry.eieio to the sync
list.

I was not familiar with syncthing and it looks intriguing. But
alas it doesn't suit my use case. I cannot rely on having two
devices running at the same time. I might have two or even
three or four, but most of the time it is just one. So
syncthing wouldn't work for me.

In any case the biggest difficulty would still be the age-old
problem of having the same file open on more than one machine
at a time. In my environment I have to be careful not to do
that or I run the risk of data loss. I don't see that
syncthing would really fix that.

My method of operation requires that I sync to/from a central
source. In my case I settled on rsync.net, who offer their
"borg" plan to users who don't require technical support at a
price affordable to a retiree on a fixed income.

-- 
Bob Newell
Honolulu, Hawai`i

- Via GNU/Linux/Emacs/Gnus/BBDB


^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: synchronizing gnus sessions on multiple computers
  2021-09-10 21:50         ` Eric Abrahamsen
  2021-09-10 22:39           ` Bob Newell
  2021-09-11  4:04           ` Roland Winkler
@ 2021-10-06  4:28           ` Roland Winkler
  2021-10-06 14:42             ` Eric Abrahamsen
  2 siblings, 1 reply; 15+ messages in thread
From: Roland Winkler @ 2021-10-06  4:28 UTC (permalink / raw)
  To: Eric Abrahamsen; +Cc: info-gnus-english

On Fri, Sep 10 2021, Eric Abrahamsen wrote:
> Yes, the .newsrc.eld file only gets written when you save Gnus ("s" in
> the *Group* buffer), and I would absolutely not try to update files for
> a Gnus that's still running, it will not handle that gracefully. It
> should, however, be enough to shut Gnus down first (you can see the
> `gnus-clear-system' function to see what happens then), rsync, then
> restart Gnus.

Thanks, it took me a while to get back to this.  It seems that
`gnus-clear-system' / quitting gnus is missing
`gnus-save-newsrc-file-last-timestamp'.  Quitting gnus, then rsync'ing
gnus followed by a restart of gnus will use .newsrc.eld file from the
other machine.  But when this .newsrc.eld file is saved again, the
timestamp of the file on disk is compared against the old timestamp of
the old file that's already gone.

I submitted a bug report (bug#51052).


^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: synchronizing gnus sessions on multiple computers
  2021-09-12  4:47 ` James Thomas
@ 2021-10-06  4:29   ` Roland Winkler
  2021-10-09  4:00     ` James Thomas
  0 siblings, 1 reply; 15+ messages in thread
From: Roland Winkler @ 2021-10-06  4:29 UTC (permalink / raw)
  To: James Thomas; +Cc: info-gnus-english, Roland Winkler

On Sun, Sep 12 2021, James Thomas wrote:
>> - How can I tell gnus (in a running emacs session) that I've rsync'ed
>>   all local user directories and gnus should update itself accordingly
>>   when, for example, articles are marked as read or expired.
>
> Perhaps you can look into the Gnus Cloud code. I haven't ever used that
> feature myself, though.

Thanks, it took me a while to get back to this.  It seems that Gnus
Cloud is a very different thing using an IMAP server to synchronize
marks and general files and data across multiple machines.  I want my
mail local, but synchronized among different machines.


^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: synchronizing gnus sessions on multiple computers
  2021-10-06  4:28           ` Roland Winkler
@ 2021-10-06 14:42             ` Eric Abrahamsen
  0 siblings, 0 replies; 15+ messages in thread
From: Eric Abrahamsen @ 2021-10-06 14:42 UTC (permalink / raw)
  To: info-gnus-english

"Roland Winkler" <winkler@gnu.org> writes:

> On Fri, Sep 10 2021, Eric Abrahamsen wrote:
>> Yes, the .newsrc.eld file only gets written when you save Gnus ("s" in
>> the *Group* buffer), and I would absolutely not try to update files for
>> a Gnus that's still running, it will not handle that gracefully. It
>> should, however, be enough to shut Gnus down first (you can see the
>> `gnus-clear-system' function to see what happens then), rsync, then
>> restart Gnus.
>
> Thanks, it took me a while to get back to this.  It seems that
> `gnus-clear-system' / quitting gnus is missing
> `gnus-save-newsrc-file-last-timestamp'.  Quitting gnus, then rsync'ing
> gnus followed by a restart of gnus will use .newsrc.eld file from the
> other machine.  But when this .newsrc.eld file is saved again, the
> timestamp of the file on disk is compared against the old timestamp of
> the old file that's already gone.
>
> I submitted a bug report (bug#51052).

And it looks like it's already fixed! Thanks a lot for doing that --
there's so much uncertainty and magical thinking around Gnus' behavior,
it's good to get one step closer to reliability.



^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: synchronizing gnus sessions on multiple computers
  2021-10-06  4:29   ` Roland Winkler
@ 2021-10-09  4:00     ` James Thomas
  0 siblings, 0 replies; 15+ messages in thread
From: James Thomas @ 2021-10-09  4:00 UTC (permalink / raw)
  To: Roland Winkler; +Cc: info-gnus-english

Roland Winkler wrote:

> On Sun, Sep 12 2021, James Thomas wrote:
>>> - How can I tell gnus (in a running emacs session) that I've rsync'ed
>>>   all local user directories and gnus should update itself accordingly
>>>   when, for example, articles are marked as read or expired.
>>
>> Perhaps you can look into the Gnus Cloud code. I haven't ever used that
>> feature myself, though.
>
> Thanks, it took me a while to get back to this.  It seems that Gnus
> Cloud is a very different thing using an IMAP server to synchronize
> marks and general files and data across multiple machines.  I want my
> mail local, but synchronized among different machines.

Right. I should have been clearer. I was just wondering out aloud
whether you could adapt its code to synchronise the marks gracefully
while Gnus is running.

Cheers


^ permalink raw reply	[flat|nested] 15+ messages in thread

end of thread, other threads:[~2021-10-09  3:59 UTC | newest]

Thread overview: 15+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-09-09 23:21 synchronizing gnus sessions on multiple computers Roland Winkler
2021-09-10  4:46 ` Pankaj Jangid
2021-09-10 11:06   ` Leo Butler
2021-09-10 19:45     ` Bob Newell
2021-09-10 21:15       ` Roland Winkler
2021-09-10 21:50         ` Eric Abrahamsen
2021-09-10 22:39           ` Bob Newell
2021-09-11  4:04           ` Roland Winkler
2021-10-06  4:28           ` Roland Winkler
2021-10-06 14:42             ` Eric Abrahamsen
2021-09-11 16:07         ` Greg Farough
2021-09-12 18:30           ` Bob Newell
2021-09-12  4:47 ` James Thomas
2021-10-06  4:29   ` Roland Winkler
2021-10-09  4:00     ` James Thomas

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).