Gnus development mailing list
 help / color / mirror / Atom feed
* "Ghost count" on nnimap group after server move
@ 2023-02-10 16:15 Steinar Bang
  2023-02-10 16:31 ` Steinar Bang
  0 siblings, 1 reply; 9+ messages in thread
From: Steinar Bang @ 2023-02-10 16:15 UTC (permalink / raw)
  To: ding

I have transferred my imap server to a different server.

Same OS both places (debian 11.6) and same IMAP server (dovecot) with
config rsync'd over from the old server (Maildir folders).

I also rsynced over the Maildir from the old server at a point where
there probably was something going on on the new one.  Because I ended
up with a "duplicate something" error message and deleted a dovecot
.index file in the maildir (as suggested by a google search hit).

But right now the nnimap group in Gnus is showing a count of 5, with no
new articles.

I have tried "M-g" and "C" on the group without losing the count.

I have also tried killing the group, stopping gnus, starting gnus and
re-adding the nnimap group, but the "ghost count" was still there.

Where do I go to fixing the count in gnus?

Suggestions about manual tinkering in .newsrc.eld are acceptable (I just
have to know what to change and where).

I'm using the emacs-27 gnus and I'm using agent in the nnimap server.

Thanks!


- Steinar



^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: "Ghost count" on nnimap group after server move
  2023-02-10 16:15 "Ghost count" on nnimap group after server move Steinar Bang
@ 2023-02-10 16:31 ` Steinar Bang
  2023-02-11 11:46   ` nnimap displays wrong article after server move (Was: "Ghost count" on nnimap group after server move) Steinar Bang
  0 siblings, 1 reply; 9+ messages in thread
From: Steinar Bang @ 2023-02-10 16:31 UTC (permalink / raw)
  To: ding

>>>>> Steinar Bang <sb@dod.no>:

> But right now the nnimap group in Gnus is showing a count of 5, with no
> new articles.

> I have tried "M-g" and "C" on the group without losing the count.

And now the count is up to "6"...

Just doing a "g" today didn't update the count today, but doing "M-g" on
the group upped the count to 10.

I entered the group and found 4 new articles.

Two of them were obvious spam, so I "B DEL"'d them.  One of them I read,
the other I ticked (an apticron report), and then I unticked and
marked-as-read an earlier message (the previous apticron report for that
machine).

And when I exited the groupd the "Ghost count" had increased from "5" to
"6". 



^ permalink raw reply	[flat|nested] 9+ messages in thread

* nnimap displays wrong article after server move (Was: "Ghost count" on nnimap group after server move)
  2023-02-10 16:31 ` Steinar Bang
@ 2023-02-11 11:46   ` Steinar Bang
  2023-02-11 11:55     ` nnimap displays wrong article after server move Steinar Bang
  0 siblings, 1 reply; 9+ messages in thread
From: Steinar Bang @ 2023-02-11 11:46 UTC (permalink / raw)
  To: ding

Summary:
 - I have moved my IMAP server to a different machine
 - Both machines running dovecot on debian 11.6.
 - I have rsynced config and mail from the old server to the new
 - I see weird behaviour in Gnus after the move

By "weird behaviour", I mean:
 - The count issues mentioned earlier in this thread
 - Missing articles (articles I thought I had seen in gnus were gone
   from the Maildir, when I looked into the Maildir on the server)
 - Re-appearing deleted articles (some spam articles I was sure I had
   deleted in gnus with "B DEL" were still in the Maildir)

I thought the problem was in the server: something in the dovecot.*
files in the Maildir directories maybe.

But then I switched off the gnus agent for that IMAP server and I saw
different kind of strange: In the group summary, I saw the same articles
I saw in the Maildir, which were not the same articles I saw with the
agent present.

I tried sending some messages from my work address.

The messages arrived, and looked right in the summary:
R  [   5: BANG Steinar           ] Test fra jobben til meg hos bang 1
R  [   5: BANG Steinar           ] Test fra jobben steinar at bang 2

But the content of the messages were older messages, eg. for the first
one (wrong subject, wrong text in the body 1and wrong date), with
anonymized email addresses:
 From: BANG Steinar <steinar.bang@mycompany.com>
 Subject: Test fra jobben
 To: "sb@mydomain.no" <sb@mydomain.no>
 Date: Tue, 7 Feb 2023 22:08:52 +0000 (3 days, 13 hours, 12 minutes ago)
 Attachment: [3. text/html]...


 Kommer denne fram?

So then I thought: is it my client?

So I installed thunderbird and added the server credentials, and yep, I
saw the same files I saw physically in the Maildir.

So yep, definitely my client.

So in case someone wonders: Moving from one dovecot to another, using
rsync for dovecot config and the mail content, and switching the server
in DNS, while keeping the config on gnus (agent disconnected until the
DNS alias had switched), probably doesn't work.

There is something funky with the article ids probably?

That's why I saw articles in the agent that no longer were present on
the server, and saw articles I thought I had deleted. I had "B DEL"'d
the wrong articles on the server.

So will need to sync up gnus to the current server somehow?

Why do I get the wrong articles?

Can gnus be loading the agent articles even though I stopped using agent
for that server?

Things to try:
 1. Stop gnus and rename the agent directory for the server
 2. Delete all groups from the server and stop gnus and then restart
    gnus on a different emacs (to ensure nothing is cached in memory)
    and readd the groups

I'll do the above anyway, but if someone have suggestions to things I
should have done instead, then I'll be happy to hear them (even if they
come too late for me).



^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: nnimap displays wrong article after server move
  2023-02-11 11:46   ` nnimap displays wrong article after server move (Was: "Ghost count" on nnimap group after server move) Steinar Bang
@ 2023-02-11 11:55     ` Steinar Bang
  2023-02-12 19:16       ` Eric Abrahamsen
  0 siblings, 1 reply; 9+ messages in thread
From: Steinar Bang @ 2023-02-11 11:55 UTC (permalink / raw)
  To: ding

>>>>> Steinar Bang <sb@dod.no>:

> Things to try:
>  1. Stop gnus and rename the agent directory for the server

I did the following:
 - I stopped gnus
 - Renamed the ~/News/agent/nnimap/privat/ directory to ~/News/agent/nnimap/privat.old/
 - Started gnus

Then gnus displayed the correct article.

So it looks like gnus is using the agent to fetch the article for an
nnimap server, even with agent switched off for that server?



^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: nnimap displays wrong article after server move
  2023-02-11 11:55     ` nnimap displays wrong article after server move Steinar Bang
@ 2023-02-12 19:16       ` Eric Abrahamsen
  2023-02-17 18:23         ` Steinar Bang
  0 siblings, 1 reply; 9+ messages in thread
From: Eric Abrahamsen @ 2023-02-12 19:16 UTC (permalink / raw)
  To: ding

Steinar Bang <sb@dod.no> writes:

>>>>>> Steinar Bang <sb@dod.no>:
>
>> Things to try:
>>  1. Stop gnus and rename the agent directory for the server
>
> I did the following:
>  - I stopped gnus
>  - Renamed the ~/News/agent/nnimap/privat/ directory to ~/News/agent/nnimap/privat.old/
>  - Started gnus
>
> Then gnus displayed the correct article.
>
> So it looks like gnus is using the agent to fetch the article for an
> nnimap server, even with agent switched off for that server?

If `gnus-agent-cache' is t, and data is present, Gnus will prefer the
agent data for that server, even if it's no longer agentized. You'd have
to shut the agent all the way off (`gnus-agent' -> nil) to ignore that
data.



^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: nnimap displays wrong article after server move
  2023-02-12 19:16       ` Eric Abrahamsen
@ 2023-02-17 18:23         ` Steinar Bang
  2023-02-18 16:58           ` Eric Abrahamsen
  2023-02-19 13:25           ` Eric S Fraga
  0 siblings, 2 replies; 9+ messages in thread
From: Steinar Bang @ 2023-02-17 18:23 UTC (permalink / raw)
  To: ding

>>>>> Eric Abrahamsen <eric@ericabrahamsen.net>:

> If `gnus-agent-cache' is t, and data is present, Gnus will prefer the
> agent data for that server, even if it's no longer agentized. You'd
> have to shut the agent all the way off (`gnus-agent' -> nil) to ignore
> that data.

Yeah, it's really, really, really, head-thumping-in-table time...

Gnus and gnus agent is not to blame.

dovecot is not to blame.

I and only I am to blame.

Yesterday's email was wiped today, and now I knew for sure I hadn't
deleted anything, even though deleted spam was back... so I looked
again, and my user's crontab on the mailserver had this thing syncing
from the old server (slightly anonymized server name):

0 5 * * * rsync -4 --archive --delete oldserver.mydomain.no:Maildir .

So basically all email received from february 6 (when I switched
servers) and until last midnight is gone, because of --delete.

And heh... the gnus agent that could have helped me, I've also deleted,
since I thought it was to blame.

Oh well! In 100 years' etc. etc.

(And yes. The rsync line has ben scrubbed from my crontab...)



^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: nnimap displays wrong article after server move
  2023-02-17 18:23         ` Steinar Bang
@ 2023-02-18 16:58           ` Eric Abrahamsen
  2023-02-19 13:25           ` Eric S Fraga
  1 sibling, 0 replies; 9+ messages in thread
From: Eric Abrahamsen @ 2023-02-18 16:58 UTC (permalink / raw)
  To: ding

Steinar Bang <sb@dod.no> writes:

>>>>>> Eric Abrahamsen <eric@ericabrahamsen.net>:
>
>> If `gnus-agent-cache' is t, and data is present, Gnus will prefer the
>> agent data for that server, even if it's no longer agentized. You'd
>> have to shut the agent all the way off (`gnus-agent' -> nil) to ignore
>> that data.
>
> Yeah, it's really, really, really, head-thumping-in-table time...
>
> Gnus and gnus agent is not to blame.
>
> dovecot is not to blame.
>
> I and only I am to blame.
>
> Yesterday's email was wiped today, and now I knew for sure I hadn't
> deleted anything, even though deleted spam was back... so I looked
> again, and my user's crontab on the mailserver had this thing syncing
> from the old server (slightly anonymized server name):
>
> 0 5 * * * rsync -4 --archive --delete oldserver.mydomain.no:Maildir .
>
> So basically all email received from february 6 (when I switched
> servers) and until last midnight is gone, because of --delete.
>
> And heh... the gnus agent that could have helped me, I've also deleted,
> since I thought it was to blame.

Oof! You're not the first to do something like this, and won't be the
last, but it always hurts to lose mail.



^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: nnimap displays wrong article after server move
  2023-02-17 18:23         ` Steinar Bang
  2023-02-18 16:58           ` Eric Abrahamsen
@ 2023-02-19 13:25           ` Eric S Fraga
  2023-02-19 16:03             ` Steinar Bang
  1 sibling, 1 reply; 9+ messages in thread
From: Eric S Fraga @ 2023-02-19 13:25 UTC (permalink / raw)
  To: ding

On Friday, 17 Feb 2023 at 19:23, Steinar Bang wrote:
> 0 5 * * * rsync -4 --archive --delete oldserver.mydomain.no:Maildir .

:-(

but it highlights that rsync promises more than it really delivers.  I
would love to be able to use it to keep two sets of files in sync
between two computers, including my nnml groups, but the --delete
doesn't work reliably enough (or possibly more likely that I don't
understand it well enough).

I'm using syncthing at the moment but it's also somewhat temperamental.

Unison is the best around but less portable and subject to breaking when
upgraded on one system and not another.

Back to gnus... sorry for the diversion (& rant ;-)).

-- 
Eric S Fraga via gnus (Emacs 30.0.50 2023-02-14) on Debian 11.5



^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: nnimap displays wrong article after server move
  2023-02-19 13:25           ` Eric S Fraga
@ 2023-02-19 16:03             ` Steinar Bang
  0 siblings, 0 replies; 9+ messages in thread
From: Steinar Bang @ 2023-02-19 16:03 UTC (permalink / raw)
  To: ding

>>>>> Eric S Fraga <e.fraga@ucl.ac.uk>:

> but it highlights that rsync promises more than it really delivers.  I
> would love to be able to use it to keep two sets of files in sync
> between two computers, including my nnml groups, but the --delete
> doesn't work reliably enough (or possibly more likely that I don't
> understand it well enough).

The reason for using --delete is the way maildir operates, with moving
articles from "new" to "tmp" and finally to "cur" and to track that you
will need to delete something once articles are read and viewed.

FWIW I see from my notes that on the last server switch, in March of
2017 I used a somewhat different rsync command:
 rsync --archive --fuzzy --delete-after rainey.bang.priv.no:Maildir .

Not sure if the --delete-after would have helped me this time...?
Probably not...?

(rsync sort of did what I had told it to do, which was to faitfully
restore the maildir to its February 6 state on the old server... so I
can't push the blame there either)

> I'm using syncthing at the moment but it's also somewhat temperamental.

> Unison is the best around but less portable and subject to breaking when
> upgraded on one system and not another.

Ok, thanks to the pointers.  I'll look into them if/when I need to move
servers again (hopefully never, but things change in the server farm
world...).

I'll put a mention of syncthing and unison in the transfer cheklist in
my org-mode notes (which will be copied in as the initial checklist if I
need to do this again).



^ permalink raw reply	[flat|nested] 9+ messages in thread

end of thread, other threads:[~2023-02-19 16:03 UTC | newest]

Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-02-10 16:15 "Ghost count" on nnimap group after server move Steinar Bang
2023-02-10 16:31 ` Steinar Bang
2023-02-11 11:46   ` nnimap displays wrong article after server move (Was: "Ghost count" on nnimap group after server move) Steinar Bang
2023-02-11 11:55     ` nnimap displays wrong article after server move Steinar Bang
2023-02-12 19:16       ` Eric Abrahamsen
2023-02-17 18:23         ` Steinar Bang
2023-02-18 16:58           ` Eric Abrahamsen
2023-02-19 13:25           ` Eric S Fraga
2023-02-19 16:03             ` Steinar Bang

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).