From: Greg Troxel <gdt@lexort.com>
To: Eric Abrahamsen <eric@ericabrahamsen.net>
Cc: ding@gnus.org
Subject: Gnus sometimes fails to see a message that is actually in IMAP
Date: Sun, 12 May 2024 07:14:48 -0400 [thread overview]
Message-ID: <rmi1q67wac7.fsf@s1.lexort.com> (raw)
In-Reply-To: <87msow9dwx.fsf@ericabrahamsen.net>
I have changed the subject, as I am not at all sure that my problem is
the same as "reports new messages but not showing them on IMAP server".
In my case, it seems to me as if there is a filter between gnus and the
IMAP server that redacts a message.
Eric Abrahamsen <eric@ericabrahamsen.net> writes:
> I was going to say, the IMAP implementation of Outlook is indeed bad!
> But that's "good" that you're hitting this issue with Dovecot running
> locally, as that's a setup with fewer confounding variables, and also
> what I run, so maybe we stand a better chance of reproducing.
Yes, it does seem more tractable.
One thing I find troublesome about modern gnus is that it is part of
emacs. I'm currently running 28.2, so that's the gnus I have. Years
before, when gnus was in a separate repo, I was running up-to-date gnus
with stable emacs. If I'm confused and we should be debugging gnus
trunk please let me know.
I probably can update to 29.x without trouble, and I should do that. I
try to stay away from bleeding edge including .0, when the older version
isn't crufty and isn't troublesome, for things I want to use rather than
hack on. I don't really want to run emacs-current.
> Maybe it's time to dive in :( Greg, if you're willing, would you provide
> as much details as possible about your setup, and anything/everything
> you can remember about what happened/happens before you see this error?
First, I have no perception of anything unusual or remarkable happening
when the problem happens. Mail arrives, apparently just like any other
message, I see it in K-9 (Android client that seems very well behaved),
and I then go to look for it in gnus (to answer, because I have a
keyboard) and find it's not there. This is infrequent, maybe once a
month, and there is nothing strange about these messages. They are from
people I've heard from before, are not necessarily even large, and "more
$filename" in the IMAP dir looks unremarkable.
When this happens, gnus feels entirely normal. I can still enter INBOX,
read arriving mail, enter with "100 <space>" to see already read mail,
etc. If I were only using gnus I would have had no idea anything was
wrong, until someone asked me why I didn't respond to some message.
The missing messages are in $HOME/IMAP/cur because they've been read by
another client. Thunderbird sees the missing messages too.
I have previously tried restarting dovecot and removing dovecot caches,
but this has never fixed the issue. I am 95% convinced that the is
issue is in gnus.
My setup is:
- computer runs NetBSD 9, postfix receiving mail from the Internet at
large, with lots of spam filtering, postfix delivering to $HOME/IMAP
as maildir, hence into $HOME/IMAP/new. While I have to mess with
spam filtering from time to time, it is not flaky - just minor
misclassification - and the messages in question are 1) not spam and
2) delivered to INBOX.
- dovecot 2.3.21 serves $HOME/IMAP. Many updates have happened since
I first noticed this -- it's been several years I think (but no
notes).
- dovecot has an x509 cert (that validates under pkix rules), and
listens only on the "implicit TLS" port, meaning that it expects TLS
negotation right away vs STARTTLS. (I'm firmly in the "cleartext
connections should be adminitratively prohibited" camp.)
- dovecot's uidvalidity as shown in "G p" is a number that when
interpreted as a timeval is noonish EDT on a Sunday in March of
2006. Indeed the IMAP spec says that servers should try not to
change that. That's wild, but probably when I installed dovecot
version 1 on a different computer. I have upgraded dovecot,
upgraded the OS version, and migrated to different hardware with a
different hostname since. The point is that this problem is almost
certainly not about gnus' IMAP implemention getting confused about
uidvalidity changes!
- emacs with gnus is left running always, and accessed via tmux. The
machine is very stable, basically getting rebooted for software
updates.
- emacs 28.2 gnus has nnimap configured
(setq gnus-secondary-select-methods
'((nnimap "foo.example.com"
(nnimap-address "bar.example.com")
(nnimap-port 993)
(nnimap-stream ssl)
(nnimap-authenticator login)
(nnir-search-engine imap)
)
(nntp "news.gwene.org")
)
)
where foo is a historical name and bar.example.com is the actual name
of the host (with emacs and the imap server both).
The connection goes over IPv6, not ::1, but the actual IP address.
I would have used localhost, but then the certificate is not valid.
(Yes, I know the gwene/gmane world is messy and that it's possible I
should be stopping this or doing it differently, but I don't think it
is relevant.)
- I had vestigial config that I just dropped, to add --insecure to
gnutls-cli, for a historical collection of self-signed certs,
connecting to localhost, emacs having the wrong path to ssl config,
and me not straightening out my
copy-of-config-in-linuxy-pathname-for-emacs. In the end, it was
just adding "--insecure" to the gnutls-cli invocation. I can't
believe that matters, but mentioning it for completeness.
> And if you do see the error crop up again, please let us know what you
> were doing immediately before, and if at all possible, leave Gnus in the
> broken state until we can do some remote debugging.
Thanks for offering to spend time on this. I'll write offlist to see
about coordinating.
It does seem like looking at group properties is useful and I'll do that.
I'll at least take better notes. I think but am not 100% sure that the
broken state survives exiting/restarting gnus, and exiting/restarting
emacs.
My previous workaround was move-out move-back.
Now, M-g on the group seems to fix things.
It would help to understand the big picture of how gnus caches imap
state. Ideally that would be in the manual; I'll take another look at
that.
It would also be interesting to have a gnus-imap-revalidate function that
for all cached information, fetches it and compares, complaining loudly
if different. I have used this technique when debugging routing code to
end up with only correct cached information, and didn't think of any
other way to find rare bugs.
Greg
next prev parent reply other threads:[~2024-05-12 11:15 UTC|newest]
Thread overview: 41+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-04-12 0:57 Gnus sometimes reports new messages but not showing them on IMAP server Xiyue Deng
2024-04-17 0:03 ` Greg Troxel
2024-04-17 6:47 ` Xiyue Deng
2024-04-17 19:51 ` Dan Christensen
2024-04-18 12:13 ` Greg Troxel
2024-04-19 22:55 ` Dan Christensen
2024-04-21 12:08 ` Arash Esbati
2024-04-21 12:43 ` Dan Christensen
2024-04-21 13:14 ` Andreas Schwab
2024-04-21 16:18 ` Eric Abrahamsen
2024-04-21 18:16 ` Arash Esbati
2024-04-22 0:00 ` Greg Troxel
2024-04-22 2:35 ` Eric Abrahamsen
2024-04-27 19:46 ` Xiyue Deng
2024-05-02 5:47 ` Xiyue Deng
2024-05-02 15:08 ` Eric Abrahamsen
2024-05-02 17:24 ` Xiyue Deng
2024-05-02 17:41 ` Dan Christensen
2024-05-02 18:41 ` Xiyue Deng
2024-05-05 20:18 ` Xiyue Deng
2024-05-05 21:01 ` Dan Christensen
2024-05-05 22:10 ` Xiyue Deng
2024-05-08 18:36 ` Xiyue Deng
2024-05-08 18:41 ` Xiyue Deng
2024-05-09 23:11 ` Andrew Cohen
2024-05-10 7:45 ` Xiyue Deng
2024-05-10 8:16 ` Andrew Cohen
2024-05-10 8:26 ` Xiyue Deng
2024-05-10 9:13 ` Andreas Schwab
2024-05-10 0:43 ` Greg Troxel
2024-05-10 1:20 ` Xiyue Deng
2024-05-10 12:24 ` Greg Troxel
2024-05-10 20:06 ` Xiyue Deng
2024-05-11 19:08 ` Greg Troxel
2024-05-11 22:33 ` Eric Abrahamsen
2024-05-12 11:14 ` Greg Troxel [this message]
2024-05-13 14:39 ` Gnus sometimes fails to see a message that is actually in IMAP Eric Abrahamsen
2024-05-24 15:58 ` Eric Abrahamsen
2024-05-24 17:29 ` Eric Abrahamsen
2024-04-24 15:23 ` Gnus sometimes reports new messages but not showing them on IMAP server James Thomas
2024-04-25 2:59 ` Eric Abrahamsen
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=rmi1q67wac7.fsf@s1.lexort.com \
--to=gdt@lexort.com \
--cc=ding@gnus.org \
--cc=eric@ericabrahamsen.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).