agent skips over emails

agent skips over emails

[Eric S Fraga]

Hello,

One of my servers is an Outlook server (office365) which I access
through IMAP.  If I don't enable the agent for this server, everything
is fine (although slow).  If I enable the agent for this server, gnus
misses some emails.  If I look at the inbox with another tool
(e.g. outlook web access), I can see emails already marked as read that
gnus has no record of.  The .overview file does not have those emails
and the .agentview file has message numbers that include those emails,
indicating that it seems to have already seen them.

I can get gnus to find the missing emails by manually removing the
.agentview file but this means that gnus has to recreate this file when
I next open the group.

And then later another (seemingly random) email will be missed.

I am happy to try to debug this but I have no idea where to start.  I
have had this problem for years now.  I am using emacs snapshot.  The
problem is not reproducible per se: i.e. I never know if emails have
been missed and only some emails are missed.  The problem usually arises
once I get an error message from gnus that tells me that there are
duplicate overview lines.  But I don't know why these duplicates appear.

Basically, the agent system is great but seems rather fragile.

Any pointers would be welcome.

Thanks,
eric

PS - I should add that this problem *only* arises with outlook and
not with any other imap server or nntp or any other backend.
-- 
Eric S Fraga via Emacs 27.0.50 & org 9.1.13 on Debian buster/sid

Re: agent skips over emails

[Steinar Bang]

Do you read main from this IMAP server from several gnus'en that all use
agent?

I've experienced that if a message is marked as read on one gnus
instance, the other gnusen can't see it, even when I look at old read
messages with C-u <number> RET.

To be able to see these messages I have to remove the agent from that
nnimap server, temporarily.

Re: agent skips over emails

[Eric S Fraga]

On Tuesday, 17 Jul 2018 at 10:17, Steinar Bang wrote:
> Do you read main from this IMAP server from several gnus'en that all use
> agent?

Hi Steinar,

yes, I do often read from different systems, but the problem arises
without doing so.  Yesterday, I turned off the agent as I needed to get
work done with stressing about missing emails.

This morning, using only one system, I turned the agent on to see if I
could start figuring out what was going on.  I had outlook web access
(OWA) on the browser in the corner of the display to monitor what was
coming in etc.

I started gnus, went into the INBOX group.  All unread emails shown
(about 30) matched what I could see in the OWA window.  I read all the
emails and went back to the group buffer by quitting the group.  So far
so good.

In the OWA window, I noticed that three new emails had come in while I
had been reading the INBOX group and, in fact, the number of unread
shown in the group buffer for that group is 3.  So I went back into that
group: no new unread emails!  Quit the group and I see, in the OWA
window, that 2 of the 3 new emails are now marked as read.  I rescan the
groups and gnus tells me that there is 1 unread email.  I go in to the
group.  I can see the new email but the two that were marked as read
that I had not actually read are nowhere to be seen (e.g. "/o 100" in
the summary buffer shows me plenty of old read emails but not those
two).

To be clear, in all of the above, all interactions are with gnus.  The
OWA window is just sitting there.  Somehow, gnus (while using the agent)
loses emails, at least in this case, when emails arrive while a group is
being read.  Maybe other times as well, of course.

> To be able to see these messages I have to remove the agent from that
> nnimap server, temporarily.

Yes but I cannot function like that.  I get many emails and I expect the
system to keep track of what I have read and what I have not; if I have
to manually go through all emails (that are marked as read) to actually
see if I have already read them, I will go crazy!

I would love to have the agent working with confidence.  I travel a lot
and offline access is important but, even when online, the use of the
agent as a cache is fantastic given the slowness of the outlook server I
have to use (for work).

thanks,
eric

-- 
Eric S Fraga via Emacs 27.0.50 & org 9.1.13 on Debian buster/sid

Re: agent skips over emails

[Eric S Fraga]

[-- Attachment #1: Type: text/plain, Size: 308 bytes --]

Hi Steinar,

one more data point: the article numbers for emails in Outlook are not
contiguous.  That is, there are gaps so that the article numbers might
be 1, 2, 5, 6, 9, 11, 12, ...  This screenshot of my INBOX summary
window (showing only article numbers, flags and the time of the message)
shows this:


[-- Attachment #2: screendump-20180717112947.png --]
[-- Type: image/png, Size: 6665 bytes --]

[-- Attachment #3: Type: text/plain, Size: 444 bytes --]


The two that went missing earlier were 522589 and 522591.

I have no idea why this non-contiguous numbering might be but I wonder
if this might affect the agent in some way given that it seems to care
what the "active" number is?  Does the agent assume contiguous values of
the article numbers?

Everything that has to do with MS is fraught with fragility...

thanks,
eric

-- 
Eric S Fraga via Emacs 27.0.50 & org 9.1.13 on Debian buster/sid