nnimap unread count broken again

nnimap unread count broken again

[Frank Schmitt]

Hello,

After Lars's rewrite of nnimap the problem that reading IMAP messages from
several Gnus instances caused wrong unread counts was fixed. However,
now it is there again and even worse then with the pre-Lars-nnimap.

If I get a new message, read it in Gnus A, then hit g in Gnus B, the
message is still unread. Even if I enter the group, the message is shown
as unread (at this point the old nnimap got it right at least). The
only way to get correct unread state and ticks is saying M-g on all nnimap
groups.

I think this problem was mentioned in previous messages already, so
before I start git bisecting or something: Is it already worked on?
The server in question is a dovecot BTW.

-- 
Have you ever considered how much text can fit in eighty columns?  Given that a
signature typically contains up to four lines of text, this space allows you to
attach a tremendous amount of valuable information to your messages.  Seize the
opportunity and don't waste your signature on bullshit that nobody cares about.

Re: nnimap unread count broken again

[Steinar Bang]

>>>>> Frank Schmitt <ich@frank-schmitt.net>:

> If I get a new message, read it in Gnus A, then hit g in Gnus B, the
> message is still unread. Even if I enter the group, the message is shown
> as unread (at this point the old nnimap got it right at least). The
> only way to get correct unread state and ticks is saying M-g on all nnimap
> groups.

> I think this problem was mentioned in previous messages already, so
> before I start git bisecting or something: Is it already worked on?
> The server in question is a dovecot BTW.

If you M-g all groups on all gnusen, and save the .newsrc.eld file on
the gnusen afterwards, it should eventually stabilize.

At least it does for me.

Note: I'm not running the gnus instaces simultanously.  Just one at a
time.

Re: nnimap unread count broken again

[Frank Schmitt]

Steinar Bang <sb@dod.no> writes:

>>>>>> Frank Schmitt <ich@frank-schmitt.net>:
>
>> If I get a new message, read it in Gnus A, then hit g in Gnus B, the
>> message is still unread. Even if I enter the group, the message is shown
>> as unread (at this point the old nnimap got it right at least). The
>> only way to get correct unread state and ticks is saying M-g on all nnimap
>> groups.
>
>> I think this problem was mentioned in previous messages already, so
>> before I start git bisecting or something: Is it already worked on?
>> The server in question is a dovecot BTW.
>
> If you M-g all groups on all gnusen, and save the .newsrc.eld file on
> the gnusen afterwards, it should eventually stabilize.
>
> At least it does for me.

Hmm. Isn't this removing symptoms instead of curing the disease?

-- 
Have you ever considered how much text can fit in eighty columns?  Given that a
signature typically contains up to four lines of text, this space allows you to
attach a tremendous amount of valuable information to your messages.  Seize the
opportunity and don't waste your signature on bullshit that nobody cares about.

Re: nnimap unread count broken again

[Steinar Bang]

>>>>> Frank Schmitt <ich@frank-schmitt.net>:

> Hmm. Isn't this removing symptoms instead of curing the disease?

I presume that if you start new gnusen with clean .newsrc.eld files, and
start subscribing to the groups in question you will never see the
issue...? 

The way I understood it, what you're seeing is caused by the .newsrc.eld
files being out of sync, and M-g'ing the group is a way of synching
them.

Once they are synched you shouldn't have the issue.

Again, the way I've understood it, this is a result of the way QRESYNC
implementation tries to keep the network traffic to a minimum (and
thereby the speed to a maximum).

Re: nnimap unread count broken again

[Dave Goldberg]

>>>>> Frank Schmitt <ich@frank-schmitt.net>:
>> Hmm. Isn't this removing symptoms instead of curing the disease?

> I presume that if you start new gnusen with clean .newsrc.eld files, and
> start subscribing to the groups in question you will never see the
> issue...? 

> The way I understood it, what you're seeing is caused by the .newsrc.eld
> files being out of sync, and M-g'ing the group is a way of synching
> them.

> Once they are synched you shouldn't have the issue.

> Again, the way I've understood it, this is a result of the way QRESYNC
> implementation tries to keep the network traffic to a minimum (and
> thereby the speed to a maximum).

I only use one instance of Gnus, and no, the count never resync's by running M-g on the group line.  I've sent Lars (directly per his instruction due to the size) the traces I got through debugging and hopefully that will help to resolve the problem.  Can say for certain that James Cloos is correct that the count has gone back to being determined by subtracting the smallest UID number from the highest.  On my cyrus server this actually works because the UIDs are consecutive.  On Exchange the numbers can be quite bizarre (a group with two unread showed over 100 in the Group buffer).  As I've noted in earlier posts, watching the IMAP traffic, the server is clearly providing the correct numbers; Gnus is no longer doing the right thing with them as it did up until a week or so ago.  I've tried
  to look at git bisect per someone's suggestion but am sufficiently unfamiliar with git that I've been unable to get anywhere.

Thanks,
-- 
Dave Goldberg
david.goldberg6@verizon.net

Re: nnimap unread count broken again

[Lars Magne Ingebrigtsen]

Using Dave's data I was finally able to debug this.  I think it's all
due to the read-only makeover.  It refused to update group that didn't
have \* in PERMANENT-FLAGS.  I've now reworked that code, so it should
probably be more correct now.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: nnimap unread count broken again

[Lars Magne Ingebrigtsen]

Frank Schmitt <ich@frank-schmitt.net> writes:

> I think this problem was mentioned in previous messages already, so
> before I start git bisecting or something: Is it already worked on?
> The server in question is a dovecot BTW.

Is this a Dovecot server that supports QRESYNC?

I haven't yet been able to figure out what's going on, or reproduce the
bug, so I welcome all help in this area.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: nnimap unread count broken again

[Frank Schmitt]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> Frank Schmitt <ich@frank-schmitt.net> writes:
>
>> I think this problem was mentioned in previous messages already, so
>> before I start git bisecting or something: Is it already worked on?
>> The server in question is a dovecot BTW.
>
> Is this a Dovecot server that supports QRESYNC?
>
> I haven't yet been able to figure out what's going on, or reproduce the
> bug, so I welcome all help in this area.

Hmm. I don't know. It says:

. CAPABILITY
* CAPABILITY IMAP4rev1 SASL-IR SORT THREAD=REFERENCES MULTIAPPEND
  UNSELECT LITERAL+ IDLE CHILDREN NAMESPACE LOGIN-REFERRALS
. OK Capability completed.

I can give you a test account on this server if you want.

-- 
Have you ever considered how much text can fit in eighty columns?  Given that a
signature typically contains up to four lines of text, this space allows you to
attach a tremendous amount of valuable information to your messages.  Seize the
opportunity and don't waste your signature on bullshit that nobody cares about.

Re: nnimap unread count broken again

[Lars Magne Ingebrigtsen]

Frank Schmitt <ich@frank-schmitt.net> writes:

> Hmm. I don't know. It says:
>
> . CAPABILITY
> * CAPABILITY IMAP4rev1 SASL-IR SORT THREAD=REFERENCES MULTIAPPEND
>   UNSELECT LITERAL+ IDLE CHILDREN NAMESPACE LOGIN-REFERRALS
> . OK Capability completed.

Without QRESYNC.  But try the fix I just pushed out.  It fixed the test
case I got from Dan.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: nnimap unread count broken again

[Lars Magne Ingebrigtsen]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> Without QRESYNC.  But try the fix I just pushed out.  It fixed the test
> case I got from Dan.

Dave.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: nnimap unread count broken again

[Frank Schmitt]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> Lars Magne Ingebrigtsen <larsi@gnus.org> writes:
>
>> Without QRESYNC.  But try the fix I just pushed out.  It fixed the test
>> case I got from Dan.
>
> Dave.

Yes, seems to be fixed for me, too. Thanks.

-- 
Have you ever considered how much text can fit in eighty columns?  Given that a
signature typically contains up to four lines of text, this space allows you to
attach a tremendous amount of valuable information to your messages.  Seize the
opportunity and don't waste your signature on bullshit that nobody cares about.

Re: nnimap unread count broken again

[Dave Goldberg]

> Lars Magne Ingebrigtsen <larsi@gnus.org> writes:
>> Without QRESYNC.  But try the fix I just pushed out.  It fixed the test
>> case I got from Dan.

> Dave.

Confirm it works for me again now.

Thanks,

-- 
Dave Goldberg
david.goldberg6@verizon.net

Re: nnimap unread count broken again

[James Cloos]

I seem to have lost the article I wanted to reply to, but this is in the
right thread....

The article counts seem to be correct, now, after a M-g, but not after g.

I do make frequent use of the cache to backup stuff which is interesting;
might that interfere with the unread counts?

-JimC
-- 
James Cloos <cloos@jhcloos.com>         OpenPGP: 1024D/ED7DAEA6

Re: nnimap unread count broken again

[Lars Magne Ingebrigtsen]

James Cloos <cloos@jhcloos.com> writes:

> I seem to have lost the article I wanted to reply to, but this is in the
> right thread....
>
> The article counts seem to be correct, now, after a M-g, but not after g.
>
> I do make frequent use of the cache to backup stuff which is interesting;
> might that interfere with the unread counts?

It might.  The interactions between the backends, the Agent and the
cache are kinda hairy.

But I've cached a few messages in a few nnimap groups, and it seems to
work OK for me.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen