New messages marked O before seeing them

New messages marked O before seeing them

[John Sullivan]

nnimap messages are sometimes being marked O before I ever see them. 

I'm using emacs from bzr, revno: 103416.

I don't know how to give any more info to help reproduce or debug this.
Messages arriving both before and after the problem ones are being
displayed unread, as expected.

Sometimes I get new messages with / N from Summary inside the folder,
sometimes with g or M-g in *Group*. 

Re: New messages marked O before seeing them

[Lars Magne Ingebrigtsen]

John Sullivan <john@wjsullivan.net> writes:

> nnimap messages are sometimes being marked O before I ever see them. 
>
> I'm using emacs from bzr, revno: 103416.
>
> I don't know how to give any more info to help reproduce or debug this.
> Messages arriving both before and after the problem ones are being
> displayed unread, as expected.

If you can determine when the "O" issue happens, that would be helpful
in trying to chase down this bug.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: New messages marked O before seeing them

[John Sullivan]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> John Sullivan <john@wjsullivan.net> writes:
>
>> nnimap messages are sometimes being marked O before I ever see them. 
>>
>> I'm using emacs from bzr, revno: 103416.
>>
>> I don't know how to give any more info to help reproduce or debug this.
>> Messages arriving both before and after the problem ones are being
>> displayed unread, as expected.
>
> If you can determine when the "O" issue happens, that would be helpful
> in trying to chase down this bug.

It happens with some new messages that arrive (to the server) while I
have the group open. 

Do you think using / N vs M-g from inside the group could make any
difference here?

Re: New messages marked O before seeing them

[Lars Magne Ingebrigtsen]

John Sullivan <john@wjsullivan.net> writes:

> It happens with some new messages that arrive (to the server) while I
> have the group open. 
>
> Do you think using / N vs M-g from inside the group could make any
> difference here?

It could well be.  I'm unable to reproduce it, though.

I send a message to my Gmail account, and then enter the INBOX.  I wait
for a while, and say `/ N'.  The message is then inserted in the summary
buffer and behaves normally, as far as I can see.

Do you have a more detailed recipe for how to tickle the bug?

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: New messages marked O before seeing them

[John Sullivan]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> John Sullivan <john@wjsullivan.net> writes:
>
>> It happens with some new messages that arrive (to the server) while I
>> have the group open. 
>>
>> Do you think using / N vs M-g from inside the group could make any
>> difference here?
>
> It could well be.  I'm unable to reproduce it, though.
>
> I send a message to my Gmail account, and then enter the INBOX.  I wait
> for a while, and say `/ N'.  The message is then inserted in the summary
> buffer and behaves normally, as far as I can see.
>
> Do you have a more detailed recipe for how to tickle the bug?

Not a for-sure recipe :/ Try sending yourself a few dozen messages with
the folder open for an hour or so. During that time, delete a few
messages, move a few messages to another group, read a few messages.
Then / N. That's closer to the conditions under which I see it.

The next thing I'd try is do the same thing, but before doing / N, do a
g in *Group* (with the group still open). I know that sometimes I do
that.

Finally, I'd try the same thing as the first recipe, but instead of
doing / N, just quit the group, re-enter it, quit it again, fetch new
mail, and enter it again.

Re: New messages marked O before seeing them

[Lars Magne Ingebrigtsen]

John Sullivan <john@wjsullivan.net> writes:

> Not a for-sure recipe :/ Try sending yourself a few dozen messages with
> the folder open for an hour or so. During that time, delete a few
> messages, move a few messages to another group, read a few messages.
> Then / N. That's closer to the conditions under which I see it.

Could you try to provoke the bug with a simpler recipe?  :-)

> Finally, I'd try the same thing as the first recipe, but instead of
> doing / N, just quit the group, re-enter it, quit it again, fetch new
> mail, and enter it again.

I haven't seen any issues with non-`/ N' IMAP group usage, though...

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: New messages marked O before seeing them

[John Sullivan]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> John Sullivan <john@wjsullivan.net> writes:
>
>> Not a for-sure recipe :/ Try sending yourself a few dozen messages with
>> the folder open for an hour or so. During that time, delete a few
>> messages, move a few messages to another group, read a few messages.
>> Then / N. That's closer to the conditions under which I see it.
>
> Could you try to provoke the bug with a simpler recipe?  :-)
>

Yeah, I'm trying, but I can't seem to make it happen, even though it
happened to 10 or so messages just today in the normal course of reading
and answering my mail. I'm baffled that other people aren't seeing this,
because I don't think I'm doing anything weird. The only thing I do
that's maybe unusual for gnus users is actually delete a fair amount of
mail.

One idea I might want to try -- making the group update after each
article deletion or move. 

Re: New messages marked O before seeing them

[Glyn Millington]

John Sullivan <john@wjsullivan.net> writes:

> Lars Magne Ingebrigtsen <larsi@gnus.org> writes:
>
>> John Sullivan <john@wjsullivan.net> writes:
>>
>>> Not a for-sure recipe :/ Try sending yourself a few dozen messages
>>> with the folder open for an hour or so. During that time, delete a
>>> few messages, move a few messages to another group, read a few
>>> messages.  Then / N. That's closer to the conditions under which I
>>> see it.
>> Could you try to provoke the bug with a simpler recipe?  :-)
>>
>
> Yeah, I'm trying, but I can't seem to make it happen, even though it
> happened to 10 or so messages just today in the normal course of
> reading and answering my mail. I'm baffled that other people aren't
> seeing this, because I don't think I'm doing anything weird. The only
> thing I do that's maybe unusual for gnus users is actually delete a
> fair amount of mail.

I'm seeing it too and have been for some time.  I believe it started when
I upgraded to Emacs 23.  Changing to No Gnus 13 improved it but did not
eradicate it completely.

atb



Glyn

Re: New messages marked O before seeing them

[John Sullivan]

Glyn Millington <wistanswick@linuxmail.org> writes:

> John Sullivan <john@wjsullivan.net> writes:
>
>> Lars Magne Ingebrigtsen <larsi@gnus.org> writes:
>>
>>> John Sullivan <john@wjsullivan.net> writes:
>>>
>>>> Not a for-sure recipe :/ Try sending yourself a few dozen messages
>>>> with the folder open for an hour or so. During that time, delete a
>>>> few messages, move a few messages to another group, read a few
>>>> messages.  Then / N. That's closer to the conditions under which I
>>>> see it.
>>> Could you try to provoke the bug with a simpler recipe?  :-)
>>>
>>
>> Yeah, I'm trying, but I can't seem to make it happen, even though it
>> happened to 10 or so messages just today in the normal course of
>> reading and answering my mail. I'm baffled that other people aren't
>> seeing this, because I don't think I'm doing anything weird. The only
>> thing I do that's maybe unusual for gnus users is actually delete a
>> fair amount of mail.
>
> I'm seeing it too and have been for some time.  I believe it started when
> I upgraded to Emacs 23.  Changing to No Gnus 13 improved it but did not
> eradicate it completely.

Okay, good to not be alone :).

Can you describe anything else about the conditions under which you see
it? Do you also do a lot of moving messages between groups and/or
deleting messages? What IMAP server?

Re: New messages marked O before seeing them

[Glyn Millington]

John Sullivan <john@wjsullivan.net> writes:

> Glyn Millington <wistanswick@linuxmail.org> writes:
>
>> John Sullivan <john@wjsullivan.net> writes:
>>
>>> Lars Magne Ingebrigtsen <larsi@gnus.org> writes:
>>>
>>>> John Sullivan <john@wjsullivan.net> writes:
>>>>
>>>>> Not a for-sure recipe :/ Try sending yourself a few dozen
>>>>> messages with the folder open for an hour or so. During that
>>>>> time, delete a few messages, move a few messages to another
>>>>> group, read a few messages.  Then / N. That's closer to the
>>>>> conditions under which I see it.
>>>> Could you try to provoke the bug with a simpler recipe?  :-)
>>>>
>>> Yeah, I'm trying, but I can't seem to make it happen, even though
>>> it happened to 10 or so messages just today in the normal course of
>>> reading and answering my mail. I'm baffled that other people aren't
>>> seeing this, because I don't think I'm doing anything weird. The
>>> only thing I do that's maybe unusual for gnus users is actually
>>> delete a fair amount of mail.
>> I'm seeing it too and have been for some time.  I believe it started
>> when I upgraded to Emacs 23.  Changing to No Gnus 13 improved it but
>> did not eradicate it completely.
>
> Okay, good to not be alone :).
>
> Can you describe anything else about the conditions under which you
> see it? Do you also do a lot of moving messages between groups and/or
> deleting messages? What IMAP server?

Not IMAP here but pop and nnml

I do delete a lot of messages, but don't shift them around between groups
much. And it only happens with mail - not with news.  What is more it is
not consistent in any way that is obvious to me.

Sorry to be rather vague!

atb



Glyn

Re: New messages marked O before seeing them

[John Sullivan]

Glyn Millington <wistanswick@linuxmail.org> writes:

> John Sullivan <john@wjsullivan.net> writes:
>
>> Glyn Millington <wistanswick@linuxmail.org> writes:
>>
>>> John Sullivan <john@wjsullivan.net> writes:
>>>
>>>> Lars Magne Ingebrigtsen <larsi@gnus.org> writes:
>>>>
>>>>> John Sullivan <john@wjsullivan.net> writes:
>>>>>
>>>>>> Not a for-sure recipe :/ Try sending yourself a few dozen
>>>>>> messages with the folder open for an hour or so. During that
>>>>>> time, delete a few messages, move a few messages to another
>>>>>> group, read a few messages.  Then / N. That's closer to the
>>>>>> conditions under which I see it.
>>>>> Could you try to provoke the bug with a simpler recipe?  :-)
>>>>>
>>>> Yeah, I'm trying, but I can't seem to make it happen, even though
>>>> it happened to 10 or so messages just today in the normal course of
>>>> reading and answering my mail. I'm baffled that other people aren't
>>>> seeing this, because I don't think I'm doing anything weird. The
>>>> only thing I do that's maybe unusual for gnus users is actually
>>>> delete a fair amount of mail.
>>> I'm seeing it too and have been for some time.  I believe it started
>>> when I upgraded to Emacs 23.  Changing to No Gnus 13 improved it but
>>> did not eradicate it completely.
>>
>> Okay, good to not be alone :).
>>
>> Can you describe anything else about the conditions under which you
>> see it? Do you also do a lot of moving messages between groups and/or
>> deleting messages? What IMAP server?
>
> Not IMAP here but pop and nnml
>
> I do delete a lot of messages, but don't shift them around between groups
> much. And it only happens with mail - not with news.  What is more it is
> not consistent in any way that is obvious to me.
>

Not even IMAP??? So, by what means do you bring new mail to gnus? Does
it talk direct to a remote POP server?

Re: New messages marked O before seeing them

[Glyn Millington]

John Sullivan <john@wjsullivan.net> writes:

> Glyn Millington <wistanswick@linuxmail.org> writes:
>
>> John Sullivan <john@wjsullivan.net> writes:
>>
>>> Glyn Millington <wistanswick@linuxmail.org> writes:
>>>
>>>> John Sullivan <john@wjsullivan.net> writes:
>>>>
>>>>> Lars Magne Ingebrigtsen <larsi@gnus.org> writes:
>>>>>
>>>>>> John Sullivan <john@wjsullivan.net> writes:
>>>>>>
>>>>>>> Not a for-sure recipe :/ Try sending yourself a few dozen
>>>>>>> messages with the folder open for an hour or so. During that
>>>>>>> time, delete a few messages, move a few messages to another
>>>>>>> group, read a few messages.  Then / N. That's closer to the
>>>>>>> conditions under which I see it.
>>>>>> Could you try to provoke the bug with a simpler recipe?  :-)
>>>>>>
>>>>> Yeah, I'm trying, but I can't seem to make it happen, even though
>>>>> it happened to 10 or so messages just today in the normal course
>>>>> of reading and answering my mail. I'm baffled that other people
>>>>> aren't seeing this, because I don't think I'm doing anything
>>>>> weird. The only thing I do that's maybe unusual for gnus users is
>>>>> actually delete a fair amount of mail.
>>>> I'm seeing it too and have been for some time.  I believe it
>>>> started when I upgraded to Emacs 23.  Changing to No Gnus 13
>>>> improved it but did not eradicate it completely.
>>> Okay, good to not be alone :).
>>> Can you describe anything else about the conditions under which you
>>> see it? Do you also do a lot of moving messages between groups
>>> and/or deleting messages? What IMAP server?
>> Not IMAP here but pop and nnml
>> I do delete a lot of messages, but don't shift them around between
>> groups much. And it only happens with mail - not with news.  What is
>> more it is not consistent in any way that is obvious to me.
>>
>
> Not even IMAP??? So, by what means do you bring new mail to gnus? Does
> it talk direct to a remote POP server?

Getmail dumps the mail into  my user's inbox and then I have this

(setq  gnus-secondary-select-methods '((nnml ""))
       mail-sources '((file :path "/home/glyn/Mail/inbox")
                      (file :path "/var/spool/mail/glyn")))

Then fancy splitting to sort the mail into groups.

atb


Glyn

Re: New messages marked O before seeing them

[John Sullivan]

Lars Magne Ingebrigtsen <larsi <at> gnus.org> writes:

> 
> John Sullivan <john <at> wjsullivan.net> writes:
> 
> > Not a for-sure recipe :/ Try sending yourself a few dozen messages with
> > the folder open for an hour or so. During that time, delete a few
> > messages, move a few messages to another group, read a few messages.
> > Then / N. That's closer to the conditions under which I see it.
> 
> Could you try to provoke the bug with a simpler recipe?  
> 
> > Finally, I'd try the same thing as the first recipe, but instead of
> > doing / N, just quit the group, re-enter it, quit it again, fetch new
> > mail, and enter it again.
> 
> I haven't seen any issues with non-`/ N' IMAP group usage, though...
> 


So, I can reproduce this with a command in cron that emails me every minute.

echo "Test" | mail me@example.com -s "`date -u`"

Out of 116 test messages sent, 17 were marked O without me seeing them.

To reproduce it, it does not seem to be enough to just have the folder open 
while the mail is arriving. I think other activity, such as moving and replying 
to mail, is also necessary. But I haven't pinned that down yet.

In an hour while I was away from the computer with the folder open and the 
program running, no mails were marked O. But while actually working in the 
folder, those 17 were.

Re: New messages marked O before seeing them

[John Sullivan]

John Sullivan <john@wjsullivan.net> writes:

> Lars Magne Ingebrigtsen <larsi <at> gnus.org> writes:
>
>> 
>> John Sullivan <john <at> wjsullivan.net> writes:
>> 
>> > Not a for-sure recipe :/ Try sending yourself a few dozen messages with
>> > the folder open for an hour or so. During that time, delete a few
>> > messages, move a few messages to another group, read a few messages.
>> > Then / N. That's closer to the conditions under which I see it.
>> 
>> Could you try to provoke the bug with a simpler recipe?  
>> 
>> > Finally, I'd try the same thing as the first recipe, but instead of
>> > doing / N, just quit the group, re-enter it, quit it again, fetch new
>> > mail, and enter it again.
>> 
>> I haven't seen any issues with non-`/ N' IMAP group usage, though...
>> 
>
>
> So, I can reproduce this with a command in cron that emails me every minute.
>
> echo "Test" | mail me@example.com -s "`date -u`"
>
> Out of 116 test messages sent, 17 were marked O without me seeing them.
>
> To reproduce it, it does not seem to be enough to just have the folder open 
> while the mail is arriving. I think other activity, such as moving and replying 
> to mail, is also necessary. But I haven't pinned that down yet.
>
> In an hour while I was away from the computer with the folder open and the 
> program running, no mails were marked O. But while actually working in the 
> folder, those 17 were.

I have a sneaking suspicion that I may have solved this by setting
(setq gnus-gcc-mark-as-read nil). It was t before, and my mails are
GCC'd to the inbox.

My suspicion is that somehow the wrong messages are being marked as
read, targeting incoming mail instead of the Gcc'd mail just written.

Since I've made that change, I have seen no more new messages marked
read, despite receiving 1 a minute and using my inbox under
approximately the same conditions as the ones which produced the
negative effects before.

-john

Re: New messages marked O before seeing them

[John Sullivan]

John Sullivan <john@wjsullivan.net> writes:

> I have a sneaking suspicion that I may have solved this by setting
> (setq gnus-gcc-mark-as-read nil). It was t before, and my mails are
> GCC'd to the inbox.
>
> My suspicion is that somehow the wrong messages are being marked as
> read, targeting incoming mail instead of the Gcc'd mail just written.
>
> Since I've made that change, I have seen no more new messages marked
> read, despite receiving 1 a minute and using my inbox under
> approximately the same conditions as the ones which produced the
> negative effects before.

I'm quite sure at this point that the Gcc settings are the cause. So
excited to have traced this down!

I should have made this connection before, because I'd noticed that
sometimes, my Gcc'd messages were not getting marked as read, and I
thought that was weird. 

Now to fix it :). Any background information about how Gcc works would
be helpful. 

-john

Re: New messages marked O before seeing them

[Lars Magne Ingebrigtsen]

John Sullivan <john@wjsullivan.net> writes:

> I have a sneaking suspicion that I may have solved this by setting
> (setq gnus-gcc-mark-as-read nil). It was t before, and my mails are
> GCC'd to the inbox.

Oh, that makes sense, I think.  I mean, it makes sense that that is
something that could trigger a bug like this.  :-)

So how do you reproduce this bug?  If you have the inbox summary buffer
open, and you post a message (with `gnus-gcc-mark-as-read' set to t),
will other messages get marked as read?

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: New messages marked O before seeing them

[John Sullivan]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> John Sullivan <john@wjsullivan.net> writes:
>
>> I have a sneaking suspicion that I may have solved this by setting
>> (setq gnus-gcc-mark-as-read nil). It was t before, and my mails are
>> GCC'd to the inbox.
>
> Oh, that makes sense, I think.  I mean, it makes sense that that is
> something that could trigger a bug like this.  :-)
>
> So how do you reproduce this bug?  If you have the inbox summary buffer
> open, and you post a message (with `gnus-gcc-mark-as-read' set to t),
> will other messages get marked as read?

Yep.

Re: New messages marked O before seeing them

[Lars Magne Ingebrigtsen]

John Sullivan <john@wjsullivan.net> writes:

>> So how do you reproduce this bug?  If you have the inbox summary buffer
>> open, and you post a message (with `gnus-gcc-mark-as-read' set to t),
>> will other messages get marked as read?
>
> Yep.

It's vaguely possible that I've fixed this now.  I found some suspicious
handling of unselected articles when extending the range when marking as
read. 

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/