This bug is killing me!

This bug is killing me!

[Dave Abrahams]

This is very serious: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=9386GNU
As an emergency workaround, I've rebound `M-g' to `gnus-summary-exit'.
And a friend of mine remarked,

,----
| I never, ever, use M-g within a summary buffer.  I have seen enough
| unpredictable behavior with it through the years that I instinctively stopped
| trusting it.  Nowadays I exit the summary buffer, hit 'g', then go back in.
`----

How can it possibly be so hard to make `M-g' equivalent to the above
procedure?

-- 
Dave Abrahams
BoostPro Computing
http://www.boostpro.com

Re: This bug is killing me!

[Tassilo Horn]

Dave Abrahams <dave@boostpro.com> writes:

Hi Dave,

> This is very serious: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=9386GNU
> As an emergency workaround, I've rebound `M-g' to `gnus-summary-exit'.
> And a friend of mine remarked,
>
> ,----
> | I never, ever, use M-g within a summary buffer.  I have seen enough
> | unpredictable behavior with it through the years that I instinctively stopped
> | trusting it.  Nowadays I exit the summary buffer, hit 'g', then go back in.
> `----

Strange, I use M-g quite frequently and never had problems.  Concerning
your bug report, I've just ticked your article and hit M-g.  What I got
was a summary with your still ticked article plus the other unread
articles.  That's how it's supposed to work.

Reading your bug report, it seems that the mark is not propagated to the
server.  But even in that case, the mark should be stored locally (in
.newsrc.eld or whatever variable holds the marks at runtime).  So
probably the local marks not saved before exiting summary while
propagating marks to the remote server is done immediately, which would
explain your issue and why it works for me.

Could you check if it works for you if you enable propagating marks?

,----[ C-h v gnus-propagate-marks RET ]
| gnus-propagate-marks is a variable defined in `gnus-sum.el'.
| Its value is t
| Original value was nil
| 
| Documentation:
| If non-nil, Gnus will store and retrieve marks from the backends.
| This means that marks will be stored both in .newsrc.eld and in
| the backend, and will slow operation down somewhat.
| 
| You can customize this variable.
| 
| [back]
`----

BTW: I think, that the default of this variable should be t nowadays,
where it's pretty common that people use different clients for accessing
their mail (Gnus on "real" computers, Whatever on their phones, the web
interfaces of their mail providers, etc.).

Bye,
Tassilo
-- 
Sent from my Emacs

Re: This bug is killing me!

[Robert Pluim]

Tassilo Horn <tassilo@member.fsf.org> writes:

> Could you check if it works for you if you enable propagating marks?
>
> ,----[ C-h v gnus-propagate-marks RET ]
> | gnus-propagate-marks is a variable defined in `gnus-sum.el'.
> | Its value is t
> | Original value was nil
> | 
> | Documentation:
> | If non-nil, Gnus will store and retrieve marks from the backends.
> | This means that marks will be stored both in .newsrc.eld and in
> | the backend, and will slow operation down somewhat.
> | 
> | You can customize this variable.
> | 
> | [back]
> `----
>
> BTW: I think, that the default of this variable should be t nowadays,
> where it's pretty common that people use different clients for accessing
> their mail (Gnus on "real" computers, Whatever on their phones, the web
> interfaces of their mail providers, etc.).


I just tried setting that to true, and all the unread counts on my nntp
groups (which I access only via Gnus), became wrong (as in much too
high). Does this mean there is stale info in the backend that I need to
regenerate or delete?

Robert

[Workaround/Solved] This bug is killing me!

[Dave Abrahams]

[long story short, you _need_ gnus-propagate-marks on]

on Mon Aug 29 2011, Tassilo Horn <tassilo-AT-member.fsf.org> wrote:

> Dave Abrahams <dave@boostpro.com> writes:
>
> Hi Dave,
>
>> This is very serious: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=9386GNU> As an emergency workaround, I've rebound `M-g' to `gnus-summary-exit'.
>> And a friend of mine remarked,
>>
>> ,----
>> | I never, ever, use M-g within a summary buffer.  I have seen enough
>> | unpredictable behavior with it through the years that I instinctively stopped
>> | trusting it.  Nowadays I exit the summary buffer, hit 'g', then go back in.
>> `----
>
> Strange, I use M-g quite frequently and never had problems.  Concerning
> your bug report, I've just ticked your article and hit M-g.  What I got
> was a summary with your still ticked article plus the other unread
> articles.  That's how it's supposed to work.
>
> Reading your bug report, it seems that the mark is not propagated to the
> server.  But even in that case, the mark should be stored locally (in
> .newsrc.eld or whatever variable holds the marks at runtime).  So
> probably the local marks not saved before exiting summary while
> propagating marks to the remote server is done immediately, which would
> explain your issue and why it works for me.
>
> Could you check if it works for you if you enable propagating marks?

Yes, it works!!  Not only that, but all the ticks on older articles that
used to persist in Gnus, but had suddenly disappeared from Gnus' view of
my server came back!

> BTW: I think, that the default of this variable should be t nowadays,
> where it's pretty common that people use different clients for accessing
> their mail (Gnus on "real" computers, Whatever on their phones, the web
> interfaces of their mail providers, etc.).

OMG, yes, please!!!

-- 
Dave Abrahams
BoostPro Computing
http://www.boostpro.com

Re: This bug is killing me!

[Tassilo Horn]

Robert Pluim <rpluim@gmail.com> writes:

Hi Robert,

>> ,----[ C-h v gnus-propagate-marks RET ]
>> | gnus-propagate-marks is a variable defined in `gnus-sum.el'.
>> | Its value is t
>> | Original value was nil
>> `----
>>
>> BTW: I think, that the default of this variable should be t nowadays,
>> where it's pretty common that people use different clients for
>> accessing their mail (Gnus on "real" computers, Whatever on their
>> phones, the web interfaces of their mail providers, etc.).
>
> I just tried setting that to true, and all the unread counts on my
> nntp groups (which I access only via Gnus), became wrong (as in much
> too high). Does this mean there is stale info in the backend that I
> need to regenerate or delete?

I don't have a clue.  But NNTP doesn't track marks at the server-side
anyway, so at least in theory, the value of that variable shouldn't have
any effect on newsgroups...

Did you change the value while gnus was running?  If so, maybe that had
some ill-effect...

And by the way: Lars changed the default for gnus-propagate-marks to t
with commit 4bf6dfbeba571f84b1da5c34d2423e3059814ccf on February, 16th,
but with his commit e5c3a381 on March, 5th it's back to nil again, but
instead an exception for IMAP is added to always propagate marks if the
server supports it.

So Lars, for what's gnus-propagate-marks good for now after this change?
IMAP is the only backend with server-marks anyway, isn't it?

And back to the original question.  Dave, what do you get when you
evaluate

  (gnus-method-option-p
    (gnus-find-method-for-group "nnimap+YourServer:INBOX")
    'server-marks)

If you get '(server-marks), then my guess that the problem is caused by
not propagating marks might be false (or some problem happens while
doing so...).

Bye,
Tassilo
-- 
Sent from my Emacs

Re: [Workaround/Solved] This bug is killing me!

[Tassilo Horn]

Dave Abrahams <dave@boostpro.com> writes:

Hi Dave,

>> Could you check if it works for you if you enable propagating marks?
>
> Yes, it works!!  Not only that, but all the ticks on older articles
> that used to persist in Gnus, but had suddenly disappeared from Gnus'
> view of my server came back!
>
>> BTW: I think, that the default of this variable should be t nowadays,
>> where it's pretty common that people use different clients for
>> accessing their mail (Gnus on "real" computers, Whatever on their
>> phones, the web interfaces of their mail providers, etc.).
>
> OMG, yes, please!!!

Please see my other message.  Propagation of marks to imap servers is
enabled by default no matter of that variable since gnus git versions
after March, 5th.  Do you use an older version?

(My git version of today also reports NoGnus v0.18, just as your
User-Agent header...)

So if you use an older version, then updating will help.  If you use a
recent gnus version, then we have to figure out why

  (gnus-method-option-p
   (gnus-find-method-for-group "nnimap+YourServer:INBOX")
   'server-marks)

returns nil.

Bye,
Tassilo

Re: [Workaround/Solved] This bug is killing me!

[Dave Abrahams]

on Tue Aug 30 2011, Tassilo Horn <tassilo-AT-member.fsf.org> wrote:

> Dave Abrahams <dave@boostpro.com> writes:
>
> Hi Dave,
>
>>> Could you check if it works for you if you enable propagating marks?
>>
>> Yes, it works!!  Not only that, but all the ticks on older articles
>> that used to persist in Gnus, but had suddenly disappeared from Gnus'
>> view of my server came back!
>>
>>> BTW: I think, that the default of this variable should be t nowadays,
>>> where it's pretty common that people use different clients for
>>> accessing their mail (Gnus on "real" computers, Whatever on their
>>> phones, the web interfaces of their mail providers, etc.).
>>
>> OMG, yes, please!!!
>
> Please see my other message.  

Which other message?

> Propagation of marks to imap servers is
> enabled by default no matter of that variable since gnus git versions
> after March, 5th.  Do you use an older version?

I'm on this version:

,----
| commit b7049858dc4463249d94613e05f6044cb5d70d6d 
| Author: Katsumi Yamaoka <yamaoka@jpl.org>
| Date:   Fri Aug 26 09:01:29 2011 +0000
`----

I have this additional change, which avoids pathological regexp
behavior:

--8<---------------cut here---------------start------------->8---
diff --git a/lisp/nnimap.el b/lisp/nnimap.el
index 2dbc465..5b7d253 100644
--- a/lisp/nnimap.el
+++ b/lisp/nnimap.el
@@ -190,7 +190,7 @@ textual parts.")
   (let (article bytes lines size string)
     (block nil
       (while (not (eobp))
-	(while (not (looking-at "\\* [0-9]+ FETCH.+UID \\([0-9]+\\)"))
+	(while (not (looking-at "\\* [0-9]+ FETCH.+?UID \\([0-9]+\\)"))
 	  (delete-region (point) (progn (forward-line 1) (point)))
 	  (when (eobp)
 	    (return)))
--8<---------------cut here---------------end--------------->8---

> (My git version of today also reports NoGnus v0.18, just as your
> User-Agent header...)
>
> So if you use an older version, then updating will help.  If you use a
> recent gnus version, then we have to figure out why
>
>   (gnus-method-option-p
>    (gnus-find-method-for-group "nnimap+YourServer:INBOX")
>    'server-marks)

Just let me know what I can do.

-- 
Dave Abrahams
BoostPro Computing
http://www.boostpro.com

Re: [Workaround/Solved] This bug is killing me!

[Dave Abrahams]

on Tue Aug 30 2011, Tassilo Horn <tassilo-AT-member.fsf.org> wrote:

> If you use a
> recent gnus version, then we have to figure out why
>
>   (gnus-method-option-p
>    (gnus-find-method-for-group "nnimap+YourServer:INBOX")
>    'server-marks)
>
> returns nil.

FWIW, it doesn't return nil, currently.  The result is

,----
| (server-marks)
`----


-- 
Dave Abrahams
BoostPro Computing
http://www.boostpro.com

Re: [Workaround/Solved] This bug is killing me!

[Tassilo Horn]

Dave Abrahams <dave@boostpro.com> writes:

Hi Dave,

>> Please see my other message.  
>
> Which other message?

  Message-ID: <8739gjqkb9.fsf@thinkpad.tsdh.de>

>> Propagation of marks to imap servers is enabled by default no matter
>> of that variable since gnus git versions after March, 5th.  Do you
>> use an older version?
>
> I'm on this version:
>
> ,----
> | commit b7049858dc4463249d94613e05f6044cb5d70d6d 
> | Author: Katsumi Yamaoka <yamaoka@jpl.org>
> | Date:   Fri Aug 26 09:01:29 2011 +0000
> `----

That should be recent enough.

> I have this additional change, which avoids pathological regexp
> behavior:
>
> diff --git a/lisp/nnimap.el b/lisp/nnimap.el
> index 2dbc465..5b7d253 100644
> --- a/lisp/nnimap.el
> +++ b/lisp/nnimap.el
> @@ -190,7 +190,7 @@ textual parts.")
>    (let (article bytes lines size string)
>      (block nil
>        (while (not (eobp))
> -	(while (not (looking-at "\\* [0-9]+ FETCH.+UID \\([0-9]+\\)"))
> +	(while (not (looking-at "\\* [0-9]+ FETCH.+?UID \\([0-9]+\\)"))
>  	  (delete-region (point) (progn (forward-line 1) (point)))
>  	  (when (eobp)
>  	    (return)))

Hm, so you use the non-greedy variant of ".+".  That should make a
difference only if there a lines like

  * 939393 FETCH something here UID 918823 UID 191929

i.e. the string UID followed by a number occurs more than once.  Not
sure if the imap spec allows that.  With the original regexp, the latter
UID would be captured, with your change, the former will be captured.


-- 
Sent from my Emacs

Re: [Workaround/Solved] This bug is killing me!

[Tassilo Horn]

Dave Abrahams <dave@boostpro.com> writes:

Hi Dave,

> FWIW, it doesn't return nil, currently.  The result is
>
> ,----
> | (server-marks)
> `----

Well, in that case, marks should be propagated no matter of the value of
gnus-propagate-marks...  Maybe edebugging `gnus-update-read-articles'
might shed some light.

Bye,
Tassilo
-- 
Sent from my Emacs

Re: [Workaround/Solved] This bug is killing me!

[James Cloos]

>>>>> "DA" == Dave Abrahams <dave@boostpro.com> writes:

DA> [long story short, you _need_ gnus-propagate-marks on]

>> BTW: I think, that the default of this variable should be t nowadays,

Mark propagation is not universally supported.  Setting it to t would
certainly break things here.

Gnus needs to work correcly with gnus-propagate-marks nil.

-JimC
-- 
James Cloos <cloos@jhcloos.com>         OpenPGP: 1024D/ED7DAEA6

Re: [Workaround/Solved] This bug is killing me!

[Dave Abrahams]

> Dave Abrahams <dave@boostpro.com> writes:
>
> Hi Dave,
>
>>> Please see my other message.  
>>
>> Which other message?
>
>   Message-ID: <8739gjqkb9.fsf@thinkpad.tsdh.de>

Ah, missed that one, thanks.

>>> Propagation of marks to imap servers is enabled by default no matter
>>> of that variable since gnus git versions after March, 5th.  Do you
>>> use an older version?
>>
>> I'm on this version:
>>
>> ,----
>> | commit b7049858dc4463249d94613e05f6044cb5d70d6d 
>> | Author: Katsumi Yamaoka <yamaoka@jpl.org>
>> | Date:   Fri Aug 26 09:01:29 2011 +0000
>> `----
>
> That should be recent enough.
>
>> I have this additional change, which avoids pathological regexp
>> behavior:
>>
>> diff --git a/lisp/nnimap.el b/lisp/nnimap.el
>> index 2dbc465..5b7d253 100644
>> --- a/lisp/nnimap.el
>> +++ b/lisp/nnimap.el
>> @@ -190,7 +190,7 @@ textual parts.")
>>    (let (article bytes lines size string)
>>      (block nil
>>        (while (not (eobp))
>> -	(while (not (looking-at "\\* [0-9]+ FETCH.+UID \\([0-9]+\\)"))
>> +	(while (not (looking-at "\\* [0-9]+ FETCH.+?UID \\([0-9]+\\)"))
>>  	  (delete-region (point) (progn (forward-line 1) (point)))
>>  	  (when (eobp)
>>  	    (return)))
>
> Hm, so you use the non-greedy variant of ".+".  That should make a
> difference only if there a lines like
>
>   * 939393 FETCH something here UID 918823 UID 191929
>
> i.e. the string UID followed by a number occurs more than once.  

Not necessarily.  On a really long line the original has to search to
the end before it can accept the match at the beginning, even if the
string appears only once.  I reported this bug 9 days ago; full details
are here: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=9338

-- 
Dave Abrahams
BoostPro Computing
http://www.boostpro.com

Re: [Workaround/Solved] This bug is killing me!

[Tassilo Horn]

Dave Abrahams <dave@boostpro.com> writes:

>>> I have this additional change, which avoids pathological regexp
>>> behavior:
>>>
>>> diff --git a/lisp/nnimap.el b/lisp/nnimap.el
>>> index 2dbc465..5b7d253 100644
>>> --- a/lisp/nnimap.el
>>> +++ b/lisp/nnimap.el
>>> @@ -190,7 +190,7 @@ textual parts.")
>>>    (let (article bytes lines size string)
>>>      (block nil
>>>        (while (not (eobp))
>>> -	(while (not (looking-at "\\* [0-9]+ FETCH.+UID \\([0-9]+\\)"))
>>> +	(while (not (looking-at "\\* [0-9]+ FETCH.+?UID \\([0-9]+\\)"))
>>>  	  (delete-region (point) (progn (forward-line 1) (point)))
>>>  	  (when (eobp)
>>>  	    (return)))
>>
>> Hm, so you use the non-greedy variant of ".+".  That should make a
>> difference only if there a lines like
>>
>>   * 939393 FETCH something here UID 918823 UID 191929
>>
>> i.e. the string UID followed by a number occurs more than once.  
>
> Not necessarily.  On a really long line the original has to search to
> the end before it can accept the match at the beginning, even if the
> string appears only once.  I reported this bug 9 days ago; full details
> are here: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=9338

Thanks.  In fact, I can reproduce that stack overflow here.

Bye,
Tassilo

Re: [Workaround/Solved] This bug is killing me!

[Dave Abrahams]

> Dave Abrahams <dave@boostpro.com> writes:
>
> Hi Dave,
>
>> FWIW, it doesn't return nil, currently.  The result is
>>
>> ,----
>> | (server-marks)
>> `----
>
> Well, in that case, marks should be propagated no matter of the value of
> gnus-propagate-marks...  Maybe edebugging `gnus-update-read-articles'
> might shed some light.

That function, being large-ish and totally unfamiliar to me, is more
than I can afford to take on right now, at least without more guidance,
as long as I have a workaround.

-- 
Dave Abrahams
BoostPro Computing
http://www.boostpro.com

Re: [Workaround/Solved] This bug is killing me!

[Dave Abrahams]

>>>>>> "DA" == Dave Abrahams <dave@boostpro.com> writes:
>
> DA> [long story short, you _need_ gnus-propagate-marks on]
>
>>> BTW: I think, that the default of this variable should be t nowadays,
>
> Mark propagation is not universally supported.  

By what? (not challenging; just not understanding yet).

> Setting it to t would certainly break things here.

Where; what things?

> Gnus needs to work correcly with gnus-propagate-marks nil.

Perhaps so.  Can you define what "correctly" means in this case?

Thanks,

-- 
Dave Abrahams
BoostPro Computing
http://www.boostpro.com

Re: [Workaround/Solved] This bug is killing me!

[Tassilo Horn]

Dave Abrahams <dave@boostpro.com> writes:

>>>> BTW: I think, that the default of this variable should be t
>>>> nowadays,
>>
>> Mark propagation is not universally supported.  
>
> By what? (not challenging; just not understanding yet).

By backends: Server-side marks are something that's basically only
supported by IMAP (and thus the nnimap backend).

>> Setting it to t would certainly break things here.
>
> Where; what things?

I don't know the answer but have another question: if it would break
things, why is it a defcustom then?  And if only nnimap supports mark
propagation, and the code already has an exception to enable MP for IMAP
servers (*), why do we have that variable anyway?

(*) ... which doesn't seem to work for Dave ...

>> Gnus needs to work correcly with gnus-propagate-marks nil.
>
> Perhaps so.  Can you define what "correctly" means in this case?

Marks should be propagated to your IMAP server, no matter the value of
that variable.  And what we've discovered so far is that this seems to
happen when you mark an article and then exit the summary, while the
mark is lost when you hit M-g, thus updating the summary without exiting
it.

Well, that's at least some perimeter of the issue.  I have no clue about
all that backend code, but hopefully someone who has is helped on with
it.

Bye,
Tassilo
-- 
Sent from my Emacs

Re: [Workaround/Solved] This bug is killing me!

[Dave Abrahams]

> Dave Abrahams <dave@boostpro.com> writes:
>
>>>>> BTW: I think, that the default of this variable should be t
>>>>> nowadays,
>>>
>>> Mark propagation is not universally supported.  
>>
>> By what? (not challenging; just not understanding yet).
>
> By backends: Server-side marks are something that's basically only
> supported by IMAP (and thus the nnimap backend).

Sure... but presumably having it set to t wouldn't hurt anything in the
case where no propagation was possible.

>>> Setting it to t would certainly break things here.
>>
>> Where; what things?
>
> I don't know the answer but have another question: if it would break
> things, why is it a defcustom then?  And if only nnimap supports mark
> propagation, and the code already has an exception to enable MP for IMAP
> servers (*), why do we have that variable anyway?
>
> (*) ... which doesn't seem to work for Dave ...
>
>>> Gnus needs to work correcly with gnus-propagate-marks nil.
>>
>> Perhaps so.  Can you define what "correctly" means in this case?
>
> Marks should be propagated to your IMAP server, no matter the value of
> that variable.  And what we've discovered so far is that this seems to
> happen when you mark an article and then exit the summary, while the
> mark is lost when you hit M-g, thus updating the summary without exiting
> it.

[I note here that the docs for `M-g' say it exits the summary and
re-enters it!]

-- 
Dave Abrahams
BoostPro Computing
http://www.boostpro.com

Re: [Workaround/Solved] This bug is killing me!

[James Cloos]

>>>>> "DA" == Dave Abrahams <dave@boostpro.com> writes:

DA> By what? (not challenging; just not understanding yet).

I use dbmail.  It only supports seen, answered, deleted, flagged, recent
and draft.  Notably there is no server-side flag to match with tick.

>> Setting it to t would certainly break things here.

DA> Where; what things?

Gnus needs to keep marks other than the above in the .newsrc.el file; if
it only tries to store them remotely it will fail.

>> Gnus needs to work correcly with gnus-propagate-marks nil.

DA> Perhaps so.  Can you define what "correctly" means in this case?

Store the flags in .newsrc.eld and use them.  As it is, ticked articles
confuse the unread counts in Group enough that gnus even tries to open
the group when it should pass over.  (There is some variation depending
on which version of gnus I was running when each article was ticked.)

Everything worked fine, incidently, until gnus started trying to store
everything on the imap server.

-JimC
-- 
James Cloos <cloos@jhcloos.com>         OpenPGP: 1024D/ED7DAEA6

Re: [The saga continues...] This bug is killing me!

[Dave Abrahams]

on Tue Aug 30 2011, Tassilo Horn <tassilo-AT-member.fsf.org> wrote:

> Dave Abrahams <dave@boostpro.com> writes:
>
>>>>> BTW: I think, that the default of this variable should be t
>>>>> nowadays,
>>>
>>> Mark propagation is not universally supported.  
>>
>> By what? (not challenging; just not understanding yet).
>
> By backends: Server-side marks are something that's basically only
> supported by IMAP (and thus the nnimap backend).
>
>>> Setting it to t would certainly break things here.
>>
>> Where; what things?
>
> I don't know the answer but have another question: if it would break
> things, why is it a defcustom then?  And if only nnimap supports mark
> propagation, and the code already has an exception to enable MP for IMAP
> servers (*), why do we have that variable anyway?
>
> (*) ... which doesn't seem to work for Dave ...
>
>>> Gnus needs to work correcly with gnus-propagate-marks nil.
>>
>> Perhaps so.  Can you define what "correctly" means in this case?
>
> Marks should be propagated to your IMAP server, no matter the value of
> that variable.  And what we've discovered so far is that this seems to
> happen when you mark an article and then exit the summary, while the
> mark is lost when you hit M-g, thus updating the summary without exiting
> it.
>
> Well, that's at least some perimeter of the issue.  I have no clue about
> all that backend code, but hopefully someone who has is helped on with
> it.

Okay, so I just exited Gnus and decided to answer `n' to the "update
summary buffer INBOX" question.  Restarted Emacs and Gnus and nearly all
my marks were gone, even in some NNTP groups.  So I set (and saved)
`gnus-propagate-marks' back to nil, and did a `M-g', and all my marks
came back.  

something-is-rotten-in-the-state-of-Denmark-ly y'rs,

-- 
Dave Abrahams
BoostPro Computing
http://www.boostpro.com

Re: [Workaround/Solved] This bug is killing me!

[Dave Abrahams]

on Tue Aug 30 2011, James Cloos <cloos-AT-jhcloos.com> wrote:

>>>>>> "DA" == Dave Abrahams <dave@boostpro.com> writes:
>
> DA> By what? (not challenging; just not understanding yet).
>
> I use dbmail.  It only supports seen, answered, deleted, flagged, recent
> and draft.  Notably there is no server-side flag to match with tick.

I thought that was "flagged"(?)

>>> Setting it to t would certainly break things here.
>
> DA> Where; what things?
>
> Gnus needs to keep marks other than the above in the .newsrc.el file; if
> it only tries to store them remotely it will fail.

IIUC gnus-propagate-marks is supposed to store them in both places.  At
least, that's what the docstring clearly implies:

,----[ C-h v gnus-propagate-marks RET ]
| gnus-propagate-marks is a variable defined in `gnus-sum.el'.
| Its value is nil
| 
| Documentation:
| If non-nil, Gnus will store and retrieve marks from the backends.
| This means that marks will be stored both in .newsrc.eld and in
| the backend, and will slow operation down somewhat.
| 
| You can customize this variable.
| 
| [back]
`----

>>> Gnus needs to work correcly with gnus-propagate-marks nil.
>
> DA> Perhaps so.  Can you define what "correctly" means in this case?
>
> Store the flags in .newsrc.eld and use them.  As it is, ticked articles
> confuse the unread counts in Group enough that gnus even tries to open
> the group when it should pass over.  (There is some variation depending
> on which version of gnus I was running when each article was ticked.)
>
> Everything worked fine, incidently, until gnus started trying to store
> everything on the imap server.

Huh.  Well, for you, I guess.  My friend had a different experience.
There really ought to be some kind of testing framework for Gnus,
shouldn't there?  I suppose someone has already been thinking about
this in more depth...(?)

-- 
Dave Abrahams
BoostPro Computing
http://www.boostpro.com

Re: [Workaround/Solved] This bug is killing me!

[Tassilo Horn]

James Cloos <cloos@jhcloos.com> writes:

Hi James,

> DA> By what? (not challenging; just not understanding yet).
>
> I use dbmail.  It only supports seen, answered, deleted, flagged,
> recent and draft.  Notably there is no server-side flag to match with
> tick.

The marks you've listed are exactly those predefined by the IMAP spec.
Looking at nnimap.el, Gnus translates the tick mark to \Flagged.

Bye,
Tassilo
-- 
Sent from my Emacs

Re: [Workaround/Solved] This bug is killing me!

[James Cloos]

>>>>> "TH" == Tassilo Horn <tassilo@member.fsf.org> writes:

TH> The marks you've listed are exactly those predefined by the IMAP spec.
TH> Looking at nnimap.el, Gnus translates the tick mark to \Flagged.

A thought-o.  Instead of the ! flag I meant the ? flag.

-JimC
-- 
James Cloos <cloos@jhcloos.com>         OpenPGP: 1024D/ED7DAEA6

Re: This bug is killing me!

[Andrew Cohen]

>>>>> "Dave" == Dave Abrahams <dave@boostpro.com> writes:

    Dave> This is very serious:
    Dave> http://debbugs.gnu.org/cgi/bugreport.cgi?bug=9386GNUAs an
 


I took a minute to look at this and think I have a fix. However I'm not
that familiar with this code and it might break something else. Any
brave souls willing to test it out, just try this replacement function
for `gnus-summary-insert-articles'. (You can just replace it on the fly;
no need to restart gnus. Put in scratch buffer and evaluate). 

(defun gnus-summary-insert-articles (articles)
  (when (setq articles
	      (gnus-sorted-difference articles
				      (mapcar (lambda (h)
						(mail-header-number h))
					      gnus-newsgroup-headers)))
    (setq gnus-newsgroup-headers
	  (gnus-merge 'list
		      gnus-newsgroup-headers
		      (gnus-fetch-headers articles)
		      'gnus-article-sort-by-number))
    (setq gnus-newsgroup-articles 
	  (gnus-merge 'list gnus-newsgroup-articles articles '<))
    ;; Suppress duplicates?
    (when gnus-suppress-duplicates
      (gnus-dup-suppress-articles))

    (if (and gnus-fetch-old-headers
	     (eq gnus-headers-retrieved-by 'nov))
	;; We might want to build some more threads first.
	(if (eq gnus-fetch-old-headers 'invisible)
	    (gnus-build-all-threads)
	  (gnus-build-old-threads))
      ;; Mark the inserted articles that are unread as unread.
      (setq gnus-newsgroup-unreads
	    (gnus-sorted-nunion
	     gnus-newsgroup-unreads
	     (gnus-sorted-nintersection
	      (gnus-list-of-unread-articles gnus-newsgroup-name)
	      articles)))
      ;; Mark the inserted articles as selected so that the information
      ;; of the marks having been changed by a user may be updated when
      ;; exiting this group.  See `gnus-summary-update-info'.
      (dolist (art articles)
	(setq gnus-newsgroup-unselected (delq art gnus-newsgroup-unselected))))
    ;; Let the Gnus agent mark articles as read.
    (when gnus-agent
      (gnus-agent-get-undownloaded-list))
    ;; Remove list identifiers from subject
    (gnus-summary-remove-list-identifiers)
    ;; First and last article in this newsgroup.
    (when gnus-newsgroup-headers
      (setq gnus-newsgroup-begin
	    (mail-header-number (car gnus-newsgroup-headers))
	    gnus-newsgroup-end
	    (mail-header-number
	     (gnus-last-element gnus-newsgroup-headers))))
    (when gnus-use-scoring
      (gnus-possibly-score-headers))))

Re: This bug is killing me!

[Dave Abrahams]

on Wed Aug 31 2011, Andrew Cohen <cohen-AT-andy.bu.edu> wrote:

>>>>>> "Dave" == Dave Abrahams <dave@boostpro.com> writes:
>
>     Dave> This is very serious:
>     Dave> http://debbugs.gnu.org/cgi/bugreport.cgi?bug=9386GNUAs an
>
> I took a minute to look at this and think I have a fix. However I'm not
> that familiar with this code and it might break something else. Any
> brave souls willing to test it out, just try this replacement function
> for `gnus-summary-insert-articles'. (You can just replace it on the fly;
> no need to restart gnus. Put in scratch buffer and evaluate). 

Thanks for your work on this.  It looks like a fairly low-risk change,
so I'll try it out.  And maybe it's a necessary bug fix for other
reasons, but still... it seems like a function
(gnus-summary-rescan-group) whose documentation says it exits the group
and re-enters it should be written to do exactly that.  Am I missing
something?

Dave

-- 
Dave Abrahams
BoostPro Computing
http://www.boostpro.com

Re: This bug is killing me!

[Andrew Cohen]

>>>>> "Dave" == Dave Abrahams <dave@boostpro.com> writes:

    Dave> Thanks for your work on this.  It looks like a fairly low-risk
    Dave> change, so I'll try it out.  And maybe it's a necessary bug
    Dave> fix for other reasons, but still... it seems like a function
    Dave> (gnus-summary-rescan-group) whose documentation says it exits
    Dave> the group and re-enters it should be written to do exactly
    Dave> that.  Am I missing something?

I can't say for sure. The function does indeed exit the group and
re-enter, but in between it checks for message updates (which is the
'rescan' bit).  This checking is significantly different from what you
would get just by hitting 'g' in the *Group* buffer (aside from the
trivial difference that it acts only on the current group). Why exactly
its a different function I can't answer, but it is significantly more
aggressive in what information about the group it tries to rescan.

The specific bug you reported should be fixed by my change---that is,
change a mark, execute 'M-g', and have the new mark persist. The issue
was just that the marks weren't being propagated to the backend (due to
a slightly subtle chain of events). The other issues that have been
reported in this thread may or may not be affected; I can't say because
I can't reproduce them (unlike the original bug report you made which
was easy to reproduce).

Re: [Workaround/Solved] This bug is killing me!

[Lars Magne Ingebrigtsen]

Dave Abrahams <dave@boostpro.com> writes:

> I have this additional change, which avoids pathological regexp
> behavior:

Thanks; applied.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: [The saga continues...] This bug is killing me!

[Lars Magne Ingebrigtsen]

Dave Abrahams <dave@boostpro.com> writes:

> Okay, so I just exited Gnus and decided to answer `n' to the "update
> summary buffer INBOX" question.  Restarted Emacs and Gnus and nearly all
> my marks were gone, even in some NNTP groups.  So I set (and saved)
> `gnus-propagate-marks' back to nil, and did a `M-g', and all my marks
> came back.  

If you have `gnus-propagate-marks' set to t, the backends will store
marks themselves, and Gnus will retrieve that data.  (For nntp it's
stored in a file.)  If you twiddle the value on and off and on, Gnus
will, of course, use the marks from the backend, which will, by that
time, be out of date.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: This bug is killing me!

[Lars Magne Ingebrigtsen]

Andrew Cohen <cohen@andy.bu.edu> writes:

> I took a minute to look at this and think I have a fix. However I'm not
> that familiar with this code and it might break something else. Any
> brave souls willing to test it out, just try this replacement function
> for `gnus-summary-insert-articles'. (You can just replace it on the fly;
> no need to restart gnus. Put in scratch buffer and evaluate). 

Could you send a patch?

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: This bug is killing me!

[Lars Magne Ingebrigtsen]

Andrew Cohen <cohen@andy.bu.edu> writes:

> I already pushed a fix a little over a week ago:
>
> commit	f6afef21c74b8654c40348fa0d8655369884d7df

Great.  :-)

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: This bug is killing me!

[Andrew Cohen]

>>>>> "Lars" == Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

    Lars> Andrew Cohen <cohen@andy.bu.edu> writes:
    >> I took a minute to look at this and think I have a fix. However
    >> I'm not that familiar with this code and it might break something
    >> else. Any brave souls willing to test it out, just try this
    >> replacement function for `gnus-summary-insert-articles'. (You can
    >> just replace it on the fly; no need to restart gnus. Put in
    >> scratch buffer and evaluate).

    Lars> Could you send a patch?

I already pushed a fix a little over a week ago:

commit	f6afef21c74b8654c40348fa0d8655369884d7df

A.