HEADS UP! Unplugged (agent) marks handling changed

HEADS UP! Unplugged (agent) marks handling changed

[Kevin Greiner]

For some months now, individuals have been reporting that marks set
while unplugged disappeared when they synchronized with the backend.
The reports usually had to do with read marks in nntp groups and,
since the nntp backend was recently updated to support marks, I
generally concluded that these were due to operator error.  That
conclusion was even more certain when I was never able to reproduce
the problem.

As it turns out, these people were correct.  I've now updated both
gnus-agent and nnagent to address the problem.  However, this fix
impacts all backends while I've only tested with nntp.

If you run into any unexpected behavior with offline flags, please
post so that I can address it.

Kevin

Re: HEADS UP! Unplugged (agent) marks handling changed

[Ralf Angeli]

* Kevin Greiner (2004-09-25) writes:

> If you run into any unexpected behavior with offline flags, please
> post so that I can address it.

Hm, it still seems to forget the marks.  So the behavior hasn't
changed in my case.

I tried to debug `gnus-agent-possibly-synchronize-flags' again and I
saw the following oddity:

Debugger entered--returning value: (nnnil)
  gnus-find-method-for-group("comp.text.tex")
* gnus-request-set-mark("comp.text.tex" (((311864) add (read))))
* gnus-agent-synchronize-group-flags("comp.text.tex" (((311864) add (read))) "news.individual.net")
* eval((gnus-agent-synchronize-group-flags "comp.text.tex" (quote (...)) "news.individual.net"))
[byte-code snipped]
* gnus-agent-synchronize-flags-server((nntp "news.individual.net"))
* gnus-agent-possibly-synchronize-flags-server((nntp "news.individual.net"))
[byte-code snipped]
* gnus-agent-possibly-synchronize-flags()

`nnil' is my primary select method, but "comp.text.tex" is used with
an NNTP server.  I haven't looked deeper into the code, but I would
have expected that `gnus-find-method-for-group' returns something like
`nntp'.  The return value of `nnil' is a result of `(gnus-get-info
"comp.text.tex")' returning nil.  If this is unrelated to the problem
please ignore my babbling.

-- 
Ralf

Re: HEADS UP! Unplugged (agent) marks handling changed

[Kevin Greiner]

Ralf Angeli <dev.null@iwi.uni-sb.de> writes:

> * Kevin Greiner (2004-09-25) writes:
>
>> If you run into any unexpected behavior with offline flags, please
>> post so that I can address it.
>
> Hm, it still seems to forget the marks.  So the behavior hasn't
> changed in my case.
>
> I tried to debug `gnus-agent-possibly-synchronize-flags' again and I
> saw the following oddity:
>
> Debugger entered--returning value: (nnnil)
>   gnus-find-method-for-group("comp.text.tex")
> * gnus-request-set-mark("comp.text.tex" (((311864) add (read))))
> * gnus-agent-synchronize-group-flags("comp.text.tex" (((311864) add (read))) "news.individual.net")
> * eval((gnus-agent-synchronize-group-flags "comp.text.tex" (quote (...)) "news.individual.net"))
> [byte-code snipped]
> * gnus-agent-synchronize-flags-server((nntp "news.individual.net"))
> * gnus-agent-possibly-synchronize-flags-server((nntp "news.individual.net"))
> [byte-code snipped]
> * gnus-agent-possibly-synchronize-flags()
>
> `nnil' is my primary select method, but "comp.text.tex" is used with
> an NNTP server.  I haven't looked deeper into the code, but I would
> have expected that `gnus-find-method-for-group' returns something like
> `nntp'.  The return value of `nnil' is a result of `(gnus-get-info
> "comp.text.tex")' returning nil.  If this is unrelated to the problem
> please ignore my babbling.

Babbling nothing.  You are right on track.  I rewrote
nnagent-request-set-mark to address this issue.  Apparently, the
problem goes deeper.  Try debugging nnagent-request-set-mark.  It's
called as you exit a summary buffer while unplugged.  The issue is
what does gnus-method-to-server do?  It's being called with your
method (nntp "news.individual.net" ...).  I expected it to return
"nntp:news.individual.net" but it appears to have returned
"news.individual.net".  Can you debug it to see what is going on?

Kevin

Re: HEADS UP! Unplugged (agent) marks handling changed

[Ralf Angeli]

* Kevin Greiner (2004-09-26) writes:

> Babbling nothing.  You are right on track.  I rewrote
> nnagent-request-set-mark to address this issue.  Apparently, the
> problem goes deeper.  Try debugging nnagent-request-set-mark.  It's
> called as you exit a summary buffer while unplugged.  The issue is
> what does gnus-method-to-server do?  It's being called with your
> method (nntp "news.individual.net" ...).  I expected it to return
> "nntp:news.individual.net" but it appears to have returned
> "news.individual.net".  Can you debug it to see what is going on?

I can see that `gnus-method-to-server' is being called with `(nntp
"news.individual.net")', so no problem here.

Inside the function it finds a match in the first `mapc' loop (right
after the "Perhaps it is already in the cache." comment).  The test is:

Debugger entered--returning value: t
  equal((nntp "news.individual.net") (nntp "news.individual.net"))

The variable `name-method' has a value of `("news.individual.net" nntp
"news.individual.net")' at this point.  That means that the `(car
name-method)' in the `throw' statement will return
"news.individual.net":

Debugger entered--returning value: "news.individual.net"
  car(("news.individual.net" nntp "news.individual.net"))
* (throw (quote server-name) (car name-method))

-- 
Ralf

Re: HEADS UP! Unplugged (agent) marks handling changed

[Ralf Angeli]

* Ralf Angeli (2004-09-26) writes:

> The variable `name-method' has a value of `("news.individual.net" nntp
> "news.individual.net")' at this point.  That means that the `(car
> name-method)' in the `throw' statement will return
> "news.individual.net":
>
> Debugger entered--returning value: "news.individual.net"
>   car(("news.individual.net" nntp "news.individual.net"))
> * (throw (quote server-name) (car name-method))

Maybe I should add that the value of `gnus-agent-covered-methods' is
`("news.individual.net" "news.gmane.org")' which is the same as the
content of the file `~/News/Agent/lib/servers'.

-- 
Ralf

Re: HEADS UP! Unplugged (agent) marks handling changed

[Kevin Greiner]

Ralf Angeli <dev.null@iwi.uni-sb.de> writes:

> * Ralf Angeli (2004-09-26) writes:
>
>> The variable `name-method' has a value of `("news.individual.net" nntp
>> "news.individual.net")' at this point.  That means that the `(car
>> name-method)' in the `throw' statement will return
>> "news.individual.net":
>>
>> Debugger entered--returning value: "news.individual.net"
>>   car(("news.individual.net" nntp "news.individual.net"))
>> * (throw (quote server-name) (car name-method))
>
> Maybe I should add that the value of `gnus-agent-covered-methods' is
> `("news.individual.net" "news.gmane.org")' which is the same as the
> content of the file `~/News/Agent/lib/servers'.

Great work.  It looks like the problem is in the gnus-server-to-method
call in gnus-agent-synchronize-group-flags.  Can you add a (debug)
expression to gnus-agent-synchronize-group-flags then try to
synchronize flags?  What I'm trying to understand is why
gnus-server-to-method returned nil when passed "news.individual.net"
as an argument.

Kevin

Re: HEADS UP! Unplugged (agent) marks handling changed

[Ralf Angeli]

* Kevin Greiner (2004-09-27) writes:

> Great work.  It looks like the problem is in the gnus-server-to-method
> call in gnus-agent-synchronize-group-flags.  Can you add a (debug)
> expression to gnus-agent-synchronize-group-flags then try to
> synchronize flags?  What I'm trying to understand is why
> gnus-server-to-method returned nil when passed "news.individual.net"
> as an argument.

`(gnus-server-to-method "news.individual.net")' doesn't return nil, it
returns `("news.individual.net" nntp "news.individual.net")':

Debugger entered--returning value: ("news.individual.net" nntp "news.individual.net")
  assoc("news.individual.net" ([IMAP server snipped] ("archive" nnfolder "archive" (nnfolder-directory "~/Mail/archive") (nnfolder-active-file "~/Mail/archive/active") (nnfolder-get-new-mail nil) (nnfolder-inhibit-expiry t)) ("native" nnnil) ("nndraft" nndraft "") ("news.gmane.org" nntp "news.gmane.org") ("news.individual.net" nntp "news.individual.net")))
  gnus-agent-synchronize-group-flags("de.comp.text.tex" (((...) add (read))) "news.individual.net")

-- 
Ralf

Re: HEADS UP! Unplugged (agent) marks handling changed

[Kevin Greiner]

Ralf Angeli <dev.null@iwi.uni-sb.de> writes:

> * Kevin Greiner (2004-09-27) writes:
>
>> Great work.  It looks like the problem is in the gnus-server-to-method
>> call in gnus-agent-synchronize-group-flags.  Can you add a (debug)
>> expression to gnus-agent-synchronize-group-flags then try to
>> synchronize flags?  What I'm trying to understand is why
>> gnus-server-to-method returned nil when passed "news.individual.net"
>> as an argument.
>
> `(gnus-server-to-method "news.individual.net")' doesn't return nil, it
> returns `("news.individual.net" nntp "news.individual.net")':
>
> Debugger entered--returning value: ("news.individual.net" nntp "news.individual.net")
>   assoc("news.individual.net" ([IMAP server snipped] ("archive" nnfolder "archive" (nnfolder-directory "~/Mail/archive") (nnfolder-active-file "~/Mail/archive/active") (nnfolder-get-new-mail nil) (nnfolder-inhibit-expiry t)) ("native" nnnil) ("nndraft" nndraft "") ("news.gmane.org" nntp "news.gmane.org") ("news.individual.net" nntp "news.individual.net")))
>   gnus-agent-synchronize-group-flags("de.comp.text.tex" (((...) add (read))) "news.individual.net")

Hmmm...  I was actually interested in com.text.tex which you reported
on 25 Sept as returning null.

Kevin

Re: HEADS UP! Unplugged (agent) marks handling changed

[Ralf Angeli]

* Kevin Greiner (2004-09-28) writes:

> Ralf Angeli <dev.null@iwi.uni-sb.de> writes:
>
>> * Kevin Greiner (2004-09-27) writes:
>>
>>> Great work.  It looks like the problem is in the gnus-server-to-method
>>> call in gnus-agent-synchronize-group-flags.  Can you add a (debug)
>>> expression to gnus-agent-synchronize-group-flags then try to
>>> synchronize flags?  What I'm trying to understand is why
>>> gnus-server-to-method returned nil when passed "news.individual.net"
>>> as an argument.
>>
>> `(gnus-server-to-method "news.individual.net")' doesn't return nil, it
>> returns `("news.individual.net" nntp "news.individual.net")':
[...]
> Hmmm...  I was actually interested in com.text.tex which you reported
> on 25 Sept as returning null.

Then you probably mean, why `gnus-get-info' returns nil.

That's because `(intern-soft "comp.text.tex" gnus-newsrc-hashtb)'
returns nil.  `gnus-newsrc-hashtb' looks like

[[...] 0 0 0 0 0 0 nntp+news\.individual\.net:comp\.text\.tex 0 [...]]

So it seems that "comp.text.tex" doesn't match this.

The variable `gnus-newsrc-alist' which is used to build
`gnus-newsrc-hashtb' is set via the `(gnus-load ding-file)' call in
the function `gnus-read-newsrc-el-file'.  (BTW, there is a variable
`gnus-newsrc-assoc' let-bound in this function which is never set, so
this seems a bit useless.)  And in ~/.newsrc.eld the groups are
specified in the long form "nntp+news.individual.net:de.comp.text.tex".

That means either the `gnus-get-info' call has to use the long form as
its argument or the groups in the ~/.newsrc.eld file have to be
specified with their short forms.

-- 
Ralf

Re: HEADS UP! Unplugged (agent) marks handling changed

[Kevin Greiner]

Ralf Angeli <dev.null@iwi.uni-sb.de> writes:

> * Kevin Greiner (2004-09-28) writes:
>
>> Ralf Angeli <dev.null@iwi.uni-sb.de> writes:
>>
>>> * Kevin Greiner (2004-09-27) writes:
>>>
>>>> Great work.  It looks like the problem is in the gnus-server-to-method
>>>> call in gnus-agent-synchronize-group-flags.  Can you add a (debug)
>>>> expression to gnus-agent-synchronize-group-flags then try to
>>>> synchronize flags?  What I'm trying to understand is why
>>>> gnus-server-to-method returned nil when passed "news.individual.net"
>>>> as an argument.
>>>
>>> `(gnus-server-to-method "news.individual.net")' doesn't return nil, it
>>> returns `("news.individual.net" nntp "news.individual.net")':
> [...]
>> Hmmm...  I was actually interested in com.text.tex which you reported
>> on 25 Sept as returning null.
>
> Then you probably mean, why `gnus-get-info' returns nil.
>
> That's because `(intern-soft "comp.text.tex" gnus-newsrc-hashtb)'
> returns nil.  `gnus-newsrc-hashtb' looks like
>
> [[...] 0 0 0 0 0 0 nntp+news\.individual\.net:comp\.text\.tex 0 [...]]
>
> So it seems that "comp.text.tex" doesn't match this.
>
> The variable `gnus-newsrc-alist' which is used to build
> `gnus-newsrc-hashtb' is set via the `(gnus-load ding-file)' call in
> the function `gnus-read-newsrc-el-file'.  (BTW, there is a variable
> `gnus-newsrc-assoc' let-bound in this function which is never set, so
> this seems a bit useless.)  And in ~/.newsrc.eld the groups are
> specified in the long form "nntp+news.individual.net:de.comp.text.tex".
>
> That means either the `gnus-get-info' call has to use the long form as
> its argument or the groups in the ~/.newsrc.eld file have to be
> specified with their short forms.

The problem is that gnus-get-info isn't consistent.  Sometimes it
needs the full name other times the 'real' name.  I've updated yet
again, please give it a try.

Kevin

Re: HEADS UP! Unplugged (agent) marks handling changed

[Ralf Angeli]

* Kevin Greiner (2004-09-30) writes:

> The problem is that gnus-get-info isn't consistent.  Sometimes it
> needs the full name other times the 'real' name.  I've updated yet
> again, please give it a try.

It still forgets the marks.

In `gnus-agent-synchronize-group-flags' you are now computing a full
group name to be passed to `gnus-get-info' when it's called the second
time.  If this value is used instead of the short group name in the
call to `gnus-request-set-mark' as well, the marks will be updated.
That's because `(gnus-find-method-for-group group)' in
`gnus-request-set-mark' won't return '(nnil) anymore but '(nntp
"news.individual.net").

-- 
Ralf

Re: HEADS UP! Unplugged (agent) marks handling changed

[test]

Ralf Angeli a écrit :

> * Kevin Greiner (2004-09-30) writes:
> 
> 
>>The problem is that gnus-get-info isn't consistent.  Sometimes it
>>needs the full name other times the 'real' name.  I've updated yet
>>again, please give it a try.
> 
> 
> It still forgets the marks.
> 
> In `gnus-agent-synchronize-group-flags' you are now computing a full
> group name to be passed to `gnus-get-info' when it's called the second
> time.  If this value is used instead of the short group name in the
> call to `gnus-request-set-mark' as well, the marks will be updated.
> That's because `(gnus-find-method-for-group group)' in
> `gnus-request-set-mark' won't return '(nnil) anymore but '(nntp
> "news.individual.net").
> 
I confirm that. I still have lost marks with a checkout from today morning.

Re: HEADS UP! Unplugged (agent) marks handling changed

[Kevin Greiner]

test <invalig@toto.com> writes:

> Ralf Angeli a écrit :
>
>> * Kevin Greiner (2004-09-30) writes:
>> 
>>>The problem is that gnus-get-info isn't consistent.  Sometimes it
>>>needs the full name other times the 'real' name.  I've updated yet
>>>again, please give it a try.
>> It still forgets the marks.
>> In `gnus-agent-synchronize-group-flags' you are now computing a full
>> group name to be passed to `gnus-get-info' when it's called the second
>> time.  If this value is used instead of the short group name in the
>> call to `gnus-request-set-mark' as well, the marks will be updated.
>> That's because `(gnus-find-method-for-group group)' in
>> `gnus-request-set-mark' won't return '(nnil) anymore but '(nntp
>> "news.individual.net").
>> 
> I confirm that. I still have lost marks with a checkout from today morning.

OK, I've fixed the group being passed to gnus-request-set-mark.  How
are things now?

BTW, Lars reversed the default setting on
gnus-agent-synchronize-flags.  Flag synchronization is now turned OFF
until you customize this variable.

Kevin

Solved ? (was: HEADS UP! Unplugged (agent) marks handling changed)

[Xavier Maillard]

On  1 oct 2004, Kevin Greiner wrote:

> OK, I've fixed the group being passed to gnus-request-set-mark.
> How are things now?

\o/ Great ! At least, it is the first time read articles are
marked as read after plugged in !

 
> BTW, Lars reversed the default setting on
> gnus-agent-synchronize-flags. Flag synchronization is now
> turned OFF until you customize this variable.

That is to say ? I have to set it to t to synchronize ? I didn't
set it to 'ask or t but my articles have the right mark (at least
in one group).
 
Regards.
-- 
Xavier MAILLARD (GnuPG: 1024D/1E028EA5)
EmacsOS user (http://emacsfr.org)
APRIL (http://www.april.org)

Re: HEADS UP! Unplugged (agent) marks handling changed

[Ralf Angeli]

* Kevin Greiner (2004-10-01) writes:

> OK, I've fixed the group being passed to gnus-request-set-mark.  How
> are things now?

The "read" marks are working now.  Ticks like "!" don't work correctly
yet.

Steps to reproduce: Start Emacs/Gnus in plugged state, toggle to
unplugged state with `J j', enter a group where you don't have any
ticked articles and tick an article with `!'.  Close Gnus and emacs.
Start Emacs/Gnus in plugged state again.  A "*" won't show up in the
group buffer at the respective group.  Close Gnus and Emacs and start
them again.  The "*" will now show up.

> BTW, Lars reversed the default setting on
> gnus-agent-synchronize-flags.  Flag synchronization is now turned OFF
> until you customize this variable.

I never understood this change.  `ask' was the correct default IMO.
Anyway, I customized it to t.

-- 
Ralf

Re: HEADS UP! Unplugged (agent) marks handling changed

[Simon Josefsson]

Ralf Angeli <dev.null@iwi.uni-sb.de> writes:

>> BTW, Lars reversed the default setting on
>> gnus-agent-synchronize-flags.  Flag synchronization is now turned OFF
>> until you customize this variable.
>
> I never understood this change.  `ask' was the correct default IMO.
> Anyway, I customized it to t.

Presumably Lars doesn't like the interactivity of it.  Perhaps we
should simply change the default to t, then?

Re: HEADS UP! Unplugged (agent) marks handling changed

[Kevin Greiner]

Ralf Angeli <dev.null@iwi.uni-sb.de> writes:

> * Kevin Greiner (2004-10-01) writes:
>
>> OK, I've fixed the group being passed to gnus-request-set-mark.  How
>> are things now?
>
> The "read" marks are working now.  Ticks like "!" don't work correctly
> yet.
>
> Steps to reproduce: Start Emacs/Gnus in plugged state, toggle to
> unplugged state with `J j', enter a group where you don't have any
> ticked articles and tick an article with `!'.  Close Gnus and emacs.
> Start Emacs/Gnus in plugged state again.  A "*" won't show up in the
> group buffer at the respective group.  Close Gnus and Emacs and start
> them again.  The "*" will now show up.
>
>> BTW, Lars reversed the default setting on
>> gnus-agent-synchronize-flags.  Flag synchronization is now turned OFF
>> until you customize this variable.
>
> I never understood this change.  `ask' was the correct default IMO.
> Anyway, I customized it to t.

Tick marks are now handled correctly.

Kevin

Re: HEADS UP! Unplugged (agent) marks handling changed

[Ralf Angeli]

* Kevin Greiner (2004-10-01) writes:

> Tick marks are now handled correctly.

Yep.  Thank you very much.

-- 
Ralf

Re: HEADS UP! Unplugged (agent) marks handling changed

[test]

Kevin Greiner a écrit :

> Tick marks are now handled correctly.

Yeah, confirmed.

Thank you.

Display of quoted-printable (was: HEADS UP! Unplugged (agent) marks handling changed)

[Ralf Angeli]

* test (2004-09-30) writes:

> Ralf Angeli a =E9crit :
>
>> * Kevin Greiner (2004-09-30) writes:
>>=20
>>=20

Am I the only one who sees this quoted-printable stuff?  Is it a
problem on Gnus' side or is the original article incorrect?

-- 
Ralf

Re: Display of quoted-printable

[Katsumi Yamaoka]

>>>>> In <cjj2vb$c1$2@sea.gmane.org> Ralf Angeli wrote:

> * test (2004-09-30) writes:

>> Ralf Angeli a =E9crit :
>>
>>> * Kevin Greiner (2004-09-30) writes:
>>>=20
>>>=20

> Am I the only one who sees this quoted-printable stuff?  Is it a
> problem on Gnus' side or is the original article incorrect?

The original article seems correct here, but I could also see
raw qp stuff when I set the mm-fill-flowed variable to nil.  I
found it is caused by the following change in No Gnus, although
I'm not sure in whether setting mm-fill-flowed to t (which is
the default) solves your problem.

2004-08-18  Florian Weimer  <fw@deneb.enyo.de>

	* gnus-sum.el (gnus-summary-force-verify-and-decrypt): Bind
	`mm-fill-flowed'.

	* mm-decode.el (mm-dissect-singlepart): Check it.

I vaguely think it might be better to use the new variable
exclusively for this control.

Re: Display of quoted-printable

[Ralf Angeli]

* Katsumi Yamaoka (2004-10-01) writes:

>>>>>> In <cjj2vb$c1$2@sea.gmane.org> Ralf Angeli wrote:
>
>> * test (2004-09-30) writes:
>
>>> Ralf Angeli a =E9crit :
>>>
>>>> * Kevin Greiner (2004-09-30) writes:
>>>>=20
>>>>=20
>
>> Am I the only one who sees this quoted-printable stuff?  Is it a
>> problem on Gnus' side or is the original article incorrect?
>
> The original article seems correct here, but I could also see
> raw qp stuff when I set the mm-fill-flowed variable to nil.  I
> found it is caused by the following change in No Gnus, although
> I'm not sure in whether setting mm-fill-flowed to t (which is
> the default) solves your problem.

Well, the quoted-printable characters are displayed correctly then.
But this will enable the IMO incorrect sqashing of quoted empty lines
again.  That means, the ">=20" or "> " lines respectively will not be
displayed and will not show up in replies to such messages.

> 2004-08-18  Florian Weimer  <fw@deneb.enyo.de>
>
> 	* gnus-sum.el (gnus-summary-force-verify-and-decrypt): Bind
> 	`mm-fill-flowed'.
>
> 	* mm-decode.el (mm-dissect-singlepart): Check it.
>
> I vaguely think it might be better to use the new variable
> exclusively for this control.

The original intention of `mm-fill-flowed' was to control filling
only.

See <URL:http://thread.gmane.org/gmane.emacs.gnus.general/55056>.

-- 
Ralf

Re: HEADS UP! Unplugged (agent) marks handling changed

[Ralf Angeli]

[I already answered this message, but the article did not show up on
Gmane for several hours, so here is a second try.]

* Kevin Greiner (2004-09-25) writes:

> If you run into any unexpected behavior with offline flags, please
> post so that I can address it.

For me it does not fix the original problem.  Gnus is still forgetting
marks with the described usage pattern.

I tried to debug `gnus-agent-possibly-synchronize-flags' again and saw
that `gnus-find-method-for-group' (called in `gnus-request-set-mark')
returns `(nnnil)' for groups which are accessed through NNTP.
(`nnnil' is my primary select method.)  The resulting backend function
to be called will be `nnnil-request-set-mark' which obviously is not
bound.  I am assuming this is the cause for the marks not getting set,
but maybe I am just babbling as I haven't looked deeper into the code.

Here is a backtrace which probably shows the cause:

Debugger entered--returning value: nil
  string-match(":" "de.comp.editoren")
* gnus-group-name-to-method("de.comp.editoren")
* gnus-find-method-for-group("de.comp.editoren")
* gnus-request-set-mark("de.comp.editoren" (((13149) add (read))))
* gnus-agent-synchronize-group-flags("de.comp.editoren" (((13149) add (read))) "news.individual.net")
* eval((gnus-agent-synchronize-group-flags "de.comp.editoren" (quote (...)) "news.individual.net"))
[byte-code snipped]
* gnus-agent-synchronize-flags-server((nntp "news.individual.net"))
* gnus-agent-possibly-synchronize-flags-server((nntp "news.individual.net"))
[byte-code snipped]
* gnus-agent-possibly-synchronize-flags()

-- 
Ralf