A T does not work in nnml ...

A T does not work in nnml ...

[Ulf Stegemann]

... but does in nntp and nnimap.  I don't know if this has something to
do with the recent changes made for making `A T' work in nnimap.

Behaviour of `A T' in nnml groups is rather strange: Most of the time it
seems to do something (`Fetching headers for...', `Generating
summary...' etc.) but the summary buffer looks exactly the same after
the command.  For some messages `A T' is able to fetch children of the
current article (sometimes not all of them, though).  However, parents
are never fetched.  Does anyone else encounter this behaviour?

Ulf

Re: A T does not work in nnml ...

[Andrew Cohen]

>>>>> "Ulf" == Ulf Stegemann <ulf-news@zeitform.de> writes:

    Ulf> ... but does in nntp and nnimap.  I don't know if this has
    Ulf> something to do with the recent changes made for making `A T'
    Ulf> work in nnimap.


Hmm, this is strange. There should have been no change for nnml. I just
checked the old function and the new one and they behave exactly the
same for my nnml groups. Are you really seeing a change in behavior?

Thread-referral has always been sub-optimal for nnml. The current (and
previous) behavior is to just download a bunch of headers and hope that
the thread is among them. For nnml I suspect there is a better way...

Re: A T does not work in nnml ...

[Ulf Stegemann]

Andrew Cohen <cohen@andy.bu.edu> wrote:

>>>>>> "Ulf" == Ulf Stegemann <ulf-news@zeitform.de> writes:
>
>     Ulf> ... but does in nntp and nnimap.  I don't know if this has
>     Ulf> something to do with the recent changes made for making `A T'
>     Ulf> work in nnimap.
>
> Hmm, this is strange. There should have been no change for nnml. I just
> checked the old function and the new one and they behave exactly the
> same for my nnml groups. Are you really seeing a change in behavior?
>
> Thread-referral has always been sub-optimal for nnml. The current (and
> previous) behavior is to just download a bunch of headers and hope that
> the thread is among them. For nnml I suspect there is a better way...

Ah, well, I wouldn't bet my house on that the behaviour changed
recently.  I rather seldom use `A T' in nnml groups so maybe this has
been buggy a long time or, err,  maybe it never worked?  Anyway, since
Larsi has been asking for things to improve in Gnus this would be
probably something to go for.  If the issue can't be addressed for any
reason it would be nice to give users a hint in the documentation that
`A T' might not always do the right thing in nnml groups.

Ulf

Re: A T does not work in nnml ...

[Andrew Cohen]

>>>>> "Ulf" == Ulf Stegemann <ulf-news@zeitform.de> writes:


    Ulf> Ah, well, I wouldn't bet my house on that the behaviour changed
    Ulf> recently.  I rather seldom use `A T' in nnml groups so maybe
    Ulf> this has been buggy a long time or, err, maybe it never worked?
    Ulf> Anyway, since Larsi has been asking for things to improve in
    Ulf> Gnus this would be probably something to go for.  If the issue
    Ulf> can't be addressed for any reason it would be nice to give
    Ulf> users a hint in the documentation that `A T' might not always
    Ulf> do the right thing in nnml groups.

OK, thanks for the info. A 30 second glance at nnml suggests that it
shouldn't be hard to make it work more reliably.

Regards,
Andy

Re: A T does not work in nnml ...

[Lars Magne Ingebrigtsen]

Andrew Cohen <cohen@andy.bu.edu> writes:

> Thread-referral has always been sub-optimal for nnml. The current (and
> previous) behavior is to just download a bunch of headers and hope that
> the thread is among them. For nnml I suspect there is a better way...

Perhaps the number of headers it downloads should be doubled or
something? 

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: A T does not work in nnml ...

[Andrew Cohen]

>>>>> "Lars" == Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

    Lars> Andrew Cohen <cohen@andy.bu.edu> writes:
    >> Thread-referral has always been sub-optimal for nnml. The current
    >> (and previous) behavior is to just download a bunch of headers
    >> and hope that the thread is among them. For nnml I suspect there
    >> is a better way...

    Lars> Perhaps the number of headers it downloads should be doubled
    Lars> or something?

Hmm, it appears there is an underlying bug which manifests whenever the
headers are retrieved as nov.  I kind of know what's going on but I
don't yet fully understand the way dependencies work.

The Problem:
Even when the headers for the articles in the thread have been retrieved
the articles don't make their way into the summary buffer. 

"A T" (aka gnus-summary-refer-thread) retrieves the headers, calls
gnus-build-all-threads and finally
gnus-summary-limit-include-thread. Now gnus-build-all-threads parses
each header (in nov format) and calls gnus-dependencies-add-header. This
function checks to see if the header is already in the dependencies
hash. If not, it adds it and then pushes the header onto
gnus-newsgroup-headers. If the header is already present it just skip
these steps.

This is the problem---there may already be an entry in the dependencies
list if the article is already present but refers to other articles in
the thread that haven't yet been retrieved. But then the article id of
these "other" articles is a negative number (reflecting the fact that
they haven't been retrieved) and never gets updated to the actual
article id. And the newly fetched headers for these articles doesn't get
pushed onto gnus-newsgroup-headers, hence the problem.

I thought I could fix it by calling gnus-dependencies-add-header with
the force-new flag. This works better, but still shows some odd
behavior. 

I'll keep plugging away at this, but someone who knows more about
dependencies and threading might find the fix faster.

Re: A T does not work in nnml ...

[Andrew Cohen]

>>>>> "Andy" == Andrew Cohen <cohen@andy.bu.edu> writes:

>>>>> "Lars" == Lars Magne Ingebrigtsen <larsi@gnus.org> writes:
    Lars> Andrew Cohen <cohen@andy.bu.edu> writes:
    >>> Thread-referral has always been sub-optimal for nnml. The
    >>> current (and previous) behavior is to just download a bunch of
    >>> headers and hope that the thread is among them. For nnml I
    >>> suspect there is a better way...

    Lars> Perhaps the number of headers it downloads should be doubled
    Lars> or something?

    Andy> Hmm, it appears there is an underlying bug which manifests
    Andy> whenever the headers are retrieved as nov.  I kind of know
    Andy> what's going on but I don't yet fully understand the way
    Andy> dependencies work.

    Andy> The Problem: Even when the headers for the articles in the
    Andy> thread have been retrieved the articles don't make their way
    Andy> into the summary buffer.


[...]


    Andy> I thought I could fix it by calling
    Andy> gnus-dependencies-add-header with the force-new flag. This
    Andy> works better, but still shows some odd behavior.

OK, I think that this does indeed fix it. I was misinterpreting what
happens when threads get "cut" based on the value of
gnus-fetch-old-headers and gnus-build-sparse-threads. 

Re: A T does not work in nnml ...

[Andrew Cohen]

>>>>> "Andrew" == Andrew Cohen <cohen@andy.bu.edu> writes:

>>>>> "Andy" == Andrew Cohen <cohen@andy.bu.edu> writes:
>>>>> "Lars" == Lars Magne Ingebrigtsen <larsi@gnus.org> writes:
    Lars> Andrew Cohen <cohen@andy.bu.edu> writes:
    >>>> Thread-referral has always been sub-optimal for nnml. The
    >>>> current (and previous) behavior is to just download a bunch of
    >>>> headers and hope that the thread is among them. For nnml I
    >>>> suspect there is a better way...

    Lars> Perhaps the number of headers it downloads should be doubled
    Lars> or something?

    Andy> Hmm, it appears there is an underlying bug which manifests
    Andy> whenever the headers are retrieved as nov.  I kind of know
    Andy> what's going on but I don't yet fully understand the way
    Andy> dependencies work.

    Andy> The Problem: Even when the headers for the articles in the
    Andy> thread have been retrieved the articles don't make their way
    Andy> into the summary buffer.

    Andy> [...]

    Andy> I thought I could fix it by calling
    Andy> gnus-dependencies-add-header with the force-new flag. This
    Andy> works better, but still shows some odd behavior.

    Andy> OK, I think that this does indeed fix it. I was
    Andy> misinterpreting what happens when threads get "cut" based on
    Andy> the value of gnus-fetch-old-headers and
    Andy> gnus-build-sparse-threads.


And one more bug fix needed: `gnus-summary-limit-include-thread' is more
"limit" than "include":). This function ultimately calls
`gnus-summary-prepare` which leads to `gnus-cut-thread`.  There is some
logic in `gnus-cut-thread` that I don't fully understand (can anyone
explain it to me?), but it has the effect of removing parts of threads
whenever `gnus-fetch-old-headers' or `gnus-build-sparse-threads' are,
well, almost anything. I assume that anyone who asks to include a thread
really, really wants to include it, and so I'm just bypassing the cuts
when preparing the summary buffer. This seems to fix almost (but not
quite) all of the issues I've been having with thread-referral.

I'll push this stuff shortly.

Andy

Re: A T does not work in nnml ...

[Steinar Bang]

FWIW I would be happy with a `G A' command opening just that particular
article in the group it was posted in, without attempting to gather the
thread.

`G T' and then C-g when it gets stuck does that particular trick, but
feels a bit... I dunno... awkward...? :-)

Re: A T does not work in nnml ...

[Andrew Cohen]

>>>>> "Steinar" == Steinar Bang <sb@dod.no> writes:

    Steinar> `G T' and then C-g when it gets stuck does that particular
    Steinar> trick, but feels a bit... I dunno... awkward...? :-)

Hmm, what kind of group is this? imap? or nntp? I'm guessing nntp...

(imap and nnml should both be very fast.)

Nntp should be fast unless you are downloading lots of headers. What is
the value of  `gnus-fetch-old-headers'?

Re: A T does not work in nnml ...

[Steinar Bang]

>>>>> Andrew Cohen <cohen@andy.bu.edu>:

> Hmm, what kind of group is this? imap? or nntp? I'm guessing nntp...

Yep.

> (imap and nnml should both be very fast.)

> Nntp should be fast unless you are downloading lots of headers. What
> is the value of `gnus-fetch-old-headers'?

`C-h v gnus-fetch-old-headers RET' says:
 gnus-fetch-old-headers is a variable defined in `gnus-sum.el'.
 Its value is nil

What I did to test it was position the cursor over gmane.discuss and
search for the string
 gwene
Then I picked the first article in the thread with the subject
 Podcasts in gwene ?
and typed `G T'

Re: A T does not work in nnml ...

[Andrew Cohen]

>>>>> "Steinar" == Steinar Bang <sb@dod.no> writes:

    Steinar> `C-h v gnus-fetch-old-headers RET' says:
    Steinar> gnus-fetch-old-headers is a variable defined in
    Steinar> `gnus-sum.el'.  Its value is nil

Oops, I meant to ask for 'gnus-refer-thread-limit'. 

    Steinar> What I did to test it was position the cursor over
    Steinar> gmane.discuss and search for the string gwene Then I picked
    Steinar> the first article in the thread with the subject Podcasts
    Steinar> in gwene ?  and typed `G T'

The only thing that should take any time is the network retrieval of
headers. The default value of gnus-refer-thread-limit is 500; can you
try with a smallish value and see if that completes faster?

I just want to make sure there isn't some other bug I'm unaware of. I'll
be checking in a modified way to do this that should do a better job of
finding the thread in any case.

Andy

Re: A T does not work in nnml ...

[Steinar Bang]

>>>>> Andrew Cohen <cohen@andy.bu.edu>:

> Oops, I meant to ask for 'gnus-refer-thread-limit'. 

Right.  Here it is:
 gnus-refer-thread-limit is a variable defined in `gnus-sum.el'.
 Its value is 500

> The only thing that should take any time is the network retrieval of
> headers. The default value of gnus-refer-thread-limit is 500; can you
> try with a smallish value and see if that completes faster?

Ok.  I will try that.  I'll try a value of a 100 first.

> I just want to make sure there isn't some other bug I'm unaware of. I'll
> be checking in a modified way to do this that should do a better job of
> finding the thread in any case.

Kewl!  I really like being able to do this: search and go to the
group.  I've been wanting to do it for a long time.

So thanks for picking this up!

Re: A T does not work in nnml ...

[Steinar Bang]

>>>>> Steinar Bang <sb@dod.no>:

>> The only thing that should take any time is the network retrieval of
>> headers. The default value of gnus-refer-thread-limit is 500; can you
>> try with a smallish value and see if that completes faster?

> Ok.  I will try that.  I'll try a value of a 100 first.

Trying with
 gnus-refer-thread-limit is a variable defined in `gnus-sum.el'.
 Its value is 100
and after emacs and gnus restart, I see the same effect:

It's hanging for a long time.  I didn't time it exactly, but I let it
run for 5 minutes or so, before I C-g'd it.  When I did, I was in the
gmane.discuss group, with the article under point as the only article.

Here's another data point for you: I run with nntp and agent on
news.gmane.org.  Maybe that messes up things?

I see one strange effect: if I have done `G-T' followed by a `C-g'
earlier, on an article, and I try to `G-T' that particular article
again, it shows up immediately in the gmane group, without trying to
gather a thread.

This persists across emacs processes, and gnus invocations, and the only
place that persisting could happen is in the agent (I think...?).

Re: A T does not work in nnml ...

[Andrew Cohen]

>>>>> "Steinar" == Steinar Bang <sb@dod.no> writes:

>>>>> Steinar Bang <sb@dod.no>:
    Steinar> It's hanging for a long time.  I didn't time it exactly,
    Steinar> but I let it run for 5 minutes or so, before I C-g'd it.
    Steinar> When I did, I was in the gmane.discuss group, with the
    Steinar> article under point as the only article.

    Steinar> Here's another data point for you: I run with nntp and
    Steinar> agent on news.gmane.org.  Maybe that messes up things?

I haven't thought at all about the agent. Maybe that's the issue.

Can you M-x toggle-debug-on-quit and see where things stand when you
C-g? 

Re: A T does not work in nnml ...

[Steinar Bang]

>>>>> Andrew Cohen <cohen@andy.bu.edu>:

> Can you M-x toggle-debug-on-quit and see where things stand when you
> C-g?

This is the stack trace from the C-g (I loaded gnus-sum.el, nntp.el,
nnir.el, and gnus-agent.el as el files before starting gnus for the
trace): 

  (if (eq gnus-headers-retrieved-by (quote nov)) (progn (with-current-buffer nntp-server-buffer ... ...) (gnus-build-all-threads)))
  (when (eq gnus-headers-retrieved-by (quote nov)) (with-current-buffer nntp-server-buffer (goto-char ...) (keep-lines ...)) (gnus-build-all-threads))
  (let ((id ...) (subject ...) (refs ...) (gnus-summary-ignore-duplicates t) (gnus-inhibit-demon t) (gnus-read-all-available-headers t) (limit ...)) (if (gnus-check-backend-function ... gnus-newsgroup-name) (setq gnus-newsgroup-headers ...) (unless ... ... ...)) (when (eq gnus-headers-retrieved-by ...) (with-current-buffer nntp-server-buffer ... ...) (gnus-build-all-threads)) (gnus-summary-limit-include-thread id))
  gnus-summary-refer-thread()
  (if (eq (car ...) (quote nnimap)) (progn (nnimap-possibly-change-group ... nil) (with-current-buffer ... ...)) (gnus-summary-read-group-1 group t t gnus-summary-buffer nil (list backend-number)) (gnus-summary-refer-thread))
  (let* ((cur ...) (group ...) (backend-number ...) (id ...) (refs ...)) (if (eq ... ...) (progn ... ...) (gnus-summary-read-group-1 group t t gnus-summary-buffer nil ...) (gnus-summary-refer-thread)))
  gnus-summary-nnir-goto-thread()
  call-interactively(gnus-summary-nnir-goto-thread nil nil)

Re: A T does not work in nnml ...

[Lars Magne Ingebrigtsen]

Andrew Cohen <cohen@andy.bu.edu> writes:

> I'll push this stuff shortly.

Sounds good.  :-)  This also sounds like the same problem I was having
with editing nnimap articles, so perhaps your fix here will fix that
problem, too.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: A T does not work in nnml ...

[Andrew Cohen]

I just pushed a bunch of changes to improve thread-referral. I think
this should fix most of the remaining idiosyncrasies. I also simplified
the code. There is no longer an nnir-specific thread-referral function;
the general one `gnus-summary-refer-thread' works, and now uses the same
binding "A T" in nnir groups (so you should stop using "G T" and just
always use "A T"). 

I've added a new function to just warp from the article in the nnir
group to the real one without fetching the thread. This is bound to "A
W". 

Finally when retrieving the headers for the referred thread I have
turned off the agent, since I haven't yet understood what the right
behavior should be. I suspect this should help with the issue that
Steinar has been having.

Andy

Re: A T does not work in nnml ...

[Steinar Bang]

>>>>> Andrew Cohen <cohen@andy.bu.edu>:

> Finally when retrieving the headers for the referred thread I have
> turned off the agent, since I haven't yet understood what the right
> behavior should be. I suspect this should help with the issue that
> Steinar has been having.

It still gets stuck.  According to the system monitors it burns CPU, but
doesn't use any significant memory, disk or network resources.

Here's the stack trace from C-g on when it's stuck on `A T' from the
nnir buffer after searching gmane.discuss for "gwene":

Debugger entered--Lisp error: (quit)
  quoted-printable-decode-string("Sj=C3=B8gren")
  byte-code("\302\303\bA@\"\203.

Re: A T does not work in nnml ...

[Andrew Cohen]

>>>>> "Steinar" == Steinar Bang <sb@dod.no> writes:

    Steinar> Here's the stack trace from C-g on when it's stuck on `A T'
    Steinar> from the nnir buffer after searching gmane.discuss for
    Steinar> "gwene":

    Steinar> Debugger entered--Lisp error: (quit)
    Steinar> quoted-printable-decode-string("Sj=C3=B8gren")
    Steinar> byte-code("\302\303\bA@\"\203.


This is hard to understand. As you have no doubt guessed, I can't
reproduce the problem. The string "Sj=C3=B8gren" is the quoted-printable
representation of Adam Sjøgren's last name. But this article is already
in the buffer right after the search for "gwene". If the
quoted-printable stuff gets decoded correctly, then why should it be any
different the next time around?

Can you edebug the function quoted-printable-decode-string and send me
the backtrace when the function is called right after the search, and
then again after the function is called following "A T"?

Thanks,
andy

Re: A T does not work in nnml ...

[Steinar Bang]

>>>>> Andrew Cohen <cohen@andy.bu.edu>:

> This is hard to understand. 

No wonder.  The stack trace was seriously chopped off in the sending.
Probably because one of the control characters gnus warned me about when
sending...

I think rfc2047-decode-encoded-words was just coincidental (just where
it was at the moment I pressed C-g).

I think it is looping higher up in the stack trace.

I will try sending it again.  Here it comes (byte code replaced with the
text "byte code removed"):

Debugger entered--Lisp error: (quit)
  quoted-printable-decode-string("Sj=C3=B8gren")
  byte-code("[byte code removed]" [word text char-equal 66 base64-decode-string rfc2047-pad-base64 2 81 quoted-printable-decode-string mm-subst-char-in-string 95 32 t] 6)
  rfc2047-decode-encoded-words((("utf-8" 81 "Sj=C3=B8gren" "=?utf-8?Q?Sj=C3=B8gren?=")))
  rfc2047-decode-string("asjo@koldfront.dk (Adam =?utf-8?Q?Sj=C3=B8gren?=)" t)
  mail-decode-encoded-address-string("asjo@koldfront.dk (Adam =?utf-8?Q?Sj=C3=B8gren?=)")
  funcall(mail-decode-encoded-address-string "asjo@koldfront.dk (Adam =?utf-8?Q?Sj=C3=B8gren?=)")
  (gnus-remove-odd-characters (funcall gnus-decode-encoded-address-function (setq x ...)))
  (condition-case nil (gnus-remove-odd-characters (funcall gnus-decode-encoded-address-function ...)) (error x))
  (make-full-mail-header number (condition-case nil (gnus-remove-odd-characters ...) (error x)) (condition-case nil (gnus-remove-odd-characters ...) (error x)) (nnheader-nov-field) (nnheader-nov-read-message-id number) (setq references (nnheader-nov-field)) (nnheader-nov-read-integer) (nnheader-nov-read-integer) (unless (eobp) (if ... ...) (nnheader-nov-field)) (nnheader-nov-parse-extra))
  (setq header (make-full-mail-header number (condition-case nil ... ...) (condition-case nil ... ...) (nnheader-nov-field) (nnheader-nov-read-message-id number) (setq references ...) (nnheader-nov-read-integer) (nnheader-nov-read-integer) (unless ... ... ...) (nnheader-nov-parse-extra)))
  (let (x) (narrow-to-region (point) eol) (unless (eobp) (forward-char)) (setq header (make-full-mail-header number ... ... ... ... ... ... ... ... ...)))
  (unwind-protect (let (x) (narrow-to-region ... eol) (unless ... ...) (setq header ...)) (widen))
  (let ((eol ...) (buffer ...) header references in-reply-to) (unwind-protect (let ... ... ... ...) (widen)) (when (and ... ... ...) (mail-header-set-references header ...)) (when gnus-alter-header-function (funcall gnus-alter-header-function header)) (gnus-dependencies-add-header header dependencies force-new))
  gnus-nov-parse-line(13689 [0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 none 0 0 0 0 ...] t)
  (inline (gnus-nov-parse-line number dependencies force-new))
  (setq header (inline (gnus-nov-parse-line number dependencies force-new)))
  (progn (setq sequence (cdr sequence)) (setq header (inline ...)))
  (and (or allp (and sequence ...)) (progn (setq sequence ...) (setq header ...)))
  (if (and (or allp ...) (progn ... ...)) (progn (push header headers)))
  (when (and (or allp ...) (progn ... ...)) (push header headers))
  (while (and (or sequence allp) (not ...)) (setq number (read cur)) (when (not allp) (while ... ...)) (when (and ... ...) (push header headers)) (forward-line 1))
  (progn (while (and ... ...) (setq number ...) (when ... ...) (when ... ...) (forward-line 1)) (goto-char (point-max)))
  (condition-case nil (progn (while ... ... ... ... ...) (goto-char ...)) (error (gnus-error 4 "Invalid data on line %d" ...) (forward-line 1)))
  (while (not (eobp)) (condition-case nil (progn ... ...) (error ... ...)))
  (gnus-parse-without-error (while (and ... ...) (setq number ...) (when ... ...) (when ... ...) (forward-line 1)))
  (save-current-buffer (set-buffer nntp-server-buffer) (subst-char-in-region (point-min) (point-max) 13 32 t) (gnus-run-hooks (quote gnus-parse-headers-hook)) (goto-char (point-min)) (gnus-parse-without-error (while ... ... ... ... ...)) (if (or ... ...) (nreverse headers) (let ... ...)))
  (with-current-buffer nntp-server-buffer (subst-char-in-region (point-min) (point-max) 13 32 t) (gnus-run-hooks (quote gnus-parse-headers-hook)) (goto-char (point-min)) (gnus-parse-without-error (while ... ... ... ... ...)) (if (or ... ...) (nreverse headers) (let ... ...)))
  (let ((mail-parse-charset gnus-newsgroup-charset) (mail-parse-ignored-charsets gnus-newsgroup-ignored-charsets) (cur nntp-server-buffer) (dependencies ...) (allp ...) number headers header) (with-current-buffer nntp-server-buffer (subst-char-in-region ... ... 13 32 t) (gnus-run-hooks ...) (goto-char ...) (gnus-parse-without-error ...) (if ... ... ...)))
  gnus-get-newsgroup-headers-xover((13763) t nil "nntp+news.gmane.org:gmane.discuss" t)
  (if (eq (quote nov) (setq gnus-headers-retrieved-by ...)) (gnus-get-newsgroup-headers-xover articles force-new dependencies gnus-newsgroup-name t) (gnus-get-newsgroup-headers dependencies force-new))
  (prog1 (if (eq ... ...) (gnus-get-newsgroup-headers-xover articles force-new dependencies gnus-newsgroup-name t) (gnus-get-newsgroup-headers dependencies force-new)) (gnus-message 5 "Fetching headers for %s...done" name))
  (let ((name ...)) (gnus-message 5 "Fetching headers for %s..." name) (prog1 (if ... ... ...) (gnus-message 5 "Fetching headers for %s...done" name)))
  gnus-fetch-headers((13763) 200 t)
  (let* ((last ...) (subject ...) (refs ...) (gnus-parse-headers-hook ...)) (gnus-fetch-headers (list last) (if ... ... limit) t))
  (if (gnus-check-backend-function (quote request-thread) gnus-newsgroup-name) (gnus-request-thread id) (let* (... ... ... ...) (gnus-fetch-headers ... ... t)))
  (gnus-merge (quote list) gnus-newsgroup-headers (if (gnus-check-backend-function ... gnus-newsgroup-name) (gnus-request-thread id) (let* ... ...)) (quote gnus-article-sort-by-number))
  (setq gnus-newsgroup-headers (gnus-merge (quote list) gnus-newsgroup-headers (if ... ... ...) (quote gnus-article-sort-by-number)))
  (let ((id ...) (gnus-inhibit-demon t) (gnus-agent nil) (gnus-summary-ignore-duplicates t) (gnus-read-all-available-headers t) (limit ...)) (setq gnus-newsgroup-headers (gnus-merge ... gnus-newsgroup-headers ... ...)) (gnus-summary-limit-include-thread id))
  gnus-summary-refer-thread(nil)
  call-interactively(gnus-summary-refer-thread nil nil)

Re: A T does not work in nnml ...

[Andrew Cohen]

>>>>> "Steinar" == Steinar Bang <sb@dod.no> writes:

>>>>> Andrew Cohen <cohen@andy.bu.edu>:
    >> This is hard to understand.

    Steinar> I will try sending it again.  Here it comes (byte code
    Steinar> replaced with the text "byte code removed"):


[...]


I'm confused by this backtrace. Most likely its just my own confusion,
but just to eliminate any possibility:

1. What is the value of gnus-parse-headers-hook

2. Are you positive that you have the latest git, fully compiled, emacs
   restarted, etc. etc. and no gnus .el files that might be pulled in
   from somewhere else? There are bits of the backtrace that don't seem
   to be anywhere in the original code.

Thanks,
Andy

Re: A T does not work in nnml ...

[Steinar Bang]

>>>>> Andrew Cohen <cohen@andy.bu.edu>:

> I'm confused by this backtrace. Most likely its just my own confusion,
> but just to eliminate any possibility:

> 1. What is the value of gnus-parse-headers-hook

 gnus-parse-headers-hook is a variable defined in `gnus-sum.el'.
 Its value is nil

> 2. Are you positive that you have the latest git, fully compiled,

Here are the commands I use:
 (cd ~/git/gnus; git pull)
 (cd ~/git/gnus; make clean; make; cd lisp; make tags)

>    emacs restarted,

Yup.

>    etc. etc. and no gnus .el files that might be pulled in from
>    somewhere else?

Ah... that I can't say with any certainity.  Except that there is a lot
of .el stuff around.  We're talking about 21 years of emacs
customization, evaluation here.

>    There are bits of the backtrace that don't seem to be anywhere in
>    the original code.

Which ones?  Should be possible to track if I have the function name(s)
(with grep, if nothing else).

Re: A T does not work in nnml ...

[Steinar Bang]

>>>>> Steinar Bang <sb@dod.no>:

> ...  We're talking about 21 years of emacs
> customization, evaluation here.

"customization evolution", I meant to say.

Re: A T does not work in nnml ...

[Andrew Cohen]

I've gone through the backtrace more carefully and I retract my
suggestion that its pulling in other code :)

Since nothing is failing (other than it not working) I can't really see
whats wrong (and since it works for me its hard to track it down).  Did
this work at some point and then stop?

Can you try two things:

1. After you break out of the loop email me the contents of the *nntpd*
   buffer.

2. Try evaling this defun and then see if things work.


(defun gnus-summary-refer-thread (&optional limit)
  "Fetch all articles in the current thread.
If no backend-specific 'request-thread function is available
fetch LIMIT (the numerical prefix) old headers. If LIMIT is nil
fetch what's specified by the `gnus-refer-thread-limit'
variable."
  (interactive "P")
  (gnus-warp-to-article)
  (let ((id (mail-header-id (gnus-summary-article-header)))
	(gnus-inhibit-demon t)
	(gnus-agent nil)
	(gnus-summary-ignore-duplicates t)
	(gnus-read-all-available-headers t)
	(limit (if limit (prefix-numeric-value limit)
		 gnus-refer-thread-limit)))
    (setq gnus-newsgroup-headers
	  (gnus-merge
	   'list gnus-newsgroup-headers
	   (if (gnus-check-backend-function
		'request-thread gnus-newsgroup-name)
	       (gnus-request-thread id)
	     (let* ((last (if (numberp limit)
			      (min (+ (mail-header-number
				       (gnus-summary-article-header))
				      limit)
				   gnus-newsgroup-highest)
			    gnus-newsgroup-highest))
		    (subject (gnus-simplify-subject
			      (mail-header-subject
			       (gnus-summary-article-header))))
		    (refs (split-string (or (mail-header-references
					     (gnus-summary-article-header))
					    ""))))
	       (gnus-fetch-headers (list last) (if (numberp limit)
						   (* 2 limit) limit) t)))
	   'gnus-article-sort-by-number))
    (gnus-summary-limit-include-thread id)))

Re: A T does not work in nnml ...

[Steinar Bang]

>>>>> Andrew Cohen <cohen@andy.bu.edu>:

> Since nothing is failing (other than it not working)
> I can't really see whats wrong (and since it works for me its hard to
> track it down). 

Well the emacs process is burning all available CPU while waiting to be
C-g'd, it hints at an eternal loop of some kind...?

> Did this work at some point and then stop?

No, it has never worked for me before.

In fact: that it actually has an article in the actual group after C-g
is a distinct improvement over not going to the actual group at
all...:-)

> Can you try two things:

> 1. After you break out of the loop email me the contents of the *nntpd*
>    buffer.

At the end of this email.  Point was at line 2 in the buffer.

> 2. Try evaling this defun and then see if things work.

Nope.  The same behaviour. 

I'll try to edebug into gnus-summary-refer-thread and see if I can see
where the loop is.

Here are the  *nntpd* buffer contents:

13688	Podcasts in gwene ?	e20100633 <e20100633@inbox.lv>	Tue, 19 Oct 2010 14:49:03 +0200	<87wrpe4abk.fsf@EVA01.wopr.org>		4704	42	Xref: news.gmane.org gmane.discuss:13688
13689	Re: Podcasts in gwene ?	asjo@koldfront.dk (Adam =?utf-8?Q?Sj=C3=B8gren?=)	Tue, 19 Oct 2010 15:01:52 +0200	<87r5fmwd33.fsf@topper.koldfront.dk>	<87wrpe4abk.fsf@EVA01.wopr.org>	4497	36	Xref: news.gmane.org gmane.discuss:13689
13690	Re: Podcasts in gwene ?	e20100633 <e20100633@inbox.lv>	Tue, 19 Oct 2010 15:48:00 +0200	<87ocaq47lb.fsf@EVA01.wopr.org>	<87wrpe4abk.fsf@EVA01.wopr.org> <87r5fmwd33.fsf@topper.koldfront.dk>	4198	41	Xref: news.gmane.org gmane.discuss:13690
13693	Re: Podcasts in gwene ?	Lars Magne Ingebrigtsen <larsi@gnus.org>	Tue, 19 Oct 2010 18:58:10 +0200	<m3sk02dsrh.fsf@quimbies.gnus.org>	<87wrpe4abk.fsf@EVA01.wopr.org>	5274	20	Xref: news.gmane.org gmane.discuss:13693 gmane.emacs.gnus.general:73299
13703	Re: Podcasts in gwene ?	e20100633 <e20100633@inbox.lv>	Tue, 19 Oct 2010 22:57:43 +0200	<871v7l529k.fsf@EVA01.wopr.org>	<87wrpe4abk.fsf@EVA01.wopr.org> <m3sk02dsrh.fsf@quimbies.gnus.org>	5736	72	Xref: news.gmane.org gmane.discuss:13703 gmane.emacs.gnus.general:73326
13704	Re: Podcasts in gwene ?	Lars Magne Ingebrigtsen <larsi@gnus.org>	Tue, 19 Oct 2010 23:25:14 +0200	<m3pqv5ub7p.fsf@quimbies.gnus.org>	<87wrpe4abk.fsf@EVA01.wopr.org> <m3sk02dsrh.fsf@quimbies.gnus.org> <871v7l529k.fsf@EVA01.wopr.org>	4694	9	Xref: news.gmane.org gmane.discuss:13704 gmane.emacs.gnus.general:73329
13706	Re: Podcasts in gwene ?	Kevin Ryde <user42@zip.com.au>	Wed, 20 Oct 2010 08:37:32 +1100	<87eiblj23n.fsf@blah.blah>	<87wrpe4abk.fsf@EVA01.wopr.org> <m3sk02dsrh.fsf@quimbies.gnus.org>	3591	14	Xref: news.gmane.org gmane.discuss:13706
13707	Re: Podcasts in gwene ?	e20100633 <e20100633@inbox.lv>	Tue, 19 Oct 2010 23:44:22 +0200	<87lj5t3ljd.fsf@EVA01.wopr.org>	<87wrpe4abk.fsf@EVA01.wopr.org> <m3sk02dsrh.fsf@quimbies.gnus.org> <871v7l529k.fsf@EVA01.wopr.org> <m3pqv5ub7p.fsf@quimbies.gnus.org>	3685	16	Xref: news.gmane.org gmane.discuss:13707 gmane.emacs.gnus.general:73331
13708	Re: Podcasts in gwene ?	Lars Magne Ingebrigtsen <larsi@gnus.org>	Wed, 20 Oct 2010 00:24:39 +0200	<m3lj5tu8go.fsf@quimbies.gnus.org>	<87wrpe4abk.fsf@EVA01.wopr.org> <m3sk02dsrh.fsf@quimbies.gnus.org> <87eiblj23n.fsf@blah.blah>	4776	11	Xref: news.gmane.org gmane.discuss:13708
13709	Re: Podcasts in gwene ?	Kevin Ryde <user42@zip.com.au>	Wed, 20 Oct 2010 10:28:51 +1100	<87aam9iwy4.fsf@blah.blah>	<87wrpe4abk.fsf@EVA01.wopr.org> <m3sk02dsrh.fsf@quimbies.gnus.org> <87eiblj23n.fsf@blah.blah> <m3lj5tu8go.fsf@quimbies.gnus.org>	3445	6	Xref: news.gmane.org gmane.discuss:13709
13710	Re: Podcasts in gwene ?	Lars Magne Ingebrigtsen <larsi@gnus.org>	Wed, 20 Oct 2010 01:32:32 +0200	<m3bp6pu5bj.fsf@quimbies.gnus.org>	<87wrpe4abk.fsf@EVA01.wopr.org> <m3sk02dsrh.fsf@quimbies.gnus.org> <87eiblj23n.fsf@blah.blah> <m3lj5tu8go.fsf@quimbies.gnus.org> <87aam9iwy4.fsf@blah.blah>	4743	12	Xref: news.gmane.org gmane.discuss:13710
13711	Re: Podcasts in gwene ?	Kevin Ryde <user42@zip.com.au>	Wed, 20 Oct 2010 10:47:24 +1100	<87y69thhir.fsf@blah.blah>	<87wrpe4abk.fsf@EVA01.wopr.org> <m3sk02dsrh.fsf@quimbies.gnus.org> <87eiblj23n.fsf@blah.blah> <m3lj5tu8go.fsf@quimbies.gnus.org> <87aam9iwy4.fsf@blah.blah> <m3bp6pu5bj.fsf@quimbies.gnus.org>	3611	8	Xref: news.gmane.org gmane.discuss:13711
13712	Re: Podcasts in gwene ?	Lars Magne Ingebrigtsen <larsi@gnus.org>	Wed, 20 Oct 2010 01:56:41 +0200	<m3y69tspmu.fsf@quimbies.gnus.org>	<87wrpe4abk.fsf@EVA01.wopr.org> <m3sk02dsrh.fsf@quimbies.gnus.org> <87eiblj23n.fsf@blah.blah> <m3lj5tu8go.fsf@quimbies.gnus.org> <87aam9iwy4.fsf@blah.blah> <m3bp6pu5bj.fsf@quimbies.gnus.org> <87y69thhir.fsf@blah.blah>	5635	32	Xref: news.gmane.org gmane.discuss:13712

Re: A T does not work in nnml ...

[Steinar Bang]

>>>>> Steinar Bang <sb@dod.no>:

> I'll try to edebug into gnus-summary-refer-thread and see if I can see
> where the loop is.

The bottom line here (the regexp-opt line), ie. line 8869 of gnus-sum.el
is where edebug hangs:
		    (gnus-parse-headers-hook
		     (lambda () (goto-char (point-min))
		       (keep-lines
			(regexp-opt (append refs (list id subject)))))))
                                                                     ^point
The point is on the third paranthesis from the end.

Re: A T does not work in nnml ...

[Andrew Cohen]

>>>>> "Steinar" == Steinar Bang <sb@dod.no> writes:

>>>>> Steinar Bang <sb@dod.no>:
    >> I'll try to edebug into gnus-summary-refer-thread and see if I
    >> can see where the loop is.

    Steinar> The bottom line here (the regexp-opt line), ie. line 8869
    Steinar> of gnus-sum.el is where edebug hangs:
    Steinar> (gnus-parse-headers-hook (lambda () (goto-char (point-min))
    Steinar> (keep-lines (regexp-opt (append refs (list id
    Steinar> subject))))))) ^point The point is on the third paranthesis
    Steinar> from the end.

Can you try this again with the version of the function I sent you
earlier? I wanted to eliminate the gnus-parse-header-hook from the
equation. 

Re: A T does not work in nnml ...

[Steinar Bang]

>>>>> Andrew Cohen <cohen@andy.bu.edu>:

> Can you try this again with the version of the function I sent you
> earlier? I wanted to eliminate the gnus-parse-header-hook from the
> equation. 

Ok, now gnus-sum.el has the following diff:
diff --git a/lisp/gnus-sum.el b/lisp/gnus-sum.el
index ad2f5b6..e3b55f0 100644
--- a/lisp/gnus-sum.el
+++ b/lisp/gnus-sum.el
@@ -8862,11 +8862,7 @@ variable."
 			       (gnus-summary-article-header))))
 		    (refs (split-string (or (mail-header-references
 					     (gnus-summary-article-header))
-					    "")))
-		    (gnus-parse-headers-hook
-		     (lambda () (goto-char (point-min))
-		       (keep-lines
-			(regexp-opt (append refs (list id subject)))))))
+					    ""))))
 	       (gnus-fetch-headers (list last) (if (numberp limit)
 						   (* 2 limit) limit) t)))
 	   'gnus-article-sort-by-number))

I have byte compiled the gnus-sum.el file, and started a fresh emacs,
and done C-u C-M-x on gnus-summary-refer-thread, then started gnus with
  M-x gnus-slave RET
(since I'm running the gnus I writing this response in at the same time,
and I don't see any difference in this behaviour from gnus to
gnus-slave)

Then I've positioned myself on the gmane.discuss group, pressed
`G G gwene RET' and then `A T' on top of the second match.

This time it got stuck on the final line here:
	       (gnus-fetch-headers (list last) (if (numberp limit)
						   (* 2 limit) limit) t)))

This time no point in the buffer and GUI completely frozen, until I do
C-g.

I did a C-g and a new search, which never seemed to complete.

So I starte a fresh emacs and instrumented up gnus-summary-refer-thread.
Then I did a new search (which completed almost immediately), and then
stepped through the function until I was in front of the
gnus-fetch-headers line, instrumented up gnus-fetch-headers, and
continued to step through the evaluation of its argument list.

It never entered gnus-fetch-headers, so I basically don't know where the
cycles are soaked up.

This time it got stuck with the point after the third limit, ie.
	       (gnus-fetch-headers (list last) (if (numberp limit)
						   (* 2 limit) limit) t)))
                                                                    ^here
and there is nothing that could make it stuck there...?

Perhaps I should just set an edebug breakpoint in gnus-fetch-headers?
Or will that also be called when setting up the nnir group and confuse
things?

Re: A T does not work in nnml ...

[Steinar Bang]

>>>>> Steinar Bang <sb@dod.no>:

> Perhaps I should just set an edebug breakpoint in gnus-fetch-headers?
> Or will that also be called when setting up the nnir group and confuse
> things?

New emacs and new gnus-slave and then edebug instrumented
gnus-fetch-headers. 

It is called twice, first from gnus-warp-to-article (which returns.
That's why I get the single article I get in the target group after C-g):

  gnus-fetch-headers((13688))
  gnus-select-newsgroup("nntp+news.gmane.org:gmane.discuss" t (13688))
  gnus-summary-read-group-1("nntp+news.gmane.org:gmane.discuss" t t #<buffer *Summary nnir:((query . "gwene") (unique-id . "87vd492dr5.fsf"))*> nil (13688))
  nnir-warp-to-article()
  gnus-warp-to-article()
  gnus-summary-refer-thread(nil)
  call-interactively(gnus-summary-refer-thread nil nil)

Then it's called from the end of gnus-summary-refer-thread where it gets
stuck: 
  gnus-fetch-headers((13765) 200 t)
  gnus-summary-refer-thread(nil)
  call-interactively(gnus-summary-refer-thread nil nil)

This time it gets stuck in the last line here:
	    (gnus-get-newsgroup-headers-xover
	     articles force-new dependencies gnus-newsgroup-name t)

I guess gnus-get-newsgroup-headers-xover is the next candidate for an
edebug instrumentation...

Re: A T does not work in nnml ...

[Steinar Bang]

New emacs and edebug of gnus-get-newsgroup-headers-xover.

This function is also called twice from `A T'.  First from
gnus-warp-to-article:
  gnus-get-newsgroup-headers-xover((13688) nil nil "nntp+news.gmane.org:gmane.discuss" t)
  gnus-fetch-headers((13688))
  gnus-select-newsgroup("nntp+news.gmane.org:gmane.discuss" t (13688))
  gnus-summary-read-group-1("nntp+news.gmane.org:gmane.discuss" t t #<buffer *Summary nnir:((query . "gwene") (unique-id . "87y6959e3i.fsf"))*> nil (13688))
  nnir-warp-to-article()
  gnus-warp-to-article()
  gnus-summary-refer-thread(nil)
  call-interactively(gnus-summary-refer-thread nil nil)

Then it is called at the end:
  gnus-get-newsgroup-headers-xover((13765) t nil "nntp+news.gmane.org:gmane.discuss" t)
  gnus-fetch-headers((13765) 200 t)
  gnus-summary-refer-thread(nil)
  call-interactively(gnus-summary-refer-thread nil nil)

In gnus-get-newsgroup-headers-xover there is this while loop:
      (gnus-parse-without-error
	(while (and (or sequence allp)
		    (not (eobp)))
	  (setq number (read cur))
          ...

I stepped through the while loop two or three times, then I tried to
step out of that form with `o'.  I pressed `o' a couple of times until
emacs froze up (and the CPU usage went up).

Then edebug came back with the stack trace below, which I don't quite
know how to interpret.  gnus-parse-without-error is in the stack trace,
and in there there seems to be some error stuff.

However when it came back to the debugger, it was in
gnus-get-newsgroup-headers-xover on the gnus-nov-parse-line in the code
below:
			      (eq number (car sequence))))
		     (progn
		       (setq sequence (cdr sequence))
		       (setq header (inline
				      (gnus-nov-parse-line
				       number dependencies force-new)))))
	    (push header headers))

Then I continued to step until I got to the 
	    (push header headers))
line and there emacs froze up again and the CPU usage went up again.

Here is the stack trace of where it came out of the attempt at `o' out
of the while loop:
  (inline (edebug-after (edebug-before 99) 103 (gnus-nov-parse-line ... ... ...)))
  (setq header (edebug-after (edebug-before 98) 104 (inline ...)))
  (progn (edebug-after (edebug-before 92) 96 (setq sequence ...)) (edebug-after (edebug-before 97) 105 (setq header ...)))
  (and (edebug-after (edebug-before 79) 90 (or ... ...)) (edebug-after (edebug-before 91) 106 (progn ... ...)))
  (if (edebug-after (edebug-before 78) 107 (and ... ...)) (progn (edebug-after ... 111 ...)))
  (when (edebug-after (edebug-before 78) 107 (and ... ...)) (edebug-after (edebug-before 108) 111 (push ... ...)))
  (while (edebug-after (edebug-before 41) 50 (and ... ...)) (edebug-after (edebug-before 51) 55 (setq number ...)) (edebug-after (edebug-before 56) 76 (when ... ...)) (edebug-after (edebug-before 77) 112 (when ... ...)) (edebug-after (edebug-before 113) 114 (forward-line 1)))
  (progn (edebug-after (edebug-before 40) 115 (while ... ... ... ... ...)) (goto-char (point-max)))
  (condition-case nil (progn (edebug-after ... 115 ...) (goto-char ...)) (error (gnus-error 4 "Invalid data on line %d" ...) (forward-line 1)))
  (while (not (eobp)) (condition-case nil (progn ... ...) (error ... ...)))
  (gnus-parse-without-error (edebug-after (edebug-before 40) 115 (while ... ... ... ... ...)))
  (save-current-buffer (set-buffer (edebug-after 0 26 nntp-server-buffer)) (edebug-after (edebug-before 27) 32 (subst-char-in-region ... ... 13 32 t)) (edebug-after (edebug-before 33) 34 (gnus-run-hooks ...)) (edebug-after (edebug-before 35) 38 (goto-char ...)) (edebug-after (edebug-before 39) 116 (gnus-parse-without-error ...)) (edebug-after (edebug-before 117) 146 (if ... ... ...)))
  (with-current-buffer (edebug-after 0 26 nntp-server-buffer) (edebug-after (edebug-before 27) 32 (subst-char-in-region ... ... 13 32 t)) (edebug-after (edebug-before 33) 34 (gnus-run-hooks ...)) (edebug-after (edebug-before 35) 38 (goto-char ...)) (edebug-after (edebug-before 39) 116 (gnus-parse-without-error ...)) (edebug-after (edebug-before 117) 146 (if ... ... ...)))
  (let ((mail-parse-charset ...) (mail-parse-ignored-charsets ...) (cur ...) (dependencies ...) (allp ...) number headers header) (edebug-after (edebug-before 25) 147 (with-current-buffer ... ... ... ... ... ...)))
  gnus-get-newsgroup-headers-xover((13765) t nil "nntp+news.gmane.org:gmane.discuss" t)
  gnus-fetch-headers((13765) 200 t)
  gnus-summary-refer-thread(nil)
  call-interactively(gnus-summary-refer-thread nil nil)

A T hangs forever in gmane nntp (Was: A T does not work in nnml ...)

[Steinar Bang]

I've determined that the cause for the extremely slow/hanging `A T' when
opening a gmane thread is caused by something in my setup.  I tried the
following (starting a new emacs and new version of gnus for each test):
 - Remove 
   (setq gnus-decay-scores t)
 - Replace
    (setq gnus-use-scoring t)
   with
    (setq gnus-use-scoring nil)
 - Remove
    (setq gnus-thread-sort-functions
         '(gnus-thread-sort-by-number gnus-thread-sort-by-total-score))
 - Remove agent from nntp server news.gmane.org
 - Move away ~/News/agent/nntp/news.gmane.org/

But nothing made any difference.  `A T' still was hanging forever when
attempting to move a thread into the group it came from. 

So I created a brand new user, called gnustest and gave it a .emacs file
containing:

(defvar running-xemacs (string-match "XEmacs\\|Lucid" (emacs-version))
  "Hold a numerical value if this is XEmacs, nil if it isn't")

(defvar git-workspace (expand-file-name "~sb/git"))

;; Git version of Gnus:
(let ((git-gnus-directory
       (if running-xemacs
	   (concat git-workspace "/gnus.xemacs")
	 (concat git-workspace "/gnus"))))
  (push (concat git-gnus-directory "/lisp") load-path)
  (let ((git-gnus-info-dir (concat git-gnus-directory "/texi")))
    (if (boundp 'Info-directory-list)
	(push git-gnus-info-dir Info-directory-list)
      (setq Info-directory-list (append
				 (list git-gnus-info-dir)
				 Info-default-directory-list)))))

and a .gnus.el containing:

; git gnus specific variables at the start. Remove when going production.
(load "gnus-load")

(setq gnus-select-method
      '(nntp "getnews"
 	     (nntp-address "news.get.no")
	     ))

(setq gnus-secondary-select-methods
      '((nntp "news.gmane.org")
	(nndiary "")
	))

Then I started emacs and gnus as the gnustest user, and browsed the
server buffer and then news.gmane.org, marked it as subscribed.  I took
catch up of all unread articles, then visited the 10 last articles and
made one of them ticked.

Then I did `G G' and searched for "gwene" (without the quotes), and
moved the cursor to the start of the thread with subject "Podcasts in
gwene?" and pressed `A T'.

And then the entire thread with that subject showed up in
gmane.discuss. 

So... it has to be something in my setup, but I have no idea what.

I guess it's back to debugging.

A T hangs forever in gmane nnir (Was: A T hangs forever in gmane nntp)

[Steinar Bang]

Continuing the edebug, where I left off:  I instrumented
gnus-parse-without-error for edebug with
	C-u C-M-x gnus-parse-without-error RET

Then I started gnus with
	M-x gnus-slave RET

Then I positioned the point on gmane.discuss and did
	G G gwene RET

Then I positioned point on the first article of the "Podcasts in gwene?"
thread, and did
	A T

And emacs went into hang, and CPU usage went up, and the
gnus-parse-without-error breakpoint was never hit.

Re: A T hangs forever in gmane nnir

[Steinar Bang]

I edebug-instrumented gnus-get-newsgroup-headers-xover and tried the
trick again (ie. search for "gwene" in gmane.discuss, and pick the
thread "Podcasts in gwene?".

I think I was confused with what `d' reports.  I think that
gnus-parse-without-error is never entered.

I think it spins eternally in this while loop in
gnus-get-newsgroup-headers-xover: 
      (gnus-parse-without-error
	(while (and (or sequence allp)
		    (not (eobp)))
	  (setq number (read cur))
	  (when (not allp)
	    (while (and sequence
			(< (car sequence) number))
	      (setq sequence (cdr sequence))))
	  (when (and (or allp
			 (and sequence
			      (eq number (car sequence))))
		     (progn
		       (setq sequence (cdr sequence))
		       (setq header (inline
				      (gnus-nov-parse-line
				       number dependencies force-new)))))
	    (push header headers))
	  (forward-line 1)))

(ie. the while loop that produces the argument for gnus-parse-without-error)

Here is the thread from the `G G' results:
 . [  42: e20100633              ] [97: gmane.discuss/13688] Podcasts in gwene ?
 .     [  36: Adam Sjøgren           ] [87: gmane.discuss/13689] Re: Podcasts in gwene ? <== Spinning here!)
 .         [  41: e20100633              ] [79: gmane.discuss/13690] Re: Podcasts in gwene ?
 .     [  20: Lars Magne Ingebrigtsen] [79: gmane.discuss/13693] Re: Podcasts in gwene ?
 .         [  72: e20100633              ] [92: gmane.discuss/13703] Re: Podcasts in gwene ?
 .             [   9: Lars Magne Ingebrigtsen] [79: gmane.discuss/13704] Re: Podcasts in gwene ?
 .                 [  16: e20100633              ] [87: gmane.discuss/13707] Re: Podcasts in gwene ?
 .         [  14: Kevin Ryde             ] [62: gmane.discuss/13706] Re: Podcasts in gwene ?
 .             [  11: Lars Magne Ingebrigtsen] [87: gmane.discuss/13708] Re: Podcasts in gwene ?
 .                 [   6: Kevin Ryde             ] [87: gmane.discuss/13709] Re: Podcasts in gwene ?
 .                     [  12: Lars Magne Ingebrigtsen] [62: gmane.discuss/13710] Re: Podcasts in gwene ?
 .                         [   8: Kevin Ryde             ] [62: gmane.discuss/13711] Re: Podcasts in gwene ?
 .                             [  32: Lars Magne Ingebrigtsen] [77: gmane.discuss/13712] Re: Podcasts in gwene ?

The while loop is spinning for some reason in Adam Sjøgren's article.  I
have no idea why (forward-line 1) doesn't advance in the buffer.

The form is evaluated.

Here are the first lines in the *nntp* buffer:
13688	Podcasts in gwene ?	e20100633 <e20100633@inbox.lv>	Tue, 19 Oct 2010 14:49:03 +0200	<87wrpe4abk.fsf@EVA01.wopr.org>		4704	42	Xref: news.gmane.org gmane.discuss:13688
13689	Re: Podcasts in gwene ?	asjo@koldfront.dk (Adam =?utf-8?Q?Sj=C3=B8gren?=)	Tue, 19 Oct 2010 15:01:52 +0200	<87r5fmwd33.fsf@topper.koldfront.dk>	<87wrpe4abk.fsf@EVA01.wopr.org>	4497	36	Xref: news.gmane.org gmane.discuss:13689
13690	Re: Podcasts in gwene ?	e20100633 <e20100633@inbox.lv>	Tue, 19 Oct 2010 15:48:00 +0200	<87ocaq47lb.fsf@EVA01.wopr.org>	<87wrpe4abk.fsf@EVA01.wopr.org> <87r5fmwd33.fsf@topper.koldfront.dk>	4198	41	Xref: news.gmane.org gmane.discuss:13690
...

The point is on the second line, even after more iterations stepped
through in the while loop, than there are articles in the thread.

The headers vector contains (as expected) multiple copies of the parse
results of the NOV line.

sequence is nil.  Is that expected?  Is that the reason the loop isn't
terminating?

Re: A T hangs forever in gmane nnir

[Steinar Bang]

Andy Cohen and I edebug'd gnus-get-newsgroup-headers-xover yesterday.
And it does the expected thing, except that 
	  (forward-line 1)))
at the end of the while loop (gnus-sum.el:6437) doesn't advance point in
the *nntp* buffer, and therefore
		    (not (eobp)))
(in gnus-sum.el:6422) never becomes nil, and the while loop never
terminates. 

(current-buffer) evaluates to
 #<buffer  *nntpd*>
which is the right one, I think.

I have verified that it is something in my configuration that causes
this behaviour (because I see it on all my gnusen, and I didn't see it
in a new test user with a minimal git gnus setup, on one of the
computers where my user fails).

But I have noe idea what could cause it.

What, in my emacs and gnus config, could tamper with (forward-line 1) ?

Re: A T does not work in nnml ...

[Ulf Stegemann]

Andrew Cohen <cohen@andy.bu.edu> wrote:

> I just pushed a bunch of changes to improve thread-referral. I think
> this should fix most of the remaining idiosyncrasies. I also simplified
> the code. There is no longer an nnir-specific thread-referral function;
> the general one `gnus-summary-refer-thread' works, and now uses the same
> binding "A T" in nnir groups (so you should stop using "G T" and just
> always use "A T").

A quick check here revealed that `A T' now works quite as expected in
nnml groups. Great! Thanks a lot.

Ulf