cannot always fetch parent article

cannot always fetch parent article

[Ansgar Burchardt]

[I posted a less detailed version of this three days ago on
 gnu.emacs.gnus [1], but got no answer so far]

Hi,

I'm using gnus to read newsgroups, but somehow the ^-command
(gnus-summary-refer-parent-article) doesn't always work.

If the parent is not in the summary buffer the command fails:
  423 No such article number in this group 
  No such article (may have expired or been canceled)

I tried setting
  (setq gnus-refer-article-method
        '(current (nnweb "google" (nnweb-type google))))
but this had no effect.  Using only Google works but is painfully slow.

Capturing traffic between Gnus and the NNTP server reveals the problem.
I have a message with this header:
  References: <vfbqdnyn4t.fsf@googlemail.com>
and try to fetch the parent.  Gnus does it like this:
  Gnus:
    HEAD <vfbqdnyn4t.fsf@googlemail.com>
  Server:
    221 0 <vfbqdnyn4t.fsf@googlemail.com> head
    [...]
    Xref: dreader10.news.xs4all.nl gnu.emacs.gnus:36849
  Gnus:
    ARTICLE 36849
  Server:
    423 No such article number in this group

The problem is that the number in the Xref header is *not* the article
number to use with the ARTICLE command.  It is even outside the range
for valid article numbers:
  LIST ACTIVE gnu.emacs.gnus
  215 Newsgroups in form "group high low flags".
  gnu.emacs.gnus 0000077358 0000077256 y

Even more fun:  The Xref header doesn't even stay the same!
  C: group gnu.emacs.gnus
  S: 211 103 77256 77358 gnu.emacs.gnus
  S: head 77357
  S: 221 77357 <vfbqdnyn4t.fsf@googlemail.com> head
  S: [...]
  S: Xref: dreader10.news.xs4all.nl gnu.emacs.gnus:77357

  C: head <vfbqdnyn4t.fsf@googlemail.com>
  S: 221 0 <vfbqdnyn4t.fsf@googlemail.com> head
  S: [...]
  S: Xref: dreader10.news.xs4all.nl gnu.emacs.gnus:37424

  C: head <vfbqdnyn4t.fsf@googlemail.com> 
  S: 221 0 <vfbqdnyn4t.fsf@googlemail.com> head
  S: [...]
  S: Xref: dreader10.news.xs4all.nl gnu.emacs.gnus:36849

This happens with both newsservers I have tried.

Is there any way to make Gnus fetch the whole article at once instead of
first fetching the header and then the whole article?  This would at
least be a workaround.

I'm using the Debian package for emacs 22.1+1 and the included version
of Gnus.

Ansgar

[1] see <gnus-86k5sciimj.fsf@wg42.homeip.net>

Re: cannot always fetch parent article

[Katsumi Yamaoka]

>>>>> Ansgar Burchardt wrote:

> [I posted a less detailed version of this three days ago on
>  gnu.emacs.gnus [1], but got no answer so far]

I read this article but I still have no idea for it.

> I'm using gnus to read newsgroups, but somehow the ^-command
> (gnus-summary-refer-parent-article) doesn't always work.

[...]

> Even more fun:  The Xref header doesn't even stay the same!
>   C: group gnu.emacs.gnus
>   S: 211 103 77256 77358 gnu.emacs.gnus
>   S: head 77357
>   S: 221 77357 <vfbqdnyn4t.fsf@googlemail.com> head
>   S: [...]
>   S: Xref: dreader10.news.xs4all.nl gnu.emacs.gnus:77357

>   C: head <vfbqdnyn4t.fsf@googlemail.com>
>   S: 221 0 <vfbqdnyn4t.fsf@googlemail.com> head
>   S: [...]
>   S: Xref: dreader10.news.xs4all.nl gnu.emacs.gnus:37424

>   C: head <vfbqdnyn4t.fsf@googlemail.com>
>   S: 221 0 <vfbqdnyn4t.fsf@googlemail.com> head
>   S: [...]
>   S: Xref: dreader10.news.xs4all.nl gnu.emacs.gnus:36849

> This happens with both newsservers I have tried.

Strange.  I did the same test with three news servers and
confirmed the Xref header never varies.

news.individual.net (INN?)
  Xref: uni-berlin.de gnu.emacs.gnus:81543

news.gnus.org (INN 2.3.2)
  Xref: quimby.gnus.org gnu.emacs.gnus:48908

the local server (INN 2.3.5)
  Xref: pathhost gnu.emacs.gnus:58643

> Is there any way to make Gnus fetch the whole article at once instead of
> first fetching the header and then the whole article?  This would at
> least be a workaround.

Even if your servers always return the right article for the command

article <vfbqdnyn4t.fsf@googlemail.com>

, I don't think there is an easy way to do that with Gnus.  Isn't
it better to look into the reason why the Xref header varies (and
to fix the problem in the servers side if possible)?

Regards,

Re: cannot always fetch parent article

[Ansgar Burchardt]

Katsumi Yamaoka wrote:
>>>>>> Ansgar Burchardt wrote:
>> Even more fun:  The Xref header doesn't even stay the same!
>> This happens with both newsservers I have tried.
>
> Strange.  I did the same test with three news servers and
> confirmed the Xref header never varies.
>
> news.individual.net (INN?)
> news.gnus.org (INN 2.3.2)
> the local server (INN 2.3.5)

newszilla.xs4all.nl and news.arcor.de both seem to run Diablo.

I found an interesting post about this [1]: apparently Diablo will
forward a request by Message-Id to a different system, which likely has
different article numbers.  If load balancing is used this might explain
why I get sometimes different article numbers.

>> Is there any way to make Gnus fetch the whole article at once instead of
>> first fetching the header and then the whole article?  This would at
>> least be a workaround.
>
> Even if your servers always return the right article for the command
>
> article <vfbqdnyn4t.fsf@googlemail.com>
>
> , I don't think there is an easy way to do that with Gnus.  Isn't
> it better to look into the reason why the Xref header varies (and
> to fix the problem in the servers side if possible)?

If I post a followup to an article to a different group (e.g. a random
*.test group) and then try to fetch the parent article it works without
problems.  The problem only occurs if the parent was posted in the
same group as the followup and Gnus tries to guess the article number.
If Gnus behaved as in the first case, everything would be fine.

Ansgar

Footnotes: 
[1]  <pan.2004.03.17.10.27.18.676702@proxad.net>

Re: cannot always fetch parent article

[Katsumi Yamaoka]

[-- Attachment #1: Type: text/plain, Size: 1750 bytes --]

>>>>> Ansgar Burchardt wrote:

> newszilla.xs4all.nl and news.arcor.de both seem to run Diablo.

> I found an interesting post about this [1]: apparently Diablo will
> forward a request by Message-Id to a different system, which likely has
> different article numbers.  If load balancing is used this might explain
> why I get sometimes different article numbers.

I see.  So, we should use only Message-IDs to refer to articles
for such news servers.

[...]

> If I post a followup to an article to a different group (e.g. a random
> *.test group) and then try to fetch the parent article it works without
> problems.  The problem only occurs if the parent was posted in the
> same group as the followup and Gnus tries to guess the article number.
> If Gnus behaved as in the first case, everything would be fine.

I studied how Gnus refers to parent articles, and realized that
Gnus uses an article number only when a parent article is in the
current group.  Could you try the patch attached below?

The new variable `nntp-xref-number-is-evil' controls whether Gnus
ignores article numbers in the Xref header.  This is a server
variable, so you can set it per server as follows:

(setq gnus-select-method
      '(nntp "newszilla"
	     (nntp-address "newszilla.xs4all.nl")
	     (nntp-xref-number-is-evil t)
	     ...
	     ))

Or you can simply use the following if your servers are all Diablo:

(setq nntp-xref-number-is-evil t)

Note that the number of a parent article in the current group
will be displayed as a negative number if you've configured
`gnus-summary-line-format', `gnus-summary-mode-line-format' or
`gnus-article-mode-line-format' to do so.

I verified the patch can be applied to both Gnus v5.11 of Emacs
22.1 and No Gnus v0.7.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: Type: text/x-patch, Size: 1032 bytes --]

--- nntp.el~	2007-07-18 12:07:39 +0000
+++ nntp.el	2007-08-08 10:12:33 +0000
@@ -203,6 +203,14 @@
 server there that you can connect to.  See also
 `nntp-open-connection-function'")
 
+(defvoo nntp-xref-number-is-evil nil
+  "*If non-nil, Gnus never trusts article numbers in the Xref header.
+Some news servers, e.g., ones running Diablo, run multiple engines
+having the same articles but article numbers are not kept synchronized
+between them.  If you connect to such a server, set this to t, and Gnus
+never uses article numbers (that appear in the Xref header and vary by
+which engine is chosen) in order to refer to articles.")
+
 (defvoo nntp-coding-system-for-read 'binary
   "*Coding system to read from NNTP.")
 
@@ -1702,7 +1710,8 @@
 		    (match-string 1 xref))
 		   (t "")))
 	  (cond
-	   ((and (setq xref (mail-fetch-field "xref"))
+	   ((and (not nntp-xref-number-is-evil)
+		 (setq xref (mail-fetch-field "xref"))
 		 (string-match
 		  (if group
 		      (concat "\\(" (regexp-quote group) "\\):\\([0-9]+\\)")

Re: cannot always fetch parent article

[Ansgar Burchardt]

Katsumi Yamaoka wrote:
>>>>>> Ansgar Burchardt wrote:
>> If I post a followup to an article to a different group (e.g. a random
>> *.test group) and then try to fetch the parent article it works without
>> problems.  The problem only occurs if the parent was posted in the
>> same group as the followup and Gnus tries to guess the article number.
>> If Gnus behaved as in the first case, everything would be fine.
>
> I studied how Gnus refers to parent articles, and realized that
> Gnus uses an article number only when a parent article is in the
> current group.  Could you try the patch attached below?

[ patch introducing nntp-xref-number-is-evil snipped ]

The patch solves the problem.  Thank you.

Ansgar

Re: cannot always fetch parent article

[Katsumi Yamaoka]

>>>>> Ansgar Burchardt wrote:

> [ patch introducing nntp-xref-number-is-evil snipped ]

> The patch solves the problem.  Thank you.

I've installed it in both the trunk and the v5-10 branch with an
Info document.