Arghh, my Gnus is going crazy !!

Arghh, my Gnus is going crazy !!

[Xavier Maillard]

Hello,

Dunno what is going on on Gnus but I have *HUGE* troubles since
yesterday and my latest upgrade (No Gnus).

Whenever I try to enter a USENET group, Gnus complains about
'duplucate line in overview file' and delete all entry from the
file.

This makes disappearing all the group articles (cached or not,
downloaded or not) and the group remains empty.

Deleting the .agentview, .overview and doing C-k in the summary
buffer fix the problem when I resubscribe to the group. Problem
is that this reappear when I next enter the group.

I checked the famous .overview file and here is a diff on it.
Quite weird:

,----
|  diff -u .overview\~1\~ .overview | more                                                                                                                [1.29 1.30 - 58% 3%]
| --- .overview~1~        2004-09-25 14:21:52 +0200
| +++ .overview   2004-09-25 14:21:52 +0200
| @@ -1,1081 +1,811 @@
|  794    Re: Facturation : Avoir raph <fraph@free.fr>    Thu, 08 Jul 2004 09:05:07 +0200 <opsas4mtkyosm168@news.nerim.net>       <ccgrk3$ish$1@biggoron.nerim.net>       2473 47       Xref: news.gnu-rox.org nerim.support:794
| -794    Re: Facturation : Avoir raph <fraph@free.fr>    Thu, 08 Jul 2004 09:05:07 +0200 <opsas4mtkyosm168@news.nerim.net>       <ccgrk3$ish$1@biggoron.nerim.net>       2473 47       Xref: news.gnu-rox.org nerim.support:794
| -795    Re: Facturation : Avoir Dominique ROUSSEAU <domi@nerim.net>     Thu, 8 Jul 2004 09:52:09 +0200  <9fuicc.ej3.ln@127.0.0.1>       <ccgrk3$ish$1@biggoron.nerim.net> <ops
| as4mtkyosm168@news.nerim.net>   1571    25      Xref: news.gnu-rox.org nerim.support:795
| +794795 Re: Facturation : Avoir Dominique ROUSSEAU <domi@nerim.net>     Thu, 8 Jul 2004 09:52:09 +0200  <9fuicc.ej3.ln@127.0.0.1>       <ccgrk3$ish$1@biggoron.nerim.net> <ops
| as4mtkyosm168@news.nerim.net>   1571    25      Xref: news.gnu-rox.org nerim.support:795
|  795
`----

Something is doing concatenation somewhere (look 794, 795 and the
794795). What is that ????

Thank you for any hint on fixing this.

-- 
      Xavier Maillard| "Stand Back! I'm a programmer!"
.0.             zedek@gnu-rox.orgz|
..0             (+33) 326 770 221 |   Webmaster, emacsfr.org
000              PGP : 0x1E028EA5 |    Membre de l' APRIL

Re: Arghh, my Gnus is going crazy !!

[Kevin Greiner]

Xavier Maillard <zedek@gnu-rox.org> writes:

> Hello,
>
> Dunno what is going on on Gnus but I have *HUGE* troubles since
> yesterday and my latest upgrade (No Gnus).
>
> Whenever I try to enter a USENET group, Gnus complains about
> 'duplucate line in overview file' and delete all entry from the
> file.
>
> This makes disappearing all the group articles (cached or not,
> downloaded or not) and the group remains empty.
>
> Deleting the .agentview, .overview and doing C-k in the summary
> buffer fix the problem when I resubscribe to the group. Problem
> is that this reappear when I next enter the group.
>
> I checked the famous .overview file and here is a diff on it.
> Quite weird:
>
> ,----
> |  diff -u .overview\~1\~ .overview | more                                                                                                                [1.29 1.30 - 58% 3%]
> | --- .overview~1~        2004-09-25 14:21:52 +0200
> | +++ .overview   2004-09-25 14:21:52 +0200
> | @@ -1,1081 +1,811 @@
> |  794    Re: Facturation : Avoir raph <fraph@free.fr>    Thu, 08 Jul 2004 09:05:07 +0200 <opsas4mtkyosm168@news.nerim.net>       <ccgrk3$ish$1@biggoron.nerim.net>       2473 47       Xref: news.gnu-rox.org nerim.support:794
> | -794    Re: Facturation : Avoir raph <fraph@free.fr>    Thu, 08 Jul 2004 09:05:07 +0200 <opsas4mtkyosm168@news.nerim.net>       <ccgrk3$ish$1@biggoron.nerim.net>       2473 47       Xref: news.gnu-rox.org nerim.support:794
> | -795    Re: Facturation : Avoir Dominique ROUSSEAU <domi@nerim.net>     Thu, 8 Jul 2004 09:52:09 +0200  <9fuicc.ej3.ln@127.0.0.1>       <ccgrk3$ish$1@biggoron.nerim.net> <ops
> | as4mtkyosm168@news.nerim.net>   1571    25      Xref: news.gnu-rox.org nerim.support:795
> | +794795 Re: Facturation : Avoir Dominique ROUSSEAU <domi@nerim.net>     Thu, 8 Jul 2004 09:52:09 +0200  <9fuicc.ej3.ln@127.0.0.1>       <ccgrk3$ish$1@biggoron.nerim.net> <ops
> | as4mtkyosm168@news.nerim.net>   1571    25      Xref: news.gnu-rox.org nerim.support:795
> |  795
> `----
>
> Something is doing concatenation somewhere (look 794, 795 and the
> 794795). What is that ????
>
> Thank you for any hint on fixing this.

This is actually a very old problem. We've made very little progress
with it, hence the gnus-agent-check-overview-buffer function, as it is
rarely reproducable.  I suspect that the actual bug is in your server
and that it is sending you bad headers.

Let's try this first:

Delete the group's .overview and .agentview so that you fetch all new
headers from nntp.

Customize nntp-nov-is-evil to set it to t.  Try to open the group, if
the problem has gone away then we know that your server's nov database
is corrupt.

If that doesn't fix it, would you be willing to do a little debugging
for me?

If so, please do the following.

Then do
M-x load-library<ret>gnus-agent.el<ret>
M-x gnus
M-x debug-on-entry<ret>gnus-agent-check-overview-buffer<ret>

Now try to open the group's summary buffer.  This should launch the
debugger.

If I've gotten this right, the stack trace should show that
gnus-agent-check-overview-buffer was called by
gnus-agent-fetch-headers.  The current buffer should contain the raw
headers returned by nntp.  How do they look?  If they look fine, the
likely culpret is the gnus-agent-braid-nov function it is about to
be called so you can just step forward from here to watch it
execute.  If that is too much to ask, you can also save the nntp
headers (the entire buffer) to a file then email it to me.  I'll
fool my gnus into parsing those headers which should let me
reproduce the bug.

Kevin

Re: Arghh, my Gnus is going crazy !!

[Xavier Maillard]

On 25 sep 2004, Kevin Greiner wrote:

> Xavier Maillard <zedek@gnu-rox.org> writes:
> 

 [ ... ]

> > Something is doing concatenation somewhere (look 794, 795 and
> > the 794795). What is that ????
> > 
> > Thank you for any hint on fixing this.
> 
> This is actually a very old problem. We've made very little
> progress with it, hence the gnus-agent-check-overview-buffer
> function, as it is rarely reproducable. I suspect that the
> actual bug is in your server and that it is sending you bad
> headers.
> 
> Let's try this first:
> 
> Delete the group's .overview and .agentview so that you fetch
> all new headers from nntp.

Problem with such tecnic is that AFAIK, it makes forget the
marks :(

> Customize nntp-nov-is-evil to set it to t. Try to open the
> group, if the problem has gone away then we know that your
> server's nov database is corrupt.

Nothing change. Err, at least, rmoving .agentview/.overview and
doing C-k on the group is the only solution I know. When doing
that, something seems to "fix" the problem.

> If that doesn't fix it, would you be willing to do a little
> debugging for me?
> 
> If so, please do the following.
> 
> Then do
> M-x load-library<ret>gnus-agent.el<ret>
> M-x gnus
> M-x debug-on-entry<ret>gnus-agent-check-overview-buffer<ret>
> 
> Now try to open the group's summary buffer. This should launch
> the debugger.

Yes it did.
 
> If I've gotten this right, the stack trace should show that
> gnus-agent-check-overview-buffer was called by
> gnus-agent-fetch-headers. The current buffer should contain the

Not exactly but no problem.

> raw headers returned by nntp. How do they look? If they look

I didn't get any "raw nntp headrs" buffer here. When it say it
entered function blah-blah, hitting on 'c' open the summary
buffer.

-- 
"sometimes i feel like we're making emacs better and better because we don't 
 know what to do with emacs once it is finished."

    -- AlexSchroeder on #emacs @OPN

Re: Arghh, my Gnus is going crazy !!

[Kevin Greiner]

Xavier Maillard <zedek@gnu-rox.org> writes:

> On 25 sep 2004, Kevin Greiner wrote:
>
>> Xavier Maillard <zedek@gnu-rox.org> writes:
>> 
>
>  [ ... ]
>
>> > Something is doing concatenation somewhere (look 794, 795 and
>> > the 794795). What is that ????
>> > 
>> > Thank you for any hint on fixing this.
>> 
>> This is actually a very old problem. We've made very little
>> progress with it, hence the gnus-agent-check-overview-buffer
>> function, as it is rarely reproducable. I suspect that the
>> actual bug is in your server and that it is sending you bad
>> headers.
>> 
>> Let's try this first:
>> 
>> Delete the group's .overview and .agentview so that you fetch
>> all new headers from nntp.
>
> Problem with such tecnic is that AFAIK, it makes forget the
> marks :(
>
>> Customize nntp-nov-is-evil to set it to t. Try to open the
>> group, if the problem has gone away then we know that your
>> server's nov database is corrupt.
>
> Nothing change. Err, at least, rmoving .agentview/.overview and
> doing C-k on the group is the only solution I know. When doing
> that, something seems to "fix" the problem.
>
>> If that doesn't fix it, would you be willing to do a little
>> debugging for me?
>> 
>> If so, please do the following.
>> 
>> Then do
>> M-x load-library<ret>gnus-agent.el<ret>
>> M-x gnus
>> M-x debug-on-entry<ret>gnus-agent-check-overview-buffer<ret>
>> 
>> Now try to open the group's summary buffer. This should launch
>> the debugger.
>
> Yes it did.
>  
>> If I've gotten this right, the stack trace should show that
>> gnus-agent-check-overview-buffer was called by
>> gnus-agent-fetch-headers. The current buffer should contain the
>
> Not exactly but no problem.
>
>> raw headers returned by nntp. How do they look? If they look
>
> I didn't get any "raw nntp headrs" buffer here. When it say it
> entered function blah-blah, hitting on 'c' open the summary
> buffer.
>
> -- 
> "sometimes i feel like we're making emacs better and better because we don't 
>  know what to do with emacs once it is finished."
>
>     -- AlexSchroeder on #emacs @OPN

I found, and fixed, the problem.  Update from CVS.

Kevin

Re: Arghh, my Gnus is going crazy !!

[test]

Kevin Greiner a écrit :

> 
> I found, and fixed, the problem.  Update from CVS.
> 
> Kevin
> 
> 

What was it ?? :/

Thank you.

Re: Arghh, my Gnus is going crazy !!

[Kevin Greiner]

test <invalig@toto.com> writes:

> Kevin Greiner a écrit :
>
>> I found, and fixed, the problem.  Update from CVS.
>> Kevin
>> 
>
> What was it ?? :/
>
> Thank you.

gnus-agent-check-overview-buffer.  Interestingly, this function was
added in 2003-01-14 to help detect obscure errors that were occurring.
Now, it's part of the problem.

Kevin

Re: Arghh, my Gnus is going crazy !!

[Xavier Maillard]

On 28 sep 2004, Kevin Greiner wrote:

> test <invalig@toto.com> writes:
> 
> > Kevin Greiner a écrit :
> > 
> > > I found, and fixed, the problem.  Update from CVS.
> > > Kevin
> > > 
> > 
> > What was it ?? :/
> > 
> > Thank you.
> 
> gnus-agent-check-overview-buffer. Interestingly, this function
> was added in 2003-01-14 to help detect obscure errors that were
> occurring. Now, it's part of the problem.

:/ Thank you for the quick and good support.

Oh BTW, I still have trouble with the synchronization of the
flags (sutff read/marked during an unplugged session is still
forgotten when sync is done). I saw it yesterday when I reentered
my gnus-bug group where I had removed (when unplugged) all the
spam messages. When back nd after a 'g' on the group, all spam
messages reappeared :(

Regards.
-- 
      Xavier Maillard| "Stand Back! I'm a programmer!"
.0.             zedek@gnu-rox.orgz|
..0             (+33) 326 770 221 |   Webmaster, emacsfr.org
000              PGP : 0x1E028EA5 |    Membre de l' APRIL

Re: Arghh, my Gnus is going crazy !!

[Wes Hardaker]

>>>>> On Tue, 28 Sep 2004 08:22:06 +0200, Xavier Maillard <zedek@gnu-rox.org> said:

zeDek> When back nd after a 'g' on the group, all spam messages
zeDek> reappeared :(

Oddly enough, I think it was the spam messages that made me stop
trying offline reading because them reappearing wasn't worth it!

-- 
"In the bathtub of history the truth is harder to hold than the soap,
 and much more difficult to find."  -- Terry Pratchett

Re: Arghh, my Gnus is going crazy !!

[Kevin Greiner]

Xavier Maillard <zedek@gnu-rox.org> writes:

> On 28 sep 2004, Kevin Greiner wrote:
>
>> test <invalig@toto.com> writes:
>> 
>> > Kevin Greiner a écrit :
>> > 
>> > > I found, and fixed, the problem.  Update from CVS.
>> > > Kevin
>> > > 
>> > 
>> > What was it ?? :/
>> > 
>> > Thank you.
>> 
>> gnus-agent-check-overview-buffer. Interestingly, this function
>> was added in 2003-01-14 to help detect obscure errors that were
>> occurring. Now, it's part of the problem.
>
> :/ Thank you for the quick and good support.
>
> Oh BTW, I still have trouble with the synchronization of the
> flags (sutff read/marked during an unplugged session is still
> forgotten when sync is done). I saw it yesterday when I reentered
> my gnus-bug group where I had removed (when unplugged) all the
> spam messages. When back nd after a 'g' on the group, all spam
> messages reappeared :(

Are you up to date?  I checked the latest patch about 24 hours ago.

If you're up to date and still having the problem, then I need more
information.  What would be best is if you could step through
gnus-agent-synchronize-group-flags.  I just added it to try to fix up
the info structures when the flags are sync'd.  It works for me.  Why
isn't it working for you?

Kevin

Re: Arghh, my Gnus is going crazy !!

[test]

>>Oh BTW, I still have trouble with the synchronization of the
>>flags (sutff read/marked during an unplugged session is still
>>forgotten when sync is done). I saw it yesterday when I reentered
>>my gnus-bug group where I had removed (when unplugged) all the
>>spam messages. When back nd after a 'g' on the group, all spam
>>messages reappeared :(
> 
> 
> Are you up to date?  I checked the latest patch about 24 hours ago.

My latest checkout was on the 28th at 6AM. Maybe I should update ?

Re: Arghh, my Gnus is going crazy !!

[Kevin Greiner]

Xavier Maillard <zedek@gnu-rox.org> writes:

> Hello,
>
> Dunno what is going on on Gnus but I have *HUGE* troubles since
> yesterday and my latest upgrade (No Gnus).
>
> Whenever I try to enter a USENET group, Gnus complains about
> 'duplucate line in overview file' and delete all entry from the
> file.
>
> This makes disappearing all the group articles (cached or not,
> downloaded or not) and the group remains empty.
>
> Deleting the .agentview, .overview and doing C-k in the summary
> buffer fix the problem when I resubscribe to the group. Problem
> is that this reappear when I next enter the group.
>
> I checked the famous .overview file and here is a diff on it.
> Quite weird:
>
> ,----
> |  diff -u .overview\~1\~ .overview | more                                                                                                                [1.29 1.30 - 58% 3%]
> | --- .overview~1~        2004-09-25 14:21:52 +0200
> | +++ .overview   2004-09-25 14:21:52 +0200
> | @@ -1,1081 +1,811 @@
> |  794    Re: Facturation : Avoir raph <fraph@free.fr>    Thu, 08 Jul 2004 09:05:07 +0200 <opsas4mtkyosm168@news.nerim.net>       <ccgrk3$ish$1@biggoron.nerim.net>       2473 47       Xref: news.gnu-rox.org nerim.support:794
> | -794    Re: Facturation : Avoir raph <fraph@free.fr>    Thu, 08 Jul 2004 09:05:07 +0200 <opsas4mtkyosm168@news.nerim.net>       <ccgrk3$ish$1@biggoron.nerim.net>       2473 47       Xref: news.gnu-rox.org nerim.support:794
> | -795    Re: Facturation : Avoir Dominique ROUSSEAU <domi@nerim.net>     Thu, 8 Jul 2004 09:52:09 +0200  <9fuicc.ej3.ln@127.0.0.1>       <ccgrk3$ish$1@biggoron.nerim.net> <ops
> | as4mtkyosm168@news.nerim.net>   1571    25      Xref: news.gnu-rox.org nerim.support:795
> | +794795 Re: Facturation : Avoir Dominique ROUSSEAU <domi@nerim.net>     Thu, 8 Jul 2004 09:52:09 +0200  <9fuicc.ej3.ln@127.0.0.1>       <ccgrk3$ish$1@biggoron.nerim.net> <ops
> | as4mtkyosm168@news.nerim.net>   1571    25      Xref: news.gnu-rox.org nerim.support:795
> |  795
> `----
>
> Something is doing concatenation somewhere (look 794, 795 and the
> 794795). What is that ????

Please ignore my preceeding post.  I've fixed the concatenation
problem.  You should still get duplicate headers from somewhere so
expect the duplicate lines messages to continue for now.

To get the fix, check out the latest gnus-agent.el.

Kevin