problem: .overview~[0-9]*~ files not being deleted

problem: .overview~[0-9]*~ files not being deleted

[wayne]

Ok, I have a wierd one that I hope someone has heard of before.  I've
poked around some, but I can't find the cause.  I confess I haven't
dug deeply into the code though.

A while back, I suddenly started to accumulate old .overview~[0-9]*~
files in my ~/News/agent/ subdirectories.  They stick around for ever,
so it isn't just keeping the last 4 files.  They seem to accumulate in
most groups, but maybe not all of them.

I think this started happening when I upgraded to gnus "No gnus v0.4",
but I'm not certain.  It took a while before I noticed that my disk
was overflowing with them.  I'm running Debian Linux and uxing "XEmacs
21.4 (patch 17) \"Jumbo Shrimp\" [Lucid] (i386-debian-linux) of Sat
May 28 2005 on penell" 


Other than the accumulation of these .overview files, things seem to
be working fine.


Any ideas?  Should I just create a cron job to delete them?


-wayne

Re: problem: .overview~[0-9]*~ files not being deleted

[Kevin Greiner]

wayne <wayne@schlitt.net> writes:

> Ok, I have a wierd one that I hope someone has heard of before.  I've
> poked around some, but I can't find the cause.  I confess I haven't
> dug deeply into the code though.
>
> A while back, I suddenly started to accumulate old .overview~[0-9]*~
> files in my ~/News/agent/ subdirectories.  They stick around for ever,
> so it isn't just keeping the last 4 files.  They seem to accumulate in
> most groups, but maybe not all of them.
>
> I think this started happening when I upgraded to gnus "No gnus v0.4",
> but I'm not certain.  It took a while before I noticed that my disk
> was overflowing with them.  I'm running Debian Linux and uxing "XEmacs
> 21.4 (patch 17) \"Jumbo Shrimp\" [Lucid] (i386-debian-linux) of Sat
> May 28 2005 on penell" 
>
>
> Other than the accumulation of these .overview files, things seem to
> be working fine.
>
>
> Any ideas?  Should I just create a cron job to delete them?

The cron job is a good idea.  The agent is generating these files
because it corrected an error in the overview file.  The original idea
was to keep them around to help with diagnosing the error.
Limiting the agent to a couple of backup files sounds like a good
idea.  I'll see about getting that in the next few days.

Kevin

Re: problem: .overview~[0-9]*~ files not being deleted

[wayne]

In <uy83x5j53.fsf@vignette.com> Kevin Greiner <kevin.greiner@vignette.com> writes:

> wayne <wayne@schlitt.net> writes:
>
>> A while back, I suddenly started to accumulate old .overview~[0-9]*~
>> files in my ~/News/agent/ subdirectories.  They stick around for ever,
>> so it isn't just keeping the last 4 files.  They seem to accumulate in
>> most groups, but maybe not all of them.
>>
>> [...]
>
> The cron job is a good idea.  The agent is generating these files
> because it corrected an error in the overview file.

I saw some discussions about the backup overview files being created
when errors were detected, so I went and deleted *all* the overview
files.  The backup files were immediately being created.  So, this
isn't a case of a rare corruption of the .overview that sticks around.
Whatever it is, the problem happens very quickly.


-wayne

Re: problem: .overview~[0-9]*~ files not being deleted

[Kevin Greiner]

wayne <wayne@schlitt.net> writes:

> In <uy83x5j53.fsf@vignette.com> Kevin Greiner <kevin.greiner@vignette.com> writes:
>
>> wayne <wayne@schlitt.net> writes:
>>
>>> A while back, I suddenly started to accumulate old .overview~[0-9]*~
>>> files in my ~/News/agent/ subdirectories.  They stick around for ever,
>>> so it isn't just keeping the last 4 files.  They seem to accumulate in
>>> most groups, but maybe not all of them.
>>>
>>> [...]
>>
>> The cron job is a good idea.  The agent is generating these files
>> because it corrected an error in the overview file.
>
> I saw some discussions about the backup overview files being created
> when errors were detected, so I went and deleted *all* the overview
> files.  The backup files were immediately being created.  So, this
> isn't a case of a rare corruption of the .overview that sticks around.
> Whatever it is, the problem happens very quickly.

Now, that's interesting.  There's quite a few agent users in the world
so this isn't a common issue.  There must be something about the nntp
headers that your server is providing.  There are messages written as
the same time as the backup is created.  Something like 'Duplicate
overview line for ...' or 'Overview buffer not sorted'.  Can you tell
me which message you are seeing?  You might need to look in the
*Messages* buffer to find it.

TIA,
Kevin

Re: problem: .overview~[0-9]*~ files not being deleted

[wayne]

In <ufyq24edn.fsf@vignette.com> Kevin Greiner <kevin.greiner@vignette.com> writes:

> wayne <wayne@schlitt.net> writes:
>
>> I saw some discussions about the backup overview files being created
>> when errors were detected, so I went and deleted *all* the overview
>> files.  The backup files were immediately being created.  So, this
>> isn't a case of a rare corruption of the .overview that sticks around.
>> Whatever it is, the problem happens very quickly.
>
> Now, that's interesting.  There's quite a few agent users in the world
> so this isn't a common issue.  There must be something about the nntp
> headers that your server is providing.  There are messages written as
> the same time as the backup is created.  Something like 'Duplicate
> overview line for ...' or 'Overview buffer not sorted'.  Can you tell
> me which message you are seeing?  You might need to look in the
> *Messages* buffer to find it.


I don't see errors very often, but that may be because the messages go
by too quickly for me to see them.  Checking my *Message-Log*, I see
this:


Retrieving newsgroup: nntp+news-server.neb.rr.com:gnu.emacs.gnus...
Opening nntp server on news-server.neb.rr.com...
Opening nntp server on news-server.neb.rr.com...done
Fetching headers for nntp+news-server.neb.rr.com:gnu.emacs.gnus...
nntp read: 10k
nntp read: 12k
nntp read: 13k
nntp read: 15k
nntp read: 16k
nntp read: 17k
nntp read: 18k
Finding sort keys...
Sorting records...
Reordering buffer...
Reordering buffer... Done
Created backup copy of overview in /home/wayne/News/agent/nntp/news-server.neb.rr.com/gnu/emacs/gnus/.overview~1~.
Duplicate overview line for 49692
Duplicate overview line for 49694
[50 more "duplicate" errors deleted]
Fetching headers for nntp+news-server.neb.rr.com:gnu.emacs.gnus...done
Scoring...
Scoring...done
Making sparse threads...
Making sparse threads...done
Generating summary...
Generating summary...done


Now, one thing I may be doing that others aren't, is that I like to
see some context with the messages so I usually add a prefix arg when
I enter a group.  That is, I type "99 space" on the group line.  I've
tried setting some variables to have this done automatically, but none
really seem to work as well.  For example, I have the following in my
init.el file:

(setq gnus-use-cross-reference t)
(setq gnus-auto-select-newsgroup nil)
(setq gnus-summary-check-current t)
(setq gnus-fetch-old-headers 200)
(add-hook 'gnus-group-mode-hook 'gnus-topic-mode)
(add-hook 'message-sent-hook 'gnus-score-followup-thread)
(setq gnus-use-cache 'passive)
(setq gnus-build-sparse-threads 'some)
(setq gnus-keep-backlog 200)
(setq gnus-read-active-file nil)
(setq gnus-large-newsgroup 1000)


Mind you, I've been using gnus since 1996 and my init.el file has
accumulated a large amount of cruft over the years.  So, this is just
part of the 764 lines in the init.el file and I don't remember why I
have all that stuff set.  About 4 years ago, I switched to using the
agent stuff so that I could read email when my laptop was off the net
and around the same time I went on a rampage of tweaking variables.


-wayne

Re: problem: .overview~[0-9]*~ files not being deleted

[Kevin Greiner]

wayne <wayne@schlitt.net> writes:

> Mind you, I've been using gnus since 1996 and my init.el file has
> accumulated a large amount of cruft over the years.  So, this is just
> part of the 764 lines in the init.el file and I don't remember why I
> have all that stuff set.  About 4 years ago, I switched to using the
> agent stuff so that I could read email when my laptop was off the net
> and around the same time I went on a rampage of tweaking variables.

Now ain't that the truth.  I have much the same trouble.

Thanks, for the details.  It would appear that your nntp server is
sending duplicate header entries in its response.  The agent tested
for unsorted headers but missed the duplicate test.  I've updated the
logic so you shouldn't see this error once you update to the
gnus-agent.el in CVS.

Kevin

Re: problem: .overview~[0-9]*~ files not being deleted

[wayne]

In <ulkztqizc.fsf@vignette.com> Kevin Greiner <kevin.greiner@vignette.com> writes:

> Thanks, for the details.  It would appear that your nntp server is
> sending duplicate header entries in its response.  The agent tested
> for unsorted headers but missed the duplicate test.  I've updated the
> logic so you shouldn't see this error once you update to the
> gnus-agent.el in CVS.

Sounds great!  Thanks much!


-wayne