Request for repost

Request for repost

[Harry Putnam]

I hope someone will remember this article and still have it available.

Karl Kleinpaste posted a reply to a message of mine here, concering what
the various gnus created files do and how they are generated.

active file
.overview
.agentview

Including detailed information about what the various field mean.

I saved it, but in the course of experiments with various archiving
methods, I've leveled my ding archive since then.  The online archive
must have this message, but I couldn't come up with a tricky enough
search string to find it.

Karl Kleinpaste;active;overview;agentview   doesn't turn it up.

Geez, I hope this wasn't posted on gnu.emacs.gnus, but I tried a few
nnweb queries there too.

Re: Request for repost

[Karl Kleinpaste]

Unearthed, and sent privately.

Re: Request for repost

[Harry Putnam]

Karl Kleinpaste <karl@charcoal.com> writes:

> Unearthed, and sent privately.
Nice to know there are real archivists around...

Any one want to take a stab at explaining the .agentview file.  Karl
did a first rate job on active, overview and history.

The .agentview files look like a list of the file numbers with this
notation (NUMBER . t) (NUMBER . t) ===> major long line.

These appear to be the same numbers as on the .overview file so not
sure how these might be used.  Is this something more than simply a
count of the article numbers that agent has retrieved?

I'd like to hear an explanation in plain words, of how gnus builds the
active and overview files.  I mean the details of how the stripping is
done. Looking at what I guess is the code, isn't very helpful to me.

What all this is about is rebuilding the active file after every rsync
transfer from my $HOME setup to an archive.  All the new stuff just
gets brute forced into the Archive directories which are a mirror
image of the directories under News/agent.

This archive is under the control of a secondary nnml server.

After hours of tinkering with command line nov regeneration 
( thanks  Shenghuo ZHU), Still something is breaking and some messages
never make it to the nov .overview file.  And consequently are
invisible to gnus.

Not sure at all how all this works inside gnus and It isn't explained
in much detail in the Info pages.

Karls detailed account of what those files do has augmented my
suspicion that the trouble is coming at the active file level.

I've pieced together a tool, that probably isn't near as good at this
as gnus own tools that do this, but couldn't decipher how gnus does it.

The tool I've made dredges through every directory containing messages
and reports the highest and lowest file number.  Then stips out all
the "/" and replaces them with "." and prints a nice alphabetically
ordered `active' file granting posting privileges  ..  : )

Useless on an nnml server but I couldn't resist.

Also worked out a tool that can generate the nov (.overview) file
But that part, gnus seems to be handling fine, long as I feed it a
nice acurate `active' file.

I've only used this a few times now but so far I don't see the problem
I was having, of messages being invisible to gnus..

This has only been tested out on 16,000+ messages some 76M.  

Whereas the original archive I ruined was some 120,000 and about
400M.