Discussion: To del cached articles upon zapping group info?

Discussion: To del cached articles upon zapping group info?

[Wesley.Hardaker]

Lars and I have been going back and forth about the following situation:

   When I zapped my group attributes (ie, numbers associated with read
   articles) gnus didn't clear my cache, which affected what it
   thought was the current article range later on.  For instance, I
   had an article cached in comp.protocols.snmp with a article number
   of 832 or something like that, but the articles in the newsgroup
   were now 1-315 or so (due to the new news server).  Gnus keyt
   saying there were no new articles because it had marked 1-832 as
   read after the first time I read the group...

We are trying to decide what it should do about the cached
articles...  In the above situation, its horrible (no new news till
after 832 appears).

I said it should delete the cached articles, simply because they are
no longer valid and will conflict with new numbered articles.  There
is simply no place in the 1-* range for them as they have been taken
up by new numbers.

I suppose you could re-order them to something like -3 -> -1 or
something similar, but I don't know how gnus would handle that (this
is a new idea Lars).

So, in the end, Lars needs other suggestions?

Ideas?

Wes

Re: Discussion: To del cached articles upon zapping group info?

[David Moore]

Wesley.Hardaker@sphys.unil.ch writes:
> I said it should delete the cached articles, simply because they are
> no longer valid and will conflict with new numbered articles.  There
> is simply no place in the 1-* range for them as they have been taken
> up by new numbers.

	Well, it certainly shouldn't delete them, although you want it
to stick them into some other archive kind of group.  You probably
cached them because they had useful information.  But you obviously
can't leave them in the cache tree, since they conflict with other
messages.  You could probably roll your old articles into an archive,
set the new real newsgroup to a highish level, and drop a virtual group
on top of them to combine them.  Although this isn't a great solution.

> I suppose you could re-order them to something like -3 -> -1 or
> something similar, but I don't know how gnus would handle that (this
> is a new idea Lars).

I kindof like that idea.  Of course a couple places in the code, it
explicitly checks for negative numbers in ranges and removes them.

	When I have to change news servers, which may happen some day,
I'd like to keep my cache as enties in the same ''group'' without it
overriding other potential articles, and also not being a seperate
entity.

-- 
David Moore <dmoore@ucsd.edu>       | Computer Systems Lab      __o
UCSD Dept. Computer Science - 0114  | Work: (619) 534-8604    _ \<,_
La Jolla, CA 92093-0114             | Fax:  (619) 534-1445   (_)/ (_)
<URL:http://oj.egbt.org/dmoore/>    | Solo Furnace Creek 508 -- 1996!

Re: Discussion: To del cached articles upon zapping group info?

[Lars Magne Ingebrigtsen]

David Moore <dmoore@UCSD.EDU> writes:

> 	Well, it certainly shouldn't delete them, although you want it
> to stick them into some other archive kind of group.  You probably
> cached them because they had useful information.  But you obviously
> can't leave them in the cache tree, since they conflict with other
> messages.  You could probably roll your old articles into an archive,
> set the new real newsgroup to a highish level, and drop a virtual group
> on top of them to combine them.  Although this isn't a great solution.

Hm.  How about just moving the cached articles to, uhm,
"~/News/cache.old/group/name", or something like that?

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@ifi.uio.no * Lars Ingebrigtsen

Re: Discussion: To del cached articles upon zapping group info?

[Wesley.Hardaker]

Lars Magne Ingebrigtsen <larsi@ifi.uio.no> writes:

> Hm.  How about just moving the cached articles to, uhm,
> "~/News/cache.old/group/name", or something like that?

Well, this would be fine too (ie, better than removing them and is an
acceptable solution).  However, it will hide them from the user will
it not?  IE, the user will never know to look there...

This should probably be solved with some sort of warning:  "Yo...  You
have cached articles in the groups you're wiping...  What should I do
with them? (del/mov/neg/abort):" with the default being mov?

Sort of a pain.  I think there is no way of doing something smart
(without implementing the negative numbers scheme, which would be
cool) without at least telling (read "warning") the user if not asking
him what to do...

???
Wes

Re: Discussion: To del cached articles upon zapping group info?

[Edward J. Sabol]

IMHO, I think the negative numbers scheme is the best solution. It completely
hides the problem from the user, and old cached articles will still appear in
the newsgroup that you cached them in. It's very neat and tidy.

Later,
Ed

Re: Discussion: To del cached articles upon zapping group info?

[David Moore]

"Edward J. Sabol" <sabol@alderaan.gsfc.nasa.gov> writes:

> IMHO, I think the negative numbers scheme is the best solution. It completely
> hides the problem from the user, and old cached articles will still appear in
> the newsgroup that you cached them in. It's very neat and tidy.

	I'm preferring the negative number scheme myself.  If you make
another directory for the old articles but still want them to appear
with the articles from the new server, you'd need to place a virtual
group on top of them.  I'm not too concerned about the performance of
doing this, but it makes maintaining your newsgroups a lot more
difficult.  You have files in multiple trees; every original newsgroup
is now 3 newgroups (1 old cache, 1 new nntp, 1 new virtual), etc.  Even
if you made the virtual groups dynmically without explicitly being in
the newsrc. you'd need to be careful about people's hooks checking the
method vs nntp and similar.

	Whereas if everything stays in the same cache file, but with
negative numbers for articles (-1..-N for N articles in orig cache), you
don't have to change any of the rest of your group setup (creating these
extra groups).

	Most things in the code would work fine if this change was made,
except a couple routines which filter out negative numbers in ranges, I
think.  Changing those to just filtering numbers outside of the
'cache+nntp' active range, as done in most other places should hopefully
make everything work.  But then, there might be some hidden dependencies
on the article numbers being positive that I'm missing.

	Another approach would be to put the articles in the cache
numbered 1..N, and store the delta N as a group parameter, and then just
add N to all article numbers returned by the backend for that group.
This adds more overhead to article fetching, and makes Xref's rather
challenging.

	Or you could let article numbers (internally to gnus) come as
dotted pairs: (component-number . article-number).  And you'd just
assign increasing component-numbers to each new nntp server you get.
Going this path means changing a lot more internal code, but you might
be able to leverage the functionality for nnvirtual and IMAP.  Although,
it's probably not worth it. ;)


-- 
David Moore <dmoore@ucsd.edu>       | Computer Systems Lab      __o
UCSD Dept. Computer Science - 0114  | Work: (619) 534-8604    _ \<,_
La Jolla, CA 92093-0114             | Fax:  (619) 534-1445   (_)/ (_)
<URL:http://oj.egbt.org/dmoore/>    | Solo Furnace Creek 508 -- 1996!

Re: Discussion: To del cached articles upon zapping group info?

[Wesley.Hardaker]

David Moore <dmoore@UCSD.EDU> writes:

> 	I'm preferring the negative number scheme myself.

Lars, you want to toss in your estimate of accomplishing this
impossiblitiy?  You'd know best of course...  This is definately the
easiest way to go and the least user intrusive.

[...]

> Although, it's probably not worth it. ;)

Thats the whole problem.  We're talking about a fix for something that
is rarely done in the first place...  Its not exactly something that
Lars should spend a lot of time on, but then...  Something has to be
done too.

Ah, well...
Wes

Re: Discussion: To del cached articles upon zapping group info?

[Lars Magne Ingebrigtsen]

David Moore <dmoore@UCSD.EDU> writes:

> 	I'm preferring the negative number scheme myself.

The problem is that Gnus has reserved negative numbers to articles
that it don't know the number of.  Articles fetched by Message-ID, for
instance, and pseudo-articles.  Lots of code depend on this.

> 	Another approach would be to put the articles in the cache
> numbered 1..N, and store the delta N as a group parameter, and then just
> add N to all article numbers returned by the backend for that group.
> This adds more overhead to article fetching, and makes Xref's rather
> challenging.

Indeed.

> 	Or you could let article numbers (internally to gnus) come as
> dotted pairs: (component-number . article-number).  And you'd just
> assign increasing component-numbers to each new nntp server you get.
> Going this path means changing a lot more internal code, but you might
> be able to leverage the functionality for nnvirtual and IMAP.  Although,
> it's probably not worth it. ;)

I don't think so either.

This is a rather obscure problem -- what to do with cached articles
when changing nntp servers.  I'm inclined to just say that this isn't
something that's outside Gnus' scope and let the user deal with it
herself. 

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@ifi.uio.no * Lars Ingebrigtsen

Re: Discussion: To del cached articles upon zapping group info?

[Per Abrahamsen]

Lars Magne Ingebrigtsen <larsi@ifi.uio.no> writes:

> This is a rather obscure problem -- what to do with cached articles
> when changing nntp servers.  I'm inclined to just say that this isn't
> something that's outside Gnus' scope and let the user deal with it
> herself. 

Maybe an explicit command to move cached articles somewhere else, and
a warning in any action that would render cached articles invalid.

Re: Discussion: To del cached articles upon zapping group info?

[Wesley.Hardaker]

Per Abrahamsen <abraham@dina.kvl.dk> writes:

> Maybe an explicit command to move cached articles somewhere else, and
> a warning in any action that would render cached articles invalid.

I think gnus *has* to do at least a move to get them out of the way so
it doesn't screw up in the first place, and it should at the very
least warn the user as well.  Preferably, with a prompt as a simple
message can disappear without the user actually ever seeing it whereas
a prompt will sit around till the specifically acknowledge it...

Wes

Re: Discussion: To del cached articles upon zapping group info?

[Lars Magne Ingebrigtsen]

Wesley.Hardaker@sphys.unil.ch writes:

> I think gnus *has* to do at least a move to get them out of the way so
> it doesn't screw up in the first place, and it should at the very
> least warn the user as well.  Preferably, with a prompt as a simple
> message can disappear without the user actually ever seeing it whereas
> a prompt will sit around till the specifically acknowledge it...

I've now added this to `gnus-group-clear-data-on-native-groups' in
0.63. 

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@ifi.uio.no * Lars Ingebrigtsen

Re: Discussion: To del cached articles upon zapping group info?

[Paul Franklin]

 > Per Abrahamsen <abraham@dina.kvl.dk> writes:

 >> Maybe an explicit command to move cached articles somewhere else, and
 >> a warning in any action that would render cached articles invalid.

>>>>> Wesley Hardaker writes:

 > I think gnus *has* to do at least a move to get them out of the way so
 > it doesn't screw up in the first place, and it should at the very
 > least warn the user as well.  Preferably, with a prompt as a simple
 > message can disappear without the user actually ever seeing it whereas
 > a prompt will sit around till the specifically acknowledge it...

Hmm.  How about this:

* Gnus asks you for permission to move them out of the way when it
  detects this problem.
* It records the directory they were moved to in the group parameters,
  along with a group count.
* There's a command to explicitly visit the old cached articles, based
  on the record in the group parameters.
* It waits until enough articles have expired on the server that
  these articles and others cached can be safely renumbered and placed
  all back in the individual group.

(These functions might also help with server migration, which has been
discussed but seems to be another one of those things that happens
occasionally enough that it isn't worth much effort.)

--Paul

Re: Discussion: To del cached articles upon zapping group info?

[Lars Magne Ingebrigtsen]

Paul Franklin <paul@cs.washington.edu> writes:

> Hmm.  How about this:
> 
> * Gnus asks you for permission to move them out of the way when it
>   detects this problem.
> * It records the directory they were moved to in the group parameters,
>   along with a group count.
> * There's a command to explicitly visit the old cached articles, based
>   on the record in the group parameters.
> * It waits until enough articles have expired on the server that
>   these articles and others cached can be safely renumbered and placed
>   all back in the individual group.

Much too much work for a rather obscure problem, in my opinion.

> (These functions might also help with server migration, which has been
> discussed but seems to be another one of those things that happens
> occasionally enough that it isn't worth much effort.)

When migrating properly between servers, the cached articles should be
renumbered properly.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@ifi.uio.no * Lars Ingebrigtsen