Changing Default News Server

Changing Default News Server

[Lal Samuel Varghese]

I want to change my default news server from x to y.
What all should I do to ensure a smooth transition?

-- 
Regards,
Lal.

-----------------------------------------------------------------
Lal Samuel Varghese                             lal@miel.mot.com.
MIEL, 33A, Ulsoor Road,                         Of: 91-80-5598615
Bangalore - 42, INDIA                           Re: 91-80-5293499
-----------------------------------------------------------------

Re: Changing Default News Server

[Lars Balker Rasmussen]

Lal Samuel Varghese <lal@max.miel.mot.com> writes:
> I want to change my default news server from x to y.
> What all should I do to ensure a smooth transition?

What's a smooth transition?  Keeping the read articles as read, and the
unread as unread?  That is not possible due to the nature of news
distribution.
-- 
Lars Balker Rasmussen
<URL:http://www.daimi.aau.dk/~gnort/>  <URL:mailto:lars@rasmussen.org>

Re: Changing Default News Server

[Darren/Torin/Who Ever...]

-----BEGIN PGP SIGNED MESSAGE-----

Lal Samuel Varghese, in an immanent manifestation of deity, wrote:
>I want to change my default news server from x to y.
>What all should I do to ensure a smooth transition?

Well, if you want to keep the read articles read, and the unread
articles unread, you should wait for Red Gnus since this is on the wish
list.

Darren
- -- 
<torin@daft.com> <http://www.daft.com/~torin> <torin@debian.org> <torin@io.com>
Darren Stalder/2608 Second Ave, @282/Seattle, WA 98121-1212/USA/+1-800-921-4996
@ Do you have your clothes on? I probably don't. Take yours off. Feel better. @
@ Sysadmin, webweaver, postmaster for hire.  C/Perl/CGI programmer and tutor. @

-----BEGIN PGP SIGNATURE-----
Version: 2.6.2
Comment: Processed by Mailcrypt 3.4, an Emacs/PGP interface

iQCVAwUBMe4eW44wrq++1Ls5AQE5yAQAmVFmP1dHhAKjr0k6eLuSOhCTMsJsJ73o
2V0O9ciiGEHzhIY5Dc0hdIwVTLeW7Jh/BLo9yZQ22mrD1uUh77Zgq36Ju4M1fJYk
U0dlodaZ/nAatH9NrMkby76Fd8whDTcN/zmG16vaWiiQB4zMSaVsgxX6nFBYNFUF
u4SAMpvBNwg=
=1BfV
-----END PGP SIGNATURE-----

Re: Changing Default News Server

[Lars Balker Rasmussen]

"Darren/Torin/Who Ever..." <torin@daft.com> writes:
> Well, if you want to keep the read articles read, and the unread
> articles unread, you should wait for Red Gnus since this is on the wish
> list.

Hmm, curious.  Lars, how do you expect to solve this problem?
-- 
Lars Balker Rasmussen
<URL:http://www.daimi.aau.dk/~gnort/>  <URL:mailto:lars@rasmussen.org>

Re: Changing Default News Server

[Jan Vroonhof]

Lars Balker Rasmussen <gnort@daimi.aau.dk> writes:

> Hmm, curious.  Lars, how do you expect to solve this problem?

The brute force solution would be to get the Message-ID's of all read
messages and then look them up on the new server.

Jan

-- 
Jan Vroonhof                    http://www.math.ethz.ch/~vroonhof/
Mathematik,                                  vroonhof@math.ethz.ch
HG E16, ETH-Zentrum,                      Tel: +41-1-6325456/25154
Raemistrasse 101, CH-8092 Zuerich.              Fax: +41-1-6321085

Re: Changing Default News Server

[Lars Balker Rasmussen]

Jan Vroonhof <vroonhof@math.ethz.ch> writes:
> Lars Balker Rasmussen <gnort@daimi.aau.dk> writes:
> > Hmm, curious.  Lars, how do you expect to solve this problem?
> 
> The brute force solution would be to get the Message-ID's of all read
> messages and then look them up on the new server.

Precisely, that's a brute force ad-hoc solution, which would absolutely
not work if e.g. my local news-server (which has perverse expiration
times) was involved.

I had considered that one, but is that the _only_ approach you have
considered?  In that case, the feature would only be useful for filling
up a feature list :-/
-- 
Lars Balker Rasmussen
<URL:http://www.daimi.aau.dk/~gnort/>  <URL:mailto:lars@rasmussen.org>

Re: Changing Default News Server

[Jan Vroonhof]

Lars Balker Rasmussen <gnort@daimi.aau.dk> writes:

> > The brute force solution would be to get the Message-ID's of all read
> > messages and then look them up on the new server.
> 
> Precisely, that's a brute force ad-hoc solution, which would absolutely
> not work if e.g. my local news-server (which has perverse expiration
> times) was involved.
> 

However it works quite well for small groups. For large groups we are
helped by the fact that the newsreading is done by a human (i.e.
slow).
If you a large group say recs.jobs.offered then most of the time
either one of these is true:

1. Only a few messages are 'read' (i.e. those you actually read).
2. Almost all are read and those that are unread are at the end
(i.e. you have used Catchup).

Case 2 is the difficult one:

Then consider say the last 100 messages of the large "read" block and
look them up on the new server. Lookup the unread articles on the new
server. Take the minum of the article numbers found This gives you a
reasonble lower bound for the unread articles.
Then proceed lower bound upwards to lookup which of the remaing
articles you have read.

This at least gives you a reasonble approximation of wat you want.

Wether this is implementable in e-lisp is another matter.

Jan

P.S. How many articles does rec.jobs.offered have on your news server?

-- 
Jan Vroonhof                    http://www.math.ethz.ch/~vroonhof/
Mathematik,                                  vroonhof@math.ethz.ch
HG E16, ETH-Zentrum,                      Tel: +41-1-6325456/25154
Raemistrasse 101, CH-8092 Zuerich.              Fax: +41-1-6321085

Re: Changing Default News Server

[Jan Vroonhof]

Jan Vroonhof <vroonhof@math.ethz.ch> writes:

> 2. Almost all are read and those that are unread are at the end
> (i.e. you have used Catchup).
> 
> Case 2 is the difficult one:
> 
> Then consider say the last 100 messages of the large "read" block and
> look them up on the new server. Lookup the unread articles on the new
> server. Take the minum of the article numbers found This gives you a
> reasonble lower bound for the unread articles.
> Then proceed lower bound upwards to lookup which of the remaing
> articles you have read.
> 
> This at least gives you a reasonble approximation of wat you want.
> 
> Whether this is implementable in e-lisp is another matter.

A trivial prototype perl program showed that perl can read in 35000
Messaged-ID,article number pairs from an xover file in an  hash table
and look them all up again in under two minutes.

-- 
Jan Vroonhof                    http://www.math.ethz.ch/~vroonhof/
Mathematik,                                  vroonhof@math.ethz.ch
HG E16, ETH-Zentrum,                      Tel: +41-1-6325456/25154
Raemistrasse 101, CH-8092 Zuerich.              Fax: +41-1-6321085

Re: Changing Default News Server

[Steven L Baur]

>>>>> "Lars" == Lars Balker Rasmussen <gnort@daimi.aau.dk> writes:
>>>>> "Lal" == Lal Samuel Varghese <lal@max.miel.mot.com> writes:

Lal> I want to change my default news server from x to y.
Lal> What all should I do to ensure a smooth transition?

Lars> What's a smooth transition?  Keeping the read articles as read,
Lars> and the unread as unread?  That is not possible due to the
Lars> nature of news distribution.

All of the mailgroup information is in the .newsrc.  I would love to
see it separated from a specific nntp server, since mail archives are
portable.
-- 
steve@miranova.com baur
Unsolicited commercial e-mail will be proofread for $250/hour.
Andrea Seastrand: For your vote on the Telecom bill, I will vote for anyone
except you in November.

Re: Changing Default News Server

[David Watson]

> Lal Samuel Varghese <lal@max.miel.mot.com> writes:
> > I want to change my default news server from x to y.
> > What all should I do to ensure a smooth transition?
>
> What's a smooth transition?  Keeping the read articles as read, and the
> unread as unread?  That is not possible due to the nature of news
> distribution.
> -- 
> Lars Balker Rasmussen
> <URL:http://www.daimi.aau.dk/~gnort/>  <URL:mailto:lars@rasmussen.org>


Years ago I wrote a quick script to read the active files on the old and
new servers as close to simultaneously as possible, then, for each
newsgroup found on the new server, take the delta between the high
article numbers on the two servers, and apply that delta as a correction
to the corresponding line in the reader's .newsrc file.  In my script, I
only changed the number of the highest article read, not all the
individual article numbers on the line, but I suppose you could alter
those too, according to the the old/new server delta, and hope that the
resulting articles weren't canceled or expired too differently on the
new server.  Anyway, the method achieved generally successful results
according to the users.  I can't find a copy of the script to post here,
but it was fairly straightforward.

  -David.

Re: Changing Default News Server

[Lars Magne Ingebrigtsen]

David Watson <watson@wink.corp.sgi.com> writes:

> Years ago I wrote a quick script to read the active files on the old and
> new servers as close to simultaneously as possible, then, for each
> newsgroup found on the new server, take the delta between the high
> article numbers on the two servers, and apply that delta as a correction
> to the corresponding line in the reader's .newsrc file. 

The problem is that articles have a nasty tendency to arrive in
different order on different servers, so this straightforward (and
fast) approach will yield less than optimal results when you apply it
to two servers that are far from each other.

I'll probably do something similar, though.  Going through all
Message-ID's of all read and/or unread articles is out of the
question.  Perhaps one could get kinda good results by going through a
few groups and trying to guess how thorough one has to be?  

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@ifi.uio.no * Lars Ingebrigtsen

Re: Changing Default News Server

[Lars Balker Rasmussen]

Jan Vroonhof <vroonhof@math.ethz.ch> writes:
> However it works quite well for small groups. For large groups we are
> helped by the fact that the newsreading is done by a human (i.e.
> slow).
> If you a large group say recs.jobs.offered then most of the time
> either one of these is true:
> 
> 1. Only a few messages are 'read' (i.e. those you actually read).
> 2. Almost all are read and those that are unread are at the end
> (i.e. you have used Catchup).

Ok, I understand this, but the original question was whether it could be
done smoothless.  I still doubt this.

> P.S. How many articles does rec.jobs.offered have on your news server?

I assume you mean misc.jobs.offered:  It currently has 13817 articles,
the oldest (top one in Gnus) being dated July 10.  That's a bit
cheating, because the groups that are slaughtered at daimi are mainly
alt and rec (sometimes articles live less than 24 hours...)

(Starting Gnus on that group took one HELL of a time on this 2
processor/plenty of ram sparcstation 20, and this emacs currently eats
21 megs of ram because of it :-) )
-- 
Lars Balker Rasmussen
<URL:http://www.daimi.aau.dk/~gnort/>  <URL:mailto:lars@rasmussen.org>

Re: Changing Default News Server

[Ken Raeburn]

Lars Magne Ingebrigtsen <larsi@ifi.uio.no> writes:

> I'll probably do something similar, though.  Going through all
> Message-ID's of all read and/or unread articles is out of the
> question.  Perhaps one could get kinda good results by going through a
> few groups and trying to guess how thorough one has to be?

Why?  It's not a common operation, it shouldn't be a big problem for
it to take a lot of time.  I think getting the Message-ID's of all
read articles is exactly the way to do it.  It may bring back some
articles that were expired on the old server and not on the new, but
that's almost certainly better than hiding unread articles, or
bringing back *lots* of read articles because the mechanism wasn't
thorough.

I've seen some perl code that operates exactly this way, though it
only works on the standard .newsrc, of course.

Re: Changing Default News Server

[Lars Magne Ingebrigtsen]

Ken Raeburn <raeburn@cygnus.com> writes:

> Why?  It's not a common operation, it shouldn't be a big problem for
> it to take a lot of time.

Well, I think if we're clever we can get the operation down to hours,
and if we do it thoroughly it might take days.  I'm probably being
overly pessimistic, though.  I think I should just write it and see
whether it'll be too slow to be useful.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@ifi.uio.no * Lars Ingebrigtsen

Re: Changing Default News Server

[David Lebel]

Lars Magne Ingebrigtsen <larsi@ifi.uio.no> writes:

: Well, I think if we're clever we can get the operation down to hours,
: and if we do it thoroughly it might take days.  I'm probably being
: overly pessimistic, though.  I think I should just write it and see
: whether it'll be too slow to be useful.

	Isn't everything needed in the XOVERVIEW file?  Just by
downloading the file from both servers, then comparing the numbers and
re-arranging the unread/read numbers and the thing is done.  Shouldn't
be too long.

	I could be wrong, though.

	...David

-- 
// d a v i d  l e b e l  <lebel@socom.com>      http://www.socom.com/~lebel/
// analyst / system administrator / hacker          socom technologies, inc.
// " thoughts from above hit the people down below, people in this world, we
//   have no place to go. "                       -- new order, _temptation_

Re: Changing Default News Server

[Ken Raeburn]

Lars Magne Ingebrigtsen <larsi@ifi.uio.no> writes:

> Well, I think if we're clever we can get the operation down to hours,
> and if we do it thoroughly it might take days.  I'm probably being
> overly pessimistic, though.  I think I should just write it and see
> whether it'll be too slow to be useful.

I located the perl script I had seen before, and ran it with my .newsrc
and my nntp server.  Now, we do have a short expire time for many groups,
and we've been off the air for a couple of days, so I've got an easy test
case, but it finished translating from newsrc to message-id list (ids plus
group names totalling 2854 items) in under 30 seconds.  The reverse
translation took less time.

This is recording only the read articles.  Ticked and replied-to articles
would need to be tracked separately.  I don't think unread articles need
to be tracked.

A typical news server probably has many more articles than we do right
now, but I would guess (possibly incorrectly?) that the time required
would be more dependent on the number of articles read, not the total
available on the server.

I would expect an elisp version to be much slower than the perl code --
take this as a challenge if you like :-) -- but if it does amount to a
large number of minutes or even an hour or more, I don't think it's a big
problem as long as progress can be monitored in the minibuffer or
something.  This *shouldn't* be a common operation, and the user shouldn't
mind waiting a long time.  Heck, she could run it in batch mode.

It takes a long time to run nnml-generate-nov-databases too, but we don't
skimp on the accuracy for performance reasons. :-)

Ken

P.S.  I don't know if this perl code is considered ripe for distribution.
I can check with the author, if anyone wants to look at it.