NNML => NNIMAP migration (and IMAP server recommendations. :)

NNML => NNIMAP migration (and IMAP server recommendations. :)

[Daniel Pittman]

I am in the position of /almost/ always wanting to access my email on my
local machine. /Almost/, though, is not quite always, which has been a
pain on a couple of occasions recently.

So, to address this, I would like to migrate from my current NNML-based
mail setup to an IMAP based setup.

This has the obvious advantage of being agnostic to the mail client used
while retaining almost all the functionality of NNML.

My current plan is to use Cyrus-IMAP as the server backend (on the local
machine, with one user). Has anyone a recommendation against using this
tool?

The U-Washington IMAP server is not interesting (doesn't scale) and
Courier is a little problematic in terms of accepting 8-bit bodies and
so forth.[1]


As part of this migration, I need to move my current mail collection
into the server I chose. This /needs/ to preserve the flagged and unread
articles in my current NNML spool.

My spool currently contains ~ 100,000 messages, all of which I want to
preserve in their current group and flagged/read/unread state during the
transition.

After using 'B c' (for testing) to spool 2,200 articles into IMAP, I am
more than a little afraid of the time it will take to do this migration
in Gnus. My estimate is ~ 48 hours, maybe more.

So, any suggestions for dealing with that issue?

        Daniel


Footnotes: 
[1]  This is based on issues encountered at my place of work while they
     were testing Courier. I am happy to hear this dismissed.

-- 
In the present-day reconstruction of physics, fragments of the Newtonian
concepts are stubbornly retained. The result is to reduce modern physics to a
sort of mystic chant over an unintelligible universe.
        -- Alfred North Whitehead, _Modes of Thought_

Re: NNML => NNIMAP migration (and IMAP server recommendations. :)

[Simon Josefsson]

Daniel Pittman <daniel@rimspace.net> writes:

> My current plan is to use Cyrus-IMAP as the server backend (on the local
> machine, with one user). Has anyone a recommendation against using this
> tool?

No.  Good choice.

> After using 'B c' (for testing) to spool 2,200 articles into IMAP, I am
> more than a little afraid of the time it will take to do this migration
> in Gnus. My estimate is ~ 48 hours, maybe more.
> 
> So, any suggestions for dealing with that issue?

There are utilities such as `imapcopy' that are faster, but the flags
are a problem.  Hm.  You could trick Gnus into setting the flags for
you, using `G E' and some hackery, but you'd need to be very careful
about article numbering.  You probably need to compact your nnml
article range first.  Seems tricky.

Or you could optimize `gnus-summary-move-article' /
`nnimap-accept-article' for bulk transfers.  This shouldn't be very
difficult, and would probably take less than 48 hours.

> [1]  This is based on issues encountered at my place of work while they
>      were testing Courier. I am happy to hear this dismissed.

8-bit _bodies_ are ok (I think), but Courier choses to (rightly)
reject articles with unencoded 8-bit characters in headers.  It's easy
to patch around this.  Cyrus used to do the same, but now it simply
replaces 8-bit characters with `X'.  Also, sendmail is able to perform
8bit stripping.

Courier does not store the original RFC822 content, from what I hear.
Also from what I understand, this breaks PGP/MIME and S/MIME.  To me,
that's more troublesome.

And oh, take backups.

Re: NNML => NNIMAP migration (and IMAP server recommendations. :)

[NAGY Andras]

On Wed, 07 Mar 2001, at 04:35, Daniel Pittman <daniel@rimspace.net> wrote:

> After using 'B c' (for testing) to spool 2,200 articles into IMAP, I am
> more than a little afraid of the time it will take to do this migration
> in Gnus. My estimate is ~ 48 hours, maybe more.
> 
> So, any suggestions for dealing with that issue?

You could use Mutt (the version that supports IMAP) to do mass
transfers.  It's MH backend can deal with Gnus' nnml format, and has
the best performance (compared to Gnus and Pine) when it comes to
_really_ large mailboxes.

But eeek, Gnus stores flags in newsrc, and Mutt expects them in the
message's Status: header.  Which makes it unsuitable for you, I think.
:(


Andras

Re: NNML => NNIMAP migration (and IMAP server recommendations. :)

[Daniel Pittman]

On 07 Mar 2001, NAGY Andras wrote:
> On Wed, 07 Mar 2001, at 04:35, Daniel Pittman <daniel@rimspace.net>
> wrote:
> 
>> After using 'B c' (for testing) to spool 2,200 articles into IMAP, I
>> am more than a little afraid of the time it will take to do this
>> migration in Gnus. My estimate is ~ 48 hours, maybe more.
>> 
>> So, any suggestions for dealing with that issue?
> 
> You could use Mutt (the version that supports IMAP) to do mass
> transfers.  It's MH backend can deal with Gnus' nnml format, and has
> the best performance (compared to Gnus and Pine) when it comes to
> _really_ large mailboxes.
> 
> But eeek, Gnus stores flags in newsrc, and Mutt expects them in the
> message's Status: header.  Which makes it unsuitable for you, I think.
> :(

True. My current plan is to write some Lisp to implement
'gnus-server-migrate-group(s)' that will do the work in Lisp without the
fluff that entering and leaving a summary buffer has.

With luck, fingers crossed, I can manage this, at least on a per-group
basis.

If that fails, though, I might have a look at the Mutt thing, together
with a hack to fill out a header on each message containing the flags...

        Daniel

-- 
You must be the change you wish to see in the world.
        -- Gandhi