compatibility of nnml and MH

compatibility of nnml and MH

[Jason R Mastaler]

How compatibile is the nnml backend with MH and how uncompatible is it
with MH?  If I have a bunch of nnml groups and save articles from
those groups to existing MH folders with `O h', do I have to worry
that those saved articles are no longer MH compliant in some way
because their origin was an nnml group?  Or, does the origin not
matter in this case since I used `O h' to save?

  `O h'
     Save the current article in mh folder format

I'm not interested in reading existing nnml groups with MH or MH
front-end, but I am interested in being able to manipulate saved
articles from those nnml groups at a later date with MH and not run
into problems.  I hope I'm making sense here.

Also, there seems to be conflicting ideas about the compatibility
of nnml with other formats (such as MH).  

On one hand nnml is supposedly not at all compatible with other
formats..

The manual says of nnml:
 
  The "nnml" spool mail format isn't compatible with any other known
  format.  It should be used with some caution.

Lars wrote to ding:

  Uhm.  Nope.  nnml is nnml, while nnmh is supposed to be mh compliant.
  nnml will never be able to coexist with mh.  That the point of nnmh. 

On the other hand..

The manual says of nnmh:

  `nnmh' is just like `nnml', except that is doesn't generate NOV
  databases and it doesn't keep an active file.  This makes `nnmh' a
  *much* slower backend than `nnml', but it also makes it easier to
  write procmail scripts for.

This seems to imply that nnml is somewhat compatable with MH since
nnmh (which is MH compliant) is "just like `nnml', except that is
doesn't generate NOV databases and it doesn't keep an active file".
Is the existance of the NOV databases and active file enough to
introduce these incompatibilities?  It really doesn't seem like it
could, so is there something else I've missed?

   Jason R. Mastaler                      jason@mastaler.com

Re: compatibility of nnml and MH

[Kai Grossjohann]

>>>>> Jason R Mastaler writes:

  Jason> How compatibile is the nnml backend with MH and how
  Jason> uncompatible is it with MH?  If I have a bunch of nnml groups
  Jason> and save articles from those groups to existing MH folders
  Jason> with `O h', do I have to worry that those saved articles are
  Jason> no longer MH compliant in some way because their origin was
  Jason> an nnml group?  Or, does the origin not matter in this case
  Jason> since I used `O h' to save?

There is two kinds of data in an nnml or nnmh group or an MH folder.

(1) The messages themselves.
(2) Other data.

(1) is the same in all three formats.  (2) is different.  In fact, for
an MH folder, (2) is empty, i.e. there is no other data.  Therefore,
you can use MH tools to read any one of these three formats but you
can't use them to write all of them.

kai
-- 
A large number of young women don't trust men with beards.
(BFBS Radio)

Re: compatibility of nnml and MH

[Jason R Mastaler]

Kai Grossjohann <grossjohann@charly.cs.uni-dortmund.de> writes:

> There is two kinds of data in an nnml or nnmh group or an MH folder.
> 
> (1) The messages themselves.
> (2) Other data.
> 
> (1) is the same in all three formats.  (2) is different.  In fact, for
> an MH folder, (2) is empty, i.e. there is no other data.  Therefore,
> you can use MH tools to read any one of these three formats but you
> can't use them to write all of them.

What exactly does (2) consist of then?  NOV databases and active file?
Something in addition?  Something different?

   Jason R. Mastaler                      jason@mastaler.com

Re: compatibility of nnml and MH

[Kai Grossjohann]

>>>>> Jason R Mastaler writes:

  Kai> (1) The messages themselves.
  Kai> (2) Other data.

  Jason> What exactly does (2) consist of then?  NOV databases and
  Jason> active file?  Something in addition?  Something different?

For nnml folders, it's the ".overview" file.  For nnmh folders, it's
nothing (as far as I know).  For MH folders, there's the .sequences
file, I'm not sure if that's updated with nnmh.

kai
-- 
A large number of young women don't trust men with beards.
(BFBS Radio)

Re: compatibility of nnml and MH

[Justin Sheehy]

Jason R Mastaler <jason@mastaler.com> writes:

> I'm not interested in reading existing nnml groups with MH or MH
> front-end

Too bad.  :-)   That's the easy part.

> but I am interested in being able to manipulate saved articles from
> those nnml groups at a later date with MH and not run into problems.

If you manipulate nnml group messages with anything but Gnus (such as
mh), it will probably work but will be very likely to cause problems
for Gnus when you next try to read said group.

If you are not interested in reading them as nnml messages again, then
you can treat them as mh messages with impunity.

> Also, there seems to be conflicting ideas about the compatibility
> of nnml with other formats (such as MH).  

[snip]

> This seems to imply that nnml is somewhat compatable with MH since
> nnmh (which is MH compliant) is "just like `nnml', except that is
> doesn't generate NOV databases and it doesn't keep an active file".
> Is the existance of the NOV databases and active file enough to
> introduce these incompatibilities?  It really doesn't seem like it
> could, so is there something else I've missed?

It is.

nnml and nnmh both use exactly the same format for storing the actual
messages, which is the mh spool format.  nnml also stores additional,
Gnus-specific data.  Altering the messages from outside of Gnus is
likely to make the nnml group misbehave.

So, you can read either one with mh.  If you are planning on switching
back and forth between Gnus and mh for _altering_ the messages, then
stick with nnmh.

-- 
Justin Sheehy

In a cloud bones of steel.
  

Re: compatibility of nnml and MH

[Jason R Mastaler]

Justin Sheehy <justin@linus.mitre.org> writes:

> If you manipulate nnml group messages with anything but Gnus (such as
> mh), it will probably work but will be very likely to cause problems
> for Gnus when you next try to read said group.

The only time the messages will be manipulated my MH is once they are
out of the the nnml group (saved to a new nnmh group).  

> If you are not interested in reading them as nnml messages again, then
> you can treat them as mh messages with impunity.

This is what I had hoped for.

> So, you can read either one with mh.  If you are planning on switching
> back and forth between Gnus and mh for _altering_ the messages, then
> stick with nnmh.

Stick with nnmh in what sense?  As the original spool backend, or
just for those cases where I plan to use both Gnus and MH to
read/alter a group's messages?  Just a sanity check here.  I want to
use nnml as the original spool backend and only use Gnus to touch
those groups.  Then, I want to save selected messages from those nnml
groups to other nnmh groups using `O h'.  I expect to be able to use
both Gnus and MH to manipulate those nnmh groups.  

   Jason R. Mastaler                      jason@mastaler.com
 

Re: compatibility of nnml and MH

[Kai Grossjohann]

>>>>> Jason R Mastaler writes:

  Jason> [...]  Then, I want to save selected messages from those nnml
  Jason> groups to other nnmh groups using `O h'.  I expect to be able
  Jason> to use both Gnus and MH to manipulate those nnmh groups.

FWIW, you can also use "B c" to copy articles from an nnml group to an
nnmh group.  Works between all kinds of groups.  Nifty way to do
format conversion.  Can convert old RMAIL (babyl) files to nnml in a
snap; point an nndoc group to the RMAIL file then mark all articles as
processable, then copy all articles to nnml group using "B c".

kai
-- 
A large number of young women don't trust men with beards.
(BFBS Radio)

Re: compatibility of nnml and MH

[Justin Sheehy]

Jason R Mastaler <jason@mastaler.com> writes:

> The only time the messages will be manipulated my MH is once they are
> out of the the nnml group (saved to a new nnmh group).  

Then you have nothing to worry about.

> Stick with nnmh in what sense?  As the original spool backend, or
> just for those cases where I plan to use both Gnus and MH to
> read/alter a group's messages? 

Just if you plan to use both.  For what you've described, your
original backend will have little bearing.

> I expect to be able to use both Gnus and MH to manipulate those nnmh
> groups.

That is the intended purpose of nnmh.

-- 
Justin Sheehy

In a cloud bones of steel.
  

Re: compatibility of nnml and MH

[Jason R Mastaler]

Justin Sheehy <justin@linus.mitre.org> writes:

> Then you have nothing to worry about.

Great.  Thanks all.

   Jason R. Mastaler                      jason@mastaler.com

 

Re: compatibility of nnml and MH

[Eze Ogwuma]

Jason R Mastaler <jason@mastaler.com> writes:

> Justin Sheehy <justin@linus.mitre.org> writes:
> 
> > Then you have nothing to worry about.
> 
> Great.  Thanks all.

Actually this depends on how many articles you have had in a group
overall. If nnmh has writes article numbers above 9999 then mh will
not see the numbers correctly. It will put a ? into the number scheme.

There was some discussion about re-numbering articles in nnm? groups a
while back but I don't remember the outcome.


-- 
Eze Ogwuma

Re: compatibility of nnml and MH

[Richard Coleman]

> Actually this depends on how many articles you have had in a group
> overall. If nnmh has writes article numbers above 9999 then mh will
> not see the numbers correctly. It will put a ? into the number scheme.
> 
> There was some discussion about re-numbering articles in nnm? groups a
> while back but I don't remember the outcome.

This is not completely true.  MH/nmh have no arbitrary limit
at 9999 and came handle messages with numbers above this.
But the default format which MH/nmh uses to represent messages
when outputing to screen only allocates space for 4 digit in the
message name.  So the display may not be correct if a message
name is higher than 9999.  This can be fixed by changing the
format to use 5 digits, or pack the folder.

Richard Coleman
coleman@math.gatech.edu

Re: compatibility of nnml and MH

[Eze Ogwuma]

Richard Coleman <coleman@math.gatech.edu> writes:

> > Actually this depends on how many articles you have had in a group
> > overall. If nnmh has writes article numbers above 9999 then mh will
> > not see the numbers correctly. It will put a ? into the number scheme.
> > 
> > There was some discussion about re-numbering articles in nnm? groups a
> > while back but I don't remember the outcome.
> 
> This is not completely true.  MH/nmh have no arbitrary limit
> at 9999 and came handle messages with numbers above this.
> But the default format which MH/nmh uses to represent messages
> when outputing to screen only allocates space for 4 digit in the
> message name.  So the display may not be correct if a message
> name is higher than 9999.  This can be fixed by changing the
> format to use 5 digits, or pack the folder.

Packing won't work if you actually do have more than 9999 articles in
the folder though.

I didn't know I could change the default. Do you have any idea how to
do that?


-- 
Eze Ogwuma

Re: compatibility of nnml and MH

[Eze Ogwuma]

Richard Coleman <coleman@math.gatech.edu> writes:

> > Actually this depends on how many articles you have had in a group
> > overall. If nnmh has writes article numbers above 9999 then mh will
> > not see the numbers correctly. It will put a ? into the number scheme.
> > 
> > There was some discussion about re-numbering articles in nnm? groups a
> > while back but I don't remember the outcome.
> 
> This is not completely true.  MH/nmh have no arbitrary limit
> at 9999 and came handle messages with numbers above this.
> But the default format which MH/nmh uses to represent messages
> when outputing to screen only allocates space for 4 digit in the
> message name.  So the display may not be correct if a message
> name is higher than 9999.  This can be fixed by changing the
> format to use 5 digits, or pack the folder.

Packing won't work if you actually do have more than 9999 articles in
the folder though.

I didn't know I could change the default format but I've had a look at
the mh-format man page.


-- 
Eze Ogwuma

Re: compatibility of nnml and MH

[Jason R Mastaler]

Eze Ogwuma <zcaceog@ucl.ac.uk> writes:

> Actually this depends on how many articles you have had in a group
> overall. If nnmh has writes article numbers above 9999 then mh will
> not see the numbers correctly. It will put a ? into the number scheme.
> 
> There was some discussion about re-numbering articles in nnm? groups a
> while back but I don't remember the outcome.

So, is there a way to re-number articles in an nnmh group? (from
within Gnus)  This would be the equivalent of "packing" a folder 
with MH/nmh.

  Compacting a Folder
     The `-pack' switch will compress the message  names  in  the
     designated  folders,  removing  holes  in message numbering.

   Jason R. Mastaler                      jason@mastaler.com

Re: compatibility of nnml and MH

[Justin Sheehy]

Jason R Mastaler <jason@mastaler.com> writes:

> So, is there a way to re-number articles in an nnmh group? (from
> within Gnus)  This would be the equivalent of "packing" a folder 
> with MH/nmh.

I don't think that there is a "nice" way.

You could, however, create a new nnmh group, process mark all of the
articles in the old one, and 'B m' them to the new one.

They would then have nice consecutive numbers, starting at 1.

But you don't want to do this as often as you might want to pack a
folder.

-- 
Justin Sheehy

In a cloud bones of steel.
  

Re: compatibility of nnml and MH

[Jason R Mastaler]

Justin Sheehy <justin@linus.mitre.org> writes:

> > So, is there a way to re-number articles in an nnmh group? (from
> > within Gnus)  This would be the equivalent of "packing" a folder 
> > with MH/nmh.
> 
> I don't think that there is a "nice" way.
> 
> You could, however, create a new nnmh group, process mark all of the
> articles in the old one, and 'B m' them to the new one.
> 
> They would then have nice consecutive numbers, starting at 1.
> 
> But you don't want to do this as often as you might want to pack a
> folder.

Right.  This works kinda-sorta, but it is somewhat of a pain.  How
feasible would it be to add a "nice" way of doing this from within
Gnus?  Could it perhaps be added to the TODO list?  I think it would
be quite useful for nnmh users.

   Jason R. Mastaler                      jason@mastaler.com

Re: compatibility of nnml and MH

[Lars Magne Ingebrigtsen]

Jason R Mastaler <jason@mastaler.com> writes:

[Renumbering articles]

> Right.  This works kinda-sorta, but it is somewhat of a pain.  How
> feasible would it be to add a "nice" way of doing this from within
> Gnus?  Could it perhaps be added to the TODO list?  I think it would
> be quite useful for nnmh users.

No, it's too much work.  Group, cache, agent, and Gnus renumbering.
My fingers hurt just by thinking about it.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: compatibility of nnml and MH

[David S. Goldberg]

There's no clean way to do a folder -pack for nnmh with Gnus (at least
there never was when I used it and I don't recall seeing anything to
that effect in the ChangeLog).  When I used nnmh, I would do it
manually; run folder -pack, either with mh-e or the shell, and then do
a G E on the group and change the article numbers to match the new
range.  Note that if you tick anything you've got to figure out how to
map the ticked articles to the new numbers.  Fortunately the -verbose
switch with folder -pack displays the mapping so you can use that.  It
occurs to me that this could also work for nnml.  I tried to figure
out how to edit the group params functionally but never could get that
to work.

--
Dave Goldberg
Post: The Mitre Corporation\MS B305\202 Burlington Rd.\Bedford, MA 01730
Phone: 617-271-3887
Email: dsg@mitre.org

Re: compatibility of nnml and MH

[Jason R Mastaler]

dsg@linus.mitre.org (David S. Goldberg) writes:
 
> There's no clean way to do a folder -pack for nnmh with Gnus (at least
> there never was when I used it and I don't recall seeing anything to
> that effect in the ChangeLog).  When I used nnmh, I would do it
> manually; run folder -pack, either with mh-e or the shell, and then do
> a G E on the group and change the article numbers to match the new
> range.  Note that if you tick anything you've got to figure out how to
> map the ticked articles to the new numbers.  Fortunately the -verbose
> switch with folder -pack displays the mapping so you can use that.  It
> occurs to me that this could also work for nnml.  I tried to figure
> out how to edit the group params functionally but never could get that
> to work.
 
Ok, I see.  However if you do it your way, Gnus will still number new
articles using the old range right?  In other words, if I have 13
messages in nnmh:inbox, and the last article is number 158 
(Xref: ashanti inbox:158), I can edit the range with `G E', but the
Xref's won't be changed to match the pack so the next new article
inc'd by Gnus will be 159, then 160, etc..  This doesn't really help
much.
 
   Jason R. Mastaler                      jason@mastaler.com

Re: compatibility of nnml and MH

[David S. Goldberg]

At least when I did this (which was only from a fairly late version of
(ding) up to a very early release of September), I didn't have any
problem.  Certainly you're right that the Xref headers wouldn't have
changed, but nnmh seemed to accept that the last article was whatever
file had the highest number in the folder and new ones were
incorporated with the next number up.

--
Dave Goldberg
Post: The Mitre Corporation\MS B305\202 Burlington Rd.\Bedford, MA 01730
Phone: 617-271-3887
Email: dsg@mitre.org

Re: compatibility of nnml and MH

[Jason R Mastaler]

dsg@linus.mitre.org (David S. Goldberg) writes:

> At least when I did this (which was only from a fairly late version of
> (ding) up to a very early release of September), I didn't have any
> problem.  Certainly you're right that the Xref headers wouldn't have
> changed, but nnmh seemed to accept that the last article was whatever
> file had the highest number in the folder and new ones were
> incorporated with the next number up.

Okay, this time I stopped and restarted Gnus after packing the folder
with nmh and editing the range with `G E' and Gnus incorporated with
the next number up.  Thanks.

   Jason R. Mastaler                      jason@mastaler.com

Re: compatibility of nnml and MH

[Eze Ogwuma]

Jason R Mastaler <jason@mastaler.com> writes:

> Eze Ogwuma <zcaceog@ucl.ac.uk> writes:
> 
> > Actually this depends on how many articles you have had in a group
> > overall. If nnmh has writes article numbers above 9999 then mh will
> > not see the numbers correctly. It will put a ? into the number scheme.
> > 
> > There was some discussion about re-numbering articles in nnm? groups a
> > while back but I don't remember the outcome.
> 
> So, is there a way to re-number articles in an nnmh group? (from
> within Gnus)  This would be the equivalent of "packing" a folder 
> with MH/nmh.
> 
>   Compacting a Folder
>      The `-pack' switch will compress the message  names  in  the
>      designated  folders,  removing  holes  in message numbering.

I might be wrong but I recall that the only solution was to re-spool
all the articles to another folder then back again. I'm not sure how
this would affect ticked articles.


-- 
Eze Ogwuma

Re: compatibility of nnml and MH

[Kai Grossjohann]

>>>>> Eze Ogwuma writes:

  Eze> I might be wrong but I recall that the only solution was to
  Eze> re-spool all the articles to another folder then back
  Eze> again. I'm not sure how this would affect ticked articles.

Not at all.  They stay ticked.  Gnus remembers which ones were ticked.

kai
-- 
A large number of young women don't trust men with beards.
(BFBS Radio)