mh backend

mh backend

[Phil Humpherys]

I'm forever converted to gnus, but am interested in using the mh
backend to gnus rather than the default nnml.  What would I need
to change in my .emacs to make that work?  Also, how would I
convert my nnml folders over to mh?  I guess I could export them
to a unix mbox and then just incorporate...

In using the mh backend, does gnus actually run the mh commands
to run the folders?  Or does it keep it all in gnus and just the
format of the mail folders is mh?

Anyone with some experience here?

--
Phil Humpherys <phumpherys@utah-inter.net>   DriverSoft
Unix Systems Administrator                   Mobile: +1.801.725.3257 
WWW/PGPkeys: http://www.spire.com/~humphery

Re: mh backend

[Kai Grossjohann]

>>>>> On 14 Sep 1998, Phil Humpherys said:

  Phil> In using the mh backend, does gnus actually run the mh commands
  Phil> to run the folders?  Or does it keep it all in gnus and just the
  Phil> format of the mail folders is mh?

Nnml is just like nnmh, except that it has additional .overview files
for speedup.  Both do not use the MH programs but directly read/write
the files.

Given that you can use MH tools to read nnml folders (be careful not
to write using MH tools, though, lest you clobber the .overview
files!), do you *really* want to use the much slower nnmh backend?

kai
-- 
OOP: object oriented programming;  OOPS: object oriented mistakes

Re: mh backend

[Rajappa Iyer]

Stainless Steel Rat <ratinox@peorth.gweep.net> writes:

> "PH" == Phil Humpherys <phumpherys@utah-inter.net> writes:
> 
> PH> I'm forever converted to gnus, but am interested in using the mh
> PH> backend to gnus rather than the default nnml.
> 
> Is there any particular reason you want to do so?  Because there is little
> difference between nnml and nnmh, other than a) for Gnus nnml is faster and
> b) you only need nnmh if you also use MH.

I switched to nnml from MH quite some time ago, but there are a few
things about MH that I miss and can't quite duplicate them with Gnus + 
nnml.

1. Message sequences.
2. Picking messages based on patterns in their body or some header
   field.
3. When I periodically delete some mail, I can folder -pack to
   renumber all the messages.  With Gnus it is a lot more painful.
-- 
Rajappa Iyer <rsi@lucent.com>		#include <std_disclaimer.h>
	We're too busy mopping the floor to turn off the faucet.

Re: mh backend

[François Pinard]

Rajappa Iyer <rsi@lucent.com> writes:

> 1. Message sequences.

What are they?

> 2. Picking messages based on patterns in their body or some header
>    field.

The usual devices for this, I think, are the limiting commands (starting
with `/') and the scoring commands (starting with `I', 'L' or 'V').
Maybe there are others?  That makes a lot already :-).

> 3. When I periodically delete some mail, I can folder -pack to
>    renumber all the messages.  With Gnus it is a lot more painful.

What is the purpose of doing such an operation?  I have the impression that
it is rather unusual that one has to enter the `nnml' hierarchy directly,
or depends on attributed numbers.  Isn't it?

-- 
François Pinard                            mailto:pinard@iro.umontreal.ca
Join the free Translation Project!    http://www.iro.umontreal.ca/~pinard

Re: mh backend

[Richard Coleman]

> > 1. Message sequences.
> 
> What are they?

In MH/nmh, you can give symbolic names to an arbitrary list of
numbers.  Then you can manipulate the messages that these numbers
represent, by using the message sequence.  All the MH/nmh will
take message sequences as arguments.  Also, there are commands
for adding or deleting messages from a sequence.  This makes it
very easy to remove, refile, search, or forward a collection of
messages.

In particular, there are some internal sequences, such as "unseen",
which tracks the messages that are unread.  When you read a message,
it is automatically removed from this sequence.

It's a very powerful tool.  I've always wondered why other mail
readers haven't done similar things.

> > 3. When I periodically delete some mail, I can folder -pack to
> >    renumber all the messages.  With Gnus it is a lot more painful.
> 
> What is the purpose of doing such an operation?  I have the impression that
> it is rather unusual that one has to enter the `nnml' hierarchy directly,
> or depends on attributed numbers.  Isn't it?

Well, it's common to repack the folder in the MH/nmh world.  The
message numbers are not hidden in the background quite as much as in
Gnus.

--
Richard Coleman (author of nmh)
coleman@math.gatech.edu

Re: mh backend

[Rajappa Iyer]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 2225 bytes --]

François Pinard <pinard@iro.umontreal.ca> writes:

> Rajappa Iyer <rsi@lucent.com> writes:
> 
> > 1. Message sequences.
> 
> What are they?

In MH, I can assign a compound sequence of messages a name and can
limit my operations to operate on specific sequences.  Look upon it as 
user customizable marks.

> > 2. Picking messages based on patterns in their body or some header
> >    field.
> 
> The usual devices for this, I think, are the limiting commands (starting
> with `/') and the scoring commands (starting with `I', 'L' or 'V').
> Maybe there are others?  That makes a lot already :-).

Yes, limiting takes care of matching on headers, but unless I'm
missing something obvious, one cannot limit based on the contents
message body.  I find this occasionally useful when searching for a
particular piece of email.  

Hmm, now that I think about it, both 1 and 2 may perhaps easily be
implemented by spawning an external command like MH's pick which will
return a sequence of message numbers which match the given criteria
and have a special limiting function which will use that list.

> > 3. When I periodically delete some mail, I can folder -pack to
> >    renumber all the messages.  With Gnus it is a lot more painful.
> 
> What is the purpose of doing such an operation?  I have the impression that
> it is rather unusual that one has to enter the `nnml' hierarchy directly,
> or depends on attributed numbers.  Isn't it?

Well, the idea is that despite all the fine procmail filters I have in
place, a lot of spam and otherwise useless messages make it to my
inbox.  If I expire them, it leaves a gap in the message numbers and
then the count of messages is inaccurate since Gnus calculates the
number of articles based on the article numbers of the first and the
last articles.  Ideally, I'd like to be able to `pack' the nnml folder
in the same way as an MH folder so that I have an accurate count of
the messages in my folder.

I do think that I shall look into adapting and implementing some of
the nifty MH features that I've been silently missing for so long. :-)

Regards,
Rajappa
-- 
Rajappa Iyer <rsi@lucent.com>		#include <std_disclaimer.h>
	We're too busy mopping the floor to turn off the faucet.

Re: mh backend

[David Hedbor]

François Pinard <pinard@iro.umontreal.ca> writes:

> Rajappa Iyer <rsi@lucent.com> writes:
> 
> > 3. When I periodically delete some mail, I can folder -pack to
> >    renumber all the messages.  With Gnus it is a lot more painful.
> 
> What is the purpose of doing such an operation?  I have the impression that
> it is rather unusual that one has to enter the `nnml' hierarchy directly,
> or depends on attributed numbers.  Isn't it?

The reason to do this is that in the list of messages, you otherwise
get an incorrect count. This is especially troublesome if you get lots 
of mails to one group, which usually expire quickly. I only have four
chars "reserved" for that column in the group buffer, so it looks very 
ugly when it goes over 10k. Of course I could change it to five, but
then it looks ugly when I have less...

Anyway, another solution, which might be easier, would be simply to
show the real number of messages instead of last # - first # (like
counting the number of lines in the overview file or simply storing
the numbers somewhere in .newsrc.eld).

I don't care what the numbers are, but I do want a correct display of
the number of messages in a group.

-- 
[ Below is a random fortune, which is unrelated to the above message. ]
This process can check if this value is zero, and if it is, it does
something child-like.
		-- Forbes Burkowski, CS 454, University of Washington

Re: mh backend

[Phil Humpherys]

Rajappa Iyer <rsi@lucent.com> writes:

> I do think that I shall look into adapting and implementing some of
> the nifty MH features that I've been silently missing for so long. :-)

This would be great!

--
Phil Humpherys <phumpherys@utah-inter.net>   DriverSoft
Unix Systems Administrator                   Mobile: +1.801.725.3257 
WWW/PGPkeys: http://www.spire.com/~humphery

Re: mh backend

[François Pinard]

> > > 1. Message sequences.

> > What are they?

> In MH/nmh, you can give symbolic names to an arbitrary list of
> numbers.  Then you can manipulate the messages that these numbers
> represent, by using the message sequence.  [...]

> In MH, I can assign a compound sequence of messages a name and can
> limit my operations to operate on specific sequences.  [...]

Could mailgroups, or submailgroups, be used to similar purposes?  Maybe not,
if a single message may be part of many sequences simultaneously.  Let's
assume for a moment it would be possible...

> All the MH/nmh will take message sequences as arguments.  Also, there are
> commands for adding or deleting messages from a sequence.  This makes it
> very easy to remove, refile, search, or forward a collection of messages.

A bit like it would be easy to handle messages in mailgroups.  `M P a'
could select all messages from a group, if I remember correctly.

> It's a very powerful tool.  I've always wondered why other mail
> readers haven't done similar things.

Sometimes, there are other but different paradigms which have similar
powerfulness?  It's always a bit difficult to compare different approaches.
Maybe you could define something that fits well in the various Gnus
paradigms, and that would give you the functionality you need?

> Well, it's common to repack the folder in the MH/nmh world.  The message
> numbers are not hidden in the background quite as much as in Gnus.

Maybe (just hypothesising :-) because you often need to look at these
numbers.  If you stopped having this need, probably the need to repack
would disappear as well?

> Yes, limiting takes care of matching on headers, but unless I'm
> missing something obvious, one cannot limit based on the contents
> message body.

For one, I do it with temporary scoring instead.  Maybe it would be
convenient that have a limiting function for that as well?  It might be slow.

> I find this occasionally useful when searching for a particular piece
> of email.

People speak highly of `nnir', maybe it could help?  Maybe it is overkill?

> If I expire them, it leaves a gap in the message numbers and then the
> count of messages is inaccurate since Gnus calculates the number of
> articles based on the article numbers of the first and the last articles.

Maybe the good approach would to have Gnus count correctly, then?
Repacking numbers looks a fairly heavy operation, around a Gnus limitation
which might be lifted, presumably.

> I do think that I shall look into adapting and implementing some of
> the nifty MH features that I've been silently missing for so long. :-)

Good for us! :-)

-- 
François Pinard                            mailto:pinard@iro.umontreal.ca
Join the free Translation Project!    http://www.iro.umontreal.ca/~pinard

Re: mh backend

[Edward J. Sabol]

Excerpts from mail: (15-Sep-98) Re: mh backend by =?ISO-8859-1?Q?Fran=E7ois Pinard?=
>>>> 1. Message sequences.
>> In MH/nmh, you can give symbolic names to an arbitrary list of
>> numbers.  Then you can manipulate the messages that these numbers
>> represent, by using the message sequence.  [...]

This sounds a lot like RMAIL labels. I know that, at one point, adding
user-customizable labels or annotations to Gnus was on the to-do list, but
it's probably way down on the list.

> Could mailgroups, or submailgroups, be used to similar purposes?  Maybe not,
> if a single message may be part of many sequences simultaneously.  Let's
> assume for a moment it would be possible...

Someone invariably suggests that when the topic of Gnus labels comes up.
Personally, I think subgroups are a very poor alternative.

>> Well, it's common to repack the folder in the MH/nmh world.  The message
>> numbers are not hidden in the background quite as much as in Gnus.
>
> Maybe (just hypothesising :-) because you often need to look at these
> numbers.  If you stopped having this need, probably the need to repack
> would disappear as well?

I think it would be great if Gnus could count articles correctly. However,
someone may have already written a function to pack an nnml group. See the
the following articles on the Gnus mailing list archive:
<http://www.gnus.org/cgi-bin/wilma_glimpse/ding?query=function+to+pack+folders&Search=Search&lineonly=on&errors=0&maxfiles=50&maxlines=10&.cgifields=lineonly&.cgifields=filelist&.cgifields=case&.cgifields=partial&.cgifields=restricttofiles>

>> Yes, limiting takes care of matching on headers, but unless I'm
>> missing something obvious, one cannot limit based on the contents
>> message body.
> For one, I do it with temporary scoring instead. Maybe it would be
> convenient that have a limiting function for that as well? It might be
> slow.

I really think limiting on the body should be added as feature. (If it's not
already. I could have sworn it was!) It would be really useful, I think.

Later,
Ed

Re: mh backend

[Kai Grossjohann]

>>>>> On 15 Sep 1998, Phil Humpherys said:

  Phil> Well, I'm thinking that I don't always want to have to fire up
  Phil> xemacs to read email.  If I'm in a hurry or whatever, I wouldn't
  Phil> mind being able to just do a "scan" from the shell, or even
  Phil> compose a quick email...

`scan' is okay, as is `show'.  But remember to do nothing that adds or
removes messages to/from folders.

kai
-- 
OOP: object oriented programming;  OOPS: object oriented mistakes

Re: mh backend

[Phil Humpherys]

Kai Grossjohann <grossjohann@amaunet.cs.uni-dortmund.de> writes:

> >>>>> On 15 Sep 1998, Phil Humpherys said:
> 
>   Phil> Well, I'm thinking that I don't always want to have to fire up
>   Phil> xemacs to read email.  If I'm in a hurry or whatever, I wouldn't
>   Phil> mind being able to just do a "scan" from the shell, or even
>   Phil> compose a quick email...
> 
> `scan' is okay, as is `show'.  But remember to do nothing that adds or
> removes messages to/from folders.

Understood.  It's possible that I'd also want to 'comp' from the
command line once in a while...

--
Phil Humpherys <phumpherys@utah-inter.net>   DriverSoft
Unix Systems Administrator                   Mobile: +1.801.725.3257 
WWW/PGPkeys: http://www.spire.com/~humphery

Re: mh backend

[Kai Grossjohann]

>>>>> On 15 Sep 1998, Rajappa Iyer said:

  Rajappa> Well, the idea is that despite all the fine procmail
  Rajappa> filters I have in place, a lot of spam and otherwise
  Rajappa> useless messages make it to my inbox.  If I expire them, it
  Rajappa> leaves a gap in the message numbers and then the count of
  Rajappa> messages is inaccurate since Gnus calculates the number of
  Rajappa> articles based on the article numbers of the first and the
  Rajappa> last articles.  Ideally, I'd like to be able to `pack' the
  Rajappa> nnml folder in the same way as an MH folder so that I have
  Rajappa> an accurate count of the messages in my folder.

You can process-mark all messages, then move them from the group to
itself.  After that, message numbers will be consecutive, but won't
start at 1.

kai
-- 
OOP: object oriented programming;  OOPS: object oriented mistakes

Re: mh backend

[Kai Grossjohann]

>>>>> On 15 Sep 1998, Rajappa Iyer said:

  Rajappa> 1. Message sequences.

Yeah, Gnus only has one sequence -- the process-marked messages.

  Rajappa> 2. Picking messages based on patterns in their body or some header
  Rajappa>    field.

May I point you to `nnir.el' which will let you search all mails using
freeWAIS-sf or Glimpse (beware of Glimpse being rather slow).  It only
works if you use the nnml or nnmh backend, though.

  Rajappa> 3. When I periodically delete some mail, I can folder -pack to
  Rajappa>    renumber all the messages.  With Gnus it is a lot more painful.

But you don't need it as often.  C-u g, then 9 9 9 9 #, then B m do
the trick.

kai
-- 
OOP: object oriented programming;  OOPS: object oriented mistakes

Re: mh backend

[Lars Magne Ingebrigtsen]

Rajappa Iyer <rsi@lucent.com> writes:

> I do think that I shall look into adapting and implementing some of
> the nifty MH features that I've been silently missing for so long. :-)

Great!

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: mh backend

[Per Abrahamsen]

Richard Coleman <coleman@math.gatech.edu> writes:

> > > 1. Message sequences.

> It's a very powerful tool.  I've always wondered why other mail
> readers haven't done similar things.

It seems to provide the same information as labels do in RMAIL, just
implemented in a different way.  Labels are on the Gnus TODO list (and
have been there for years).

Re: mh backend

[Justin Sheehy]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset=us-ascii, Size: 2674 bytes --]

François Pinard <pinard@iro.umontreal.ca> writes:

> Maybe (just hypothesising :-) because you often need to look at these
> numbers.  If you stopped having this need, probably the need to repack
> would disappear as well?

As far as I can see, there is only one reason why one would ever need
to pack the messages in a Gnus group.  When the article numbers become
very sparse, with relatively few articles spread over a large
numerical range, some Gnus operations become slow.  At this time, one
simply moves all of the articles in that group into the same group,
and the problem is solved.  

One must be careful not to miss dormant or otherwise unseen messages,
or it will do no good.  This should not need to be done very often
except perhaps in extremely high-volume groups.

> Maybe the good approach would to have Gnus count correctly, then?

Count?  Hm.  

For some reason, a lot of people seem to think that the handy little
number in the *Group* buffer is a count of articles, which they then
conclude is incorrect.  It is not incorrect, as it is not a "count" of
articles.  It is simply the difference between the highest and lowest
numbered articles in the group plus one, nothing more.
Coincidentally, this can be generated _much_ more quickly than an
article count and is thus used instead.

If you really want a count of articles there instead, you could
probably manage to write a user-defined specifier.  The information
needed is not readily available and might have to be generated quite
expensively each time this specifier is used.  If you do so, it will
certainly be _much_ slower than the 't' specifier.

Why exactly does everyone need an exact count of all of their groups,
anyway?  That is, how will the knowledge of exactly how many messages
you have change what actions you take?

Phil Humpherys <phumpherys@utah-inter.net> writes:

> Understood.  It's possible that I'd also want to 'comp' from the
> command line once in a while...

What's stopping you?

The only MH things that you shouldn't do if you use nnml are those
which move or delete messages.  Gnus will never know if you comp a
message.

Kai Grossjohann <grossjohann@amaunet.cs.uni-dortmund.de> writes:

>   Rajappa> 1. Message sequences.
> 
> Yeah, Gnus only has one sequence -- the process-marked messages.

Between scoring, limiting and process-marking, I've never missed
sequences in Gnus.  (I switched from MH...)

> May I point you to `nnir.el' which will let you search all mails using
> freeWAIS-sf or Glimpse (beware of Glimpse being rather slow).  

Is freeWAIS-sf significantly faster?  If so, I may stop using glimpse.

-- 
Justin Sheehy

In a cloud bones of steel.
  

Re: mh backend

[Hrvoje Niksic]

Justin Sheehy <justin@linus.mitre.org> writes:

> François Pinard <pinard@iro.umontreal.ca> writes:
> 
> > Maybe (just hypothesising :-) because you often need to look at these
> > numbers.  If you stopped having this need, probably the need to repack
> > would disappear as well?
> 
> As far as I can see, there is only one reason why one would ever need
> to pack the messages in a Gnus group.

As you said yourself, it is trivial to pack the messages in a Gnus
group.  `M P b B m GROUP' has always worked for me, GROUP being the
current group.

What is not trivial to do is make the numbers start out from 1 again,
but after thinking about it, I really can't find a reason why anyone
would want to do that.

> When the article numbers become very sparse, with relatively few
> articles spread over a large numerical range, some Gnus operations
> become slow.  At this time, one simply moves all of the articles in
> that group into the same group, and the problem is solved.
> 
> One must be careful not to miss dormant or otherwise unseen messages,
> or it will do no good.  This should not need to be done very often
> except perhaps in extremely high-volume groups.

Good point.  I use total expiry for my mailing lists, and I stopped
using dormant articles for this very reason -- it makes the total
expiry process extremely slow.

David Moore proposed a solution for this, which is for Gnus to have
more than one "active" range of articles per group.  This would
effectively solve all these slow-downs.

> Why exactly does everyone need an exact count of all of their
> groups, anyway?  That is, how will the knowledge of exactly how many
> messages you have change what actions you take?

Oh, the first part is easy to answer.  If you see a number that you
assume to be an article count, it would be logical for it to *be* the
article count.  Call it "principle the least surprise", or however you
wish.  I, for one, would like the number to be "correct" in the sense
of displaying the actual number of new articles, although I'm not
sweating over it.

-- 
Hrvoje Niksic <hniksic@srce.hr> | Student at FER Zagreb, Croatia
--------------------------------+--------------------------------
Call CIA and tell them that you have placed a bomb in a 7-11 shop!  Be
sure to let them trace you. Spend 10 years in jail and then regret it.

Re: mh backend

[Alan Shutko]

>>>>> "J" == Justin Sheehy <justin@linus.mitre.org> writes:

J> For some reason, a lot of people seem to think that the handy
J> little number in the *Group* buffer is a count of articles, which
J> they then conclude is incorrect.

Probably because the manual says it's the "estimated total number of
articles" rather than "a useless number dependant on other useless
numbers you don't care about".


J> Why exactly does everyone need an exact count of all of their
J> groups, anyway?  That is, how will the knowledge of exactly how
J> many messages you have change what actions you take?

If you have 50 articles in a group, and it says you have 2000, you'll
have an extra prompt on entry, which could scare you so you don't want
to waste the time and RAM that entering a 2000 msg group would give.
I also seem to remember that somewhere along the line Gnus will
consume much more memory entering a group where it thinks it has lots
of articles than where it knows it has 50.  I've had Gnus run out of
swap on my machine when our newsserver reset article numbers to some
arbitrarily high-value, even though there might have been only 50
articles... Gnus thoughtt there were a few million.

You can't make Gnus count correctly for _this_ instance, but if you
could for mail groups I'm sure things would be much happier.  Just
saying "Move all the messages back into the group" is bad advice,
because as you mentioned, you have to take special care of dormant or
ticked articles, cached articles, etc.  If it's going to mean that
much work for the user, why not fix the program?

-- 
Alan Shutko <ats@acm.org> - By consent of the corrupted
Don't read everything you believe.

Re: mh backend

[Kai Grossjohann]

>>>>> On 22 Sep 1998-400, Justin Sheehy said:

  Justin> Is freeWAIS-sf significantly faster?  If so, I may stop
  Justin> using glimpse.

I have approx 300MB - 400MB of mail.  I tried running a specific
freeWAIS-sf query by running waissearch from the command line.  It
took less than one second.  I tried running a similar glimpse query
from the command line (used -w for speedup, too).  It took more than
one minute.

kai
-- 
OOP: object oriented programming;  OOPS: object oriented mistakes

Re: mh backend

[Matt Pharr]

Kai Grossjohann <grossjohann@amaunet.cs.uni-dortmund.de> writes:
> >>>>> On 22 Sep 1998-400, Justin Sheehy said:
> 
>   Justin> Is freeWAIS-sf significantly faster?  If so, I may stop using
>   Justin>glimpse.
> 
> I have approx 300MB - 400MB of mail.  I tried running a specific
> freeWAIS-sf query by running waissearch from the command line.  It took
> less than one second.  I tried running a similar glimpse query from the
> command line (used -w for speedup, too).  It took more than one minute.

Hmm.  Something seems wrong with that.  I have 80MB of mail indexed with
glimpse, and searches take <1 second (on an 4 year old machine, at that.)
Try building a bigger index than the default 'tiny' 2-3% sized index. (I
use 'small' (5-10%), which the -o option to glimpseindex enables, I think,
and there's also 'medium' (20-30%) which is preusmably even faster.

-matt
-- 
Matt Pharr                                   mmp@graphics.stanford.edu
<URL:http://graphics.stanford.edu/~mmp>

Re: mh backend

[Kai Grossjohann]

>>>>> On 23 Sep 1998-700, Matt Pharr said:

  Matt> Hmm.  Something seems wrong with that.  I have 80MB of mail
  Matt> indexed with glimpse, and searches take <1 second (on an 4
  Matt> year old machine, at that.)  Try building a bigger index than
  Matt> the default 'tiny' 2-3% sized index. (I use 'small' (5-10%),
  Matt> which the -o option to glimpseindex enables, I think, and
  Matt> there's also 'medium' (20-30%) which is preusmably even
  Matt> faster.

Argh!  I misunderstood the documentation of the `-B' option for
glimpseindex.  I have now added `-o' to my glimpseindex cron job and
will see what happens.

kai
-- 
OOP: object oriented programming;  OOPS: object oriented mistakes