Questions on nnmaildir

Questions on nnmaildir

[Thomas Neumann]

hiya

little introduction: So far I used Gnus with nnimap-Backend to access a
courier-imap server on a different or the same box. [Reading mail from
2 hosts]. Today I came around fiddling with nnmaildir. I was attempting to
remove the courier-part when reading mail locally.

First surprise:
  Although courier and nnmaildir use Maildirs, they're not quite
  compatible with each other.


But that's not the reason, why I write this mail. It seems that
nnmaildir is quite an inode-hogger.

Let's suppose there is a maildir with 500 mails, 15 are ticked, 480 are
read, none 'new' or 'temporary'.

This results in:
  [..]/cur                     -> 500 files
  [..]/new                     ->  no files
  [..]/tmp                     ->  no files
  [..]/.nnmaildir/marks/read   -> 480 files
  [..]/.nnmaildir/marks/ticked ->  15 files
  [..]/.nnmaildir/nov          -> 500 files
  [..]/.nnmaildir/num          -> 500 files

Files in read, ticked and num are all 0 byte in size. 
Each nov files has a single line. [Some of my maildirs contain
mailinglists with up to 11K messages *shudder*]

Wouldn't it make more sense to handle the marks in a fashion close to
courier? [appending them to the filename] and generate a single
overview-file containg all novs?

Well I'm not convinced that I could hack nnmaildir to my liking. So
either someone else is so kind and takes a stab at it, or I better
forget having heard of nnmaildir.



I'm using this setup for nnmaildir:
------------------------------------------------------------------------
(setq
 ;; Read mail with nnimap
 ;; The same for the other methods, kind of.
 gnus-secondary-select-methods 
 '([...]
   (nnmaildir "thomas"
	      (directory "~/Maildirs")
	      (create-directory "../nnmaildir")
	      (directory-files nnheader-directory-files-safe)
	      (get-new-mail t)
	      )
   [...]
   )
------------------------------------------------------------------------
'[...]' are omittet parts



tschüß
thomas

Re: Questions on nnmaildir

[Josh Huber]

Thomas Neumann <blacky_tn@gmx.de> writes:

> But that's not the reason, why I write this mail. It seems that
> nnmaildir is quite an inode-hogger.

They're hardlinks.

-- 
Josh Huber

Re: Questions on nnmaildir

[Kai Großjohann]

Thomas Neumann <blacky_tn@gmx.de> writes:

> Wouldn't it make more sense to handle the marks in a fashion close to
> courier? [appending them to the filename] and generate a single
> overview-file containg all novs?

nnml stores marks in the .marks file in that directory, and stores
novs in .overview.  Seems like nnml does what you're asking for.

But do note the other comment about the size-zero files being hard
links to each other -- I think that way only one inode is used, even
for many directory entries.
-- 
A preposition is not a good thing to end a sentence with.

Re: Questions on nnmaildir

[Thomas Neumann]

Josh Huber <huber@alum.wpi.edu> writes:

> Thomas Neumann <blacky_tn@gmx.de> writes:
>
>> But that's not the reason, why I write this mail. It seems that
>> nnmaildir is quite an inode-hogger.

> They're hardlinks.

Hmm.

Output of `df -i`

1) /dev/hda5             611648   61281  550367   11% /home
2) /dev/hda5             611648   61297  550351   11% /home
3) /dev/hda5             611648   61302  550346   11% /home
4) /dev/hda5             611648   61419  550229   11% /home
5) /dev/hda5             611648   61548  550100   11% /home

1) nothing done so far
2) B c message to new maildir group.test, which did not exist so far
3) B c different message to maildir group.test
4) C-u 50 # B c nnmaildir+thomas:group.Inbox
   (group.Inbox didn't exist before)
5) Copy 50 other mails to group.Inbox


Seems like I have about 250K mails left before I run out of Inodes on
/home.  Which is not as bad as it looked first, but still less then I
expected.


Is it really that much faster for nnmaildir to pick a specific nov-file
and grab its contents then to search through a single file containing
all novs and look for a match? [Or what's the reason behind doing it
this way?]

tschüß
thomas

Re: Questions on nnmaildir

[Paul Jarc]

Something seems to be up with my subscription; I haven't received any
ding messages since Friday, when there were DNS changes for
multivac.cwru.edu.

Thomas Neumann <blacky_tn@gmx.de> wrote:
> It seems that nnmaildir is quite an inode-hogger.

The files in */.nnmaildir/marks/*/ are hard links.  By default, they
use one inode per group, but if you create */.nnmaildir/markfile
yourself as a hard link, you can use just one inode total.

The files in */.nnmaildir/num/ are hard links.  One extra inode per
group.

The files in */.nnmaildir/nov/ are separate inodes; nnmaildir uses two
inodes per message, where nnml and nnmh use one.

> Wouldn't it make more sense to handle the marks in a fashion close to
> courier? [appending them to the filename]

The current method makes it easier to manipulate marks from outside
Gnus, and for nnmaildir to pick up changes on the fly (whether made
from a shell, or from a concurrent Gnus).  Gnus also has more marks
than the standard maildir format.

You add a hook function to sync nnmaildir's marks with Courier's with
a script: when scanning for new messages, update nnmaildir's marks;
when exiting a group, update Courier's.

> Is it really that much faster for nnmaildir to pick a specific nov-file
> and grab its contents then to search through a single file containing
> all novs and look for a match?

The task here is to fetch some persistently-stored information indexed
by a string (the unique part of the message's filename).  That is
precisely what the filesystem is meant for.  Historically, many
filesystems didn't do a great job of it, and if you're stuck with one
of those, maybe nnmaildir isn't for you.  But if you're not, ReiserFS
or another modern filesystem may be just what you need.


paul