Re: [9fans] store 9p session-only values using lib9p

["Fco. J. Ballesteros" <nemo@lsub.org>]

Funny, I've done the same in a different way.
see mail2fs in contrib/nemo.
Also, I have some proposal, skip to the end of the mail and let me know
what you think :)

In any case, I'd love to see/try your version of upas/fs et al.

Instead of adapting upas/fs, I use a mail2fs program that uses
upas to convert mail into an "unpacked" form. Each mail is a directory.
A "text" file contains the message text right as you would see it in a mail
reader (including relative paths for attachments). Each attach is decoded
and kept in the mail's directory ready to be copied, printed, etc.; if possible,
using the same file name reported by the attachment.

I use the very same approach for sending mail. A directory /mail/box/nemo/out
is spooled to send whichever file is placed on it as a mail. The format for the
outgoing messages is again similar to what you'd type in a mail writer.

The whole point is that now you editor (plus couple of scripts) becomes a mail
reader/writer without understanding anything about mail.

My proposal:
Why don't we reach an agreement and start using the very
same format. I suggest keeping my mbox format but adapting upas/fs to
understand it, which is a good idea. But I'm open to suggestions.

thanks

>  From: quanstro@quanstro.net
>  To: 9fans@9fans.net
>  Reply-To: 9fans@9fans.net
>  Date: Thu Jun 12 12:10:32 CET 2008
>  Subject: Re: [9fans] store 9p session-only values using lib9p
>
>  > at the moment i'm playing only with mbox not imap or pop, i have a version with cache per message that 'works', upas/nedmail and acme/mail are able to read messages 'nicely', but attachments are not decoded.
>
>  i have a couple of reservations about mbox format. first, a majority
>  of users that i need to support have inboxes >10mb and some
>  have mailboxes >100mb. thus deleting an older message will require
>  rewriting the whole file, and if it's not the last message, will take
>  up quite a bit of storage, even on venti.
>
>  second, it's very difficult to cache. for example, suppose i have two
>  instances of my mail fs interpreting the same mbox at the same time.
>  suppose that i delete the 5th out of 500 messages with the first. the
>  second will then have to reread the entire mbox to get its pointers
>  right. even if i write an index with the first, the entire index
>  needs to be reread.
>
>  third, since large messages tend to be really stinking huge, it is not
>  efficient to read entire messages to deduce the mime structure. (ned
>  reads the mime structure of each doc on startup.) that last mime part
>  may be tens of mb into the
>
>  you'll notice that if one uses email in the way it was used
>  traditionally in unix environments, that a 500 message, inbox or 15mb
>  message is unreasonable. for traditional uses, mbox format and
>  loading the whole stinking thing into memory is a great idea.
>
>  unfortunately, that's now how people use email these days.
>  our two machines running upas/fs burn up to 5gb of ram.
>  old rewritten mail adds several hundred mb/day to the dump.
>
>  > also file lengths is going to be a problem if i'm going to decode files within the fs.
>  >
>  > i've put what i have in /n/sources/contrib/gabidiaz/wip/mboxfs
>
>  i've adapted upas/fs to use a directory for mailboxes (mdir) by default.
>  mailbox types that support caching (mdir and imap4) keep their cache
>  below Maxcache or Σ of (actively) referenced messages, whichever is bigger.
>
>  i've also adapted the rest of upas, including deliver and marshal to
>  understand mdir format. imap4d should be done soon.
>
>  this upas/fs keeps an index, so in most cases, emails are not reread unless they are
>  viewed. the index contains some information -- like flags -- not recorded
>  in the mail.
>
>  i'd be happy to share a working copy to those interested. but it's
>  not ready to be inflicted on the world yet. the "From " line date is
>  unresolved and imap4d still thinks it knows how to append a message
>  to a mailbox.
>
>  > about the From line, qmail man page about mbox format says it is composed of From space sender@... space current date, and it is generated by the delivery agent, but moving one message from a box to another doesn't use the delivery agent :-?
>
>  why not? this seems an arbitrary distinction between the inbox and
>  every other mailbox.
>
>  - erik
>
>