From mboxrd@z Thu Jan 1 00:00:00 1970 Message-ID: <4c624048dd2bc06d5db04171f51e3dd3@quanstro.net> To: 9fans@9fans.net From: erik quanstrom Date: Thu, 12 Jun 2008 06:04:19 -0400 In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 8bit Subject: Re: [9fans] store 9p session-only values using lib9p Topicbox-Message-UUID: bbd41d12-ead3-11e9-9d60-3106f5b1d025 > at the moment i'm playing only with mbox not imap or pop, i have a version with cache per message that 'works', upas/nedmail and acme/mail are able to read messages 'nicely', but attachments are not decoded. i have a couple of reservations about mbox format. first, a majority of users that i need to support have inboxes >10mb and some have mailboxes >100mb. thus deleting an older message will require rewriting the whole file, and if it's not the last message, will take up quite a bit of storage, even on venti. second, it's very difficult to cache. for example, suppose i have two instances of my mail fs interpreting the same mbox at the same time. suppose that i delete the 5th out of 500 messages with the first. the second will then have to reread the entire mbox to get its pointers right. even if i write an index with the first, the entire index needs to be reread. third, since large messages tend to be really stinking huge, it is not efficient to read entire messages to deduce the mime structure. (ned reads the mime structure of each doc on startup.) that last mime part may be tens of mb into the you'll notice that if one uses email in the way it was used traditionally in unix environments, that a 500 message, inbox or 15mb message is unreasonable. for traditional uses, mbox format and loading the whole stinking thing into memory is a great idea. unfortunately, that's now how people use email these days. our two machines running upas/fs burn up to 5gb of ram. old rewritten mail adds several hundred mb/day to the dump. > also file lengths is going to be a problem if i'm going to decode files within the fs. > > i've put what i have in /n/sources/contrib/gabidiaz/wip/mboxfs i've adapted upas/fs to use a directory for mailboxes (mdir) by default. mailbox types that support caching (mdir and imap4) keep their cache below Maxcache or Σ of (actively) referenced messages, whichever is bigger. i've also adapted the rest of upas, including deliver and marshal to understand mdir format. imap4d should be done soon. this upas/fs keeps an index, so in most cases, emails are not reread unless they are viewed. the index contains some information -- like flags -- not recorded in the mail. i'd be happy to share a working copy to those interested. but it's not ready to be inflicted on the world yet. the "From " line date is unresolved and imap4d still thinks it knows how to append a message to a mailbox. > about the From line, qmail man page about mbox format says it is composed of From space sender@... space current date, and it is generated by the delivery agent, but moving one message from a box to another doesn't use the delivery agent :-? why not? this seems an arbitrary distinction between the inbox and every other mailbox. - erik