From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.io/gmane.emacs.gnus.general/5428 Path: main.gmane.org!not-for-mail From: tom@smart.ruhr.de (Thomas Neumann) Newsgroups: gmane.emacs.gnus.general Subject: Request for "maildir" support Date: 04 Mar 1996 14:47:06 +0100 Organization: Ruhr.DE Message-ID: <87ag1xj7hh.fsf@smart.ruhr.de> NNTP-Posting-Host: coloc-standby.netfonds.no X-Trace: main.gmane.org 1035146032 32762 80.91.224.250 (20 Oct 2002 20:33:52 GMT) X-Complaints-To: usenet@main.gmane.org NNTP-Posting-Date: Sun, 20 Oct 2002 20:33:52 +0000 (UTC) Return-Path: ding-request@ifi.uio.no Original-Received: from ifi.uio.no (ifi.uio.no [129.240.64.2]) by deanna.miranova.com (8.7.3/8.6.9) with SMTP id GAA17006 for ; Mon, 4 Mar 1996 06:52:45 -0800 Original-Received: from smart.ruhr.de (smart.ruhr.de [193.100.176.36]) by ifi.uio.no with SMTP (8.6.11/ifi2.4) id for ; Mon, 4 Mar 1996 14:47:16 +0100 Original-Received: (qmail-queue invoked by uid 1000); 4 Mar 1996 13:47:09 GMT Original-To: ding@ifi.uio.no Original-Lines: 295 Xref: main.gmane.org gmane.emacs.gnus.general:5428 X-Report-Spam: http://spam.gmane.org/gmane.emacs.gnus.general:5428 A guy named D.J. Bernstein is currently developing "qmail", a new MTA. qmail features a new mailbox format called "maildir", which is different from std. UNIX mailbox files, different from MH folders and different from whatever you care to name. Nevertheless, D.J.'s maildir is rather well designed and potentially much more reliable than most existing formats, and I think it would be nice if Red Gnus supports it as a mail format. I'm appending a manpage (maildir.5) that describes the format, as well as some general qmail blurb. I'm sure D.J. would be more than willing to help in case there are any questions regading maildir semantics. -t *** BLURB *** qmail is a secure, reliable, efficient, simple message transfer agent. It is meant as a replacement for the entire sendmail-binmail system on typical Internet-connected UNIX hosts. Secure: Security isn't just a goal, but an absolute requirement. Mail delivery is critical for users; it cannot be turned off, so it must be completely secure. (This is why I started writing qmail: I was sick of the security holes in sendmail and other MTAs.) Reliable: qmail's straight-paper-path philosophy guarantees that a message, once accepted into the system, will never be lost. qmail also supports maildir, a new, super-reliable user mailbox format. Maildirs, unlike mbox files and mh folders, won't be corrupted if the system crashes during delivery. Even better, not only can a user safely read his mail over NFS, but any number of NFS clients can deliver mail to him at the same time. Efficient: On a Pentium under BSD/OS, qmail can easily sustain 200000 local messages per day---that's separate messages injected and delivered to mailboxes in a real test! Although remote deliveries are inherently limited by the slowness of DNS and SMTP, qmail overlaps 20 simultaneous deliveries by default, so it zooms quickly through mailing lists. (This is why I finished qmail: I had to get a big mailing list set up.) Simple: qmail is vastly smaller than any other Internet MTA. Some reasons why: (1) Other MTAs have separate forwarding, aliasing, and mailing list mechanisms. qmail has one simple forwarding mechanism that lets users handle their own mailing lists. (2) Other MTAs offer a spectrum of delivery modes, from fast+unsafe to slow+queued. qmail-send is instantly triggered by new items in the queue, so the qmail system has just one delivery mode: fast+queued. (3) Other MTAs include, in effect, a specialized version of inetd that watches the load average. qmail's design inherently limits the machine load, so qmail-smtpd can safely run from your system's inetd. Replacement for sendmail: qmail supports host and user masquerading, full host hiding, virtual domains, null clients, list-owner rewriting, relay control, double-bounce recording, arbitrary RFC 822 address lists, cross-host mailing list loop detection, per-recipient checkpointing, etc. In short, it's up to speed on modern MTA features. (If you need another feature, let me know.) qmail includes a drop-in ``sendmail'' wrapper so that it will be used transparently by your current UAs. *** maildir.5 *** .TH maildir 5 .SH "NAME" maildir \- directory for incoming mail messages .SH "INTRODUCTION" .I maildir is a structure for directories of incoming mail messages. It solves the reliability problems that plague .I mbox files and .I mh folders. .SH "RELIABILITY ISSUES" A machine may crash while it is delivering a message. For both .I mbox files and .I mh folders this means that the message will be silently truncated. Even worse: for .I mbox format, if the message is truncated in the middle of a line, it will be silently joined to the next message. The mail transport agent will try again later to deliver the message, but it is unacceptable that a corrupted message should show up at all. In .IR maildir , every message is guaranteed complete upon delivery. A machine may have two programs simultaneously delivering mail to the same user. The .I mbox and .I mh formats require the programs to update a single central file. If the programs do not use some locking mechanism, the central file will be corrupted. There are several .I mbox and .I mh locking mechanisms, none of which work portably and reliably. In contrast, in .IR maildir , no locks are ever necessary. Different delivery processes never touch the same file. A user may try to delete messages from his mailbox at the same moment that the machine delivers a new message. For .I mbox and .I mh formats, the user's mail-reading program must know what locking mechanism the mail-delivery programs use. In contrast, in .IR maildir , any delivered message can be safely updated or deleted by a mail-reading program. Many sites use Sun's .B Network F\fPa\fBil\fPur\fBe System (NFS), presumably because the operating system vendor does not offer anything else. NFS exacerbates all of the above problems. Some NFS implementations don't provide .B any reliable locking mechanism. With .I mbox and .I mh formats, if two machines deliver mail to the same user, or if a user reads mail anywhere except the delivery machine, the user's mail is at risk. .I maildir works without trouble over NFS. .SH "THE MAILDIR STRUCTURE" A directory in .I maildir format has three subdirectories, all on the same filesystem: .BR tmp , .BR new , and .BR cur . Each file in .B new is a newly delivered mail message. The modification time of the file is the delivery date of the message. The message is delivered .I without an extra UUCP-style .EX From djb Mon Jul 3 01:55:29 1995 .EE line, .I without any .B >From quoting, and .I without an extra blank line at the end. The message is probably in RFC 822 format, but it could contain arbitrary binary data. It might not even end with a newline. Files in .B cur are just like files in .BR new . The big difference is that files in .B cur are no longer new mail: they have been seen by the user's mail-reading program. .SH "HOW A MESSAGE IS DELIVERED" The .B tmp directory is used to ensure reliable delivery, as discussed here. A program delivers a mail message in six steps. First, it .B chdir()\fPs to the .I maildir directory. Second, it .B stat()s the name .BR tmp/\fItime.pid.host , where .I time is the number of seconds since the beginning of 1970 GMT, .I pid is the program's process ID, and .I host is the host name. Third, if .B stat() returned anything other than ENOENT, the program sleeps for two seconds, updates .IR time , and tries the .B stat() again, a limited number of times. Fourth, the program creates .BR tmp/\fItime.pid.host . Fifth, the program .I NFS-writes the message to the file. Sixth, the program .BR link() s the file to .BR new/\fItime.pid.host . At that instant the message has been successfully delivered. The delivery program is required to start a 24-hour timer before creating .BR tmp/\fItime.pid.host , and to abort the delivery if the timer expires. Upon error, timeout, or normal completion, the delivery program may attempt to .B unlink() .BR tmp/\fItime.pid.host . .I NFS-writing means (1) as usual, checking the number of bytes returned from each .B write() call; (2) calling .B fsync() and checking its return value; (3) calling .B close() and checking its return value. (Standard NFS implementations handle .B fsync() incorrectly but make up for it by abusing .BR close() .) .SH "HOW A MESSAGE IS READ" A mail reader operates as follows. It looks through the .B new directory for new messages. Say there is a new message, .BR new/\fItime.pid.host . The reader may freely display the contents of .BR new/\fItime.pid.host , delete .BR new/\fItime.pid.host , or rename .B new/\fItime.pid.host as .BR cur/\fItime.pid.host:info . The meaning of .I info is up to the mail reader. The reader is also expected to look through the .B tmp directory and to clean up any old files found there. A file in .B tmp may be safely removed if it has not been accessed in 36 hours. .SH "ENVIRONMENT VARIABLES" Mail readers supporting .I maildir use the .B MAILDIR environment variable as the name of the user's primary mail directory. .SH "SEE ALSO" qmail-alias(8)