maildir in Gnus vs. clock corrections and PID recycling

maildir in Gnus vs. clock corrections and PID recycling

[Matthias Andree]

Hi,

there's been a lot of discussion rolling through MTA lists recently,
Postfix, qmail, Courier namely, about PID reuse by the kernel and
maildir deliveries and maildir file name collisions.

Some background: maildir is a protocol invented by DJB, which is meant
to provide for locking-free mail delivery even across NFS. It delivers
to one mail per file. The process of mail delivery includes writing to a
file tmp/<time>.<unique>.<hostname> which is then linked to
new/<time>.<unique>.<hostname>. Mail reading means moving the file from
new/... to cur/<time>.<unique>.<hostname>,<flags>.

Evidently, when <unique> is not unique or <time> steps back, there may
be a collision between file names in new/ and cur/ sharing the same
"<time>.<unique>.<hostname>" part. This has been shown to cause mail
loss with both mutt and qmail-pop3d, and applications in cooperation
with Courier have shown this problem to be very real, so this is not
theoretical.

What does nnmaildir.el and possibly other maildir code in Gnus do when
it is about to move a mail from new/ to cur/? Does it rely on uniqueness
and will it -- as mutt -- accidentally kill an already-present file in
cur/? Or will it detect the collision and create a new file name for the
cur/ file, to avoid mail loss?



Further background:

The first edition of the maildir protocol documentation, maildir(5) as
shipped with qmail-1.0x, x<=3, was incomplete and used the process ID
for <unique>, which is insufficient because the PID can be recycled by
the kernel within one second.

Mate Wierdl has code to let the PID counter cycle in well below 0.2
seconds on a GHz-class IA32 Linux machine.

Some operating systems randomize their process IDs and may have even
shorter cycles between the same PID being given to a process.

The supposedly later http://cr.yp.to/proto/maildir.html refines the
documentation, and demands that the middle part does not recur within
one second, which is still insufficient given that a computer clock
might be stepped back, for example when ntpd resynchronizes after a
network outage and the clock offset is beyond the slew/step threshold.

It seems that Postfix, Courier and Maildrop will settle on using the
tmp/* file inode number for the <unique> part in new/, so these
applications will rename from tmp/time.pid_count.hostname to
new/time.inode.hostname. Dan J. Bernstein has not mentioned any fix, he
claimed qmail-pop3d was safe, but Mate Wierdl posted counter evidence to
the qmail mailing list. Sendmail 8.12 doesn't to Maildir deliveries. I'm
unsure about Exim and Procmail.

Feel free to forward this mail to the procmail and exim mailing lists
and report about the response.

-- 
Matthias Andree

Re: maildir in Gnus vs. clock corrections and PID recycling

[Paul Jarc]

Matthias Andree <ma@dt.e-technik.uni-dortmund.de> wrote:
> Mail reading means moving the file from new/... to
> cur/<time>.<unique>.<hostname>,<flags>.

Actually, it's cur/<time>.<unique>.<hostname>:<flags>.

> What does nnmaildir.el and possibly other maildir code in Gnus do when
> it is about to move a mail from new/ to cur/? Does it rely on uniqueness
> and will it -- as mutt -- accidentally kill an already-present file in
> cur/?

Yes.

> The supposedly later http://cr.yp.to/proto/maildir.html refines the
> documentation, and demands that the middle part does not recur within
> one second, which is still insufficient given that a computer clock
> might be stepped back,

Stepping a clock backwards produces so many problems that it is better
to ignore those problems, run the clock monotonically, and find a
different way to deal with any problems that stepping backwards was
supposed to solve.

> It seems that Postfix, Courier and Maildrop will settle on using the
> tmp/* file inode number for the <unique> part in new/, so these
> applications will rename from tmp/time.pid_count.hostname to
> new/time.inode.hostname.

That sounds like a good idea.  As long as all maildir writers do that,
readers (programs that move messages from new/ to cur/) won't have to
be changed.  nnmaildir does deliver new messages to tmp/ sometimes, so
I'll change that.  But the problem isn't likely to affect nnmaildir
anyway, since Emacs is long-lived and nnmaildir is not likely to
deliver new messages within 1 second of Emacs starting or exiting.


paul

Re: maildir in Gnus vs. clock corrections and PID recycling

[Matthias Andree]

Paul Jarc schrieb am Sonntag, den 19. Januar 2003:

> > The supposedly later http://cr.yp.to/proto/maildir.html refines the
> > documentation, and demands that the middle part does not recur within
> > one second, which is still insufficient given that a computer clock
> > might be stepped back,
> 
> Stepping a clock backwards produces so many problems that it is better
> to ignore those problems, run the clock monotonically, and find a
> different way to deal with any problems that stepping backwards was
> supposed to solve.

This would mean rebooting systems when the clocks are fast and cannot be
just slowed reasonably -- this cannot be expected, so applications must
be prepared to clocks being stepped backwards. It'd be fine with me if
the program would just complain, the ultime goal is avoiding EVER
overwriting a file.

> That sounds like a good idea.  As long as all maildir writers do that,
> readers (programs that move messages from new/ to cur/) won't have to
> be changed.  nnmaildir does deliver new messages to tmp/ sometimes, so
> I'll change that.  But the problem isn't likely to affect nnmaildir
> anyway, since Emacs is long-lived and nnmaildir is not likely to
> deliver new messages within 1 second of Emacs starting or exiting.

Agreed.

-- 
Matthias Andree

Re: maildir in Gnus vs. clock corrections and PID recycling

[Paul Jarc]

Matthias Andree <matthias.andree@uni-dortmund.de> wrote:
> Paul Jarc schrieb am Sonntag, den 19. Januar 2003:
>> Stepping a clock backwards produces so many problems that it is better
>> to ignore those problems, run the clock monotonically, and find a
>> different way to deal with any problems that stepping backwards was
>> supposed to solve.
>
> This would mean rebooting systems when the clocks are fast and cannot be
> just slowed reasonably

No.  It means the clock would be run slow *all the time*, to keep it
from getting ahead in the first place.  This is how clockspeed works.

> It'd be fine with me if the program would just complain, the ultime
> goal is avoiding EVER overwriting a file.

I'll probably add that too.


paul

Re: maildir in Gnus vs. clock corrections and PID recycling

[Matthias Andree]

prj@po.cwru.edu (Paul Jarc) writes:

> No.  It means the clock would be run slow *all the time*, to keep it
> from getting ahead in the first place.  This is how clockspeed works.

clockspeed is imperfect and assumes the oscillator runs at the same
temperature all the time, or it relies on getting outside input.

ntpd relies on outside input.

If this outside input is unavailable, because your receiver is faulty,
the transmission is jammed, the network blacked out, then the clocks
will start drifting, and it may have drifted some seconds when the
clocks can be synchronized again. You can't prevent that, but software
should be prepared to deal with it.

How gracefully the software deals with this condition, is up to the
programmer, but overwriting existing files with different content is
certainly not desirable in any case.

>> It'd be fine with me if the program would just complain, the ultime
>> goal is avoiding EVER overwriting a file.
>
> I'll probably add that too.

Thank you.

-- 
Matthias Andree

Re: maildir in Gnus vs. clock corrections and PID recycling

[Paul Jarc]

Matthias Andree <ma@dt.e-technik.uni-dortmund.de> wrote:
> It seems that Postfix, Courier and Maildrop will settle on using the
> tmp/* file inode number for the <unique> part in new/, so these
> applications will rename from tmp/time.pid_count.hostname to
> new/time.inode.hostname.

What I saw in the qmail archives was new/time.inode.pid.hostname.
time.inode.hostname is extremely broken, and must not be used: there
might be two delivery programs running at once.  One might be an old
one using pid and the other might be a new one using inode.  If the
old one's pid is the same as the new one's inode, you can get a
collision.  So it should be <time>.i<inode>.<hostname> instead, to
ensure that it cannot be confused with the old format.  Please pass
this on to anyone who is planning to use <time>.<inode>.<hostname>.


paul

Re: maildir in Gnus vs. clock corrections and PID recycling

[Paul Jarc]

I wrote:
> Matthias Andree <ma@dt.e-technik.uni-dortmund.de> wrote:
>> What does nnmaildir.el and possibly other maildir code in Gnus do when
>> it is about to move a mail from new/ to cur/? Does it rely on uniqueness
>> and will it -- as mutt -- accidentally kill an already-present file in
>> cur/?
>
> Yes.

I was wrong.  nnmaildir will raise an error if the file already
exists.

>> It seems that Postfix, Courier and Maildrop will settle on using the
>> tmp/* file inode number for the <unique> part in new/, so these
>> applications will rename from tmp/time.pid_count.hostname to
>> new/time.inode.hostname.
>
> That sounds like a good idea.

After reading more in the qmail archives, I think I'll do what
qmail-pop3d does: leave message in new/ (so that attempted deliveries
with the same filename will fail) until the second is over.


paul

Re: maildir in Gnus vs. clock corrections and PID recycling

[Matthias Andree]

On Wed, 22 Jan 2003, Paul Jarc wrote:

> I was wrong.  nnmaildir will raise an error if the file already
> exists.

Good, so it won't wreak havoc.

> After reading more in the qmail archives, I think I'll do what
> qmail-pop3d does: leave message in new/ (so that attempted deliveries
> with the same filename will fail) until the second is over.

Given that I commonly see machines reset their clocks by one second,
"until the file is two seconds old" might be more appropriate, if I may
suggest so.

-- 
Matthias Andree