Sgnus .12 just lost my mail.

Sgnus .12 just lost my mail.

[Robert Nicholson]

I have a procmail log entry indicating a message was written to disk
but the message and it's Incomming file is no where to be seen.

Also, occasionally I get ^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@
appended to messages. Where are these nulls comming from?

-- 
             "Mary ate a little lamb and punk rock isn't dead"
            (PGP key: send email with Subject: request pgp key)

Re: Sgnus .12 just lost my mail.

[Steven L. Baur]

>>>>> "Robert" == Robert Nicholson <robert@steffi.accessone.com> writes:

    Robert> I have a procmail log entry indicating a message was
    Robert> written to disk but the message and it's Incomming file is
    Robert> no where to be seen.

Ed Sabol sent a definitive message about this at the end of August
which I misfiled, so it doesn't appear in the hypertext archive.  I'm
reposting it in its entirety, since I found that the default settings
in both XEmacs and Emacs configurations (for Linux) will lead to mail
lossage if not corrected, and Ed provides a very nice cookbook method
to determine whether or not you are at risk.

Date: Wed, 30 Aug 1995 15:45:03 -0400
From: "Edward J. Sabol" <sabol@thuban.gsfc.nasa.gov>
Message-Id: <199508301945.PAA01678@thuban.gsfc.nasa.gov>
To: ding Gnus Mailing List <ding@ifi.uio.no>
In-reply-to: <9508301606.AA13278@sunscipw.cern.ch> (message from Lars Magne Ingebrigtsen on Wed, 30 Aug 95 18:06:09 +0200)
Subject: Re: procmail v.s ding mail splitting
Lines: 94
Xref: diana.miranova.com dgnus-list:478

Excerpts from mail: (30-Aug-95) Re: procmail v.s ding mail splitting by Lars Magne Ingebrigtsen
Manoj> Umm, I'm wondering what would happen if mailagent/procmail were
Manoj> writing to a spool file at the same time that ding was reading it, the
Manoj> sudden truncation might give mailagent/procmail hissy fits (I mean,
Manoj> ding doesn't really look at the locking mechanism that
Manoj> mailagent/procmail uses, does it?)

Lars> Gnus uses "movemail" to, uhm, move the mail from the spool files.
Lars> movemail and procmail coexists rather peacefully, don't they?

Yes, but only if both movemail and procmail are configured correctly to use
at least one of the same locking methods.

Manoj> I just checked ... both movemail and mailagent are aware of
Manoj> MAIL_USE_FLOCK or MAIL_USE_LOCKF, and mailagent in particular will try
Manoj> and use _both_ if possible. so there should be no interference.

Except neither flock() or lockf() work reliably on files located on
NFS-mounted volumes for a variety of reasons. If the directory where you have
procmail deliver your split mail resides on an NFS-mounted volume, the only
reliable locking method is .lock semaphore files. Procmail can simultaneously
use lockf(), flock(), and .lock semaphore files (maybe even fcntl(), but I'm
not sure) all at the same time. Movemail, unfortunately, can only be
configured to use *one* of those locking methods. Procmail's installation
script automatically tests various directories and each of the locking
methods to see which ones your system needs. Movemail's locking method is set
by the system defaults that the GNU people ship with Emacs. The GNU configure
script doesn't, for example, test to see whether your home directory or your
mail spool directory are NFS-mounted. The problem is really with movemail
which cannot be configured to use both lockf()/flock() *and* .lock files at
the same time. Here's how to test to see if your movemail will work correctly
with .lock files: (Do this in the directory where you keep your mail files
before (ding) gets them.)

% echo "Testing." > foo
% lockfile foo.lock
% ls -l
total 2
-rw-r--r--  1 sabol           9 Aug 30 14:28 foo
-r--r--r--  1 sabol           1 Aug 30 14:29 foo.lock
% /usr/local/lib/emacs/19.28/sparc-sun-sunos4.1.3_U1/movemail foo bar
% ls -l
total 1
-rw-r--r--  1 sabol           9 Aug 30 14:35 bar
-rw-r--r--  1 sabol           0 Aug 30 14:35 foo

Movemail just ignored lockfile's .lock semaphore file! This means movemail
was compiled to use either lockf() or flock(). If your home directory or mail
spool directory are NFS-mounted, you should reconfigure and recompile
movemail to use .locks, like so:

% ls -l
total 2
-rw-r--r--  1 sabol           9 Aug 30 14:35 foo
-r--r--r--  1 sabol           1 Aug 30 14:39 foo.lock
% ~/bin/movemail foo bar &
[1] 187
% ls -l
total 2
-rw-r--r--  1 sabol           9 Aug 30 14:41 foo
-r--r--r--  1 sabol           1 Aug 30 14:41 foo.lock
% rm -f foo.lock 
% [1]    Done                   ~/bin/movemail foo bar
ls -l
total 1
-rw-r--r--  1 sabol           9 Aug 30 14:41 bar
-rw-r--r--  1 sabol           0 Aug 30 14:41 foo

That's the way it's supposed to work, but even this method isn't perfect.
Movemail will wait around for about twenty seconds and then go ahead and move
the file anyway, thinking the lock is stale, which in the case of some rather
complex mail processing might not be the case. This timeout is not
configurable either without changing the source code. Fortunately, it's
usually not a common problem.

Either way, movemail could definitely use some improvement to make it
NFS-resistant. Perhaps the most reliable method is to create a wrapper script
around a movemail configured to use either lockf() and flock(). The wrapper
script would recognize and create .lock semaphore files using procmail's
'lockfile'. This would give you the best of both worlds and be the most
reliable. I actually did this for a while, but it really slows down fetching
of new mail. I finally got fed up with it and asked my system administrators
for the movemail source code and compiled my own version that uses .lock
files.

And if you use procmail and RMAIL, make sure you

(setq rmail-spool-directory (expand-file-name "~/Mail/"))

(or whereever you have procmail deliver your split mail). Otherwise, RMAIL
won't even bother to use movemail at all. Actually, that may have been fixed
in Emacs 19.29, I'm not sure.

Later,
Ed


-- 
steve@miranova.com baur