MAIL, MAILPATH and maildir support

MAIL, MAILPATH and maildir support

[Clint Adams]

This patch is from Miquel van Smoorenburg <miquels@cistron.nl>.

It corrects the treatment of maildir-format mailbox directories
in MAILPATH.

-----
diff -ruN zsh-3.1.5.pws19.b4/Src/mailstat.c zsh-3.1.5.pws19/Src/mailstat.c
--- zsh-3.1.5.pws19.b4/Src/mailstat.c	Thu Jan  1 01:00:00 1970
+++ zsh-3.1.5.pws19/Src/mailstat.c	Wed Jun  2 12:29:32 1999
@@ -0,0 +1,98 @@
+/*
+ *	Stat a file. If it's a maildir, check all messages
+ *	in the maildir and present the grand total as a file.
+ *	The fields in the 'struct stat' are from the mail directory.
+ *	The following fields are emulated:
+ *
+ *	st_nlink	always 1
+ *	st_size		total number of bytes in all files
+ *	st_blocks	total number of messages
+ *	st_atime	access time of newest file in maildir
+ *	st_mtime	modify time of newest file in maildir
+ *	st_mode		S_IFDIR changed to S_IFREG
+ *
+ *	This is good enough for most mail-checking applications.
+ */
+int
+mailstat(char *path, struct stat *st)
+{
+	DIR			*dd;
+	struct			dirent *fn;
+	struct stat		st_ret, st_tmp;
+	static struct stat	st_new_last, st_ret_last;
+	char			dir[PATH_MAX * 2];
+	char			file[PATH_MAX * 2];
+	int			i, l;
+	time_t			atime = 0, mtime = 0;
+
+	/* First see if it's a directory. */
+	if ((i = stat(path, st)) != 0 || !S_ISDIR(st->st_mode))
+		return i;
+	if (strlen(path) > sizeof(dir) - 5) {
+		errno = ENAMETOOLONG;
+		return -1;
+	}
+
+	st_ret = *st;
+	st_ret.st_nlink = 1;
+	st_ret.st_size  = 0;
+	st_ret.st_blocks  = 0;
+	st_ret.st_mode  &= ~S_IFDIR;
+	st_ret.st_mode  |= S_IFREG;
+
+	/* See if cur/ is present */
+	sprintf(dir, "%s/cur", path);
+	if (stat(dir, &st_tmp) || !S_ISDIR(st_tmp.st_mode)) return 0;
+	st_ret.st_atime = st_tmp.st_atime;
+
+	/* See if tmp/ is present */
+	sprintf(dir, "%s/tmp", path);
+	if (stat(dir, &st_tmp) || !S_ISDIR(st_tmp.st_mode)) return 0;
+	st_ret.st_mtime = st_tmp.st_mtime;
+
+	/* And new/ */
+	sprintf(dir, "%s/new", path);
+	if (stat(dir, &st_tmp) || !S_ISDIR(st_tmp.st_mode)) return 0;
+	st_ret.st_mtime = st_tmp.st_mtime;
+
+	/* Optimization - if new/ didn't change, nothing else did. */
+	if (st_tmp.st_dev == st_new_last.st_dev &&
+	    st_tmp.st_ino == st_new_last.st_ino &&
+	    st_tmp.st_atime == st_new_last.st_atime &&
+	    st_tmp.st_mtime == st_new_last.st_mtime) {
+		*st = st_ret_last;
+		return 0;
+	}
+	st_new_last = st_tmp;
+
+	/* Loop over new/ and cur/ */
+	for (i = 0; i < 2; i++) {
+		sprintf(dir, "%s/%s", path, i ? "cur" : "new");
+		sprintf(file, "%s/", dir);
+		l = strlen(file);
+		if ((dd = opendir(dir)) == NULL)
+			return 0;
+		while ((fn = readdir(dd)) != NULL) {
+			if (fn->d_name[0] == '.' ||
+			    strlen(fn->d_name) + l >= sizeof(file))
+				continue;
+			strcpy(file + l, fn->d_name);
+			if (stat(file, &st_tmp) != 0)
+				continue;
+			st_ret.st_size += st_tmp.st_size;
+			st_ret.st_blocks++;
+			if (st_tmp.st_atime != st_tmp.st_mtime &&
+			    st_tmp.st_atime > atime)
+				atime = st_tmp.st_atime;
+			if (st_tmp.st_mtime > mtime)
+				mtime = st_tmp.st_mtime;
+		}
+		closedir(dd);
+	}
+
+	if (atime) st_ret.st_atime = atime;
+	if (mtime) st_ret.st_mtime = mtime;
+
+	*st = st_ret_last = st_ret;
+	return 0;
+}
diff -ruN zsh-3.1.5.pws19.b4/Src/utils.c zsh-3.1.5.pws19/Src/utils.c
--- zsh-3.1.5.pws19.b4/Src/utils.c	Sun May 16 17:32:15 1999
+++ zsh-3.1.5.pws19/Src/utils.c	Wed Jun  2 12:30:03 1999
@@ -29,6 +29,7 @@
 
 #include "zsh.mdh"
 #include "utils.pro"
+#include "mailstat.c"
 
 /* Print an error */
 
@@ -711,7 +712,7 @@
 	if (**s == 0) {
 	    *v = c;
 	    zerr("empty MAILPATH component: %s", *s, 0);
-	} else if (stat(unmeta(*s), &st) == -1) {
+	} else if (mailstat(unmeta(*s), &st) == -1) {
 	    if (errno != ENOENT)
 		zerr("%e: %s", *s, errno);
 	} else if (S_ISDIR(st.st_mode)) {

Re: MAIL, MAILPATH and maildir support

[Bart Schaefer]

On Jun 3, 12:00am, Clint Adams wrote:
} Subject: MAIL, MAILPATH and maildir support
}
} This patch is from Miquel van Smoorenburg <miquels@cistron.nl>.
} 
} It corrects the treatment of maildir-format mailbox directories
} in MAILPATH.

I was about to gripe about how horribly inefficient this bit of code is,
when I realized I'd forgotten that the existing support for directories
is if anything worse.  Recursively stat() an entire directory tree??
Does anyone really make use of this feature?

Regardless of efficiency, I think it's a bad idea to start including
support for specific mail delivery formats in the shell.  It's one thing
to get the standard system information about a file and report that the
file has changed; it's quite another to have knowledge of the internals
of the file format.

Perhaps checkmailpath is a candidate for some kind of loadable module ...

-- 
Bart Schaefer                                 Brass Lantern Enterprises
http://www.well.com/user/barts              http://www.brasslantern.com

Re: MAIL, MAILPATH and maildir support

[Clint Adams]

> I was about to gripe about how horribly inefficient this bit of code is,
> when I realized I'd forgotten that the existing support for directories
> is if anything worse.  Recursively stat() an entire directory tree??
> Does anyone really make use of this feature?

Yes, people use this.

> Regardless of efficiency, I think it's a bad idea to start including
> support for specific mail delivery formats in the shell.  It's one thing
> to get the standard system information about a file and report that the
> file has changed; it's quite another to have knowledge of the internals
> of the file format.

Well, it's MAILPATH, not GENERICFILEPATH.  There are a limited number of
mailbox formats that pose significance to the shell.

> Perhaps checkmailpath is a candidate for some kind of loadable module ...

Fair enough.

Re: MAIL, MAILPATH and maildir support

[Oliver Kiddle]

Bart Schaefer wrote:
> 
> Perhaps checkmailpath is a candidate for some kind of loadable module ...

In my opinion, it would be useful to put all of the mail related
handling in a loadable module and then make that module quite powerful
(with lots of mush style commands etc). I think it would complement
stuff like the zftp module very well. I suggested writing such a module
about 2 years ago and got a mostly negative response. What does anyone
think?

Oliver Kiddle

Re: MAIL, MAILPATH and maildir support

[Peter Stephenson]

Oliver Kiddle wrote:
> Bart Schaefer wrote:
> > 
> > Perhaps checkmailpath is a candidate for some kind of loadable module ...
> 
> In my opinion, it would be useful to put all of the mail related
> handling in a loadable module and then make that module quite powerful
> (with lots of mush style commands etc). I think it would complement
> stuff like the zftp module very well. I suggested writing such a module
> about 2 years ago and got a mostly negative response. What does anyone
> think?

A module with more powerful mail commands is a reasonable idea, but I have
some reservations.  First, if this is going to be a largish add-on, basic
UNIX mail-file checking (as it currently exists) should probably stay in
the basic shell, since pretty much everybody (? I presume) uses that ---
hence exporting it just gets you all the stuff in the module and doesn't in
practice save anything.  Second, with the plethora of mail systems (the
maildir one is just one example), a general module for mail handling is a
big undertaking.  Third, anybody who wants to use individual commands
rather than a monolothic program to get at the mail is probably well
advised to use MH or some other existing system (the problem with mainting
a connection that made zftp useful as a module doesn't exist).
Nonetheless, if you've got some good ideas for what should go in such a
module --- or even some code --- it would be interesting to see them.

As for things like the patch that started all this: it looks to me like it
could be written efficiently enough as a shell function to be run from one
of the functions periodic or precmd.  You could even divide it into two
parts: first, shell code to replace the checkmailpath() functionality,
which is pretty simple (apart from recursive directory checking, but you
can use **/*(.mm???) to get regular files modified in the last ???  minutes
--- maybe (ms???) should work too, but it doesn't at the moment); second,
any ad-hoc functions to be called for special mail systems.  The only
difficulty with implementing mailstat() in that way is that looping over a
lot of files is slow in zsh functions, but I don't think you need to: if
you're just looking for the most recently modified/accessed file, *(om[1])
and *(oa[1]) are efficient (or will be when 3.1.6 appears), and if you want
the total size (do you? checkmailpath() simply looks to see if it's
non-zero),
    ls -l new/* cur/* | awk '{ tot += $5 } END { print tot }'
works (maybe $4 for BSD).  Such functions could certainly be distributed
with the source.

-- 
Peter Stephenson <pws@ibmth.df.unipi.it>       Tel: +39 050 844536
WWW:  http://www.ifh.de/~pws/
Dipartimento di Fisica, Via Buonarroti 2, 56127 Pisa, Italy

Re: MAIL, MAILPATH and maildir support

[Bart Schaefer]

On Jun 3,  8:02am, Clint Adams wrote:
} Subject: Re: MAIL, MAILPATH and maildir support
}
} > Recursively stat() an entire directory tree??
} > Does anyone really make use of this feature?
} 
} Yes, people use this.

Never with upwards-pointing symlinks, obviously.

} Well, it's MAILPATH, not GENERICFILEPATH.  There are a limited number of
} mailbox formats that pose significance to the shell.

I know of at least four that aren't simple files, and there's always
going to be yet another one.  I'd just rather not start down the road.

On Jun 3,  2:20pm, Oliver Kiddle wrote:
} Subject: Re: MAIL, MAILPATH and maildir support
}
} Bart Schaefer wrote:
} > 
} > Perhaps checkmailpath is a candidate for some kind of loadable module ...
} 
} In my opinion, it would be useful to put all of the mail related
} handling in a loadable module and then make that module quite powerful

On Jun 3,  4:08pm, Peter Stephenson wrote:
} Subject: Re: MAIL, MAILPATH and maildir support
}
} A module with more powerful mail commands is a reasonable idea, but I have
} some reservations.  First, if this is going to be a largish add-on, basic
} UNIX mail-file checking (as it currently exists) should probably stay in
} the basic shell, since pretty much everybody (? I presume) uses that ---
} hence exporting it just gets you all the stuff in the module and doesn't in
} practice save anything.  Second, with the plethora of mail systems (the
} maildir one is just one example), a general module for mail handling is a
} big undertaking.
 
One possibility is to make an IMAP module along the lines of the zftp
module.  Another is to make a "bridge module" that links zsh with, say,
c-client (the U. of Washington's mail client library, which is itself
extensible to new formats and already supports at least 10 plus three
revisions of IMAP and two revisions of POP -- but not maildir).

Neither of those address the issue of getting the appropriate part of
such a module invoked automatically when mailpath is checked.

} As for things like the patch that started all this: it looks to me like it
} could be written efficiently enough as a shell function to be run from one
} of the functions periodic or precmd.

The mail check is performed at approximately the same time as precmd or
periodic, so perhaps that's good enough.  However, maybe we should
consider having a special function name (like precmd and preexec) that,
if it's defined, is called in place of the built-in mail-check procedure.
That would provide the necessary hook to run a zmodload'd builtin for
mail checks (by calling the module from the special function).

Or we could make checkmailpath into a builtin command that happens to be
run every MAILCHECK seconds, so that a module could override it.  I can
say with a fair degree of confidence that previous zsh maintainers would
vote in favor of using the existing precmd/periodic mechanism, but the
problem with that is that you can't set it up "transparently" to use the
standard set of shell variables (because you can't -stop- zsh from doing
the check on its own if those variables are set).

-- 
Bart Schaefer                                 Brass Lantern Enterprises
http://www.well.com/user/barts              http://www.brasslantern.com

Re: MAIL, MAILPATH and maildir support

[Peter Stephenson]

"Bart Schaefer" wrote:
> However, maybe we should
> consider having a special function name (like precmd and preexec) that,
> if it's defined, is called in place of the built-in mail-check procedure.
> ...
> I can
> say with a fair degree of confidence that previous zsh maintainers would
> vote in favor of using the existing precmd/periodic mechanism, but the
> problem with that is that you can't set it up "transparently" to use the
> standard set of shell variables (because you can't -stop- zsh from doing
> the check on its own if those variables are set).

Yes, indeed, I was thinking about that.  It doesn't stop you using a
different set of variables and unsetting MAILCHECK, of course, but that's a
little crufty.  On the other hand, we could introduce a special syntax,
e.g.  MAILCHECK could be (say) 60?checkname where checkname was the
function to run instead of the standard code.  How does that sound?  I
don't see any compatibility problems --- or is there something that could
fail because MAILCHECK is a scalar rather than an integer?

If I implemented this, I would try and produce a function to reproduce the
current checkmailpath to make it easier for people to hack it.  I suspect
this would use the stat module, otherwise it's going to be messy, but maybe
there are some tricks with globs.

-- 
Peter Stephenson <pws@ibmth.df.unipi.it>       Tel: +39 050 844536
WWW:  http://www.ifh.de/~pws/
Dipartimento di Fisica, Via Buonarroti 2, 56127 Pisa, Italy

Re: MAIL, MAILPATH and maildir support

[Oliver Kiddle]

Peter Stephenson wrote:
> 
> A module with more powerful mail commands is a reasonable idea, but I have
> some reservations.  First, if this is going to be a largish add-on, basic
> UNIX mail-file checking (as it currently exists) should probably stay in
> the basic shell, since pretty much everybody (? I presume) uses that ---

Fair enough, having a huge module loaded for simple checking would be
wasteful. I don't actually use the mail-file checking myself though.

> big undertaking.  Third, anybody who wants to use individual commands
> rather than a monolothic program to get at the mail is probably well
> advised to use MH or some other existing system (the problem with mainting
> a connection that made zftp useful as a module doesn't exist).

I'll start by having a look at MH and see if that will serve my purposes
well. If not I'll have a go at writing something myself (unless someone
else beats me to it). I've got a lot of other things to do before so
don't expect to hear anything more about this from me in for quite a
long time.

> Nonetheless, if you've got some good ideas for what should go in such a
> module --- or even some code --- it would be interesting to see them.

My ideas are basically to first have most of the basic functionality of
mailx and mush: listing and displaying a message. In addition there
would be powerful commands for selecting messages (like the mush pick
command) and operating on dynamic folders.

Oliver Kiddle

Re: MAIL, MAILPATH and maildir support

[Bruce Stephens]

"Bart Schaefer" <schaefer@candle.brasslantern.com> writes:

> I was about to gripe about how horribly inefficient this bit of code
> is, when I realized I'd forgotten that the existing support for
> directories is if anything worse.  Recursively stat() an entire
> directory tree??  Does anyone really make use of this feature?

Well, in a limited way, yes.  I have procmail deliver to a number of
files in a specific directory, and zsh checks that directory.  I'm
sure I'm not that unusual (I think it's a reasonably popular way to
set up Gnus, for example).

Re: MAIL, MAILPATH and maildir support

[Sven Wischnowsky]

Peter Stephenson wrote:

> A module with more powerful mail commands is a reasonable idea, but I have
> some reservations.  First, if this is going to be a largish add-on, basic
> UNIX mail-file checking (as it currently exists) should probably stay in
> the basic shell, since pretty much everybody (? I presume) uses that

Err... ahem. I always feared that I'm not normal.

> --- maybe (ms???) should work too, but it doesn't at the moment);

Now that's a weird omission. Why didn't I... (computers were slower at 
that time, but...)


Bye
 Sven

--- os/glob.c	Tue Jun  1 11:28:26 1999
+++ Src/glob.c	Fri Jun  4 11:40:04 1999
@@ -97,6 +97,7 @@
 #define TT_MINS 2
 #define TT_WEEKS 3
 #define TT_MONTHS 4
+#define TT_SECONDS 5
 
 #define TT_BYTES 0
 #define TT_POSIX_BLOCKS 1
@@ -1615,6 +1616,8 @@
 				units = TT_WEEKS, ++s;
 			    else if (*s == 'M')
 				units = TT_MONTHS, ++s;
+			    else if (*s == 's')
+				units = TT_SECONDS, ++s;
 			}
 			/* See if it's greater than, equal to, or less than */
 			if ((range = *s == '+' ? 1 : *s == '-' ? -1 : 0))

--
Sven Wischnowsky                         wischnow@informatik.hu-berlin.de