Re: [9fans] find(1) revisited.

Re: [9fans] find(1) revisited.

[a]

this is great, dan. i need to play with it a bit more, but it
looks really nice. got any suggestion for using sor to find
files older/newer than, or with a modification/access time
newer/older than a certain date? that and file ownership is
probably what i used find for most.
ア

Re: [9fans] find(1) revisited.

[Fco.J.Ballesteros]

This happen before, but just to help, this is what i use.
It's just du, wrapped to operate on your files.

For example, to walk /usr/nemo to find files with execute permission:

; walk /usr/nemo 'test -x $f && echo $f'

you can pass any command and use $f where you want the file name.

#!/bin/rc

rfork e

if (~ $#* 0) {
	echo 'usage: walk file [cmd...]' >[1=2]
	exit usage
}

file=$1
shift
if (~ $#* 0)
	cmd='echo $f'
if not
	cmd=$*
cd `{basename -d $file}
exec du -a $file | awk '{print $2}' | while (f=`{read}) { eval $cmd  }

Re: [9fans] find(1) revisited.

[Dan Cross]

Well, now that there's a perl port out there....  :-)

	- Dan C.

Re: [9fans] find(1) revisited.

[Russ Cox]

use /386/bin/mtime (just pushed out to sources)
to get the mtime as a decimal, and then do whatever
you want.

Re: [9fans] find(1) revisited.

[Boyd Roberts]

Been there, done that :-)

Shame you have to write walk in C, but at least it's small.

Re: [9fans] find(1) revisited.

[Boyd Roberts]

Geoff Collyer wrote:

>I'm just curious: what sorts of things did you do with find that you
>find impossible or too awkward to do with du?
>
I guess, in the old days, find made it quicker, but now we don't care,
'cos du | grep is pretty quick.

Re: [9fans] find(1) revisited.

[Dan Cross]

It was the longer permutations; mainly what I missed was finding
files owned by certain people (for instance, what do I own in
/sys/src?).  Looking for files with certain permissions is also
useful (what's world writable?).  That sort of stuff.  Also, I
kind of wanted to be able to limit the depth of a search, and du
seemed inappropriate for that.

	- Dan C.

Re: [9fans] find(1) revisited.

[Geoff Collyer]

I'm just curious: what sorts of things did you do with find that you
find impossible or too awkward to do with du?  The main things I can
recall using find for were finding set-id files (not an issue on Plan
9), finding core files (ditto), or finding files older than some date
(I use "du -ta" for that).  I haven't missed find.

Re: [9fans] find(1) revisited.

[Dan Cross]

Oh!  I forgot to say how to use these.  Sorry, my bad.  Basically,
compile walk and put it and sor in your bin directories.  Walk takes
``-d n'' to limit the depth of the walk to the nth level in the target
directory (walk -d 0 prints nothing), and ``-q'' to quote the output in
the style of ls(1).  These can of course be used in concert.  For
example, `walk -d 3 -q /dev' prints out everything up to 3 levels deep
in /dev, quoting everything that needs to be.  `walk -d 3 -q /dev /env'
might be a better demonstration, as I think it's a rare thing indeed
that something in /dev needs to be quoted.

Sor works sort of like grep, except that you pass it tests.  For
instance, `walk -d 3 -q /dev /env | sor 'test -d' '' will print out
the names of all the directories up to depth 3 in /dev and /env.
Multiple tests can be put on the same line, effectively creating a
logical `or' operator.  For example,

	walk -d 3 -q /sys/lib | sor 'test -d' 'test -x'

(interestingly, this shows /sys/lib/ghostscript/font/mkfile is
executable on my system; how odd.)

Of course, the usual combinations with grep, sed, etc, will work.
A few Unix find idioms and their translations are:

unix% find $home -name '*foo*' -print
term% walk $home | grep foo

unix% find $home -name '*foo*' -type d -print
term% walk $home | grep foo | sor 'test -d'

unix% find $home '(' -name '*foo*' -o -name '*bar*' ')' -type d -print
term% walk $home | grep '(foo|bar)' | sor 'test -d'

unix% find $home/* $home/*/* -prune '(' -name '*foo*' -o -name '*bar*' ')' -type d -print
term% walk -d 2 $home | grep '(foo|bar)' | sor 'test -d'

unix% find $home -name foo.c -exec ls '{}' ';'
term% ls `{walk $home | grep '^foo.c$'}

Incidentally, sor internally sets the environment variable ``file'' to
the name of the current file being examined.  This can be useful for
some tests.  For example:

term% walk -d 1 | sor '/sys/lib/texmf/bin/386/newer $file'

Which is basically a goofy way of printing the names of files in
the current directory.  Or the perhaps more useful:

term% walk -d 1 | sor 'grep README $file > /dev/null'

Which is a goofy way of doing: grep -l README `{walk -d 1}

Anyway, further examples exist, but already the combination of walk and
sor is just as expressive, but a lot shorter and simpler than find, et
al.  Also, it's easy to write other programs that perform tests similar
to the rest of the find predicates (I have some simple scripts, for
instance, that pull the size and owner/group of a file out of ls -l's
output).

	- Dan C.

[9fans] find(1) revisited.

[Dan Cross]

[-- Attachment #1: Type: text/plain, Size: 2914 bytes --]

So find sucks, but is inarguably useful, and I've found myself missing
it before.  What would be a better solution, I've wondered?  Here's a
proposal for one I think fits in much nicer.

Find basically does the following job:  Walk a set of specified
directory trees, apply a set of predicates to the contents, and perform
some action if the result is true.  However, find could be thought of
as a pipeline of filters that operate on a stream of filenames;
discarding some that don't find some criteria, and printing the rest to
the next stage.  The final stage of the pipeline performs some action,
such as printing the file names or making them arguments to some
command.

So, I wrote two commands: walk, and sor.  Walk walks over a directory
tree, printing it's contents, possibly with quoting in the style of the
new "%q" format (using ls's ``lsquote'' routine), and possibly with a
limitation on the depth to which it'll descend.  Sor (Stream OR, get
it?) is an rc script that reads a set of filenames from it's input, and
applies a set of tests to them, echoing those names that pass a test,
discarding the rest.  The effect is that one can now create arbitrary
pipelines that mimic what find does under Unix, only are much more
flexible.  It isn't the fastest thing in the world, but it works, and
seems to work pretty well.

Now, I think sor is genuinely useful, but why walk when we have du -a?
Two reasons.  (1) du is the program to collect disk usage statistics,
not the program to do general walks of a file tree.  As such, it misses
a feature to limit depth, and contains logic to make sure it doesn't
``count'' a file twice.  That's useful for du, but not really for a
general file walker; what if I want to see all names of a file?  What
if I want to look for multiple names with the same qid?  It certainly
makes no sense to augment du with that functionality: du is the disk
usage summarizer, not the general file tree walker.  Also, not being
able to limit the depth of one's search can be annoying.  What if I
want to look for a file, but only in the first three levels of a
hierarchy?  It can be argued that it might be useful to summarize the
disk usage information of everything up to n levels in a hierarchy, but
I'd argue that that's not generally useful enough to warrant a change
to du.  Besides, what if I wanted to find all the names of a given file
within n levels in a hierarchy?  It could be argued that du could be
used along with some clever sed script to achieve the effect of depth
limiting, but we still loose the ability to see a file more than once.
Hence walk.

Anyway, I append both here in case others find them useful.  It'd be
nice to see them go into the distrubition if there's enough general
appeal (and if they don't suck too badly).  In any event, some feedback
on the idea and the tools would be nice.

	- Dan C.


[-- Attachment #2: Type: text/plain, Size: 2515 bytes --]

/*
 *  Walk a directory tree, in the style of du(1),
 *  but with some additional flourishes.
 *
 *  Dan Cross <cross@math.psu.edu>
 */

#include <u.h>
#include <libc.h>

static int	mkdepth(int);
static char	*mkname(char *, int *, char *, char *);
static void	walk(char *, int, int);
static void	walkname(char *, int, int);
static int	walkquote(int c);

char	*fmt;

void
main(int argc, char *argv[])
{
	char	*dir;
	int	depth;
	Dir	*d;

	dir = ".";
	fmt = "%s\n";
	depth = -1;
	ARGBEGIN {
	case 'd':
		depth = atoi(ARGF());
		break;
	case 'q':
		quotefmtinstall();
		doquote = walkquote;
		fmt = "%q\n";
		break;
	}ARGEND
	if (argc == 0)
		walkname(".", depth, 1);
	else {
		for (dir = *argv; dir; dir = *++argv) {
			if ((d = dirstat(dir)) == nil) {
				fprint(2, "dirstat %s: %r\n", dir);
				continue;
			}
			walkname(dir, depth, d->mode & DMDIR);
			free(d);
		}
	}

	exits(0);
}

/*  Cribbed from ls(1) source.  */
static int
walkquote(int c)
{

	if (c <= ' ' || strchr("`^#*[]=|\?${}()'", c))
		return(1);
	return(0);
}

static void
walkname(char *dirname, int depth, int isdir)
{
	int	fd;

	if (strcmp(dirname, ".") != 0 && strcmp(dirname, "..") != 0)
		print(fmt, dirname);
	if (isdir) {
		fd = open(dirname, OREAD);
		if (fd < 0) {
			fprint(2, "open %s: %r\n", dirname);
			return;
		}
		walk(dirname, fd, depth);
		close(fd);
	}
}

static char *
mkname(char *name, int *l, char *basename, char *filename)
{
	char	*nname;
	int	t;

	t = strlen(basename) + 1 + strlen(filename) + 1;
	if (*l == 0 || name == nil) {
		*l = t;
		name = malloc(t);
		if (name == nil)
			sysfatal("malloc %d: %r\n", l);
	} else if (*l < t) {
		nname = realloc(name, t);
		if (nname == nil) {
			free(name);
			sysfatal("malloc %d: %r\n", l);
		}
		*l = t;
		name = nname;
	}
	snprint(name, t, "%s/%s", basename, filename);
	cleanname(name);

	return(name);
}

static int
mkdepth(int depth)
{

	return((depth == -1) ? depth : depth - 1);
}

static void
walk(char *dirname, int fd, int depth)
{
	Dir	*dir, *dp;
	char	*name, *nname;
	int	i, l, n, t;

	if (depth == 0)
		return;
	l = 0;
	name = nil;
	n = dirreadall(fd, &dir);
	for (dp = dir, i = 0; i < n; dp++, i++) {
		if (strcmp(dp->name, ".") == 0 || strcmp(dp->name, "..") == 0)
			continue;
		name = mkname(name, &l, dirname, dp->name);
		walkname(name, mkdepth(depth), dp->mode & DMDIR);
	}
	free(dir);
	if (name != nil)
		free(name);
}

[-- Attachment #3: Type: text/plain, Size: 193 bytes --]

#!/bin/rc
rfork e
fn runtests {
	file=$1; shift
	while (! ~ $#* 0 && ! eval $1 ''''^$file^'''')
		shift
	if (! ~ $#* 0)
		echo $file
}
while (file = `{read}) {
	runtests $file $*
}