[9fans] 'find'

[9fans] 'find'

[D. Brownlee]

	grep pattern `{du -a /}

leads to something like "grep: virtual memory allocation failure."
Sorry, but Plan 9 isn't running right now, so that's from memory.
That used to work in 3rd. edition. I found a better 'find' in
Russ' scripts tarball, but it is an interesting problem.

I suspect that the message may be generated on behalf of 'grep'.
I looked at the 'grep' source and the only potential problem (?)
seen was a failure to close a file before search() returns
to main(), if read() returns less than zero; however, the
print() associated with that error isn't seen in grep's
output before the vm failure occurs. I'll find out where
the message comes from next time Plan 9 is up.

Re: [9fans] 'find'

[Sape Mullender]

> 	grep pattern `{du -a /}
>
> leads to something like "grep: virtual memory allocation failure."

Because the argument list to grep gets impossibly long.

Re: [9fans] 'find'

[rog]

> > 	grep pattern `{du -a /}
> >
> > leads to something like "grep: virtual memory allocation failure."
>
> Because the argument list to grep gets impossibly long.

there's a little version of xargs that i tend to use in these cases,
that i've attached, despite having posted it before as it hadn't been
updated for 4e and changes to the libraries mean that it can be
smaller than before.

e.g.
	du -a / | sed 's/^[^ 	]*[ 	]//' | xargs grep pattern

(note that awk '{print $1}' won't necessarily work instead of
the above sed since filenames can contain spaces).

#include <u.h>
#include <libc.h>
#include <bio.h>

#define NUMARGS 470
#define MAXSIZE 20000

void
run(char **cmd)
{
	char buf[256];
	int npid, pid;

	pid = fork();
	if (pid == -1) {
		perror("couldn't fork");
		exits("no forkage");
	}
	if (pid == 0) {
		if (cmd[0][0] != '/' && strncmp(cmd[0], "./", 2)) {
			snprint(buf, sizeof(buf), "/bin/%s", cmd[0]);
			exec(buf, cmd);
		} else {
			exec(cmd[0], cmd);
		}
		perror(cmd[0]);
		exits("no exec");
	}
	while ((npid = waitpid()) != pid && npid != -1)
		;
}

void
main(int argc, char **argv)
{
	char **cmd;
	int i, n, m, size, eof;
	Biobuf stdin;

	if (argc < 2) {
		fprint(2, "Usage: xargs cmd [args...]\n");
		exits("usage");
	}
	if (Binit(&stdin, 0, OREAD) == -1) {
		fprint(2, "couldn't init stdin\n");
		exits("error");
	}
	cmd = malloc((argc - 1 + NUMARGS + 1) * sizeof(char *));
	n = 0;
	for (i = 1; i < argc; i++)
		cmd[n++] = argv[i];

	for (;;) {
		eof = 0;
		size = 0;
		for (m = 0; m < NUMARGS && size < MAXSIZE; m++) {
			if ((cmd[n + m] = Brdstr(&stdin, '\n', 1)) == 0) {
				eof = 1;
				break;
			}
			size += Blinelen(&stdin);
		}
		cmd[n + m] = 0;
		if (m > 0)
			run(cmd);
		for (i = 0; i < m; i++)
			free(cmd[n+i]);
		if (eof)
			break;
	}
}

Re: [9fans] 'find'

[Scott Schwartz]

| (note that awk '{print $1}' won't necessarily work instead of
| the above sed since filenames can contain spaces).

Apparently awk needs a way to split on quoted tokens, not just whitespace,
now.

Re: [9fans] 'find'

[rog]

> Apparently awk needs a way to split on quoted tokens, not just whitespace,
> now.

and i guess awk printf should be given a %q option
and maybe echo should have a -q option too
and du should print its output quoted
and pwd should too
and maybe cp should have an option to convert
spaces when copying from a filesystem that can have
spaces (e.g. ftpfs) to a filesystem that doesn't (e.g. the
plan 9 fileserver)
and other things too... i'm sure.

Re: [9fans] 'find'

[Douglas A. Gwyn]

rog@vitanuova.com wrote:
> and i guess awk printf should be given a %q option
> and maybe echo should have a -q option too
> and du should print its output quoted
> and pwd should too
> and maybe cp should have an option ...

Does this suggest why quoting (prompted by a limitation
at the "shell" level) was the wrong idea?

Re: [9fans] 'find'

[Fco.J.Ballesteros, nemo]

[-- Attachment #1: Type: text/plain, Size: 322 bytes --]

Dont know if it was good or bad.
What I know is that I ended up using trfs
because in the end 'this stuff' didnt work
as expected (think of acme and others).

Sure, it's not perfect, but I need no extra
options, and all the programs work as
expected (while at the same time they remain
using blanks all the times).

[-- Attachment #2: Type: message/rfc822, Size: 2260 bytes --]

From: "Douglas A. Gwyn" <DAGwyn@null.net>
To: 9fans@cse.psu.edu
Subject: Re: [9fans] 'find'
Date: Thu, 3 Jul 2003 09:42:20 GMT
Message-ID: <3F00600D.4060005@null.net>

rog@vitanuova.com wrote:
> and i guess awk printf should be given a %q option
> and maybe echo should have a -q option too
> and du should print its output quoted
> and pwd should too
> and maybe cp should have an option ...

Does this suggest why quoting (prompted by a limitation
at the "shell" level) was the wrong idea?

Re: [9fans] 'find'

[rob pike, esq.]

> Does this suggest why quoting (prompted by a limitation
> at the "shell" level) was the wrong idea?

it may have been wrong to do anything at the shell level,
but it was put in to clean up kernel interfaces and there i
think it was a success.  with textual command syntax,
you need some help.

Re: [9fans] 'find'

[matt]

Douglas A. Gwyn wrote:

> rog@vitanuova.com wrote:
>
>> and i guess awk printf should be given a %q option
>> and maybe echo should have a -q option too
>> and du should print its output quoted
>> and pwd should too
>> and maybe cp should have an option ...
>
>
> Does this suggest why quoting (prompted by a limitation
> at the "shell" level) was the wrong idea?

maybe we should have stuck with " as the delimeter

"don't you mean \" "

"that's what I said "" "

"oh sorry I thought you said " "

"%22, are you mad?"

"what's wrong with Igo= ?"

'gah, I''ve had enough of this. I''m going'

Re: [9fans] 'find'

[Douglas A. Gwyn]

matt wrote:
> maybe we should have stuck with " as the delimeter

I think delimiters should be chosen from the set of
characters that one is willing to "live without" in
the delimited components.  That includes control
characters but not space, ", or /.  Control characters
can be displayed with some reasonable graphic and of
course are snarfable etc.

Re: [9fans] 'find'

[matt]

Douglas A. Gwyn wrote:

> matt wrote:
>
>> maybe we should have stuck with " as the delimeter
>
>
> I think delimiters should be chosen from the set of
> characters that one is willing to "live without" in
> the delimited components.  That includes control
> characters but not space, ", or /.  Control characters
> can be displayed with some reasonable graphic and of
> course are snarfable etc.
>
>
yes, I wasn't being serious re "

It has been mentioned before so I didn't think it worth repeating that a
non-human usable glyph is desirable as a universal field delimeter

My whimsy was to demonstrated the problem of writing about the glyph
without using the glyph so that any software the processes my emails
doesn't get confused by the glyph I can't mention.

Re: [9fans] 'find'

[Dan Cross]

Scott Schwartz <schwartz@bio.cse.psu.edu> writes:
>
> | (note that awk '{print $1}' won't necessarily work instead of
> | the above sed since filenames can contain spaces).
>
> Apparently awk needs a way to split on quoted tokens, not just whitespace,
> now.

One wonders if FS can't be set appropriately by negating the things
that would make for valid tokens.  Even if possible, it'd be an ugly
ERE.

	- Dan C.

Re: [9fans] 'find'

[Kenji Arisawa]

Hello,

I have "repeat".

usage example:
du | repeat -X 'ls -l $2'

Kenji Arisawa


#!/bin/rc
# coded by Kenar
# Kenar: arisawa@aichi-u.ac.jp
#
usage='
repeat: repeat commands in the argument reading data from input.
usage: repeat [-X] ''COMMANDLINE''
option: "-X" enables COMMANDLINE to be executed.
  If this option is absent, COMMANDLINE is only echoed.
'
xopt=0
br=0
while (~  $1 -* ){
	switch($1){
	case	-X
		xopt=1
	case	-FS
		fs=$2
	case	-*
		echo $usage
		exit usage
	}
	shift
}
if(! ~ $#* 1) {
	echo $usage
	exit usage
}

cmd=$"*
# Read fields from stdin
while(*=`{read}){
	# echo TEST $#* $*
	switch($xopt){
	case 1
		eval $cmd
	case 0
		eval echo $cmd
	}
}

Re: [9fans] 'find'

[Jack Johnson]

On Sun, 2003-06-29 at 09:57, D. Brownlee wrote:
> 	grep pattern `{du -a /}

This is a general scripting question.

I've been berated for my admittedly poor command usage, namely for
excessive usage of the "useless cat" and similar techniques:

	cat foo | grep pattern

namely because when I change my mind I've always found it easier to
change the end of a line rather than some arbitrary section of the
middle (whether using some other shell's history or sweeping with the
mouse).

Is anyone else prone to this problem, or do you have some other
technique that satisfies your own personal laziness or ineptitude?

-Jack

Re: [9fans] 'find'

[Dan Cross]

> On Sun, 2003-06-29 at 09:57, D. Brownlee wrote:
> > 	grep pattern `{du -a /}
>
> This is a general scripting question.
>
> I've been berated for my admittedly poor command usage, namely for
> excessive usage of the "useless cat" and similar techniques:
>
> 	cat foo | grep pattern

There are times when the ``useless cat'' is actually quite useful.  For
instance, when one wants to pipe more than one file to tr and the files
are passed as command line arguments.  Doing the logic to figure out
whether you need cat or not would be more expensive than just using cat
in all cases.

> namely because when I change my mind I've always found it easier to
> change the end of a line rather than some arbitrary section of the
> middle (whether using some other shell's history or sweeping with the
> mouse).

I do this on the command line all the time.  In scripts, not so
frequently.

> Is anyone else prone to this problem, or do you have some other
> technique that satisfies your own personal laziness or ineptitude?

Usually, I just give up and go get a cup of coffee or something
instead.

But seriously, complaints about the useless cat are often overblown,
and not using cat often makes the resulting script really ugly.  This
isn't 1985; I'll pick a cleaner more maintainable script over saving
a process.

	- Dan C.

Re: [9fans] 'find'

[Ralph Corderoy]

Hi Jack,

>     cat foo | grep pattern
>
> namely because when I change my mind I've always found it easier to
> change the end of a line rather than some arbitrary section of the
> middle

Take your pick.

    grep pattern <foo
    grep <foo pattern
    <foo grep pattern

Cheers,

--
Ralph Corderoy.      http://inputplus.co.uk/ralph/     http://troff.org/

Re: [9fans] 'find'

[Dan Cross]

> Take your pick.
>
>     grep pattern <foo
>     grep <foo pattern
>     <foo grep pattern

err, this *does* beg the question; what's wrong with `grep pattern foo'?

	- Dan C.

Re: [9fans] 'find'

[boyd, rounin]

> err, this *does* beg the question; what's wrong with `grep pattern foo'?

yes

Re: [9fans] 'find'

[Douglas A. Gwyn]

boyd, rounin wrote:
>>err, this *does* beg the question; what's wrong with `grep pattern foo'?
> yes

What's wrong with grep pattern foo is that it acts differently
from grep pattern foo1 foo2.  Whether or not the filename is
displayed as a prefix should be controlled by a command option,
not by how many file arguments there are.

Re: [9fans] 'find'

[boyd, rounin]

> What's wrong with grep pattern foo is that it acts differently
> from grep pattern foo1 foo2.  Whether or not the filename is
> displayed as a prefix should be controlled by a command option,
> not by how many file arguments there are.

perhaps there should be an option, but i think the way it works
at the moment is more useful.  of course, if run from xargs:

    ... | xargs grep foo /dev/null

does the same and adds no extra code to grep.

but you [doug] knew that anyway.

Re: [9fans] 'find'

[Douglas A. Gwyn]

"boyd, rounin" wrote:
>     ... | xargs grep foo /dev/null
> but you [doug] knew that anyway.

Yeah, in fact I have to do that a lot.
Clearly this is a kludge.  Why should /dev/null have to get involved?

Re: [9fans] 'find'

[rob pike, esq.]

> What's wrong with grep pattern foo is that it acts differently
> from grep pattern foo1 foo2.  Whether or not the filename is
> displayed as a prefix should be controlled by a command option,
> not by how many file arguments there are.

i tend to agree.  nowadays i use bald 'grep' only on stdin and use a
wrapper shell script to look for things in the file system.  the
script adds the -n option and a /dev/null on the end to force file
names.

but that's after years (hell, decades) of use.  i also agree that the
original design was brilliant.  in an age when command lines were the
only user interface, grep's behavior was a model of thinking of how
the user would use the tool.  i don't think most users even noticed
the behavior for a long time after they became adept with it.

today, you'd never write grep the same way.  it would be more regular,
cleaner, probably noisier, and certainly less user-friendly.

-rob

Re: [9fans] 'find'

[Dan Cross]

> today, you'd never write grep the same way.  it would be more regular,
> cleaner, probably noisier, and certainly less user-friendly.

That scares me.  I don't wish to become a slave to the user interface.
Even though I already am, I'd like to spare my next life if I can.

	- Dan C.

Re: [9fans] 'find'

[David Presotto]

[-- Attachment #1: Type: text/plain, Size: 50 bytes --]

When are you adding 'browse' as an option to grep?

[-- Attachment #2: Type: message/rfc822, Size: 2360 bytes --]

From: Dan Cross <cross@math.psu.edu>
To: 9fans@cse.psu.edu
Subject: Re: [9fans] 'find'
Date: Mon, 07 Jul 2003 15:18:21 -0400
Message-ID: <200307071918.h67JIL715585@augusta.math.psu.edu>

> today, you'd never write grep the same way.  it would be more regular,
> cleaner, probably noisier, and certainly less user-friendly.

That scares me.  I don't wish to become a slave to the user interface.
Even though I already am, I'd like to spare my next life if I can.

	- Dan C.

Re: [9fans] 'find'

[Dan Cross]

> When are you adding 'browse' as an option to grep?

Right after I add support for GNU-style long options and build
in a LISP interpreter so I can turn it into an emacs module.

	- Dan C.

Re: [9fans] 'find'

[Dan Cross]

> >>err, this *does* beg the question; what's wrong with `grep pattern foo'?
> > yes
>
> What's wrong with grep pattern foo is that it acts differently
> from grep pattern foo1 foo2.  Whether or not the filename is
> displayed as a prefix should be controlled by a command option,
> not by how many file arguments there are.

grep pattern <file* doesn't work.  cat file* | grep pattern does.
My question originally referred to replacing indirection, not with
worrying about multiple arguments.

	- Dan C.

Re: [9fans] 'find'

[Dan Cross]

> grep pattern <file* doesn't work.  cat file* | grep pattern does.

Just a note lest some pedant jump at me from behind the shadows while
I'm otherwise preoccupied.  I'm assuming, of course, that file* matches
more than one file in the above examples.

	- Dan C.

Re: [9fans] 'find'

[Douglas A. Gwyn]

Dan Cross wrote:
> > grep pattern <file* doesn't work.  cat file* | grep pattern does.
> Just a note lest some pedant jump at me from behind the shadows while
> I'm otherwise preoccupied.  I'm assuming, of course, that file* matches
> more than one file in the above examples.

Actually in most Bourne shells the first file* is never expanded.

Re: [9fans] 'find'

[Dan Cross]

> Actually in most Bourne shells the first file* is never expanded.

Like I said, it doesn't work.

	- Dan C.

Re: [9fans] 'find'

[Jack Johnson]

On Sat, 2003-07-05 at 11:38, Dan Cross wrote:
> > Take your pick.
> >
> >     grep pattern <foo
> >     grep <foo pattern
> >     <foo grep pattern
>
> err, this *does* beg the question; what's wrong with `grep pattern foo'?

My original question was about prefering to edit the end of the line
when you change your mind (or make a mistake) in pattern.

So, for me, the last two make for a good alternative to the "useless
cat".

-Jack

Re: [9fans] 'find'

[matt]

>
>
>>err, this *does* beg the question; what's wrong with `grep pattern foo'?
>>

Sorry to be pedantic it would raise the question not beg it.

Begging the question is another name for a circular argument

A is true because B is true, and B is true because A is true.

Besides which, it doesn't do either  8)

Re: [9fans] 'find'

[Ralph Corderoy]

Hi Dan,

> > Take your pick.
> >
> >     grep pattern <foo
> >     grep <foo pattern
> >     <foo grep pattern
>
> err, this *does* beg the question; what's wrong with `grep pattern
> foo'?

Read my post again, including the part I quoted and replied to.  The OP
wants the pattern to be at the end of the line for easy editing.

Cheers,

--
Ralph Corderoy.      http://inputplus.co.uk/ralph/     http://troff.org/

Re: [9fans] 'find'

[Dan Cross]

> > >     <foo grep pattern
> >
> > err, this *does* beg the question; what's wrong with `grep pattern
> > foo'?
>
> Read my post again, including the part I quoted and replied to.  The OP
> wants the pattern to be at the end of the line for easy editing.

Aha!  I should pay more attention to what's going on.

	- Dan C.

Re: [9fans] 'find'

[Fco.J.Ballesteros, nemo]

> 	grep pattern `{du -a /}
>
> leads to something like "grep: virtual memory allocation failure."

For that case you can use
	walk / 'grep pattern $f'

PS: walk is a script sent before in the list.

Re: [9fans] 'find'

[boyd, rounin]

find was always a horrible thing; the syntax was dreadful and it was around
in the the days when you didn't have the cpu/disk speed to use it except
in very limited cases or root stuff run out of cron [being very careful].

on lunix i'll do a "find / ..." 'cos it won't take that long and there are so
many damn files that cd and ls just won't cut it.

i'm glad it's gone.

Re: [9fans] 'find'

[Dan Cross]

[-- Attachment #1: Type: text/plain, Size: 356 bytes --]

> grep pattern `{du -a /}

Hmm, this is exactly what I wrote walk and sor for; check the
archives if you're interested.  Actually, I've updated them for
4th Edition, so I'll include them here (I have been meaning to
add something that ignores duplicates in walk, though).  The
archives still have a nice cookbook for how to use them.

	- Dan C.


[-- Attachment #2: Type: text/plain, Size: 193 bytes --]

#!/bin/rc
rfork e
fn runtests {
	file=$1; shift
	while (! ~ $#* 0 && ! eval $1 ''''^$file^'''')
		shift
	if (! ~ $#* 0)
		echo $file
}
while (file = `{read}) {
	runtests $file $*
}

[-- Attachment #3: Type: text/plain, Size: 2336 bytes --]

/*
 *  Walk a directory tree, in the style of du(1),
 *  but with some additional flourishes.
 *
 *  Dan Cross <cross@math.psu.edu>
 */

#include <u.h>
#include <libc.h>

static int	mkdepth(int);
static char	*mkname(char *, int *, char *, char *);
static void	walk(char *, int, int);
static void	walkname(char *, int, int);

char	*fmt;

void
main(int argc, char *argv[])
{
	char	*dir;
	int	depth;
	Dir	*d;

	dir = ".";
	fmt = "%s\n";
	depth = -1;
	ARGBEGIN {
	case 'd':
		depth = atoi(ARGF());
		break;
	case 'q':
		quotefmtinstall();
		doquote = needsrcquote;
		fmt = "%q\n";
		break;
	}ARGEND
	if (argc == 0)
		walkname(".", depth, 1);
	else {
		for (dir = *argv; dir; dir = *++argv) {
			if ((d = dirstat(dir)) == nil) {
				fprint(2, "dirstat %s: %r\n", dir);
				continue;
			}
			walkname(dir, depth, d->mode & DMDIR);
			free(d);
		}
	}

	exits(0);
}

static void
walkname(char *dirname, int depth, int isdir)
{
	int	fd;

	if (strcmp(dirname, ".") != 0 && strcmp(dirname, "..") != 0)
		print(fmt, dirname);
	if (isdir) {
		fd = open(dirname, OREAD);
		if (fd < 0) {
			fprint(2, "open %s: %r\n", dirname);
			return;
		}
		walk(dirname, fd, depth);
		close(fd);
	}
}

static char *
mkname(char *name, int *l, char *basename, char *filename)
{
	char	*nname;
	int	t;

	t = strlen(basename) + 1 + strlen(filename) + 1;
	if (*l == 0 || name == nil) {
		*l = t;
		name = malloc(t);
		if (name == nil)
			sysfatal("malloc %d: %r\n", l);
	} else if (*l < t) {
		nname = realloc(name, t);
		if (nname == nil) {
			free(name);
			sysfatal("malloc %d: %r\n", l);
		}
		*l = t;
		name = nname;
	}
	snprint(name, t, "%s/%s", basename, filename);
	cleanname(name);

	return(name);
}

static int
mkdepth(int depth)
{

	return((depth == -1) ? depth : depth - 1);
}

static void
walk(char *dirname, int fd, int depth)
{
	Dir	*dir, *dp;
	char	*name, *nname;
	int	i, l, n, t;

	if (depth == 0)
		return;
	l = 0;
	name = nil;
	n = dirreadall(fd, &dir);
	for (dp = dir, i = 0; i < n; dp++, i++) {
		if (strcmp(dp->name, ".") == 0 || strcmp(dp->name, "..") == 0)
			continue;
		name = mkname(name, &l, dirname, dp->name);
		walkname(name, mkdepth(depth), dp->mode & DMDIR);
	}
	free(dir);
	if (name != nil)
		free(name);
}