how to?

how to?

[Scott Lipcon]

Here's something I figure is possible in zsh, but I don't know how -
I'd like to be able to do an ls in a directory of source code while
its being built, and basically say: show me all the .c files for which
there doesn't exist a .o file.   Right now I'm using:

ls -1 *.[co] | cut -f 1 -d"." | uniq -c | grep 1

but there has to be a zsh way to do that without 3 pipes.  Any ideas?

Scott

Re: how to?

[Phil Pennock]

[-- Attachment #1: Type: text/plain, Size: 617 bytes --]

On 2002-08-23 at 09:35 -0400, Scott Lipcon wrote:
> Here's something I figure is possible in zsh, but I don't know how -
> I'd like to be able to do an ls in a directory of source code while
> its being built, and basically say: show me all the .c files for which
> there doesn't exist a .o file.   Right now I'm using:
> 
> ls -1 *.[co] | cut -f 1 -d"." | uniq -c | grep 1
> 
> but there has to be a zsh way to do that without 3 pipes.  Any ideas?

print -l -- *.c(e:[[ ! -f '${REPLY%.c}.o' ]]:)

See zshexpn(1) and "Glob Qualifiers" therein.
-- 
Whatever kind of look you were going for, you missed.

[-- Attachment #2: Type: application/pgp-signature, Size: 187 bytes --]

Re: how to?

[Bart Schaefer]

On Aug 23,  9:35am, Scott Lipcon wrote:
}
} I'd like to be able to do an ls in a directory of source code while
} its being built, and basically say: show me all the .c files for which
} there doesn't exist a .o file.

c=(*.c) o=(*.o(N)) eval 'ls ${${c:#(${~${(j:|:)${o:r}}}).c}:?done}'

-- 
Bart Schaefer                                 Brass Lantern Enterprises
http://www.well.com/user/barts              http://www.brasslantern.com

Zsh: http://www.zsh.org | PHPerl Project: http://phperl.sourceforge.net   

Re: how to?

[Bart Schaefer]

On Aug 23,  8:22pm, Phil Pennock wrote:
} 
} On 2002-08-23 at 15:59 +0000, Bart Schaefer wrote:
} > c=(*.c) o=(*.o(N)) eval 'ls ${${c:#(${~${(j:|:)${o:r}}}).c}:?done}'
} 
} (for reference, mine was:
}  print -l -- *.c(e:[[ ! -f '${REPLY%.c}.o' ]]:)
} )
} 
} One thing that I try to avoid is building up the first list in an array.
} I keep hitting machine limits on argv, so I tend to be cautious about
} _anything_ which builds lists

Your caution is somewhat misplaced.  The glob isn't enumerated anywhere,
except internally to zsh when processing the assignment; machine limits on
argv will only come into play when an external command is involved (e.g.,
there might be a problem with the "ls ${c}" in my solution, but the spec
said "do an ls" not "do a print -l", and your solution passes exactly the
same number of arguments to "print -l --" as mine passes to "ls").

In other words, you might have reason to be cautious about anything that
*expands* a list, but just building one (i.e., array assignment) should
not be an issue unless you're hitting stacksize or memoryuse limits.

The size of the environment also has an effect -- you might try exporting
as little as possible, if you frequently hit argv limits.

} Out of curiosity: the (e::) thing builds the parse tree beforehand and
} is pretty efficient, isn't it?  I've never had cause to try it on huge
} sets of data ...

I timed your solution and mine using repeated runs on about 400 files
(after changing mine to also use the "print" builtin) and they're almost
exactly the same.  Yours uses a little more system time, mine a little
more user time (file tests vs. string manipulation, I suppose).

} Which leads to a question: how much hassle is it to have a glob modifier
} be able to duplicate the Simple Command which is calling it?

A glob modifier, just about impossible.  A precommand modifier or option,
perhaps.  The problem is, by the time the E2BIG error comes back from
execve(2), it's too late to do much except croak -- so zsh would need a
heuristic to predict whether/how to split up the arguments, so it could
be done sooner.

-- 
Bart Schaefer                                 Brass Lantern Enterprises
http://www.well.com/user/barts              http://www.brasslantern.com

Zsh: http://www.zsh.org | PHPerl Project: http://phperl.sourceforge.net   

Re: how to?

[Phil Pennock]

[-- Attachment #1: Type: text/plain, Size: 3465 bytes --]

On 2002-08-24 at 03:47 +0000, Bart Schaefer wrote:
> On Aug 23,  8:22pm, Phil Pennock wrote:

Most of my mail was curiosity.  I'm not too sure of most of this, hence
I didn't CC the list but instead made it a private reply.  Sorry for any
offense caused -- I see that I didn't tone my wording well.

> there might be a problem with the "ls ${c}" in my solution, but the spec
> said "do an ls" not "do a print -l", and your solution passes exactly the
> same number of arguments to "print -l --" as mine passes to "ls").

True, but it was "ls -1" so I added another optimisation, to avoid the
extra stat()s.

> In other words, you might have reason to be cautious about anything that
> *expands* a list, but just building one (i.e., array assignment) should
> not be an issue unless you're hitting stacksize or memoryuse limits.

Noted.  I still try to avoid risking hitting stacksize limits, though.

Influence of years ago doing Comp Sci work on an Amiga, when others used
Unix.  I tend to try to explicitly free memory that I allocate.  I try
to avoid techniques which chew a lot of stack or other RAM.  I might do
something quick and dirty, but will try to fix it before it goes into a
script.

> The size of the environment also has an effect -- you might try exporting
> as little as possible, if you frequently hit argv limits.

Ah, it's usually not envp subtracting from argv, but  more a case of
having my mind tracking a few things, finding no tags file for
/usr/src/sys/ on a BSD system and trying a grep for something across all
the kernel source, with **/*.[chyl] -- the failure leads to my blinking,
grumbling, writing a find/xargs version and then trying to remember what
I was looking at before.

Also dealing with mailboxes which have been forged as the envelope
sender in spam.  60,000 mails, two files each.  I wrote tools to handle
it more efficiently in Perl, since the regular easy shell stuff kept
barfing.

> I timed your solution and mine using repeated runs on about 400 files
> (after changing mine to also use the "print" builtin) and they're almost
> exactly the same.  Yours uses a little more system time, mine a little
> more user time (file tests vs. string manipulation, I suppose).

*nods*  Thanks.  Sorry, I was asking more if you knew of the top of your
head; I should have said so, to avoid you running tests which I was too
lazy to try.

> } Which leads to a question: how much hassle is it to have a glob modifier
> } be able to duplicate the Simple Command which is calling it?
> 
> A glob modifier, just about impossible.  A precommand modifier or option,
> perhaps.  The problem is, by the time the E2BIG error comes back from
> execve(2), it's too late to do much except croak -- so zsh would need a
> heuristic to predict whether/how to split up the arguments, so it could
> be done sooner.

Are there sufficient hooks to allow this to be done as a module?  I have
enough interest in this to actually go back to looking at zsh internals
and writing a module.  sysconf(_SC_ARG_MAX) should return the space
available.  More overhead in tracking the sizes of all strings
explicitly and summing them before a command, but then that can be
restricted to just the case when a glob is used, so it wouldn't normally
slow things.

But, uhm, not for a month or so.
-- 
"Markets can remain irrational longer than you can remain solvent"
 -- John Maynard Keynes

[-- Attachment #2: Type: application/pgp-signature, Size: 187 bytes --]

Re: how to?

[Bart Schaefer]

On Aug 24, 11:56pm, Phil Pennock wrote:
}
} I didn't CC the list but instead made it a private reply.  Sorry for any
} offense caused -- I see that I didn't tone my wording well.

Oh, dear.  I completely missed that your message wasn't Cc'd to the list;
I thought I'd just accidentally hit "reply" instead of "replyall," so I
added the list back again.  You didn't cause any offense, nor did I have
any complaint about your tone; and I certainly didn't intend for my reply
to give such an impression.

Well, as we're here already ...

} > } Which leads to a question: how much hassle is it to have a glob modifier
} > } be able to duplicate the Simple Command which is calling it?
} > 
} > A glob modifier, just about impossible.  A precommand modifier or option,
} > perhaps.
} 
} Are there sufficient hooks to allow this to be done as a module?

Definitely not for the option route, but maybe for a something that works
like a precommand modifier -- or, really, just a special builtin that
chops up the argument list and calls argv[0] repeatedly with subsets.

The trouble, of course, is that it's very difficult to generalize, such
that you know which arguments have to be kept with argv[0] on every call
and which can be split across calls.  You end up having to "invert" the
syntax, i.e., put the splittable args first, then some flag value (such
as "--"), and finally the command with its fixed args:

	zargs **/*.c -- ls -l

That could actually be written as a shell function:

	function zargs {
	    emulate -RL zsh
	    local command end=$argv[(i)--]
	    command=( $argv[end+1,-1] )
	    # Hmm: argv[end,-1]=() doesn't work ...
	    set -- $argv[1,end-1]
	    if ((ARGC == 0))
	    then $command
	    else
		while ((ARGC))
		do
		    for (( end=ARGC; ${(c)#argv[1,end]} > 20480; end/=2 )) :
		    $command $argv[1,end]
		    shift $end
		done
	    fi
	}

Aside to zsh-workers:  "emulate -R" should not reset the XTRACE option,
but it does ...

-- 
Bart Schaefer                                 Brass Lantern Enterprises
http://www.well.com/user/barts              http://www.brasslantern.com

Zsh: http://www.zsh.org | PHPerl Project: http://phperl.sourceforge.net   

Re: how to?

[Phil Pennock]

[-- Attachment #1: Type: text/plain, Size: 997 bytes --]

On 2002-08-25 at 05:27 +0000, Bart Schaefer wrote:
> The trouble, of course, is that it's very difficult to generalize, such
> that you know which arguments have to be kept with argv[0] on every call
> and which can be split across calls.  You end up having to "invert" the
> syntax, i.e., put the splittable args first, then some flag value (such
> as "--"), and finally the command with its fixed args:

If it were, say, a glob modifier, then it could be documented to mean
"anything not a glob with this modifier will be repeated" -- that should
cover the majority of the situations and be a clear enough trade-off.

If it's a pre-command modifier, then it would simply be "anything not a
glob is repeated".

Ie "don't mix specified files and globs which might be repeated".

> 	zargs **/*.c -- ls -l
> 
> That could actually be written as a shell function:

That covers most of it, really.  I'll give that a go first.

Thanks,
-- 
Magizine: A weekly read for wise men

[-- Attachment #2: Type: application/pgp-signature, Size: 187 bytes --]

Re: how to?

[Roman Neuhauser]

> From: Scott Lipcon <slipcon@mercea.net>
> To: zsh-users@sunsite.dk
> Subject: how to?
> Date: Fri, 23 Aug 2002 09:35:10 -0400
> 
> Here's something I figure is possible in zsh, but I don't know how -
> I'd like to be able to do an ls in a directory of source code while
> its being built, and basically say: show me all the .c files for which
> there doesn't exist a .o file.   Right now I'm using:
> 
> ls -1 *.[co] | cut -f 1 -d"." | uniq -c | grep 1
> 
> but there has to be a zsh way to do that without 3 pipes.  Any ideas?

    This made me look in the uguide, and looks like there's a variation
    of this is (requires EXTENDED_GLOB):

    % print *.c~f*

    it works in this form, but isn't usable for the original task.
    Neither my copy of zsh manual nor user guide mention that this
    shouldn't work, but it doesn't:

    roman@freepuppy ~/tmp/foo 1030:0 > ls
    bar.c   bar.o   baz.c   baz.o   foo.c
    roman@freepuppy ~/tmp/foo 1031:0 > ls *.c~f*
    bar.c   baz.c
    roman@freepuppy ~/tmp/foo 1032:0 > ls *.c~*.o
    bar.c   baz.c   foo.c
    roman@freepuppy ~/tmp/foo 1033:0 > 

    could anyone explain this to me?

-- 
FreeBSD 4.6-STABLE
12:19PM up 6 days, 18:12, 19 users, load averages: 0.04, 0.07, 0.02

Re: how to?

[Phil Pennock]

[-- Attachment #1: Type: text/plain, Size: 1489 bytes --]

On 2002-08-27 at 12:34 +0200, Roman Neuhauser wrote:
> From: Scott Lipcon <slipcon@mercea.net>
> > I'd like to be able to do an ls in a directory of source code while
> > its being built, and basically say: show me all the .c files for which
> > there doesn't exist a .o file.   Right now I'm using:
> > 
> > ls -1 *.[co] | cut -f 1 -d"." | uniq -c | grep 1

>     This made me look in the uguide, and looks like there's a variation
>     of this is (requires EXTENDED_GLOB):
> 
>     % print *.c~f*
> 
>     it works in this form, but isn't usable for the original task.
>     Neither my copy of zsh manual nor user guide mention that this
>     shouldn't work, but it doesn't:
> 
>     roman@freepuppy ~/tmp/foo 1030:0 > ls
>     bar.c   bar.o   baz.c   baz.o   foo.c
>     roman@freepuppy ~/tmp/foo 1031:0 > ls *.c~f*
>     bar.c   baz.c
>     roman@freepuppy ~/tmp/foo 1032:0 > ls *.c~*.o
>     bar.c   baz.c   foo.c
>     roman@freepuppy ~/tmp/foo 1033:0 > 
> 
>     could anyone explain this to me?

The extended glob used excludes those filenames which match the second
glob.  The first one excludes filenames starting with "f".  The second
excludes those ending ".o".

If a filename ends ".c" then it can not also, at the same time, end
".o".

This says nothing about whether or not there exists a different filename
which has the same basename, but ends ".o" instead of ".c".
-- 
Some mornings it's just not worth gnawing through the straps.

[-- Attachment #2: Type: application/pgp-signature, Size: 187 bytes --]

Re: how to?

[Roman Neuhauser]

> Date: Tue, 27 Aug 2002 19:20:20 +0200
> From: Phil Pennock <Phil.Pennock@globnix.org>
> To: zsh-users@sunsite.dk
> Subject: Re: how to?
> 
> On 2002-08-27 at 12:34 +0200, Roman Neuhauser wrote:
> >     roman@freepuppy ~/tmp/foo 1030:0 > ls
> >     bar.c   bar.o   baz.c   baz.o   foo.c
> >     roman@freepuppy ~/tmp/foo 1031:0 > ls *.c~f*
> >     bar.c   baz.c
> >     roman@freepuppy ~/tmp/foo 1032:0 > ls *.c~*.o
> >     bar.c   baz.c   foo.c
> >     roman@freepuppy ~/tmp/foo 1033:0 > 
> > 
> >     could anyone explain this to me?
> 
> The extended glob used excludes those filenames which match the second
> glob.  The first one excludes filenames starting with "f".  The second
> excludes those ending ".o".
> 
> If a filename ends ".c" then it can not also, at the same time, end
> ".o".
> 
> This says nothing about whether or not there exists a different filename
> which has the same basename, but ends ".o" instead of ".c".

    ah, now it's clear. thanks!

-- 
FreeBSD 4.6-STABLE
7:59PM up 7 days, 1:52, 17 users, load averages: 0.00, 0.00, 0.00