I've committed "zargs" ...

I've committed "zargs" ...

[Bart Schaefer]

I got a bit carried away and implemented the entire set of options from
GNU xargs 4.1.  Rather than post the result here, I've just committed it.
I didn't write any doc for contrib.yo ... does anyone think I should?

Here's `zargs --help':

----------
Usage: zargs [options --] [input-args] [-- command [initial-args]]

If command and initial-args are omitted, "print -r --" is used.

Options:
--eof[=eof-str], -e[eof-str]
    Change the end-of-input-args string from "--" to eof-str.  If
    given as --eof=, an empty argument is the end; as --eof or -e,
    with no (or an empty) eof-str, all arguments are input-args.
--exit, -x
    Exit if the size (see --max-chars) is exceeded.
--help
    Print this summary and exit.
--interactive, -p
    Prompt before executing each command line.
--max-args=max-args, -n max-args
    Use at most max-args arguments per command line.
--max-chars=max-chars, -s max-chars
    Use at most max-chars characters per command line.
--max-lines[=max-lines], -l[max-lines]
    Use at most max-lines of the input-args per command line.
    This option is misnamed for xargs compatibility.
--max-procs=max-procs, -P max-procs
    Run up to max-procs command lines in the background at once.
--no-run-if-empty, -r
    Do nothing if there are no input arguments before the eof-str.
--null, -0
    Split each input-arg at null bytes, for xargs compatibility.
--replace[=replace-str], -i[replace-str]
    Substitute replace-str in the initial-args by each initial-arg.
    Implies --exit --max-lines=1.
--verbose, -t
    Print each command line to stderr before executing it.
--version
    Print the version number of zargs and exit.
----------

I spent much more time than I should have on getting the best --max-procs
algorithm I could come up with.  Comments welcome.

-- 
Bart Schaefer                                 Brass Lantern Enterprises
http://www.well.com/user/barts              http://www.brasslantern.com

Zsh: http://www.zsh.org | PHPerl Project: http://phperl.sourceforge.net   

Re: Occasional 'job table full or recursion limit exceeded'

[Felix Rosencrantz]

--- Bart Schaefer <schaefer@brasslantern.com> wrote:

> I spent much more time than I should have on getting the best
> --max-procs algorithm I could come up with.  Comments welcome.

Not really sure if this is the feedback you want.  Since I mostly
only see it in long running shells, where my suspended processes have
accrued, it seems like increasng it by small or fractional increments
would be more conservative of memory usage than a doubling strategy.

When I see the problem I typically have 10 to 15 processes.  It's unlikely
I would have much more than 20, because that is getting in the range where it
is difficult to manage.

-FR.



__________________________________________________
Do You Yahoo!?
Yahoo! Finance - Get real-time stock quotes
http://finance.yahoo.com

Re: Occasional 'job table full or recursion limit exceeded'

[Bart Schaefer]

On Sep 9,  7:41am, Felix Rosencrantz wrote:
} Subject: Re: Occasional 'job table full or recursion limit exceeded'
}
} --- Bart Schaefer <schaefer@brasslantern.com> wrote:
} 
} > I spent much more time than I should have on getting the best
} > --max-procs algorithm I could come up with.  Comments welcome.
} 
} Not really sure if this is the feedback you want.

I'm not really sure what to say, because the message you quoted doesn't
have anything to do with the thread you're replying to.  (The excerpt
above refers to the number of background jobs spawned by my "zargs"
function, not to the value of MAXJOBS in the C code.)

} Since I mostly only see it in long running shells, where my suspended
} processes have accrued, it seems like increasng it by small or
} fractional increments would be more conservative of memory usage than
} a doubling strategy.

Part of the problem is that the value of MAXJOBS was established when
each job table entry represented a real unix process.  Now that we make
job table entries for internal shell control structures, we need more
table space.  It still shouldn't need to grow without bound (though I
believe we have other anti-infinite recursion mechanisms in place now).

-- 
Bart Schaefer                                 Brass Lantern Enterprises
http://www.well.com/user/barts              http://www.brasslantern.com

Zsh: http://www.zsh.org | PHPerl Project: http://phperl.sourceforge.net   

Re: Occasional 'job table full or recursion limit exceeded'

[Felix Rosencrantz]

--- Bart Schaefer <schaefer@brasslantern.com> wrote:
> I'm not really sure what to say, because the message you quoted doesn't
> have anything to do with the thread you're replying to.  (The excerpt
> above refers to the number of background jobs spawned by my "zargs"
> function, not to the value of MAXJOBS in the C code.)
I'm a wee bit red faced... :)

> Part of the problem is that the value of MAXJOBS was established when
> each job table entry represented a real unix process.  Now that we make
> job table entries for internal shell control structures, we need more
> table space.  It still shouldn't need to grow without bound (though I
> believe we have other anti-infinite recursion mechanisms in place now).
I didn't really appreciate that MAXJOBS is sort of taking on the role of a
control structures stack.   In which case, a dynamic solution seems reasonable.

Currently I'm building with MAXJOBS doubled, figuring that should cover it for
me.   Though I suspect most users have prebuilt versions of zsh, and so might
hit this problem.

-FR.

__________________________________________________
Yahoo! - We Remember
9-11: A tribute to the more than 3,000 lives lost
http://dir.remember.yahoo.com/tribute