question about zargs

question about zargs

[Han Pingtian]

Hello,

I just learnt that there is a function 'zargs' which just like 'xargs'.
So why we need the 'zargs' as we have 'xargs' already? 

As a example, this works just fine with 'xargs':

    % print -N **/* | xargs -n1 -0 ls

But this one will cause "(eval):2: fork failed: cannot allocate memory"
error on my laptop:

    % zargs **/* -- ls

Thanks in advance.

Re: question about zargs

[Peter Stephenson]

On Wed, 31 Oct 2012 21:40:07 +0800
Han Pingtian <hanpt@linux.vnet.ibm.com> wrote:
> I just learnt that there is a function 'zargs' which just like
> 'xargs'. So why we need the 'zargs' as we have 'xargs' already? 
> 
> As a example, this works just fine with 'xargs':
> 
>     % print -N **/* | xargs -n1 -0 ls

It works, but with more processes.  zargs allows you to have things
(though not ls) running completely in the shell.  In that case, you
aren't sensitive to the size of the argument list passed to an external
process.

> But this one will cause "(eval):2: fork failed: cannot allocate
> memory" error on my laptop:
> 
>     % zargs **/* -- ls

ls is run as an external process, so the argument list is limited.
zargs doesn't have a builtin system-dependent limit to the number it'll
pass in one go, unlike xargs, as far as I can see, so needs to be told
how to limit it.

Hmm... in principle, you can do:

zargs -n2 **/* -- ls

so that it executes ls and one additional argument each time.  This is
equivalent to the -n1 you gave to xargs.

However, with a lot of iles this is running incredibly slowly for me and
after a few dozen files have been processed I hit:

279: mem.c:1180: MEM: allocation error at sbrk, size 589824.
zargs:279: fatal error: out of memory

(Luckily this is in a subshell.)

It looks like this is hitting some pathology in memory management to do
with the argument array (which is being shifted at that line).

pws

Re: question about zargs

[Bart Schaefer]

Preface for Han Pingtian:  If the command you're going to run is external
to the shell, then I recommend you use xargs.  However, in some cases it
may be desirable to get xargs-like behavior when passing arguments to a
shell function or builtin, which is why there is zargs.

On Oct 31,  2:25pm, Peter Stephenson wrote:
} Subject: Re: question about zargs
}
} Hmm... in principle, you can do:
} 
} zargs -n2 **/* -- ls
} 
} so that it executes ls and one additional argument each time.  This is
} equivalent to the -n1 you gave to xargs.

Well, yes and no.  zargs -n2 is not "equivalent to" xargs -n1 in the
general case.  In the previous thread about this, it was pointed out that

    zargs -n2 **/* -- ls -l

is equivalent to

    zlargs -l1 **/* -- ls -l

But the only reason you need -n2 is because of the "-l" that is already
trailing the "ls" command.  If there are no trailing arguments (that is,
the command is a name only) then -n1 and -l1 are equivalent.
 
} However, with a lot of iles this is running incredibly slowly for me and
} after a few dozen files have been processed I hit:
} 
} 279: mem.c:1180: MEM: allocation error at sbrk, size 589824.
} zargs:279: fatal error: out of memory

Hmm, "a few dozen" isn't nearly enough to trigger this for me, though if
I run it on a sufficiently large set of files I can watch the memory
usage of the shell grow steadily throughout.

And yes, it can be slow.

If I try

    zlargs -l1 **/* -- print > /dev/null

the memory use jumps much faster, probably because no forking is involved.

-- 
Barton E. Schaefer

Re: question about zargs

[Han Pingtian]

On Wed, Oct 31, 2012 at 02:25:46PM +0000, Peter Stephenson wrote:
> On Wed, 31 Oct 2012 21:40:07 +0800
> Han Pingtian <hanpt@linux.vnet.ibm.com> wrote:
> > I just learnt that there is a function 'zargs' which just like
> > 'xargs'. So why we need the 'zargs' as we have 'xargs' already?
> >
> > As a example, this works just fine with 'xargs':
> >
> >     % print -N **/* | xargs -n1 -0 ls
>
> It works, but with more processes.  zargs allows you to have things
> (though not ls) running completely in the shell.  In that case, you
> aren't sensitive to the size of the argument list passed to an
> external
> process.
>

On Wed, Oct 31, 2012 at 09:30:17AM -0700, Bart Schaefer wrote:
> Preface for Han Pingtian:  If the command you're going to run is external
> to the shell, then I recommend you use xargs.  However, in some cases it
> may be desirable to get xargs-like behavior when passing arguments to a
> shell function or builtin, which is why there is zargs.
> 

Thanks so much, Peter and Bart.

delete key :(

[Ray Andrews]

All,

Just a quick mini-bitch. Playing with the Salix distro, I imported all 
my ~/.z* files, and got things more or less working ok, except that a 
few standard keys didn't work, like 'delete' prints a tilde, etc.  I've 
since figured out that this is because several keys are assigned to 
their normal function in the '/etc/zsh' directory of Debian based 
distros.  Why on Earth would keys as standard as 'delete' not have their 
normal function ROOTB? Why must we work to get something so basic?  Even 
bash gives us those keys by default.

Re: delete key :(

[Peter Stephenson]

On Wed, 31 Oct 2012 17:10:19 -0700
Ray Andrews <rayandrews@eastlink.ca> wrote:
> Just a quick mini-bitch. Playing with the Salix distro, I imported
> all my ~/.z* files, and got things more or less working ok, except
> that a few standard keys didn't work, like 'delete' prints a tilde,
> etc.

It needs to query termcap (which is the interface we're still using, but
is mostly good enough) for the information about the key and then
bind it.  There's already special-case code in zle_keymap.c to
add the bindings for cursor keys.  Needs some modification of
zsh.h and init.c to add the termcap strings (kD delete, kI insert, kh
home, can't see end offhand although it's there in terminfo ...hmm, we
do have a terminfo library add-on, could use it to query for kend).

Apart from arranging to use terminfo, which needs something thinking
about loading dynamic libraries, it should be a relatively easy job for
a volunteer.

pws

Re: delete key :(

[Ray Andrews]

On 01/11/12 05:54 AM, Peter Stephenson wrote:
> Apart from arranging to use terminfo, which needs something thinking
> about loading dynamic libraries, it should be a relatively easy job for
> a volunteer.
I've been trying to work up the guts to jump in for a while now, but I 
always end up chickening out ;-)
> pws
>