zsh-users
 help / color / mirror / code / Atom feed
From: Bart Schaefer <schaefer@brasslantern.com>
To: zsh-users@zsh.org
Subject: Re: How to add string to end of each array element without loops
Date: Sat, 26 Mar 2011 00:14:15 -0700	[thread overview]
Message-ID: <110326001415.ZM31882@torch.brasslantern.com> (raw)
In-Reply-To: <b3f5af84c21435d21fed3f7705180821.squirrel@gameframe.net>

On Mar 26,  4:39am, nix@myproxylists.com wrote:
}
} Line 143: print -l ${(n)PROXIES} >> $Proxy_List causes segfault. I can't
} see where the problem is, if you try IP range up to one million, it will
} work good.

Aha.  I'm able to reproduce this with

typeset -A a
a=({5000000..1} {1..5000000})
print -l $a >> /dev/null

What's actually failing is builtin.c line 283, the declaration

	VARARR(char *, argarr, argc + 1);

This is attempting to allocate the entire argument list of "print"
(which in my example is 5 million words) on the C function call
stack.  Even with "unlimit stacksize" this is likely to overflow.
The program merely doesn't discover that this has failed until a
few lines later when it tries to use the first element of the array.

There's really nothing that can be done about this.  The whole shell
paradigm of passing arguments as an argv[] array means that $a has
to be expanded and then "print" called with the words by value.

So what you have to do to work around this is NOT rely on "print -l"
to insert the newlines, and instead insert them yourself:

print ${(F)PROXIES} >> $Proxy_List

This passes a single giant word with embedded newlines to "print" and
thus avoids allocating all that space on the C stack.

You're really past the design limits here of what a language with the
semantic rules of a command shell is meant to deal with.  If "print"
were not a builtin you'd have blown out the limits of argument passing
before even getting as far as you did; even with the (F) trick, using
/bin/echo will fail with "argument list too long".

I noticed you loaded the mapfile module even though you don't use it.
It should work to do

mapfile[$Proxy_list]=${(F)PROXIES}

but whether that's actually faster than "print >>" I haven't checked.


  reply	other threads:[~2011-03-26  7:14 UTC|newest]

Thread overview: 9+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2011-03-25 21:32 nix
2011-03-25 21:45 ` Mikael Magnusson
2011-03-25 21:47   ` nix
2011-03-25 22:03 ` Bart Schaefer
2011-03-26  1:37   ` nix
2011-03-26  2:10     ` Bart Schaefer
2011-03-26  2:39       ` nix
2011-03-26  7:14         ` Bart Schaefer [this message]
2011-03-26 14:04           ` nix

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=110326001415.ZM31882@torch.brasslantern.com \
    --to=schaefer@brasslantern.com \
    --cc=zsh-users@zsh.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://git.vuxu.org/mirror/zsh/

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).