sorting/uniq-ing an array?

sorting/uniq-ing an array?

[Sweth Chandramouli]

	while playing around with zsh completion, i noticed that the
ports variable set in the comctl-examples file doesn't distinguish
between tcp and udp services in /etc/services, resulting in a non-unique
array (e.g. multiple instances of "echo").  i've come up with a way
to remove redundant entries from an array (the appropriate part of
my .zshrc is at the end of this message), but it's very slow--on
a single-user ultrasparc/30, it adds about two seconds to my login
time.
	is there a faster way to do this?  for some uses, it's okay--
i've got a function called addpath, for example, that uses this sort
of looping to scan $PATH to see if the argument passed to addpath is
already in it, and iff it isn't, adds it, and a second or two to
process that function (usually almost unnoticeable, actually, since
$PATH is so much shorter than $ports) doesn't seem strange.  as an
addition to the login process, however, it seems like it takes an
eternity.  i was thinking of creating a file that stored the array,
and on login, first populate $ports with the contents of that file, and
then spawned a bg process to do the parsing from /etc/services,
write the output out to the file, and then repopulate $ports.  any
suggestions on how to do that, or of other (quicker) ways to do the 
parsing of the array?
	(as always, of course, comments on style are also appreciated.
i'm fairly certain that i still don't understand when ${= is needed
and when it isn't, so i just tend to use it everywhere that splitting
might be needed.  did i do it right here?)

	tia,
	sweth.

--- begin quoted text ---
#### ports is made unique in an ugly way.  is this fixable?
portnames=("${${${(f)$(</etc/services)}:#\#*}%%[        ]*}")
portnums=("${${${${${(f)$(</etc/services)}:#\#*}%%/*}##*        }##* }")
ports=(${=portnums} ${=portnames})
# unique-ify ports
for i in $ports ; do
   unset append;
   for b in ${=tmpvar} ; do
      if [[ $i == $b ]] ; then
         append='false';
      fi;
   done;
   if [[ -z $append ]] ; then
      tmpvar=($tmpvar $i);
   fi;
done;
ports=(${=tmpvar});
--- end quoted text ---

-- 
Sweth Chandramouli
IS Coordinator, The George Washington University
<sweth@gwu.edu> / (202) 994 - 8521 (V) / (202) 994 - 0458 (F)
<a href="http://astaroth.nit.gwu.edu/~sweth/disc.html">*</a>

Re: sorting/uniq-ing an array?

[Bart Schaefer]

On Aug 9,  2:56am, Sweth Chandramouli wrote:
: Subject: sorting/uniq-ing an array?
:
: i've come up with a way
: to remove redundant entries from an array (the appropriate part of
: my .zshrc is at the end of this message), but it's very slow
:
: 	is there a faster way to do this?

Yes;

typeset -U ports

but the completion code makes its internal list of completions unique,
so it doesn't really help to remove the redundant entries from arrays
in advance when using them only for completion.

: 	(as always, of course, comments on style are also appreciated.
: i'm fairly certain that i still don't understand when ${= is needed
: and when it isn't, so i just tend to use it everywhere that splitting
: might be needed.  did i do it right here?)

You almost never need ${=...} when the "..." refers to an array.  Using
${=...} on an array causes the individual elements of the array to be
split into words, which is not very often what is wanted.  So no, you
didn't do it right here:

: portnames=("${${${(f)$(</etc/services)}:#\#*}%%[        ]*}")
: portnums=("${${${${${(f)$(</etc/services)}:#\#*}%%/*}##*        }##* }")
: ports=(${=portnums} ${=portnames})

However, you're over-processing.  You have both the port numbers and names
at one point during the portnums= computation, but then you throw the names
away.  Keep them:

ports=("${=${${(f)$(</etc/services)}:#\#*}%%/*}")

(Note that there I used ${=...} to re-split the ${(f)...} array of lines,
so sometimes that _is_ what you want.  I did say _almost_ never.)

: for i in $ports ; do
:    unset append;
:    for b in ${=tmpvar} ; do
:       if [[ $i == $b ]] ; then
:          append='false';
:       fi;
:    done;
:    if [[ -z $append ]] ; then
:       tmpvar=($tmpvar $i);
:    fi;
: done;

You probably want to be sure that $tmpvar doesn't already have a value
at the time you start any kind of loop like that.  Also, zsh (and most
other shells) has multi-level loop break/continue, so you can restart
the outer loop without setting/testing $append from the inner loop:

tmpvar=()
for i in $ports ; do
  for b in $tmpvar ; do
    [[ $i == $b ]] && continue 2
  done
  tmpvar=($tmpvar $i)
done

-- 
Bart Schaefer                                 Brass Lantern Enterprises
http://www.well.com/user/barts              http://www.brasslantern.com

Re: Re: sorting/uniq-ing an array?

[Sweth Chandramouli]

On Sun, Aug 09, 1998 at 04:41:23AM -0700, Bart Schaefer wrote:
> On Aug 9,  2:56am, Sweth Chandramouli wrote:
> : Subject: sorting/uniq-ing an array?
> :
> : i've come up with a way
> : to remove redundant entries from an array (the appropriate part of
> : my .zshrc is at the end of this message), but it's very slow
> :
> : 	is there a faster way to do this?
> 
> Yes;
> 
> typeset -U ports
> 
> but the completion code makes its internal list of completions unique,
> so it doesn't really help to remove the redundant entries from arrays
> in advance when using them only for completion.
	that's good to know, though there are other places where it would
be useful (such as the addpath fn i mentioned) to unique-ify.  also, i was
using portnums and portnames elsewhere, for non-completion purposes (see
below).
	i actually had thought that there was some sort of typeset switch
to do this, but did a man typeset, and didn't see any.  i now realize that
the typeset manpage on my machine is for the ksh and csh versions, and that
the zsh one is only in the zshbuiltins manpage.  what do people think of
breaking the manpages for the builtins out of zshbuiltins (or rather, copying
them out, since i think man zshall should still always be the reference of
last resort), and maybe merging them in with the equivalent pages for other
shell builtins?  if there's an interest in it, i might take a stab at it; i
just downloaded yodl, so i need a reason to learn it, and this is as good as
any others i could conjure up.

> You almost never need ${=...} when the "..." refers to an array.  Using
> ${=...} on an array causes the individual elements of the array to be
> split into words, which is not very often what is wanted.  So no, you
> didn't do it right here:
	upon further consideration, this is a leftover ksh-ism in my brain--
i wasn't thinking of these as being in an array context, because in ksh an
array without a subscript is actually just the last scalar in that array;
i was then assuming that i needed to break the scalars that i was referencing
up--hence the ${= constructs.

> 
> : portnames=("${${${(f)$(</etc/services)}:#\#*}%%[        ]*}")
> : portnums=("${${${${${(f)$(</etc/services)}:#\#*}%%/*}##*        }##* }")
> : ports=(${=portnums} ${=portnames})
> 
> However, you're over-processing.  You have both the port numbers and names
> at one point during the portnums= computation, but then you throw the names
> away.  Keep them:
> 
> ports=("${=${${(f)$(</etc/services)}:#\#*}%%/*}")
> 
> (Note that there I used ${=...} to re-split the ${(f)...} array of lines,
> so sometimes that _is_ what you want.  I did say _almost_ never.)
	this works, which surprises me, since that implies that SH_WORD_SPLIT
splits on either spaces or tabs (there are both in my /etc/services as
delimiters between portnames and portnumbers); i thought that IFS was by
default just a space.  or am i just misunderstanding what you are doing here
entirely?

	-- sweth.
-- 
Sweth Chandramouli
IS Coordinator, The George Washington University
<sweth@gwu.edu> / (202) 994 - 8521 (V) / (202) 994 - 0458 (F)
<a href="http://astaroth.nit.gwu.edu/~sweth/disc.html">*</a>

Re: sorting/uniq-ing an array?

[Bart Schaefer]

On Aug 9, 12:36pm, Sweth Chandramouli wrote:
} Subject: Re: Re: sorting/uniq-ing an array?
}
} 	i actually had thought that there was some sort of typeset switch
} to do this, but did a man typeset, and didn't see any.  i now realize that
} the typeset manpage on my machine is for the ksh and csh versions, and that
} the zsh one is only in the zshbuiltins manpage.  what do people think of
} maybe merging [zsh builtins manpages] with the equivalent pages for other
} shell builtins?

I don't think this is a good idea.  For one thing, it means replacing the
installed man pages of another app with the man pages from zsh, which may
be impossible for many zsh installers.  For another, it makes the manpages
differ significantly from the info documentation.  Then there's the issue
of "merging" itself -- when do you propose that this happen?  At the time
zsh manual pages are installed?  Even if we could somehow find, parse, and
merge with the existing other shell's manpages, what happens when the other
shell or its manpages are updated?

} > ports=("${=${${(f)$(</etc/services)}:#\#*}%%/*}")
} > 
} > (Note that there I used ${=...} to re-split the ${(f)...} array of lines,
} > so sometimes that _is_ what you want.  I did say _almost_ never.)
} 	this works, which surprises me, since that implies that SH_WORD_SPLIT
} splits on either spaces or tabs (there are both in my /etc/services as
} delimiters between portnames and portnumbers); i thought that IFS was by
} default just a space.  or am i just misunderstanding what you are doing
} here entirely?

No, you understand; IFS is by default space, tab, newline, and nul:

zagzig% echo $#IFS
4
zagzig% echo x${IFS}x | cat -v
x 
^@x
zagzig% 

I don't think other shells have nul in IFS by default, but I'm not sure.

-- 
Bart Schaefer                                 Brass Lantern Enterprises
http://www.well.com/user/barts              http://www.brasslantern.com

Re: sorting/uniq-ing an array?

[Bruce Stephens]

Sweth Chandramouli <sweth@astaroth.nit.gwu.edu> writes:

> i actually had thought that there was some sort of typeset switch to
> do this, but did a man typeset, and didn't see any.  i now realize
> that the typeset manpage on my machine is for the ksh and csh
> versions, and that the zsh one is only in the zshbuiltins manpage.

Ah, you want Tom Phelp's TkMan, which shows when there are multiple
manpages with the same name, and provides a menu of them.
<URL:ftp://ftp.cs.berkeley.edu/ucb/people/phelps/tcltk/>.

"typeset -U" works for arrays, and so works for path.  Unfortunately,
it doesn't work with (for example) LD_LIBRARY_PATH, which is a
colon-separated list.  Annoying: this really needs configurableness.