triviality regarding $# counts

triviality regarding $# counts

[Ray Andrews]

[-- Attachment #1: Type: text/plain, Size: 2568 bytes --]

This practice text is in a variable 'temp' and I want to count the 
number of lines inside a function that will also print the text exactly 
as shown, and do a bunch of other stuff.

--------------------------------------------------------------------
xz-utils:
/usr/bin/xzgrep
/usr/share/man  /de/man1/xzgrep.1.gz    # Hafta be careful not to split 
here.
/usr/share/man  /ko/man1/xzgrep.1.gz    # ... or here.
/usr/share/man/man1/xzgrep.1.gz
/usr/share/man/pt_BR/man1/xzgrep.1.gz
/usr/share/man/ro/man1/xzgrep.1.gz
/usr/share/man/uk/man1/xzgrep.1.gz
/usr/bin/xzegrep
/usr/bin/xzfgrep
/usr/share/man/de/man1/lzegrep.1.gz
/usr/share/man/de/man1/lzfgrep.1.gz
/usr/share/man/uk/man1/xzegrep.1.gz
/usr/share/man/uk/man1/xzfgrep.1.gz

zsh-common:
/usr/share/doc/zsh-common/examples/Functions/zpgrep
/usr/share/zsh/functions/Completion/Debian/_grep-excuses
/usr/share/zsh/functions/Completion/Unix/_grep
/usr/share/zsh/functions/Completion/Unix/_ngrep
/usr/share/zsh/functions/Completion/Unix/_pgrep

zstd:
/usr/bin/zstdgrep
/usr/share/man/man1/zstdgrep.1.gz
-----------------------------------------------------------------------------------

This works: (Again, this is deep in a function and I have to use eval.)

     output=$( eval "$@" )   # '$@' will expand to 'print -l $temp' 
which is the text above.
     temp=( $( eval $@ ) )    # To get the correct count I need to force 
an array.
     linecount=$#temp
     print -rl -- $output
     print  $linecount

Various experiments trying to get the correct linecount (23) *and* get 
it printing correctly all end up with waterboarding problems. If I do it 
in two stages as above, it's ok, but is seems very clumsy.  Different 
efforts at quoting or using ' ${(@f) ....} ' and various other tricks 
yield me a linecount of 1, 3, 23, 25, 26, or 738.  And output that 
deletes the blank lines, or forces everything into one 'line/element'.  
Basically I need the array form to to get the line count, but it won't 
print properly as an array.  Not that it's worth much trouble, but is it 
possible to get the variable  to print correctly *and* show the count of 
lines without having to eval it twice?  schematically:

     output=????$( eval ???)??? # Whatever the correct code might be.
     linecount=$#output
     print -rl -- $output
     print  $linecount

... as it is, it seems that '$#' never counts the lines of output as it 
actually prints.  In practice it's hardly a problem but still it bothers 
me.  '$#' seems mostly to want to count characters (738).



[-- Attachment #2: Type: text/html, Size: 3390 bytes --]

Re: triviality regarding $# counts

[Lawrence Velázquez]

On Wed, Apr 10, 2024, at 8:56 PM, Ray Andrews wrote:
> This works: (Again, this is deep in a function and I have to use eval.)

If eval really is necessary (to be frank, I don't trust your judgment
on this), show us examples that require it, instead of the misleading
Rube Goldberg machines you've been offering.


>     output=$( eval "$@" )   # '$@' will expand to 'print -l $temp' 
> which is the text above.
>     temp=( $( eval $@ ) )    # To get the correct count I need to force 
> an array.
>     linecount=$#temp
>     print -rl -- $output
>     print  $linecount

The count is correct by accident.  Your unquoted command substitution
drops the two empty lines but splits the two lines you say should not
be split.

	% orig='abc
	quote> 
	quote> def ghi
	quote> jkl mno
	quote> 
	quote> pqr'
	% arr=($(print -r -- $orig))
	% typeset -p arr
	typeset -a arr=( abc def ghi jkl mno pqr )


> Various experiments trying to get the correct linecount (23)

There are 25 lines, not 23.


> Different efforts at quoting or using ' ${(@f) ....} ' and various 
> other tricks yield me a linecount of 1, 3, 23, 25, 26, or 738.  And 
> output that deletes the blank lines, or forces everything into one 
> 'line/element'.  Basically I need the array form to to get the line 
> count, but it won't print properly as an array.  Not that it's worth 
> much trouble, but is it possible to get the variable  to print 
> correctly *and* show the count of lines without having to eval it 
> twice?

	% cat foo.zsh
	orig='abc

	def ghi
	jkl mno

	pqr'

	# The sensible way to split on LFs.
	#arr=("${(@f)orig}")

	# A very silly way to split on LFs.  Use double quotes to
	# prevent the result of $(...) from being split and to retain
	# empty words in the result of ${(@)...}.
	arr=("${(@f)$(print -r -- $orig)}")

	typeset -p arr
	print -r -- $#arr

	# Use double-quoted $arr[@] to retain empty elements.  Use
	# "print -C1" to avoid printing an empty line if "arr" is empty.
	print -rC1 -- "$arr[@]"

	% zsh ./foo.zsh
	typeset -a arr=( abc '' 'def ghi' 'jkl mno' '' pqr )
	6
	abc

	def ghi
	jkl mno

	pqr
	%


> ... as it is, it seems that '$#' never counts the lines of output as it 
> actually prints.

You both populate and print your array incorrectly.  For the umpteenth
time, you should use "typeset -p" to inspect your variables' values.


-- 
vq

Re: triviality regarding $# counts

[Ray Andrews]

[-- Attachment #1: Type: text/plain, Size: 3005 bytes --]



On 2024-04-11 21:55, Lawrence Velázquez wrote:

> If eval really is necessary (to be frank, I don't trust your judgment
> on this), show us examples that require it, instead of the misleading
> Rube Goldberg machines you've been offering.
I don't trust it neither, and I taught Rube everything he knows. 
Seriously, you won't sympathize, but the tinkerer's approach was the 
only one available to me at least until I had enough ad hoc code built 
up that sorta works to begin to start to try to understand it formally.  
And my efforts to find patterns of behavior -- as you know better than 
anyone -- lead me to false conclusions all the time.  The manual is 
useless as pedagogy.  If there was 'zsh school' I'd enroll and learn my 
shell ABC's properly from the ground up. But there isn't.  So I hack 
away.  But please understand I *hate* being me.  I like competence :(
> The count is correct by accident.  Your unquoted command substitution
> drops the two empty lines but splits the two lines you say should not
> be split.
Right, I'm aware of that possibility at least in theory.  Word splitting 
lurks.
> 	% orig='abc
> 	quote>
> 	quote> def ghi
> 	quote> jkl mno
> 	quote>
> 	quote> pqr'
> 	% arr=($(print -r -- $orig))
> 	% typeset -p arr
> 	typeset -a arr=( abc def ghi jkl mno pqr )
It will take me a long time to understand that.

> 	% cat foo.zsh
> 	orig='abc
>
> 	def ghi
> 	jkl mno
>
> 	pqr'
>
> 	# The sensible way to split on LFs.
> 	#arr=("${(@f)orig}")
>
> 	# A very silly way to split on LFs.  Use double quotes to
> 	# prevent the result of $(...) from being split and to retain
> 	# empty words in the result of ${(@)...}.
> 	arr=("${(@f)$(print -r -- $orig)}")
>
> 	typeset -p arr
> 	print -r -- $#arr
>
> 	# Use double-quoted $arr[@] to retain empty elements.  Use
> 	# "print -C1" to avoid printing an empty line if "arr" is empty.
> 	print -rC1 -- "$arr[@]"
>
> 	% zsh ./foo.zsh
> 	typeset -a arr=( abc '' 'def ghi' 'jkl mno' '' pqr )
> 	6
> 	abc
>
> 	def ghi
> 	jkl mno
>
> 	pqr
> 	%
I think I can chew on that, backatcha later.

> You both populate and print your array incorrectly. For the umpteenth
> time, you should use "typeset -p" to inspect your variables' values.
>
I keep typeset -p close at all times now.  But there are times when I'm 
still baffled.  Like an alchemist.  Sheesh, just yesterday I was 
attempting to transform this:

typeset -g var="
abc

     def ghi
     jkl mno

     pqr
"

... into an array that retained the same 'shape' -- that prints verbatim 
-- and typeset -p showed newlines being transubstantiated into dollar 
signs!  God knows how.

For now, what I'm wishing for is a nice, focused essay: "How to use '$#' 
to count whatever it is you want to count -- characters, words, lines, 
paragraphs (at one point my count was '3' which I'd approximate to being 
a paragraph) ... and all with or without spaces and/or empty lines. " 
Should run about  ten pages but it would be everything there is to know 
about '$#'.


[-- Attachment #2: Type: text/html, Size: 4464 bytes --]

Re: triviality regarding $# counts

[Bart Schaefer]

On Fri, Apr 12, 2024 at 7:49 AM Ray Andrews <rayandrews@eastlink.ca> wrote:
>
> For now, what I'm wishing for is a nice, focused essay: "How to use '$#' to count whatever it is you want to count -- characters, words, lines, paragraphs (at one point my count was '3' which I'd approximate to being a paragraph) ... and all with or without spaces and/or empty lines. "

None of that is about $#.  $# just counts either characters in a
string or elements in an array, full stop.

What you're asking for is a dissertation on how to split text into
arrays.  I have no idea what you mean by "without spaces" and there's
no inherent definition of a "paragraph" so the best you could get from
this is characters, words, and lines -- but even for words you need to
explain whether you mean "shell words" (separated by $IFS characters)
or something else, including whether quotes matter.  Further, you seem
to be starting from scalar text sometimes, and text that's already
split into an array other times -- in the latter case you have to
explain whether and how you want the array re-joined into a block of
text before re-splitting.

The shell is not a word processor and doesn't understand your
conceptualization of text formatting.

Re: triviality regarding $# counts

[Ray Andrews]

[-- Attachment #1: Type: text/plain, Size: 2398 bytes --]


On 2024-04-12 12:09, Bart Schaefer wrote:
> None of that is about $#. $# just counts either characters in a
> string or elements in an array, full stop.
Right, it's a question of getting the splitting correct.
> What you're asking for is a dissertation on how to split text into
> arrays.  I have no idea what you mean by "without spaces" and there's
> no inherent definition of a "paragraph"
I know it, thus the air-quotes, In one of my tests I had three 
consecutive outputs from aptitude written to the array and at some point 
the count was 3, so somehow each separate output got merged into one 
element.
> so the best you could get from
> this is characters, words, and lines -- but even for words you need to
> explain whether you mean "shell words" (separated by $IFS characters)
> or something else, including whether quotes matter.
It is a bit confusing.

>    Further, you seem
> to be starting from scalar text sometimes, and text that's already
> split into an array other times -- in the latter case you have to
> explain whether and how you want the array re-joined into a block of
> text before re-splitting.
I've got a whole bunch of that figured out today.  What I thought were 
arrays were scalars that just happened to print nicely on separate lines 
where wanted but not due to '\n''s.  And what I thought line splitting 
-- (f) -- did was enter '\n's into the body of the variable -- a guy 
might be forgiven for thinking that -- but no it's dollar signs.  And 
efforts to force '\n's' were disastrous. Some quality time with typedef 
-p really helped.  Sorta funny, everything was working fine, but there 
were hidden disasters lurking that surfaced for that most trivial of 
reasons -- but forced me to redo quite a bit of stuff where things 
really are arrays and nevermind the '\n's.  And I get my line counts 
honestly now :-)  One thing: it sure is hard to hang on to blank lines.  
I wish there was some option to default to preserving them.
> The shell is not a word processor and doesn't understand your
> conceptualization of text formatting.
Very true.  But I focus on what I can see and if it looks right it's 
easy to think it is right.  Anyway, thanks to Lawrence much as been 
learned. BTW $# is a very convenient way of detecting how things are 
split, now that I know that scalars always return character counts and 
arrays, element counts.  Easy!

[-- Attachment #2: Type: text/html, Size: 3604 bytes --]

Re: triviality regarding $# counts

[Mark J. Reed]

[-- Attachment #1: Type: text/plain, Size: 2942 bytes --]

More context would clarify what you're trying to do and how best to do it.
Without that context, the approach you're currently taking feels deeply
weird. Can you not provide sample input and desired output? Maybe with all
of your code rather than a few snippets from "so deep they need eval",
whatever that means...

On Fri, Apr 12, 2024 at 9:13 PM Ray Andrews <rayandrews@eastlink.ca> wrote:

>
> On 2024-04-12 12:09, Bart Schaefer wrote:
>
> None of that is about $#. $# just counts either characters in a
>
> string or elements in an array, full stop.
>
> Right, it's a question of getting the splitting correct.
>
> What you're asking for is a dissertation on how to split text into
> arrays.  I have no idea what you mean by "without spaces" and there's
> no inherent definition of a "paragraph"
>
> I know it, thus the air-quotes, In one of my tests I had three consecutive
> outputs from aptitude written to the array and at some point the count was
> 3, so somehow each separate output got merged into one element.
>
> so the best you could get from
> this is characters, words, and lines -- but even for words you need to
> explain whether you mean "shell words" (separated by $IFS characters)
> or something else, including whether quotes matter.
>
> It is a bit confusing.
>
>   Further, you seem
> to be starting from scalar text sometimes, and text that's already
> split into an array other times -- in the latter case you have to
> explain whether and how you want the array re-joined into a block of
> text before re-splitting.
>
> I've got a whole bunch of that figured out today.  What I thought were
> arrays were scalars that just happened to print nicely on separate lines
> where wanted but not due to '\n''s.  And what I thought line splitting --
> (f) -- did was enter '\n's into the body of the variable -- a guy might be
> forgiven for thinking that -- but no it's dollar signs.  And efforts to
> force '\n's' were disastrous.  Some quality time with typedef -p really
> helped.  Sorta funny, everything was working fine, but there were hidden
> disasters lurking that surfaced for that most trivial of reasons -- but
> forced me to redo quite a bit of stuff where things really are arrays and
> nevermind the '\n's.  And I get my line counts honestly now :-)  One thing:
> it sure is hard to hang on to blank lines.  I wish there was some option to
> default to preserving them.
>
> The shell is not a word processor and doesn't understand your
> conceptualization of text formatting.
>
> Very true.  But I focus on what I can see and if it looks right it's easy
> to think it is right.  Anyway, thanks to Lawrence much as been learned. BTW
> $# is a very convenient way of detecting how things are split, now that I
> know that scalars always return character counts and arrays, element
> counts.  Easy!
>
>

-- 
Mark J. Reed <markjreed@gmail.com>

[-- Attachment #2: Type: text/html, Size: 4020 bytes --]

Re: triviality regarding $# counts

[Bart Schaefer]

On Fri, Apr 12, 2024 at 6:13 PM Ray Andrews <rayandrews@eastlink.ca> wrote:
>
> I've got a whole bunch of that figured out today.  What I thought were arrays were scalars that just happened to print nicely on separate lines where wanted but not due to '\n''s.  And what I thought line splitting -- (f) -- did was enter '\n's into the body of the variable -- a guy might be forgiven for thinking that -- but no it's dollar signs.

You're definitely confusing yourself again there somehow.  (f) does
not "enter" anything into the "body" of a variable -- it splits a
string into an array, converting newlines into the breaks between
array elements.  I have no idea where you think dollar signs are
coming from ... unless possibly because "typeset -p" will output array
elements using $'...' quoting if there are certain non-printable
characters in the element?

>  One thing: it sure is hard to hang on to blank lines.  I wish there was some option to default to preserving them.

Again this is not making much sense.  There are only two cases:
Strings with newline characters in them, and array elements where a
blank line might become an empty element if you got there by splitting
a string.  Empty elements generally get removed from argument lists if
you reference the array without proper quoting, especially if you're
forcing an additional layer of interpretation by using "eval", but
they won't be removed from the array itself.

Re: triviality regarding $# counts

[Ray Andrews]

On 2024-04-12 18:33, Mark J. Reed wrote:
> More context would clarify what you're trying to do and how best to do 
> it. Without that context, the approach you're currently taking feels 
> deeply weird. Can you not provide sample input and desired output? 
> Maybe with all of your code rather than a few snippets from "so deep 
> they need eval", whatever that means...
Context would be very difficult, trust me.  You'd hafta run the thing.  
But yes, it could all be deeply weird, I have nothing to build from but 
my own notions of how things should work.

Re: triviality regarding $# counts

[Lawrence Velázquez]

On Fri, Apr 12, 2024, at 10:28 PM, Ray Andrews wrote:
> On 2024-04-12 18:33, Mark J. Reed wrote:
>> More context would clarify what you're trying to do and how best to do 
>> it. Without that context, the approach you're currently taking feels 
>> deeply weird. Can you not provide sample input and desired output? 
>> Maybe with all of your code rather than a few snippets from "so deep 
>> they need eval", whatever that means...
> Context would be very difficult, trust me.  You'd hafta run the thing.  
> But yes, it could all be deeply weird, I have nothing to build from but 
> my own notions of how things should work.

We all understand that it can be difficult to boil a problem down
to a tractable, self-contained example.  But the alternative you've
chosen -- presenting baffling examples without necessary context,
describing your debugging attempts in vague terms without concrete
details, and inventing deeply bogus theories instead of consulting
the documentation first -- is making it difficult for us to help.

-- 
vq

Re: triviality regarding $# counts

[Ray Andrews]

[-- Attachment #1: Type: text/plain, Size: 1376 bytes --]



On 2024-04-12 20:25, Lawrence Velázquez wrote:
> We all understand that it can be difficult to boil a problem down
> to a tractable, self-contained example.  But the alternative you've
> chosen -- presenting baffling examples without necessary context,
> describing your debugging attempts in vague terms without concrete
> details, and inventing deeply bogus theories instead of consulting
> the documentation first -- is making it difficult for us to help.
On the contrary, your help has been most effective.  I wouldn't know how 
to expand the context without expanding it 'all the way out' and that's 
more context than you want, trust me.  And sorry about the deeply bogus 
theories, I know they're deeply bogus but I'm a pattern seeking animal 
and I'll attempt to model what I think is going on knowing full well 
that I'm usually wrong.

Bart:

>  I have no idea where you think dollar signs are
coming from ... unless possibly because "typeset -p" will output array
elements using $'...' quoting if there are certain non-printable
characters in the element?



I dunno.  I've lost the example that I saved. Never mind, I'll keep an eye on it, and figure out next time. Red herring for now.  It's a bit confusing looking into typeset -p output. All this 'splitting' stuff is not simple. For now everything works and I'm quite sure it's a bit more orthodox.

    

[-- Attachment #2: Type: text/html, Size: 1924 bytes --]

Re: triviality regarding $# counts

[Ray Andrews]

[-- Attachment #1: Type: text/plain, Size: 2719 bytes --]

> I dunno.  I've lost the example that I saved. Never mind, I'll keep an eye on it, and figure out next time. Red herring for now.  It's a bit confusing looking into typeset -p output. All this 'splitting' stuff is not simple. For now everything works and I'm quite sure it's a bit more orthodox.
>
>     
A script:
---------------------------------------------------------------------------------

redline () { echo -e "$red$@$nrm" }

typeset -ga aaa=("
abc

     def ghi
     jkl mno

     pqr
")

redline "\naaa:"
print -l $aaa
redline "\naaa[1]:"
print -l $aaa[1]
redline "linecount of aaa is: $#aaa ... Single element!"

ccc=( ${(@f)aaa} )

redline '\nccc=( ${(@f)aaa} )'
print -l $ccc
redline "linecount of ccc is: $#ccc ... split, but blank lines gone."

ddd=( "${(@f)aaa}" )

redline '\nddd=( "${(@f)aaa}" ) ... it seems like a lot of trouble to 
copy the array as it is.'
print -l "$ddd[@]"  #... don't forget the quotes, idiot!"
redline "linecount of ddd is: $#ddd ... split, but blanks preserved and 
counted. NB EIGHT NOT SIX!"

redline "\nddd[1]: That's right it's blank, just as it should be."
print -l $ddd[1]
redline "\nddd[2]:"
print -l $ddd[2]

redline "And now the typesets\n"
typeset -p aaa
typeset -p ccc
typeset -p ddd

# Verbatim:

# typeset -a aaa=( $'\nabc\n\n\tdef ghi\n\tjkl mno\n\n\tpqr\n' )
# typeset -a ccc=( abc $'\tdef ghi' $'\tjkl mno' $'\tpqr' )
# typeset -a ddd=( '' abc '' $'\tdef ghi' $'\tjkl mno' '' $'\tpqr' '' )

# Aligned: Note the steeenking dollars ;-) ... and how they seem to 
replace the newlines. No?

# typeset -a aaa=( $'\nabc  \n\n\tdef ghi  \n\tjkl mno \n\n\tpqr\n' 
)     #Original with blanks
# typeset -a ccc=(     abc    $'\tdef ghi' $'\tjkl mno   ' $'\tpqr' )  
          #No blanks.
# typeset -a ddd=(  '' abc '' $'\tdef ghi' $'\tjkl mno' '' $'\tpqr' '' 
)    #Blanks retained BUT not the same structure!

-----------------------------------------------------------------------------------

... so what about the dollars?

Anyway, it comes clear: '${(@f)....}' doesn't ADD (or retain) newlines 
it removes them!  (Tho splitting the array where they used to be.) 
Doesn't need them cuz elements aren't separated by newlines (like I 
thought) but by ... however it's done.  Single quotes indicate 
elements.  And, best of all, copying an array, blanks and all, (not 
shown above) turns out to be simple:

eee=( $aaa ) and:

eee=( "{$(@f)aaa}" )

... turns out to be a false equivalence where newlines are REMOVED. 
Don't be fooled by 'printf' -- it might not show you the genuine state 
of the array.  Scalars show their blank lines automatically but we need:

print -l -- "$eee[@]"

... with arrays.  Trust typedef -p.


[-- Attachment #2: Type: text/html, Size: 3679 bytes --]

Re: triviality regarding $# counts

[Mark J. Reed]

[-- Attachment #1: Type: text/plain, Size: 7123 bytes --]

Hm.

*typeset* -ga aaa=(*"*

*abc*


*    def ghi*

*    jkl mno*


*    pqr*

*"*)

This makes *aaa* an array with a grand total of 1 element. Then you do this:

redline *"linecount of aaa is: *$#aaa* ... Single element!"*


Your prefix text is wrong; calling  *$# *a "linecount" is a gross
mischaracterization. It never ever ever counts lines.  Never. It counts
characters on a scalar variable, elements of an array.  Lines have nothing
to do with it.

There are various transformations you can do that will map between lines on
the one hand and array elements on the other, but that's always some
command or special expansion flag doing an explicit transformation of the
data - which is to say, changing it. Inside an array there is no concept of
"lines" anywhere. By the same token, a piece of text containing newlines is
not inherently any sort of array at all. (Well, zsh lets you treat a
non-array variable as an array of individual characters, so in that sense
it is an array, but each character is its own element in that case, and
newlines are not in any way special.)

After your above declaration, the variable *aaa* is an array. The
parentheses in the assignment would make it one even if you hadn't
done *typeset
-a*. Unless you un-array-ify the varaible, *$#aaa* will always be *the
number of elements in the array*. It doesn't matter whether any of those
elements contain newlines or not.  Your initialization has exactly one
element, so *$#aaa* is 1. End of story.

You display things with *print -l* *$array* a lot; that's one of those
transformations I mentioned above. When you do that to an array (and *printf
'%\n' "${array[@]}"* does much the same thing in a POSIX-standard way), you
will get output that *starts *each element of the array on a separate line.
But that's all it does. Any newlines inside any of the array elements still
show up as newlines in the output. So there is no guarantee that e.g. *print
-l $ary | wc -l * will produce the same number as *$#ary*.  In your
*aaa* example,
*print -l $aaa *gets you 8 lines. But it's still a single-element array.
There is no *$aaa[put anything in here]* expression that will isolate any
of the lines of text; your only option is the whole thing, because it's all
one element.  (You can use the "a string is an array of characters" trick
to get substrings of that one element, e.g. *${aaa[1][2,4]}* is "abc".  But
again, the inner array is of *characters**, *not lines.)

Then you do this, using another transformation: the *f* expansion flag:

ccc=( ${(@f)aaa} )


By not putting double quotes around the expansion of aaa, you are throwing
away all the blank lines in its expansion. Since this is followed by a
version using the quotes, I assume that's intentional. But as far as I can
tell, the lack of quotation marks also removes any point for including the
*@* flag, since that flag's job is to maintain the identity of the separate
elements of an array when the expansion is quoted.  If it does anything at
all when not quoted, I'm unaware of it, but I admit that I could just be
ignorant of some subtlety.

ddd=( *"*${(@f)aaa}*"* )


Now you're keeping the blank lines, so the (f) gives you a separate array
element for every line that you get from the expansion - 8 of them, the
same as the output of *print -l $aaa | wc -l*.


redline *'\nddd=( "${(@f)aaa}" ) ... it seems like a lot of trouble to copy
the array as it is.'*


That comment confuses me; you aren't copying the array as it is at all!
You've gone from *aaa, *which has one element, to *ddd *having 8 elements.
That's a far cry from "copying as it is".

Your next comment says "8 not 6". I assume the expected 6 would be the
middle lines, without the leading and trailing blank ones, but I know of no
expansion that will produce that. You can get 6 elements out of *aaa*, like
this:

*eee=( $=aaa )*


But it gets there by throwing away all space, including newlines.  So the
six elements will be simply "abc", "def", "ghi", "jkl", "mno", and "pqr"


On Sat, Apr 13, 2024 at 11:14 AM Ray Andrews <rayandrews@eastlink.ca> wrote:

> I dunno.  I've lost the example that I saved. Never mind, I'll keep an eye on it, and figure out next time. Red herring for now.  It's a bit confusing looking into typeset -p output. All this 'splitting' stuff is not simple. For now everything works and I'm quite sure it's a bit more orthodox.
>
>
>
> A script:
>
> ---------------------------------------------------------------------------------
>
> redline () { echo -e "$red$@$nrm" }
>
> typeset -ga aaa=("
> abc
>
>     def ghi
>     jkl mno
>
>     pqr
> ")
>
> redline "\naaa:"
> print -l $aaa
> redline "\naaa[1]:"
> print -l $aaa[1]
> redline "linecount of aaa is: $#aaa ... Single element!"
>
> ccc=( ${(@f)aaa} )
>
> redline '\nccc=( ${(@f)aaa} )'
> print -l $ccc
> redline "linecount of ccc is: $#ccc ... split, but blank lines gone."
>
> ddd=( "${(@f)aaa}" )
>
> redline '\nddd=( "${(@f)aaa}" ) ... it seems like a lot of trouble to copy
> the array as it is.'
> print -l "$ddd[@]"  #... don't forget the quotes, idiot!"
> redline "linecount of ddd is: $#ddd ... split, but blanks preserved and
> counted. NB EIGHT NOT SIX!"
>
> redline "\nddd[1]: That's right it's blank, just as it should be."
> print -l $ddd[1]
> redline "\nddd[2]:"
> print -l $ddd[2]
>
> redline "And now the typesets\n"
> typeset -p aaa
> typeset -p ccc
> typeset -p ddd
>
> # Verbatim:
>
> # typeset -a aaa=( $'\nabc\n\n\tdef ghi\n\tjkl mno\n\n\tpqr\n' )
> # typeset -a ccc=( abc $'\tdef ghi' $'\tjkl mno' $'\tpqr' )
> # typeset -a ddd=( '' abc '' $'\tdef ghi' $'\tjkl mno' '' $'\tpqr' '' )
>
> # Aligned: Note the steeenking dollars ;-) ... and how they seem to
> replace the newlines. No?
>
> # typeset -a aaa=( $'\nabc  \n\n\tdef ghi  \n\tjkl mno   \n\n\tpqr\n'
> )     #Original with blanks
> # typeset -a ccc=(     abc    $'\tdef ghi' $'\tjkl mno   ' $'\tpqr' )
>          #No blanks.
> # typeset -a ddd=(  '' abc '' $'\tdef ghi' $'\tjkl mno' '' $'\tpqr' ''
> )    #Blanks retained BUT not the same structure!
>
>
> -----------------------------------------------------------------------------------
>
> ... so what about the dollars?
>
> Anyway, it comes clear: '${(@f)....}' doesn't ADD (or retain) newlines it
> removes them!  (Tho splitting the array where they used to be.) Doesn't
> need them cuz elements aren't separated by newlines (like I thought) but by
> ... however it's done.  Single quotes indicate elements.  And, best of all,
> copying an array, blanks and all, (not shown above) turns out to be simple:
>
> eee=( $aaa ) and:
>
> eee=( "{$(@f)aaa}" )
>
> ... turns out to be a false equivalence where newlines are REMOVED.  Don't
> be fooled by 'printf' -- it might not show you the genuine state of the
> array.  Scalars show their blank lines automatically but we need:
>
> print -l -- "$eee[@]"
>
> ... with arrays.  Trust typedef -p.
>
>
>

-- 
Mark J. Reed <markjreed@gmail.com>

[-- Attachment #2: Type: text/html, Size: 17714 bytes --]

Re: triviality regarding $# counts

[Mark J. Reed]

[-- Attachment #1: Type: text/plain, Size: 847 bytes --]

Small followup I meant to type but didn't:

On Sat, Apr 13, 2024 at 1:19 PM Mark J. Reed <markjreed@gmail.com> wrote:

> redline *'\nddd=( "${(@f)aaa}" ) ... it seems like a lot of trouble to
> copy the array as it is.'*
>
>
> That comment confuses me; you aren't copying the array as it is at all!
> You've gone from *aaa, *which has one element, to *ddd *having 8
> elements. That's a far cry from "copying as it is".
>

If you want to copy an array as it is, this is all you need:

*fff=("${(@)aaa}")*


Or, if you prefer the POSIX syntax:

*fff=("${aaa[@]}")*


After either of the above assignments, *fff* is an exact copy of *aaa*,
including any empty elements.

(If there aren't any empty elements or you want to strip them, you can use
the even simpler *fff=($aaa)*.)
-- 
Mark J. Reed <markjreed@gmail.com>

[-- Attachment #2: Type: text/html, Size: 2477 bytes --]

Re: triviality regarding $# counts

[Ray Andrews]

[-- Attachment #1: Type: text/plain, Size: 1188 bytes --]



On 2024-04-13 10:27, Mark J. Reed wrote:
> Small followup I meant to type but didn't:
>
> On Sat, Apr 13, 2024 at 1:19 PM Mark J. Reed <markjreed@gmail.com> wrote:
>
>         redline *'\nddd=( "${(@f)aaa}" ) ... it seems like a lot of
>         trouble to copy the array as it is.'*
>
>
>     That comment confuses me; you aren't copying the array as it is at
>     all! You've gone from *aaa, *which has one element, to *ddd
>     *having 8 elements. That's a far cry from "copying as it is".
>
Ah, but Mark, I figured all that out by the bottom of the post.  At that 
point I was still laboring under 'visual thinking' .  I dare say I'm 
straightened out.  I had thought that 'split on newlines' meant 'add 
\n's where needed to demarcate element boundaries'.  It's actually sorta 
the opposite -- there is no demarcation character and \n's will in fact 
be removed.  But I'm still not happy with the dollars.  I had thought 
that "  $'...'  "  was grammatical but we have " ' ' "  -- meaning an 
empty line -- without the leading dollar, so the closest thing I could 
pattern was that they replaced the newlines.   If anyone on the planet 
can get it wrong, it's me.


[-- Attachment #2: Type: text/html, Size: 3114 bytes --]

Re: triviality regarding $# counts

[Bart Schaefer]

On Sat, Apr 13, 2024 at 11:08 AM Ray Andrews <rayandrews@eastlink.ca> wrote:
>
> But I'm still not happy with the dollars.  I had thought that "  $'...'  "  was grammatical but we have " ' ' "  -- meaning an empty line

There are three* forms of quoting:

1) "X" -- expand variable references and other substitutions in X, but
do not split on $IFS **
2) 'X' -- do not do any expansions, X is literal
3) $'X' -- interpret \n \t \e etc., but everything else is literal, as
is the final result

This is all explained in the manual section "Quoting".  "typeset -p"
will output either no quoting at all, or #2, or #3, depending on which
is most appropriate to the value of the parameter, such that the
entire output is printable without the possibility of events like
terminals interpreting escape sequences.  If the parameter is an
array, "typeset -p" may output some elements in #2 form and others in
#3 form depending on the value of each element.

Note that because #3 is quoting, not substitution, it does not expand
inside #1.  Conversely, backticks `X` only look like quoting, they are
actually substitution equivalent to $(X) except without nesting, so
they do execute a command when appearing inside double quotes (#1).

* Four if you count using a backslash to protect a single following character.
** $@ and $ary[@] and ${(@)ary} get special handling, their array-ness
is preserved with each element quoted separately.

Let's look at your "redline" function:

redline () { echo -e "$red$@$nrm" }

The special handling of $@ means that for example:

redline a b c

is interpreted as

echo -e "$red$1" "$2" "$3$nrm"

So you probably should have written

redline () { echo -e "$red$*$nrm" }

because $* is the same as $@ except without that special
interpretation inside double quotes.

Re: triviality regarding $# counts

[Mark J. Reed]

[-- Attachment #1: Type: text/plain, Size: 5144 bytes --]

There still seems to be some confusion, so I'm going to back up a bit.
Apologies in advance if I tell you something you already know.

Quotes are part of the shell syntax. They aren't part of the quoted value.
If you do this:

name='Ray'


then the value stored into the variable *name* is just 3 characters: *R*,
*a*, and *y*.  There aren't any quotes anymore; they were just there long
enough to tell the shell that you want the characters in between them to be
taken literally. In this simple case, they aren't even needed; this has
exactly the same result:

name=Ray


It's important to keep the distinction in mind between the *values* you're
working with and the particular way you represent those values in the
*syntax* of the language.

Here's an example where the quotes are actually required:

multiline='hello,
there'


The above command is two physical lines of text, but only one shell
command, and that command is assigning one string value. The result is not
an array; it's a single, scalar parameter containing 12 characters, one of
which happens to be a newline:  *h*, *e*, *l*, another* l*, *o*, a comma, a
newline, *t*, *h*, *e*, *r*, and finally another *e*. You can access all of
them with subscripts; *$multiline[1]* is *h*, *$multiline[2]* is *e*, etc.
The quotes, despite being required, are still not part of the value.

The ability to do this also highlights the fact that newline is just
another character. Just like a letter of the alphabet or a digit or a
punctuation mark, it's just a number, but in the context of text that
number is interpreted as a character. The number 65 means *A*, 44 means
comma, 48 means *0*, and 10 means newline.  After the above assignment, the
newline may be found at *$multiline[7]*.

Here's another way of assigning exactly the same value to the same variable:

multiline=$'hello,\nthere'

Here we have a single-line command assigning a multiline value. The ANSI
quotes *$'*...*' *let us use the ANSI C escape sequence *\n* to mean a
newline. The result is the same: the value of *multline* is 12 characters
long, and *$multiline[7] *is a newline.

This assignment gets you something completely different:

oneline='hello,\nthere'


Now there is no newline character anywhere in the string. It's 13
characters long, not 12; *$oneline[7] *is not a newline but a backslash,
and *$oneline[8]* is an *n*.

After the above pair of assignments, you might be confused by the fact
that *echo
$multiline *and *echo $oneline* produce the same result, but that's because
*echo* is doing the same kind of translation of ANSI C escapes that the
shell does when you use *$'*...*'* quotes. If you turn that off by using *echo
-E*, then you'll get two different answers:

$ echo -E "$multiline"

hello,

there

$ echo -E "$oneline"

hello,\nthere


(I used double quotes above out of long habit; they're not strictly
necessary, especially not in zsh.)

A pair of quotes – of any variety, single, double, or ANSI – with nothing
between them represents the empty string. The empty string is nothing at
all; if you print it out by itself, it's the same as not even executing the
print command. Despite this, it is sometimes a useful thing to store in a
variable:

empty=''


After the above assignment, *$#empty* is 0. It literally contains nothing.

Again, you don't actually need the quotes; this also works:

empty=


Which reinforces the role of the quotes as syntax rather than value.

And again, an empty string is not an empty *line. *If, when you print out
the empty string, you do so with a command that adds a newline at the end,
the *output* will be an empty line, but it's the printing that makes it a
line. The value inside the variable has no line-ness.

On Sat, Apr 13, 2024 at 2:08 PM Ray Andrews <rayandrews@eastlink.ca> wrote:

> On 2024-04-13 10:27, Mark J. Reed wrote:
>
> Small followup I meant to type but didn't:
>
> On Sat, Apr 13, 2024 at 1:19 PM Mark J. Reed <markjreed@gmail.com> wrote:
>
>> redline *'\nddd=( "${(@f)aaa}" ) ... it seems like a lot of trouble to
>> copy the array as it is.'*
>>
>>
>> That comment confuses me; you aren't copying the array as it is at all!
>> You've gone from *aaa, *which has one element, to *ddd *having 8
>> elements. That's a far cry from "copying as it is".
>>
> Ah, but Mark, I figured all that out by the bottom of the post.  At that
> point I was still laboring under 'visual thinking' .  I dare say I'm
> straightened out.  I had thought that 'split on newlines' meant 'add \n's
> where needed to demarcate element boundaries'.  It's actually sorta the
> opposite -- there is no demarcation character and \n's will in fact be
> removed.  But I'm still not happy with the dollars.  I had thought that "
> $'...'  "  was grammatical but we have " ' ' "  -- meaning an empty line --
> without the leading dollar, so the closest thing I could pattern was that
> they replaced the newlines.   If anyone on the planet can get it wrong,
> it's me.
>
>
>

-- 
Mark J. Reed <markjreed@gmail.com>

[-- Attachment #2: Type: text/html, Size: 11239 bytes --]

Re: triviality regarding $# counts

[Ray Andrews]

On 2024-04-13 12:45, Bart Schaefer wrote:
> On Sat, Apr 13, 2024 at 11:08 AM Ray Andrews <rayandrews@eastlink.ca> wrote:
>> But I'm still not happy with the dollars.  I had thought that "  $'...'  "  was grammatical but we have " ' ' "  -- meaning an empty line
> There are three* forms of quoting:
Many thanks, it's a brief page of explanation like this that would 
feature in my imaginary 'Idiot's Guide to Shells'.  It's not the sort of 
information you can go looking for  when you don't know it because you 
don't even know what you're looking for.  It must be brought to your 
attention.
> 1) "X" -- expand variable references and other substitutions in X, but
> do not split on $IFS **
> 2) 'X' -- do not do any expansions, X is literal
Those two I pretty much understand -- except IFS which I've learned not 
to touch.

> 3) $'X' -- interpret \n \t \e etc., but everything else is literal, as
> is the final result
This is new to me.  Never used it.  Looks useful for strings.
> This is all explained in the manual section "Quoting".  "typeset -p"
> will output either no quoting at all, or #2, or #3,
I remember chafing over that some years back.
> Note that because #3 is quoting, not substitution, it does not expand
> inside #1.  Conversely, backticks `X` only look like quoting, they are
> actually substitution equivalent to $(X) except without nesting, so
> they do execute a command when appearing inside double quotes (#1).
Heavy.
> * Four if you count using a backslash to protect a single following character.
Right.
> ** $@ and $ary[@] and ${(@)ary} get special handling, their array-ness
> is preserved with each element quoted separately.
That needs pondering.
> Let's look at your "redline" function:
> ...
> So you probably should have written
>
> redline () { echo -e "$red$*$nrm" }
>
> because $* is the same as $@ except without that special
> interpretation inside double quotes.
So, compared to the previous expansion, how would '$*' expand there?  
I'm unclear as to the practical  difference.

Anyway .... when my typedef -p line showed those "   ' '    " with no 
dollar -- which seemed to me to forbid that "   $'...'  " grammar -- 
what I should read is that that element has nothing quoted ergo no 
'controller' -- no expansion flag -- is needed.  Yes?  IOW the dollar 
might have been put there, but it has no work to do so is simply 
omitted.  Logical.  Do I have that right, for once?

Re: triviality regarding $# counts

[Bart Schaefer]

On Sat, Apr 13, 2024 at 8:14 AM Ray Andrews <rayandrews@eastlink.ca> wrote:
>
> Anyway, it comes clear: '${(@f)....}' doesn't ADD (or retain) newlines it removes them!  (Tho splitting the array where they used to be.) Doesn't need them cuz elements aren't separated by newlines (like I thought) but by ... however it's done.

(Liberties taken with historical accuracy below, do not feel compelled
to leap in with sequencing corrections, etc.)

In the beginning, there were only environment strings ($SHELL, $PATH,
etc.) and positional parameters ($1, $2, etc.).  When a command was
run, it got its arguments as an "array" of positional parameters.
This is how "variables" came to be called "parameters" in shell
jargon.  This mapped directly onto C main() argc ($#) and argv[] ($@)
with the phantom $0 as the name of the command.

Referencing an environment string or an individual positional
parameter without quoting in a command argument caused the value to be
split on whitespace (as defined by the value of $IFS) to convert it
into an argv array.  Putting it in double quotes protected it from
that splitting but still permitted it to be replaced by its value.
Single quotes were for literal strings.  To reference an entire argv
without further splitting but also without losing its array property,
"$@" got the special meaning to quote each element individually.  $*
would give the entire argv but also split it further on whitespace,
and "$*" gave the entire argv joined back together with with spaces to
make it a single string.

It follows from this that it's useful to be able to assign names to
strings other than environment strings and to arrays other than the
positional parameters.  Thus we get scalar parameters and array
parameters, but they still follow the rules corresponding to
environment strings and positional parameters including the $* and $@
references to the full arrays, except now the name is added as in
${ary[3]} and ${ary[*]} and ${ary[@]}.  From this point forward shells
began to vary on details like whether named arrays begin with index 0
or 1 and whether it was necessary to wrap the reference in braces and
(in zsh's case) whether splitting on whitespace happens by default.

The salient point here is that regardless of the internal
representation of an array, you can think of it like an argv[] in C.
The elements are separated by conceptually being different string
pointers, and anything you "see" from e.g. "typeset -p" is language
syntax, not representation.

When you use (f) and other syntax to split something, you're creating
(temporarily) that same kind of array.  It might collapse to a string
again almost immediately based on context, which is why you might need
e.g. (@f) to specifically preserve it, but it exists however briefly
in that form.

>  Single quotes indicate elements.

Thus, that's an effect of "typeset -p" rather than anything about the
internal representation.

Re: triviality regarding $# counts

[Bart Schaefer]

On Sat, Apr 13, 2024 at 1:36 PM Ray Andrews <rayandrews@eastlink.ca> wrote:
>
> On 2024-04-13 12:45, Bart Schaefer wrote:
> > redline () { echo -e "$red$*$nrm" }
> >
> So, compared to the previous expansion, how would '$*' expand there?

redline a b c

would mean

echo -e "$red$1 $2 $3$nrm"

One word, with two quoted spaces, instead of three words.

> I'm unclear as to the practical  difference.

For "echo" it doesn't matter because echo is also going to paste its
arguments together, but it might matter to some other command.

> Anyway .... when my typedef -p line showed those "   ' '    " with no
> dollar -- which seemed to me to forbid that "   $'...'  " grammar --
> what I should read is that that element has nothing quoted ergo no
> 'controller' -- no expansion flag -- is needed.  Yes?  IOW the dollar
> might have been put there, but it has no work to do so is simply
> omitted.  Logical.  Do I have that right, for once?

You do, though I would caution not to think of the $ as an "expansion
flag" -- it's just part of the quoting syntax.

Re: triviality regarding $# counts

[Ray Andrews]

[-- Attachment #1: Type: text/plain, Size: 711 bytes --]



On 2024-04-13 13:53, Bart Schaefer wrote:
> On Sat, Apr 13, 2024 at 8:14 AM Ray Andrews<rayandrews@eastlink.ca>  wrote:
> In the beginning, there were only environment strings ($SHELL, $PATH,
> etc.) and positional parameters ($1, $2, etc.).  When a command was
> run, it got its arguments as an "array" of positional parameters.
> This is how "variables" came to be called "parameters" in shell
> jargon.  This mapped directly onto C main() argc ($#) and argv[] ($@)
> with the phantom $0 as the name of the command.
>
Some might say it's esoteric trivia but I disagree.  When you elucidate 
the history, and the mapping onto C, things make more sense.  The shells 
had to grow organically, didn't they?

[-- Attachment #2: Type: text/html, Size: 1282 bytes --]

Re: triviality regarding $# counts

[Ray Andrews]

[-- Attachment #1: Type: text/plain, Size: 534 bytes --]



On 2024-04-13 14:01, Bart Schaefer wrote:
> redline a b c
> would mean
>
> echo -e "$red$1 $2 $3$nrm"
>
> One word, with two quoted spaces, instead of three words.
That simple example makes it trivially easy to understand.  I remember 
trying to figure that out previously via the manual and I just got more 
confused.  There's no merit to obfuscation tho you'd not know that 
reading some Linux/Unix man pages.  It's a critical lesson: what you see 
is not what you get, you hafta understand the hidden structure of your 
data.

[-- Attachment #2: Type: text/html, Size: 991 bytes --]

Re: triviality regarding $# counts

[Lawrence Velázquez]

On Sat, Apr 13, 2024, at 5:01 PM, Bart Schaefer wrote:
> On Sat, Apr 13, 2024 at 1:36 PM Ray Andrews <rayandrews@eastlink.ca> wrote:
>>
>> On 2024-04-13 12:45, Bart Schaefer wrote:
>> > redline () { echo -e "$red$*$nrm" }
>> >
>> So, compared to the previous expansion, how would '$*' expand there?
>
> redline a b c
>
> would mean
>
> echo -e "$red$1 $2 $3$nrm"
>
> One word, with two quoted spaces, instead of three words.
>
>> I'm unclear as to the practical  difference.
>
> For "echo" it doesn't matter because echo is also going to paste its
> arguments together, but it might matter to some other command.

A different output method should make this clearer:

	% f1() { printf '<%s>' "x$@x"; echo; }
	% f2() { printf '<%s>' "x$*x"; echo; }
	% f1 a b c
	<xa><b><cx>
	% f2 a b c
	<xa b cx>

The command "printf '<%s>' ..." outputs each argument enclosed in
"<...>" (other than the format string itself).  Observe that
double-quoted $@ can expand to multiple words, while double-quoted
$* always expands to one.

This actually makes a difference to nearly every command you might
think of, with "echo" being a rare exception (and even then, only
when the first character of IFS is a space).

	% arr=(.zshenv .zprofile .zshrc)
	% set -x
	% wc -c "$arr[@]"
	+zsh:44> wc -c .zshenv .zprofile .zshrc
	    1063 .zshenv
	    1023 .zprofile
	    3807 .zshrc
	    5893 total
	% wc -c "$arr[*]"
	+zsh:45> wc -c '.zshenv .zprofile .zshrc'
	wc: .zshenv .zprofile .zshrc: open: No such file or directory


-- 
vq

Re: triviality regarding $# counts

[Ray Andrews]

[-- Attachment #1: Type: text/plain, Size: 299 bytes --]



On 2024-04-13 17:30, Lawrence Velázquez wrote:
> A different output method should make this clearer:

Yes, thanks.  Up till now I've basically ignored $* completely. Gotta 
give it another look.  Man, the things I don't know I don't know ... not 
to mention the things I do know ... wrongly.



[-- Attachment #2: Type: text/html, Size: 718 bytes --]

Re: triviality regarding $# counts

[Lawrence Velázquez]

On Sat, Apr 13, 2024, at 11:26 PM, Ray Andrews wrote:
> On 2024-04-13 17:30, Lawrence Velázquez wrote:
>> A different output method should make this clearer: 
> Yes, thanks.  Up till now I've basically ignored $* completely.

In a sense, you've been using it all along.  Given an array "arr",
$arr is the same as $arr[*] (by default), which is why double-quoting
it produces a single word.

	% arr=(a '' b 'c   d' '' e)
	% printf '<%s>' $arr; echo
	<a><b><c   d><e>
	% printf '<%s>' $arr[*]; echo
	<a><b><c   d><e>
	% printf '<%s>' $arr[@]; echo
	<a><b><c   d><e>
	% printf '<%s>' ${(@)arr}; echo
	<a><b><c   d><e>
	% printf '<%s>' "$arr"; echo
	<a  b c   d  e>
	% printf '<%s>' "$arr[*]"; echo
	<a  b c   d  e>
	% printf '<%s>' "$arr[@]"; echo
	<a><><b><c   d><><e>
	% printf '<%s>' "${(@)arr}"; echo
	<a><><b><c   d><><e>

-- 
vq

Re: triviality regarding $# counts

[Bart Schaefer]

On Sat, Apr 13, 2024 at 8:49 PM Lawrence Velázquez <larryv@zsh.org> wrote:
>
> In a sense, you've been using it all along.  Given an array "arr",
> $arr is the same as $arr[*] (by default), which is why double-quoting
> it produces a single word.

This puts me in mind of a couple of other details.

Because zsh does not split parameter expansions by default, $ary and
$ary[*] and $ary[@] are equivalent when not quoted, as Lawrence
demonstrated.

However, in all three of those cases, empty elements are typically
discarded, as they would be in a shell that defaults to splitting.
That's why "$ary[@]" is still useful:  It quotes each element
individually, so empty elements are preserved.

(Aside, in bash and ksh, $ary is equivalent to ${ary[0]}, not to
${ary[*]}.  A little oddly, this is also true for their associative
arrays, that is, the value for key 0 is used.)

Re: triviality regarding $# counts

[Ray Andrews]

[-- Attachment #1: Type: text/plain, Size: 2967 bytes --]



On 2024-04-13 21:57, Bart Schaefer wrote:
> Because zsh does not split parameter expansions by default, $ary and
> $ary[*] and $ary[@] are equivalent when not quoted, as Lawrence
> demonstrated.
>
> However, in all three of those cases, empty elements are typically
> discarded, as they would be in a shell that defaults to splitting.
> That's why "$ary[@]" is still useful:  It quotes each element
> individually, so empty elements are preserved.
>
> (Aside, in bash and ksh, $ary is equivalent to ${ary[0]}, not to
> ${ary[*]}.  A little oddly, this is also true for their associative
> arrays, that is, the value for key 0 is used.)
I know it's far too late to fret this stuff, still it's interesting to 
contemplate these design level issues.  P.F. did the right thing not 
splitting by default but he should have left the empties in by default 
too, IMHO.  Options should be positive not negative as a fundamental 
principle.  Don't do helpful things automatically, and force me to stop 
you from doing them.  Much better to leave things alone, and if I want, 
say, empty elements removed, I'll ask for them to be removed.  A=B 
should mean that A is identical to B, not B with blanks removed, nor 
sorted, nor capitalized, nor duplicates removed, nor checked for 
spelling, nor ... nothing.

"$ary[@]" should be the default with some ${(?)...} flag to indicate 
that I'd like blanks removed or  duplicates or ... whatever massaging 
I'd want.  Cool that all those manipulations are available of course, 
but as Raymond Sensei  (E. S. Raymond) explains so well, the 'doctrine 
of least surprise' should prevail.  Equals equals equals, not 'equals 
minus blanks'.  So we have:

aaa=( "${(@f)bbb}" )

... which is code fighting itself.  The quotes normally mean 'join 
together' but they also mean 'preserve spaces' and so then '(@f)' 
contradicts the joining so we retain multiple elements.  IMHO it should 
look like this:

aaa=( $bbb )

Equals equals equals.

But as we have  it:

----------------------------------------------------------------------------------

% print $ddd
abc     def ghi     jkl mno     pqr

% fff=( $ddd )

% print $fff
abc     def ghi     jkl mno     pqr

# Looks ok, donit? But don't be a chump:

% typeset -p ddd; typeset -p fff
typeset -a ddd=( '' abc '' $'\tdef ghi' $'\tjkl mno' '' $'\tpqr' '' )
typeset -a fff=( abc $'\tdef ghi' $'\tjkl mno' $'\tpqr' )

# It's not a zebra, it's a horse with painted stripes!
# If you want a COPY ... yes, a Xerox copy:

% fff=( "${(@f)ddd}" )

% typeset -p ddd; typeset -p fff
typeset -a ddd=( '' abc '' $'\tdef ghi' $'\tjkl mno' '' $'\tpqr' '' )
typeset -a fff=( '' abc '' $'\tdef ghi' $'\tjkl mno' '' $'\tpqr' '' )

# That's a COPY ... and don't even think about a simpler way, we are on 
this earth to suffer.

------------------------------------------------------------------------------------

Just sayin', I know there's nothing to be done about it now.



[-- Attachment #2: Type: text/html, Size: 4000 bytes --]

Re: triviality regarding $# counts

[Roman Perepelitsa]

On Sun, Apr 14, 2024 at 3:25 PM Ray Andrews <rayandrews@eastlink.ca> wrote:
>
> # If you want a COPY ... yes, a Xerox copy:
>
> % fff=( "${(@f)ddd}" )
>
> # That's a COPY ... and don't even think about a simpler way, we are on this earth to suffer.

That won't make a copy if any of the ddd elements have a newline in
them. Recall what (f) does.

Roman.

Re: triviality regarding $# counts

[Ray Andrews]

On 2024-04-14 06:35, Roman Perepelitsa wrote:
> On Sun, Apr 14, 2024 at 3:25 PM Ray Andrews <rayandrews@eastlink.ca> wrote:
>> # If you want a COPY ... yes, a Xerox copy:
>>
>> % fff=( "${(@f)ddd}" )
>>
>> # That's a COPY ... and don't even think about a simpler way, we are on this earth to suffer.
> That won't make a copy if any of the ddd elements have a newline in
> them. Recall what (f) does.
But:

% ggg=( 'abc\n' '\n' 'def\n' '\n' 'ghi\n' )

% hhh=( "${(@f)ggg}" )

% typeset -p hhh; typeset -p ggg
typeset -a hhh=( 'abc\n' '\n' 'def\n' '\n' 'ghi\n' )
typeset -a ggg=( 'abc\n' '\n' 'def\n' '\n' 'ghi\n' )

... nuts yesterday I thought I had it figured out that the newlines 
would be removed but look at the above.

Re: triviality regarding $# counts

[Mark J. Reed]

[-- Attachment #1: Type: text/plain, Size: 4110 bytes --]

>
> # It's not a zebra, it's a horse with painted stripes!
> # If you want a COPY ... yes, a Xerox copy:
> % fff=( "${(@f)ddd}" )


What? No.  Why do you keep bringing (f) into this? That flag has nothing to
do with copying; in fact it intentionally changes things: anywhere there
used to be a single string containing a newline,you get two strings
instead.

But if you drop the f, you do get your Xerox copy:

% fff=( "${(@)ddd}" )

Would it be nicer if you could just do *fff=$ddd* and not have to include
the parens and quotes and @? Sure. You could even make a case that it *should
*work that way, since we're in Zshland where *$ddd* expands to the whole
array instead of a single element. But that's not the way assignment works.
Though, as I said, if the array has no empty elements, you can get away
with just *fff=($ddd)*.






On Sun, Apr 14, 2024 at 9:24 AM Ray Andrews <rayandrews@eastlink.ca> wrote:

>
>
> On 2024-04-13 21:57, Bart Schaefer wrote:
>
> Because zsh does not split parameter expansions by default, $ary and
> $ary[*] and $ary[@] are equivalent when not quoted, as Lawrence
> demonstrated.
>
> However, in all three of those cases, empty elements are typically
> discarded, as they would be in a shell that defaults to splitting.
> That's why "$ary[@]" is still useful:  It quotes each element
> individually, so empty elements are preserved.
>
> (Aside, in bash and ksh, $ary is equivalent to ${ary[0]}, not to
> ${ary[*]}.  A little oddly, this is also true for their associative
> arrays, that is, the value for key 0 is used.)
>
> I know it's far too late to fret this stuff, still it's interesting to
> contemplate these design level issues.  P.F. did the right thing not
> splitting by default but he should have left the empties in by default too,
> IMHO.  Options should be positive not negative as a fundamental principle.
> Don't do helpful things automatically, and force me to stop you from doing
> them.  Much better to leave things alone, and if I want, say, empty
> elements removed, I'll ask for them to be removed.  A=B should mean that A
> is identical to B, not B with blanks removed, nor sorted, nor capitalized,
> nor duplicates removed, nor checked for spelling, nor ... nothing.
>
> "$ary[@]" should be the default with some ${(?)...} flag to indicate that
> I'd like blanks removed or  duplicates or ... whatever massaging I'd want.
> Cool that all those manipulations are available of course, but as Raymond
> Sensei  (E. S. Raymond) explains so well, the 'doctrine of least surprise'
> should prevail.  Equals equals equals, not 'equals minus blanks'.  So we
> have:
>
> aaa=( "${(@f)bbb}" )
>
> ... which is code fighting itself.  The quotes normally mean 'join
> together' but they also mean 'preserve spaces' and so then '(@f)'
> contradicts the joining so we retain multiple elements.  IMHO it should
> look like this:
>
> aaa=( $bbb )
>
> Equals equals equals.
>
> But as we have  it:
>
>
> ----------------------------------------------------------------------------------
>
> % print $ddd
> abc     def ghi     jkl mno     pqr
>
> % fff=( $ddd )
>
> % print $fff
> abc     def ghi     jkl mno     pqr
>
> # Looks ok, donit? But don't be a chump:
>
> % typeset -p ddd; typeset -p fff
> typeset -a ddd=( '' abc '' $'\tdef ghi' $'\tjkl mno' '' $'\tpqr' '' )
> typeset -a fff=( abc $'\tdef ghi' $'\tjkl mno' $'\tpqr' )
>
> # It's not a zebra, it's a horse with painted stripes!
> # If you want a COPY ... yes, a Xerox copy:
>
> % fff=( "${(@f)ddd}" )
>
> % typeset -p ddd; typeset -p fff
> typeset -a ddd=( '' abc '' $'\tdef ghi' $'\tjkl mno' '' $'\tpqr' '' )
> typeset -a fff=( '' abc '' $'\tdef ghi' $'\tjkl mno' '' $'\tpqr' '' )
>
> # That's a COPY ... and don't even think about a simpler way, we are on
> this earth to suffer.
>
>
> ------------------------------------------------------------------------------------
>
> Just sayin', I know there's nothing to be done about it now.
>
>
>
>

-- 
Mark J. Reed <markjreed@gmail.com>

[-- Attachment #2: Type: text/html, Size: 6285 bytes --]

Re: triviality regarding $# counts

[Roman Perepelitsa]

On Sun, Apr 14, 2024 at 4:06 PM Ray Andrews <rayandrews@eastlink.ca> wrote:
>
> % ggg=( 'abc\n' '\n' 'def\n' '\n' 'ghi\n' )

There are no newlines in there. If this isn't clear, check the post by
Mark J. Reed in this thread where he elaborated on this topic in great
detail.

Roman.

Re: triviality regarding $# counts

[Ray Andrews]

[-- Attachment #1: Type: text/plain, Size: 1773 bytes --]



On 2024-04-14 07:06, Mark J. Reed wrote:
>
> What? No.  Why do you keep bringing (f) into this? That flag has 
> nothing to do with copying; in fact it intentionally changes things: 
> anywhere there used to be a single string containing a newline,you get 
> two strings instead.
% hhh=( "${(@f)iii}" )
% typeset -p hhh; typeset -p iii
typeset -a hhh=( 'abc\ndef\nghi' )
typeset iii='abc\ndef\nghi'

... newlines still there.  One element.


Besides it worked as I showed.
>
> But if you drop the f, you do get your Xerox copy:
>
>     % fff=( "${(@)ddd}" )
>
... so far so good with tests.  The (f) seems not to have removed 
anything -- I was sure it did, yesterday :( -- but that was then. But 
yours seems to work and it's simpler, more intuitive.  The (f) does not 
seem to be missed, either.  And to think that just yesterday I thought I 
had this all sorted.

> Would it be nicer if you could just do *fff=$ddd* and not have to 
> include the parens and quotes and @? Sure. You could even make a case 
> that it /should /work that way, since we're in Zshland where *$ddd* 
> expands to the whole array instead of a single element. But that's not 
> the way assignment works. Though, as I said, if the array has no empty 
> elements, you can get away with just *fff=($ddd)*.
'IF' ... what I'm wanting is some universal copy that has no gotchas.

BTW, just philosophically speaking, the parens are an interesting 
question.  Whereas I'd say that a copy is a copy is a copy:

aaa=$bbb

... so whatever bbb may happen to be, aaa will become the same.  But if 
aaa is a scalar, should it be promoted 'silently' ?  Seems to me the 
parens are explicit that aaa will become an array so I like them.  One 
can never be too clear as to what's happening.

cut


[-- Attachment #2: Type: text/html, Size: 3212 bytes --]

Re: triviality regarding $# counts

[Ray Andrews]

On 2024-04-14 07:15, Roman Perepelitsa wrote:
> On Sun, Apr 14, 2024 at 4:06 PM Ray Andrews <rayandrews@eastlink.ca> wrote:
>> % ggg=( 'abc\n' '\n' 'def\n' '\n' 'ghi\n' )
> There are no newlines in there.
% ggg=( ' abc\n' '\n' 'def\n' '\n' 'ghi\n' )

% print $ggg
  abc

  def

  ghi


% typeset -p ggg
typeset -a ggg=( ' abc\n' '\n' 'def\n' '\n' 'ghi\n' )

Re: triviality regarding $# counts

[Mark J. Reed]

[-- Attachment #1: Type: text/plain, Size: 1328 bytes --]

On Sun, Apr 14, 2024 at 10:48 Ray Andrews <rayandrews@eastlink.ca> wrote:

>
>
>
>
> % hhh=( "${(@f)iii}" )
> % typeset -p hhh; typeset -p iii
> typeset -a hhh=( 'abc\ndef\nghi' )
> typeset iii='abc\ndef\nghi'
>
> ... newlines still there.  One element.
>

No newlines nowhere! You've got backslashes and n's, but nary a newline to
be found. Which is the only reason (f) isn't completely mucking things up.
See my last message.

BTW, just philosophically speaking, the parens are an interesting
> question.  Whereas I'd say that a copy is a copy is a copy:
>
> aaa=$bbb
>
> ... so whatever bbb may happen to be, aaa will become the same.  But if
> aaa is a scalar, should it be promoted 'silently' ?  Seems to me the parens
> are explicit that aaa will become an array so I like them.  One can never
> be too clear as to what's happening.
>

The parens are needed because of the way assignment works. At least
conceptually (though the implementation may optimize), the right hand side
is first expanded, and then the result of that expansion, treated as a
literal value, is what gets assigned. The shell doesn't just copy the
innards of the variable value from one spot in its symbol table to another;
it takes a detour through first  expanding the value and then doing some
subset of its normal parsing on that expanded value.

[-- Attachment #2: Type: text/html, Size: 2296 bytes --]

Re: triviality regarding $# counts

[Mark J. Reed]

[-- Attachment #1: Type: text/plain, Size: 1807 bytes --]

As I said about *echo* way back in my first response: the *print* command
is what is turning the backslash-n sequences into newlines. If you use *-r* to
turn off that behavior, you can see the strings unmangled:

% print -r $ggg

 abc\n \n def\n \n ghi\n


The -r is telling print not to do something it normally does. The result is
the raw string (which I think is what *-r* stands for).

To restate: if your goal is to print out a newline, you have options. If
you do this:

print $'\n'

Then you are creating in the shell's memory a string containing a literal
newline character (one byte with value 10), which the shell then passes to
*print*, which just echoes it verbatim, doing no translation on it
whatsoever.  Job done. The translation from \n to newline was done by the
shell before *print* ever ran.

If you instead do this:

print '\n'

Then the string you create and pass to *print* has no newline in it. It is
not one byte of value 10, but two bytes, one with value 92 (backslash) and
the second with value 110 (lowercase *n*).  The code implementing the
*print* command recognizes this sequence as code for "print a newline", so
it does. But that's the command doing that. The string itself has no
newline in it.



Mark J. Reed <markjreed@gmail.com>


On Sun, Apr 14, 2024 at 10:53 Ray Andrews <rayandrews@eastlink.ca> wrote:

>
>
> On 2024-04-14 07:15, Roman Perepelitsa wrote:
> > On Sun, Apr 14, 2024 at 4:06 PM Ray Andrews <rayandrews@eastlink.ca>
> wrote:
> >> % ggg=( 'abc\n' '\n' 'def\n' '\n' 'ghi\n' )
> > There are no newlines in there.
> % ggg=( ' abc\n' '\n' 'def\n' '\n' 'ghi\n' )
>
> % print $ggg
>   abc
>
>   def
>
>   ghi
>
>
> % typeset -p ggg
> typeset -a ggg=( ' abc\n' '\n' 'def\n' '\n' 'ghi\n' )
>
>
>
>

[-- Attachment #2: Type: text/html, Size: 4423 bytes --]

Re: triviality regarding $# counts

[Bart Schaefer]

On Sun, Apr 14, 2024 at 6:24 AM Ray Andrews <rayandrews@eastlink.ca> wrote:
>
> I know it's far too late to fret this stuff, still it's interesting to contemplate these design level issues.  P.F. did the right thing not splitting by default but he should have left the empties in by default too, IMHO.
> [...]
> the 'doctrine of least surprise' should prevail.

The trouble with that "doctrine" is that what is surprising is
subjective and context-dependent both in usage and in time.  When
these decisions were made, the largest number of zsh adoptees would
have been more surprised by having empty elements kept than removed.
Mysteriously receiving error messages like

ls: : No such file or directory

would have been the surprising thing.  Further, many commands neither
understand nor do anything useful with empty string arguments, and zsh
was intended first to be an interactive command shell and only second
a programming language (just a better one than BSD csh).

Re: triviality regarding $# counts

[Ray Andrews]

[-- Attachment #1: Type: text/plain, Size: 1441 bytes --]



On 2024-04-14 08:11, Mark J. Reed wrote:
> As I said about *echo* way back in my first response: the 
> *print* command is what is turning the backslash-n sequences into 
> newlines. If you use *-r* to turn off that behavior, you can see the 
> strings unmangled:
>
>     % print -r $ggg
>
>     abc\n \n def\n \n ghi\n
>
>
> The -r is telling print not to do something it normally does. The 
> result is the raw string (which I think is what *-r* stands for).

Ah!  Threw me off the scent there.  So print is ... yes I get it. Ok, so 
that's what had me fooled.  So, one more thing and then I think I'll get 
back to straight and level:  How do we separate the elements?  I've 
always seen it with ticks.  Or ... now that I think ... with
$'...'
Yes?

If you instead do this:
>
>     print '\n'
>
> Then the string you create and pass to *print* has no newline in it. 
> It is not one byte of value 10, but two bytes, one with value 92 
> (backslash) and the second with value 110 (lowercase *n*).  The code 
> implementing the *print* command recognizes this sequence as code for 
> "print a newline", so it does. But that's the command doing that. The 
> string itself has no newline in it.

Ok, that blows the fog away.  We have the 'real' newline, we have the 
string '\n' ... which print helpfully translates into the former.  Yes, 
if in doubt 'print -r'.

Sheesh, I fall into every pothole there could possibly be.



[-- Attachment #2: Type: text/html, Size: 4115 bytes --]

Re: triviality regarding $# counts

[Ray Andrews]

[-- Attachment #1: Type: text/plain, Size: 1962 bytes --]



On 2024-04-14 08:51, Bart Schaefer wrote:
> On Sun, Apr 14, 2024 at 6:24 AM Ray Andrews<rayandrews@eastlink.ca>  wrote:
> The trouble with that "doctrine" is that what is surprising is
> subjective and context-dependent both in usage and in time.  When
> these decisions were made, the largest number of zsh adoptees would
> have been more surprised by having empty elements kept than removed.
> Mysteriously receiving error messages like
Absolutely.  It is subjective therefore a design decision, not anything 
like right or wrong.  I think the 'no negative options' rule is more 
objective tho.  Still, even recognizing the rule, there will be 
exceptions and maybe this is one of them.
> ls: : No such file or directory
>
> would have been the surprising thing.  Further, many commands neither
> understand nor do anything useful with empty string arguments, and zsh
> was intended first to be an interactive command shell and only second
> a programming language (just a better one than BSD csh).
That might be enough to close the case.  When I think of my data arrays 
it doesn't occur to me that 'ls' is going to receive it's arguments 
using the same rules.  Yeah, I've run into that -- piping array contents 
to 'ls' and having it barf at a blank.  So even as a theoretical 
discussion, what I'm suggesting would have to be unique to personal 
data, not arguments sent to ls ... but that might not be possible so ... 
I loose.  Or ... the speculative: 'no empty elements' flag would have to 
be applied to all utilities.  Anyway it's just philosophy.

But this morning's mixup is cured.  print might take "  \n  " and give 
me a new line but as far as "  ${(f)....}  " ... is concerned they are 
not the same thing and "  '\n'  " will not be removed but "   $'\n'   " 
will be removed.  I get it.  And in an assignment, it's spaces that 
delimit elements. Hard spaces will be inside ticks or $'' .

Thanks for everyone's patience.

[-- Attachment #2: Type: text/html, Size: 2786 bytes --]

Re: triviality regarding $# counts

[Mark J. Reed]

[-- Attachment #1: Type: text/plain, Size: 1425 bytes --]

On Sun, Apr 14, 2024 at 1:22 PM Ray Andrews <rayandrews@eastlink.ca> wrote:

> Hard spaces will be inside ticks or $'' .
>

.. or double-quotes; don't forget about them.   *" "*, *' '*, and *$' '* are
all literal spaces. It's other characters where the difference shows up. In
*"*...*"*, *$*-expansions still happen; in the others, they don't.  In *$'*
...*'*, ANSI C backslash escapes work; in the others, they don't.  Inside
*'*...*'*, absolutely everything is literal; there's not even a way to
include a literal *'* in the string.

One thing about the shell that makes it different from other programming
languages, however, is that the quotation marks themselves are not word
delimiters.  So you can switch between types of quoted string... including
entirely unquoted... without introducing a new shell "word".  That means
all of these assignments store exactly the same thing in the variable *x*:

*x=this\ is\ all\ one\ word*
*x="this is all one word"*
*x='this is all one word'*
*x=$'this is all one word'*
*x=this\ is' 'all" "one$' 'word*

The ability to mix and match gives us the old way to get an apostrophe into
an otherwise-single-quoted string: to close the quotes, add a
backslashed apostrophe, and then go back into quotes:

*x='That'\''s all, folks!'*


But it's easier to just use ANSI quotes:

*x=$'That\'s all, folks!'*



-- 
Mark J. Reed <markjreed@gmail.com>

[-- Attachment #2: Type: text/html, Size: 3030 bytes --]

Re: triviality regarding $# counts

[Bart Schaefer]

On Sun, Apr 14, 2024 at 10:42 AM Mark J. Reed <markjreed@gmail.com> wrote:
>
> On Sun, Apr 14, 2024 at 1:22 PM Ray Andrews <rayandrews@eastlink.ca> wrote:
>>
>> Hard spaces will be inside ticks or $'' .
>
> .. or double-quotes; don't forget about them.

However, "typeset -p" never outputs double-quotes for this purpose.

> But it's easier to just use ANSI quotes:
>
> x=$'That\'s all, folks!'

Which typeset will output "the old way"

'That'\''s all, folks!'

Re: triviality regarding $# counts

[Ray Andrews]

[-- Attachment #1: Type: text/plain, Size: 1184 bytes --]



On 2024-04-14 10:42, Mark J. Reed wrote:
> ...  That means all of these assignments store exactly the same thing 
> in the variable *x*:
>
>     *x=this\ is\ all\ one\ word*
>
That one is new to me.  So a quoted or escaped space is no longer a word 
delimiter.  Yeah ... that's why in typeset, any naked space is a 
delimiter, even a dozen of them make no difference, they are 'just' 
delimiters.  Yup, this kind of thing won't catch me again.

>     *x="this is all one word"*
>     *x='this is all one word'*
>     *x=$'this is all one word'*
>     *x=this\ is' 'all" "one$' 'word*
>     *
>     *
>
> The ability to mix and match gives us the old way to get an apostrophe 
> into an otherwise-single-quoted string: to close the quotes, add a 
> backslashed apostrophe, and then go back into quotes:
>
>     *x='That'\''s all, folks!'*
>

Geez ... that's dicing with the law. I'm seeing:
'That'
then an escaped apostrophe, so literal,  then
's all, folks!"
... three ... dunno how to say it ... three words joined into one.
>
>
> But it's easier to just use ANSI quotes:
>
>     *x=$'That\'s all, folks!'*
>
>
Yes, that's clear.
>
> -- 
> Mark J. Reed <markjreed@gmail.com>

[-- Attachment #2: Type: text/html, Size: 4038 bytes --]