zsh-users
 help / color / mirror / code / Atom feed
* array with newlines preserved as literal text
@ 2021-02-02 19:16 Ray Andrews
  2021-02-02 19:26 ` Roman Perepelitsa
  2021-02-02 20:25 ` Bart Schaefer
  0 siblings, 2 replies; 14+ messages in thread
From: Ray Andrews @ 2021-02-02 19:16 UTC (permalink / raw)
  To: Zsh Users

I create some nasty output involving a grep search, save to to a 
variable, send it to a function which ends up calling 'eval' which 
receives the input and executes it like this:

     @ eval "$@"
     echo "\nYou (hard return)
     will\\nregret'\x0a'\nthe day you '\n' were born."

... I can save the output to a file perfectly:
     $ eval "$@" >! junk

     $ cat junk
     echo "\nYou (hard return)
     will\\nregret'\x0a'\nthe day you '\n' were born."   # Perfectly 
identical.

... and I can recapture that text from the file to a variable like this:

     $ typeset -a array1=( "${(q+)$(<junk)}" )

     $ print -l $array1
     $'    echo "\nYou (hard return)
     will\\nregret'\x0a'\nthe day you '\n' were born."'

... pretty good.  And there's probably a way of stripping off the outer: 
$'...'.
But if I try to capture the output to a variable directly:

     $ array2=$(eval "$@")

... the results are spectacularly  bad no matter how I dress it up with 
' ${(f)....}' or quotes  any other invocation I can think of. But 
surely  there's a way.   I understand newlines are hard to ignore.  
Various ideas would treat the text: ' \n ' as text alright but also 
ignore the 'real' newline that is invisible in an editor but must of 
course be there in the real string so everything would end up on one 
line.  Instead of using '(q+)' I tried '(F)' and stripped out the 
newlines like this: 'array2=${array2//'\n'/-NL-}' and it sorta works but 
it's vulgar.  The ' \x0a' is forgivable. Note that the input of all this 
is grep searches in code, so the lines found are bound to contain all 
sorts of rotten stuff, but I want literal output.
How do I pull this off?  There will be a way.



^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: array with newlines preserved as literal text
  2021-02-02 19:16 array with newlines preserved as literal text Ray Andrews
@ 2021-02-02 19:26 ` Roman Perepelitsa
  2021-02-03  1:10   ` Ray Andrews
  2021-02-02 20:25 ` Bart Schaefer
  1 sibling, 1 reply; 14+ messages in thread
From: Roman Perepelitsa @ 2021-02-02 19:26 UTC (permalink / raw)
  To: Ray Andrews; +Cc: Zsh Users

On Tue, Feb 2, 2021 at 8:16 PM Ray Andrews <rayandrews@eastlink.ca> wrote:
>
> ... I can recapture that text from the file to a variable like this:
>
>      $ typeset -a array1=( "${(q+)$(<junk)}" )

This doesn't just "recapture" the text, it also quotes it.

>      $ print -l $array1

This unquotes the text partially before printing (because print is
invoked without -r).

> But if I try to capture the output to a variable directly:
>
>      $ array2=$(eval "$@")

This is fine. You can verify that the content is as expected:

    print -r -- $array2

Note that array2 is a scalar and not an array. array1 is an array with
one element. If you want array2 to also be an array with one element,
do this:

    array2=("$(eval "$@")")

Roman.

P.S.

Use `typeset -p var` to see the value of parameter `var`.

    typeset -p array2


^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: array with newlines preserved as literal text
  2021-02-02 19:16 array with newlines preserved as literal text Ray Andrews
  2021-02-02 19:26 ` Roman Perepelitsa
@ 2021-02-02 20:25 ` Bart Schaefer
  2021-02-02 23:31   ` Ray Andrews
  1 sibling, 1 reply; 14+ messages in thread
From: Bart Schaefer @ 2021-02-02 20:25 UTC (permalink / raw)
  To: Ray Andrews; +Cc: Zsh Users

On Tue, Feb 2, 2021 at 11:16 AM Ray Andrews <rayandrews@eastlink.ca> wrote:
>
> I create some nasty output involving a grep search, save to to a
> variable, send it to a function which ends up calling 'eval' which
> receives the input and executes it like this:
>
>      @ eval "$@"
>      echo "\nYou (hard return)
>      will\\nregret'\x0a'\nthe day you '\n' were born."

I'm curious why you want to save this in the form of a command to be
eval'd rather than just save the argument string?

Pretty much any time I start to write an "eval" I go back to make sure
I didn't do something wrong earlier. **

Roman's answers are otherwise spot-on.

** Of about 90 instances of "eval" in the contributed Functions
directory, 45 are one of these three cases:
   1) eval "var_$suffix=value" (there are 36 of these)
   2) eval "[[ ... ]]"
   3) eval "(( ... ))"
Most eval'd assignments could probably be replaced with { typeset
var_$suffix=value } (sometimes adding -g).


^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: array with newlines preserved as literal text
  2021-02-02 20:25 ` Bart Schaefer
@ 2021-02-02 23:31   ` Ray Andrews
  2021-02-03  2:21     ` Bart Schaefer
  0 siblings, 1 reply; 14+ messages in thread
From: Ray Andrews @ 2021-02-02 23:31 UTC (permalink / raw)
  To: zsh-users

On 2021-02-02 12:25 p.m., Bart Schaefer wrote:
> On Tue, Feb 2, 2021 at 11:16 AM Ray Andrews <rayandrews@eastlink.ca> wrote:
>>
>> I'm curious why you want to save this in the form of a command to be
>> eval'd rather than just save the argument string?
Complicated.  It goes back to my first bitch with zsh, namely not being 
able to capture the literal invocation of a function (along with it's 
tail) unexpanded, just as you'd see in history.  (History would work 
fine except for the fact that several commands can end up on the same 
line.)  Basically my wrappers do some heavy duty massaging of output and 
both for debugging and or for modification it's good to be able to 
recall the actual commands executed.  My function '_execute' does that, 
it calls 'eval' but does other stuff too like store to history.  Here's 
a sample:

$  l ,Hs c,*   # 'l' wraps 'ls' and colorizes and formats the output as 
I like:

LISTING of "c,*": all file types, INsensitive. Sorting upside down by: 
File Size:

  10746 [2020-12-20--14:48] c,34
  11168 [2020-12-27--10:39] c,35,localized help
  11219 [2020-12-27--10:45] c,36
  11348 [2020-12-28--13:30] c,38,delete deleted directories
  11428 [2020-12-28--10:46] c,37,trivial
  11665 [2020-12-28--16:17] c,39
  11833 [2020-12-28--16:48] c,40,misc
  13253 [2020-12-28--19:48] c,41,polishing
  13376 [2020-12-28--20:09] c,42,messages improved
  13383 [2020-12-29--07:02] c,43,before improved cd
  14045 [2021-01-31--11:08] c,55,more one line bugs
  14053 [2021-01-31--13:05] c,56,trivial edits
  14156 [2021-01-30--10:48] c,54,rationalize switches
  14184 [2021-01-24--20:47] c,52,interim
  14186 [2021-01-29--06:17] c,53,more sky trouble. big code purge
  14494 [2020-12-30--06:17] c,44,better cd to incomplete name
  14925 [2020-12-30--11:55] c,47,massive edit old code purged
  15493 [2020-12-30--09:41] c,45,working huge edit
  15741 [2021-01-31--10:48] c,48,good old Sky above was truncated
  16080 [2020-12-30--10:06] c,46,bad
  16394 [2021-01-07--08:49] c,50

Items found: 1
Total bytes in this directory: 2.9M
Total including subdirs: 15M

... the ',H' switch stores the actual working code to history.  Hit the 
up arrow and:

$ ls -AFrGgdS  --time-style='+[%F--%H:%M]' --group-directories-first   
--color=always  (#i)c,* 2> /dev/null   | sed -r "s|^(.{10} [[:digit:]] 
)| |"  | egrep -v '^total' | egrep -i --color=always "^|] c,*" | sed -r 
"s/\x1b\[01;31m\x1b\[K\] /\] \x1b\[01;31m\x1b\[K/g"

... hit ENTER and exactly the same output occurs.  Or modify to suit, or 
perhaps to chase down a bug.
But it requires execution to be put off, thus the need for an eval 
contained within '_execute'.   Meanwhile let's see what Roman has for me.





^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: array with newlines preserved as literal text
  2021-02-02 19:26 ` Roman Perepelitsa
@ 2021-02-03  1:10   ` Ray Andrews
  0 siblings, 0 replies; 14+ messages in thread
From: Ray Andrews @ 2021-02-03  1:10 UTC (permalink / raw)
  To: zsh-users

On 2021-02-02 11:26 a.m., Roman Perepelitsa wrote:
>
> This is fine. You can verify that the content is as expected:
>
>      print -r -- $array2
Bloody hell, it was the missing '-r' there :(  the array was fine, just 
improper output.  Past time I knew that.

> Use `typeset -p var` to see the value of parameter `var`.
>
>      typeset -p array2
Thanks I knew there was something like that but I forgot it.  That's 
what happens when I don't code for over a year, I forget everything.  
I've been busy columnizing grep output.  the 'column' command doesn't 
quite do it because it takes all those '\n' that you might find in code 
literally which is why I had to sort that out.

|$ g ,w4 'msg "\n' g,* ||
||
||g,32,pplain in place......................120.. ! [ "$pplain" ] && 
actionmsg "\n$msg"||
||g,32,pplain in place......................168..    ! [ "$pplain" ] && 
actionmsg "\n$msg"||
||g,33,better formatting and fix rregexp....121.. ! [ "$bbare" ] && 
actionmsg "\n$msg"||
||g,33,better formatting and fix rregexp....169..    ! [ "$bbare" ] && 
actionmsg "\n$msg"||
||g,34,light edits..........................104.. ! [ "$bbare" ] && 
actionmsg "\n$msg"||
||g,34,light edits..........................152..    ! [ "$bbare" ] && 
actionmsg "\n$msg"||
||g,35,interim..............................106.. ! [ "$bbare" ] && 
actionmsg "\n$msg"||
||g,35,interim..............................154..    ! [ "$bbare" ] && 
actionmsg "\n$msg"||
||g,36,g_column in place interim............107.. ! [ "$bbare" ] && 
actionmsg "\n$msg"||
||g,36,g_column in place interim............155..    ! [ "$bbare" ] && 
actionmsg "\n$msg"||
||g,37,g_columns in place and looks good....108.. ! [ "$bbare" ] && 
actionmsg "\n$msg"||
||g,37,g_columns in place and looks good....156..    ! [ "$bbare" ] && 
actionmsg "\n$msg"||
|
Thanks Roman



^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: array with newlines preserved as literal text
  2021-02-02 23:31   ` Ray Andrews
@ 2021-02-03  2:21     ` Bart Schaefer
  2021-02-03 19:29       ` Ray Andrews
  0 siblings, 1 reply; 14+ messages in thread
From: Bart Schaefer @ 2021-02-03  2:21 UTC (permalink / raw)
  To: Ray Andrews; +Cc: Zsh Users

On Tue, Feb 2, 2021 at 3:31 PM Ray Andrews <rayandrews@eastlink.ca> wrote:
>
> Complicated.  It goes back to my first bitch with zsh, namely not being
> able to capture the literal invocation of a function (along with it's
> tail) unexpanded, just as you'd see in history.

I'm not entirely sure I understand what that means, but maybe it's this?

_fn2hist() {
  emulate -L zsh
  local cmd=$1
  shift
  if [[ -n $functions[$cmd] ]]
  then
    print -rS "${$(functions -x 2 $cmd)#$cmd} ${${(qq)@}}"
  fi
  $cmd "$@"
}

Intended to be used like this:

myfn() { : do whatever myfn does }
alias myfn='_fn2hist myfn'

What this does is turn the body of "myfn" into an anonymous function,
and stuff that into the history followed by the original arguments,
and then run the function.

At this point if you up-history, you'll get something like

ubuntu%  () {
  : do whatever myfn does
} 'original' 'arguments'

That could be a very long history entry, if "myfn" is complicated ...
but if you hit "enter", it should behave just like "myfn" did.

With one caveat:  If the function depends on getting its own name from
$0 or some similar trick involving $funcstack, etc., that will fail.


^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: array with newlines preserved as literal text
  2021-02-03  2:21     ` Bart Schaefer
@ 2021-02-03 19:29       ` Ray Andrews
  2021-02-03 21:19         ` Bart Schaefer
  0 siblings, 1 reply; 14+ messages in thread
From: Ray Andrews @ 2021-02-03 19:29 UTC (permalink / raw)
  To: zsh-users

On 2021-02-02 6:21 p.m., Bart Schaefer wrote:
> _fn2hist() {
>    emulate -L zsh
>    local cmd=$1
>    shift
>    if [[ -n $functions[$cmd] ]]
>    then
>      print -rS "${$(functions -x 2 $cmd)#$cmd} ${${(qq)@}}"
>    fi
>    $cmd "$@"
> }
>
> Intended to be used like this:
>
> myfn() { : do whatever myfn does }
> alias myfn='_fn2hist myfn'
Cool.  Only thing is that when you pop it out of history it contains the 
'_fn2hist' as well as the subsequent command.  This could obviate all my 
" alias g='noglob _g' " stuff. Can the function and its tail be saved to 
a variable instead of to history?  Mind I'm being lazy I'm sure I can 
figure that out easy enough. Seems '(qq)' is our friend there for 
keeping things unexpanded.



^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: array with newlines preserved as literal text
  2021-02-03 19:29       ` Ray Andrews
@ 2021-02-03 21:19         ` Bart Schaefer
  2021-02-04  2:07           ` Ray Andrews
  2021-02-04 21:55           ` Ray Andrews
  0 siblings, 2 replies; 14+ messages in thread
From: Bart Schaefer @ 2021-02-03 21:19 UTC (permalink / raw)
  To: Ray Andrews; +Cc: Zsh Users

On Wed, Feb 3, 2021 at 11:29 AM Ray Andrews <rayandrews@eastlink.ca> wrote:
>
> Cool.  Only thing is that when you pop it out of history it contains the
> '_fn2hist' as well as the subsequent command.

That doesn't happen to me.  Do you have something else also modifying
the history?  Can you show an example?

> Can the function and its tail be saved to
> a variable instead of to history?

Yes, replace
  print -rS ....
with
  print -rv REPLY ...
or whatever other variable you want (it will be a string, not an array).

The phrase "its tail" still leaves me slightly doubtful.


^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: array with newlines preserved as literal text
  2021-02-03 21:19         ` Bart Schaefer
@ 2021-02-04  2:07           ` Ray Andrews
  2021-02-05  2:57             ` Bart Schaefer
  2021-02-04 21:55           ` Ray Andrews
  1 sibling, 1 reply; 14+ messages in thread
From: Ray Andrews @ 2021-02-04  2:07 UTC (permalink / raw)
  To: zsh-users

On 2021-02-03 1:19 p.m., Bart Schaefer wrote:
> On Wed, Feb 3, 2021 at 11:29 AM Ray Andrews <rayandrews@eastlink.ca> wrote:
Nuts what am I doing wrong trying to save to a variable:

_fn2hist() {
   emulate -L zsh
   local cmd=$1
   shift
   if [[ -n $functions[$cmd] ]]
   then
     print -rS "${$(functions -x 2 $cmd)#$cmd} ${${(qq)@}}"
     print -rv _com "${$(functions -x 2 $cmd)#$cmd} ${${(qq)@}}"
   fi
   $cmd "$@"
   print -rl "_com is: $_com"  # Nothing in the variable.
}



^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: array with newlines preserved as literal text
  2021-02-03 21:19         ` Bart Schaefer
  2021-02-04  2:07           ` Ray Andrews
@ 2021-02-04 21:55           ` Ray Andrews
  2021-02-05  3:22             ` Bart Schaefer
  1 sibling, 1 reply; 14+ messages in thread
From: Ray Andrews @ 2021-02-04 21:55 UTC (permalink / raw)
  To: zsh-users

On 2021-02-03 1:19 p.m., Bart Schaefer wrote:

  print -rv REPLY ...
> or whatever other variable you want (it will be a string, not an array).
>
> The phrase "its tail" still leaves me slightly doubtful.
>
Isn't 'tail' the normal term for whatever follows a command? I learned 
it as 'command tail' back in the day.

Anyway, more fiddling: I added " print -lr $@ " to my 'preexec()' and, 
mirabile dictu, it seems to capture input exactly as typed:

3 /aWorking/Zsh/Source/Wk 3 $ g '*$1*'
g '*$1*'
noglob _g '*$1*'  # 'g' is the alias that calls '_g' so that's 
understandable.
noglob _g '*$1*'  # Why the duplication?

... various tests show nothing expanded.  One thing tho, why the 
duplicate line?  In one case:

3 /aWorking/Zsh/Source/Wk 3 $ s g tiny bug 'grep -' line 88
s g tiny bug 'grep -' line 88
s g tiny bug 'grep -' line 88
s g tiny bug 'grep -' line 88

... three times?   And 's' is not an alias, that function goes straight 
in.  But here is my capture point for the literal command as it is 
typed, it seems.  "print ,lrv LAST_COMMAND $1  " and I'm good, the first 
line seems always correct.





^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: array with newlines preserved as literal text
  2021-02-04  2:07           ` Ray Andrews
@ 2021-02-05  2:57             ` Bart Schaefer
  0 siblings, 0 replies; 14+ messages in thread
From: Bart Schaefer @ 2021-02-05  2:57 UTC (permalink / raw)
  To: Ray Andrews; +Cc: Zsh Users

On Wed, Feb 3, 2021 at 6:07 PM Ray Andrews <rayandrews@eastlink.ca> wrote:
>
> Nuts what am I doing wrong trying to save to a variable:

I don't know, it looks fine to me (and works fine for me):

ubuntu% myfn() { print -r Args: "$@" }
ubuntu% alias myfn='_fn2hist myfn'
ubuntu% myfn 1 "2 3" 4
Args: 1 2 3 4
_com is:  () {
  print -r Args: "$@"
} '1' '2 3' '4'

Given that you also have trouble with what's placed in the history by
"print -rS", I'd suggest you start looking around for other things in
your configuration that are messing with command execution.


^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: array with newlines preserved as literal text
  2021-02-04 21:55           ` Ray Andrews
@ 2021-02-05  3:22             ` Bart Schaefer
  2021-02-05  4:17               ` Ray Andrews
  0 siblings, 1 reply; 14+ messages in thread
From: Bart Schaefer @ 2021-02-05  3:22 UTC (permalink / raw)
  To: Ray Andrews; +Cc: Zsh Users

On Thu, Feb 4, 2021 at 1:55 PM Ray Andrews <rayandrews@eastlink.ca> wrote:
>
> Isn't 'tail' the normal term for whatever follows a command? I learned
> it as 'command tail' back in the day.

I've never called what I think you mean anything but the "arguments"
or "argument list".  Unless "tail" means more than that.

> Anyway, more fiddling: I added " print -lr $@ " to my 'preexec()' and,
> mirabile dictu, it seems to capture input exactly as typed:

"Exactly as typed" is going to include the entire (possibly
multi-line, pipelined, etc.) command input, not just one command.  How
much of that is the "tail"?

> noglob _g '*$1*'  # Why the duplication?

RTM?  Preexec gets three strings in $@.

$1 is the string seen by the history mechanism
$2 is is a single-line summary of the command
$3 contains the full text that is being executed

Note, $1 will be empty when history is disabled.


^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: array with newlines preserved as literal text
  2021-02-05  3:22             ` Bart Schaefer
@ 2021-02-05  4:17               ` Ray Andrews
  2021-02-05  4:24                 ` Bart Schaefer
  0 siblings, 1 reply; 14+ messages in thread
From: Ray Andrews @ 2021-02-05  4:17 UTC (permalink / raw)
  To: zsh-users

On 2021-02-04 7:22 p.m., Bart Schaefer wrote:
> I've never called what I think you mean anything but the "arguments"
> or "argument list".  Unless "tail" means more than that.
Maybe just obsolete culture.  I learned that the tail was switches 
followed by arguments.  But if that's not how we speak now then that's 
not how we speak.
>> Anyway, more fiddling: I added " print -lr $@ " to my 'preexec()' and,
>> mirabile dictu, it seems to capture input exactly as typed:
> "Exactly as typed" is going to include the entire (possibly
> multi-line, pipelined, etc.) command input, not just one command.  How
> much of that is the "tail"?
Good question.  I'll have to wait for the variable to contain something 
monstrous and then I'll know what can go wrong.  It sure looks good 
right now tho.
> noglob _g '*$1*'  # Why the duplication?
> RTM?
God no!  Seriously I'm reading Peter's introduction finally, then I 
might have enough Latin to attempt the manual.  You guys don't know how 
baffling it is for someone who doesn't already understand it. Worst 
thing is just knowing where to look.

> Preexec gets three strings in $@.
>
> $1 is the string seen by the history mechanism
> $2 is is a single-line summary of the command
> $3 contains the full text that is being executed
Excellent, thanks. That's quite marvelous, that level of finesse.  I 
remember beating my head against that wall when I first got involved.  
Turns out to be that simple.

Or more than three:

7 /aWorking/Zsh/Source 3 $ echo $PREV_COMMAND
l g; l f; l s
noglob _l g; noglob _l f; noglob _l s
noglob _l g
noglob _l f
noglob _l s

... very nice.





^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: array with newlines preserved as literal text
  2021-02-05  4:17               ` Ray Andrews
@ 2021-02-05  4:24                 ` Bart Schaefer
  0 siblings, 0 replies; 14+ messages in thread
From: Bart Schaefer @ 2021-02-05  4:24 UTC (permalink / raw)
  To: Ray Andrews; +Cc: Zsh Users

On Thu, Feb 4, 2021 at 8:17 PM Ray Andrews <rayandrews@eastlink.ca> wrote:
>
> Maybe just obsolete culture.

You can't be that much older than me. :-D

> I learned that the tail was switches
> followed by arguments.

Ah, "switches".  DOS background instead of UNIX, I wager.

> > RTM?
> God no!

You could at least look up "preexec" in the index ...

> > Preexec gets three strings in $@.
>
> Or more than three:

No, just three.  If it looks like more, that's an artifact of the way
you're printing it.


^ permalink raw reply	[flat|nested] 14+ messages in thread

end of thread, other threads:[~2021-02-05  4:25 UTC | newest]

Thread overview: 14+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-02-02 19:16 array with newlines preserved as literal text Ray Andrews
2021-02-02 19:26 ` Roman Perepelitsa
2021-02-03  1:10   ` Ray Andrews
2021-02-02 20:25 ` Bart Schaefer
2021-02-02 23:31   ` Ray Andrews
2021-02-03  2:21     ` Bart Schaefer
2021-02-03 19:29       ` Ray Andrews
2021-02-03 21:19         ` Bart Schaefer
2021-02-04  2:07           ` Ray Andrews
2021-02-05  2:57             ` Bart Schaefer
2021-02-04 21:55           ` Ray Andrews
2021-02-05  3:22             ` Bart Schaefer
2021-02-05  4:17               ` Ray Andrews
2021-02-05  4:24                 ` Bart Schaefer

Code repositories for project(s) associated with this public inbox

	https://git.vuxu.org/mirror/zsh/

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).