test for newline in a variable--unexpected results sometimes

test for newline in a variable--unexpected results sometimes

[lilydjwg]

I get output from the skim tool. Then I test if it's multiline or not,
but it fails mysteriously for some cases.

My test is like this:

[[ $cmd = *\n* ]]

It doesn't work for the following data (I've echoed $cmd to the file):

> cat -E log
cd $
cd /sys/class/leds$

This fails too:

[[ $(<log) = *\n* ]]

But manually set the variable to the same text it works:

cmd2="cd \ncd /sys/class/leds"
[[ $cmd2 = *\n* ]] # this succeeded

Here's the hex dump of the file:

> xxd log
00000000: 6364 200a 6364 202f 7379 732f 636c 6173  cd .cd /sys/clas
00000010: 732f 6c65 6473 0a                        s/leds.

What's going on? I'm confused....


-- 
Best regards,
lilydjwg

Re: test for newline in a variable--unexpected results sometimes

[Daniel Shahaf]

lilydjwg wrote on Mon, Sep 17, 2018 at 01:27:17 +0800:
> I get output from the skim tool. Then I test if it's multiline or not,
> but it fails mysteriously for some cases.
> 
> My test is like this:
> 
> [[ $cmd = *\n* ]]
> 

In this context, «\n» does not mean a newline; it means either the letter 'n',
or the two-character sequence «\n», but I don't remember which.

> cmd2="cd \ncd /sys/class/leds"

Ditto..  If you try «xxd <<<$cmd2» you'll see the variable's value contains the
two-character sequence «\n».

What you're looking for is [[ $cmd == *$'\n'* ]].  «\n» is an escape sequence
in $''-strings, but not in other kinds of strings.

Cheers,

Daniel

Re: test for newline in a variable--unexpected results sometimes

[dana]

On 16 Sep 2018, at 15:44, Daniel Shahaf <d.s@daniel.shahaf.name> wrote:
>In this context, «\n» does not mean a newline; it means either the letter 'n',
>or the two-character sequence «\n», but I don't remember which.

I was about to hit send on an answer to this but you addressed it. Here's a more
detailed explanation of this part (the effects of quoting):

* In an UNQUOTED string, all slash-escaped characters produce the literal
  version of the character being escaped:

  \\  ->  \
  \$  ->  $
  \1  ->  1
  \n  ->  n

* In a DOUBLE-QUOTED string, slash-escaped shell meta-characters produce the
  literal version of the character being escaped. Any other back-slash is left
  as-is:

  "\\"  ->  \
  "\$"  ->  $
  "\1"  ->  \1
  "\n"  ->  \n

* In a SINGLE-QUOTED string, all back-slashes are left as-is:

  '\\'  ->  \\
  '\$'  ->  \$
  '\1'  ->  \1
  '\n'  ->  \n

* In a DOLLAR-QUOTED string, escape sequences are interpreted according to the
  default behaviour of the print built-in, which supports various C-style
  sequences amongst other things. Sequences that aren't handled specifically are
  treated as they are in unquoted strings:

  $'\\'  ->  \
  $'\$'  ->  $
  $'\1'  ->  (ASCII character with octal code point 001)
  $'\n'  ->  (ASCII new-line)

This is mostly covered by the Shell Grammar section of the manual, though it's
not quite as explicit

dana

Dangling reference in zshroadmap(1) (was: Re: test for newline in a variable--unexpected results sometimes)

[Daniel Shahaf]

dana wrote on Sun, 16 Sep 2018 15:53 -0500:
> This is mostly covered by the Shell Grammar section of the manual, though it's
> not quite as explicit

zshroadmap(1) refers to a "Shell Grammar" section, but there's no such
section in the man pages:

% cd $prefix/share/man/man1
% grep -i 'shell grammar' zsh*.1 | me
zshroadmap.1:General shell syntax is introduced in the section `Shell Grammar\&' in
% grep -i grammar zshmisc.1
.\" Yodl file: Zsh/grammar.yo
% 

Checking the yodl source, I see a chapter(Shell Grammar) directive,
followed by the contents that ends up at the top of zshmisc(1).  That is: the
content is there, but we have a dangling reference in zshroadmap(1) to
a non-existent section header.

Cheers,

Daniel

Re: Dangling reference in zshroadmap(1) (was: Re: test for newline in a variable--unexpected results sometimes)

[dana]

On 16 Sep 2018, at 16:44, Daniel Shahaf <d.s@daniel.shahaf.name> wrote:
>zshroadmap(1) refers to a "Shell Grammar" section, but there's no such
>section in the man pages:

Oh, i didn't notice because i always use the HTML version:

http://zsh.sourceforge.net/Doc/Release/Shell-Grammar.html

I guess it's designed with the sections that get separate pages (like
'Parameters') in mind? The top-level heading seems to be omitted if the first
immediate descendant is a sub-section. The same is true of other sections like
'Invocation', 'Files', and 'Jobs & Signals'.

Confusingly, the first two sections mentioned above each contain a sub-section
with the same name, so the exact part of the documentation being referred to is
unclear/inconsistent. (In the man pages, the top-level headings are missing, so
that leaves only the sub-headings. But in the HTML version, the links take you
to the top of the entire section.)

Not sure how to address that

dana

Re: Dangling reference in zshroadmap(1) (was: Re: test for newline in a variable--unexpected results sometimes)

[Daniel Shahaf]

dana wrote on Sun, 16 Sep 2018 17:16 -0500:
> I guess it's designed with the sections that get separate pages (like
> 'Parameters') in mind? The top-level heading seems to be omitted if the first
> immediate descendant is a sub-section.

I don't understand the last sentence?

> The same is true of other sections like 'Invocation', 'Files', and 'Jobs & Signals'.
> 
> Confusingly, the first two sections mentioned above each contain a sub-section
> with the same name, so the exact part of the documentation being referred to is
> unclear/inconsistent.

On the other hand, redirect.yo explicitly adds a sect() in man page format
only.  The net effect is that the heading "Redirection" is shown as a
chapter heading in html/pdf but as a section heading in man.

> (In the man pages, the top-level headings are missing,

That's because zman.yo defines chapter() to expand to nothing.

> so that leaves only the sub-headings. But in the HTML version, the
> links take you to the top of the entire section.)
> 

Yes, the texinode() directives are placed on the chapter(), not on the
sect() with the same name.

The mapping between chapter() nodes and man pages is not one to one;
there are more chapters than man pages.

> Not sure how to address that

What end result would we like to achieve?  Once we nail that down we can
think how to implement it.

At least, we could grep for any instances of ifnzman(see the section
`foo'...) where `foo' is a texinode() of a chapter(), and fix them.

Or we could hack at the yodl to reduce the differences between the
various output formats.

There are probably other options 

Daniel

Re: test for newline in a variable--unexpected results sometimes

[lilydjwg]

On Sun, Sep 16, 2018 at 08:44:33PM +0000, Daniel Shahaf wrote:
> In this context, «\n» does not mean a newline; it means either the letter 'n',
> or the two-character sequence «\n», but I don't remember which.

Thanks! Problem solved.

> > cmd2="cd \ncd /sys/class/leds"
> 
> Ditto..  If you try «xxd <<<$cmd2» you'll see the variable's value contains the
> two-character sequence «\n».

I echoed it, and it turns out that echo itself turned all \n into real newlines.

-- 
Best regards,
lilydjwg