Different behaviour when interpreting a here-document between 5.3 and 5.5.1

Different behaviour when interpreting a here-document between 5.3 and 5.5.1

[Enrico Maria Crisostomo]

Hi,

I'm struggling with an apparently simple problem.  A script of mine has a function that writes a here-document whose lines end with a backslash.  The script has been working fine until zsh has been updated recently.

The relevant part of the code is the following

  mp_calculate_checksums()
  {
    cat <<EOF
  checksums           rmd160 ${CHECKSUM_RMD160} \\
                      sha256 ${CHECKSUM_SHA256} \\
                      size   ${CHECKSUM_SIZE}
  EOF
  }

The expected output is something like:

  checksums           rmd160 281d97a37125fca07ff81a72ac337f6a23f5d2b0 \
                      sha256 233cd8724fd95e10b82ed62c692e7e79cca9114b49be834e65cd05b529d8ba70 \
                      size   31

When running the same script on 5.5.1, I get a single line of output, as if the last backslash of \\ were escaping the newline.

  checksums           rmd160 281d97a37125fca07ff81a72ac337f6a23f5d2b0 \                    sha256 233cd8724fd95e10b82ed62c692e7e79cca9114b49be834e65cd05b529d8ba70 \                    size   31

I haven't been able to troubleshoot it so far, and the zsh documentation seems to imply that the correct behaviour is the former, not the latter.

Thanks in advance for you help,
-- 
Enrico

Re: Different behaviour when interpreting a here-document between 5.3 and 5.5.1

[Bart Schaefer]

On Thu, Apr 19, 2018 at 2:52 PM, Enrico Maria Crisostomo
<enrico.m.crisostomo@gmail.com> wrote:
>     cat <<EOF
>   checksums           rmd160 ${CHECKSUM_RMD160} \\
>                       sha256 ${CHECKSUM_SHA256} \\
>                       size   ${CHECKSUM_SIZE}
>   EOF

Yeah, this is commit 47aa6095 aka workers/42355.  It's recognizing
backslash-newline but not whether the backslash is itself escaped.

The NEWS file should have mentioned the change to here-documents, I
think.  However, not catching the double backslash is a bug.  However,
part 2, figuring this out requires scanning backwards to see if you
have an even or odd number of trailing backlashes, or scanning forward
and tokenizing while consuming the document.  Bleah.

Re: Different behaviour when interpreting a here-document between 5.3 and 5.5.1

[Enrico Maria Crisostomo]

> On 19 Apr 2018, at 23:52, Enrico Maria Crisostomo <enrico.m.crisostomo@gmail.com> wrote:
> 
> Hi,
> 
> I'm struggling with an apparently simple problem.  A script of mine has a function that writes a here-document whose lines end with a backslash.  The script has been working fine until zsh has been updated recently.
> 
> The relevant part of the code is the following
> 
>  mp_calculate_checksums()
>  {
>    cat <<EOF
>  checksums           rmd160 ${CHECKSUM_RMD160} \\
>                      sha256 ${CHECKSUM_SHA256} \\
>                      size   ${CHECKSUM_SIZE}
>  EOF
>  }
> 
> The expected output is something like:
> 
>  checksums           rmd160 281d97a37125fca07ff81a72ac337f6a23f5d2b0 \
>                      sha256 233cd8724fd95e10b82ed62c692e7e79cca9114b49be834e65cd05b529d8ba70 \
>                      size   31
> 
> When running the same script on 5.5.1, I get a single line of output, as if the last backslash of \\ were escaping the newline.
> 
>  checksums           rmd160 281d97a37125fca07ff81a72ac337f6a23f5d2b0 \                    sha256 233cd8724fd95e10b82ed62c692e7e79cca9114b49be834e65cd05b529d8ba70 \                    size   31
> 
> I haven't been able to troubleshoot it so far, and the zsh documentation seems to imply that the correct behaviour is the former, not the latter.
> 
> Thanks in advance for you help,
> -- 
> Enrico

Thanks for your answer, Bart.  Apologies for manually quoting it, but I haven't received a mail from the mailing list, so I can't reply:

> Yeah, this is commit 47aa6095 aka workers/42355.  It's recognizing
> backslash-newline but not whether the backslash is itself escaped.
> 
> The NEWS file should have mentioned the change to here-documents, I
> think.  However, not catching the double backslash is a bug.  However,
> part 2, figuring this out requires scanning backwards to see if you
> have an even or odd number of trailing backlashes, or scanning forward
> and tokenizing while consuming the document.  Bleah.

Should I file a bug report, write to zsh-workers, or do something else?  Or may I consider that your answering the mail is equivalent to a bug having been open?

Thanks,
-- 
Enrico