Little problem while converting from bash (quoting/splitting?)

Little problem while converting from bash (quoting/splitting?)

[krasnal]

Hello,

I'm right now moving to zsh from bash which supposed to be straightforward
but I've encountered one problem.  Here's the log:
krasnal@szuflandia ~
$ for f in $(cleartool lsvob -short | head -10); do case $f in *gli* ) echo $f
| xxd;; esac; done
0000000: 5c67 6c69 320d 0a                        \gli2..
krasnal@szuflandia ~
$ bash
krasnal@szuflandia ~
$ for f in $(cleartool lsvob -short | head -10); do case $f in *gli* ) echo $f
| xxd;; esac; done
0000000: 5c67 6c69 320a                           \gli2.

This is with 4.2.4 cygwin version of zsh and 2.05b.0(1)-release of bash.
cleartool is windows program producing \r\n line terminated output.  Note
that zsh has both \r and \n in f variable while bash has only \n.  This is
a bit problematic for me because I'm eventually using these values for output
and the \r chars screw it up.  I've already learned that by default echo
interprets \a,\b,... sequences which I also have so I have to either use -E
or set bsd_echo option but this one is a bit puzzle for me.  Is there an
option for this (I thought sh_word_split would be the right one but there's
no change).

How should I solve this?  I mean I know I can try to strip \r which would work
for both windows and unix version but isn't there a simpler/more elegant way?

Best regards
-- 
    ____   _  ___ 
   /  | \_/ |/ _ \   Andrzej Marek Ostruszka 
  / _ |     | (_) |       MPSC (Cracow)
 /_/ L|_|V|_|\___/  (GnuPG key ID: 0x3D9C498A)

Re: Little problem while converting from bash (quoting/splitting?)

[Peter Stephenson]

"krasnal" wrote:
> This is with 4.2.4 cygwin version of zsh and 2.05b.0(1)-release of bash.
> cleartool is windows program producing \r\n line terminated output.  Note
> that zsh has both \r and \n in f variable while bash has only \n.  This is
> a bit problematic for me because I'm eventually using these values for output
> and the \r chars screw it up.

Well, you could add \r to IFS, but unfortunately that will generate extra
blank lines, for example:

% (IFS=$' \t\n\000\r'; for f in $(print "foo\r"); do print $f | xxd; done)
0000000: 666f 6f0a                                foo.
0000000: 0a                                       .

Because you're selecting using a "case", it's possible these don't
affect you, however.

zsh makes it relatively straightforward to remove the trailing \r by
surrounding the command substitution with a parameter-style expansion:

% for f in ${$(print "foo\r")%%$'\r'}; do print $f | xxd; done
0000000: 666f 6f0a                                foo.

(That only works if \r is *not* in IFS.)

There's no mechanism for automatically turning \r\n into \n, however.

-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR PLC, Churchill House, Cambridge Business Park, Cowley Road
Cambridge, CB4 0WZ, UK                          Tel: +44 (0)1223 692070



**********************************************************************
This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they
are addressed. If you have received this email in error please notify
the system manager.

**********************************************************************

Re: Little problem while converting from bash (quoting/splitting?)

[Bart Schaefer]

On May 12,  9:28am, krasnal wrote:
}
} This is with 4.2.4 cygwin version of zsh and 2.05b.0(1)-release of bash.
} cleartool is windows program producing \r\n line terminated output.  Note
} that zsh has both \r and \n in f variable while bash has only \n.

Bash apparently reads stdin/stdout pipes in text mode, whereas zsh does so
in binary mode.  It's somewhat reasonable to assume pipes are text in a
shell context, but because zsh has so many ways to manipulate parameters,
I seem to recall a there was a conscious decision made to use text mode as
little as possible; in effect, it's used only when reading from files.

Which suggests a possible workaround; I don't have a cygwin zsh handy to
try it:

    for f in $(<=(cleartool lsvob -short | head -10)); do ...

Re: Little problem while converting from bash (quoting/splitting?)

[krasnal]

On Thu, 12 May 2005 10:39:08 +0100, Peter Stephenson wrote
> Well, you could add \r to IFS, but unfortunately that will generate extra
> blank lines, for example:

I haven't thought about that but I'd like to avoid that

> zsh makes it relatively straightforward to remove the trailing \r by
> surrounding the command substitution with a parameter-style expansion:

Yes I knew that :) but I was just wondering if there's some option to make
zsh act more like bash in that respect.  Thank you anyway!

Best regards
Andrzej
-- 
    ____   _  ___ 
   /  | \_/ |/ _ \   Andrzej Marek Ostruszka 
  / _ |     | (_) |       MPSC (Cracow)
 /_/ L|_|V|_|\___/  (GnuPG key ID: 0x3D9C498A)

Re: Little problem while converting from bash (quoting/splitting?)

[krasnal]

On Thu, 12 May 2005 16:29:11 +0000, Bart Schaefer wrote
> Bash apparently reads stdin/stdout pipes in text mode, whereas zsh 
> does so in binary mode.  It's somewhat reasonable to assume pipes 
> are text in a shell context, but because zsh has so many ways to 
> manipulate parameters, I seem to recall a there was a conscious 
> decision made to use text mode as little as possible; in effect, 
> it's used only when reading from files.

I understand the difference but I'm not sure I follow your arguments
here :)).  Why reading in text mode would be problematic?

> Which suggests a possible workaround; I don't have a cygwin zsh 
> handy to try it:
> 
>     for f in $(<=(cleartool lsvob -short | head -10)); do ...

Yes it helps.  Thank you.

Best regards
Andrzej
-- 
    ____   _  ___ 
   /  | \_/ |/ _ \   Andrzej Marek Ostruszka 
  / _ |     | (_) |       MPSC (Cracow)
 /_/ L|_|V|_|\___/  (GnuPG key ID: 0x3D9C498A)

Re: Little problem while converting from bash (quoting/splitting?)

[Bart Schaefer]

On May 15,  4:48pm, krasnal wrote:
} Subject: Re: Little problem while converting from bash (quoting/splitting?
}
} On Thu, 12 May 2005 16:29:11 +0000, Bart Schaefer wrote
} > Bash apparently reads stdin/stdout pipes in text mode, whereas zsh 
} > does so in binary mode.
} 
} I understand the difference but I'm not sure I follow your arguments
} here :)).  Why reading in text mode would be problematic?

Suppose you have 7663 bytes of data containing 97 \r\n sequences.  You
don't know whether it's text or binary data.

Read it in text mode, where \r\n is automatically converted to \n, and
stuff the entire contents into a variable.  That variable will contain
only 7566 bytes.  Information has been lost.

Zsh is capable of dealing sensibly with non-text values stored in
variables, so it was deemed more appropriate to treat text as binary
(no information loss) than the other way around.  The exception for
reading files is made because the Cygwin platform supports filesystems
that force all reads and writes to be in text mode.  Zsh doesn't know
what kind of filesystem it's reading/writing for any given file, so the
only way to assure that all file reads and writes behave the same is
to treat them all as text, regardless of the filesystem type.

Once the data is in a pipe, though, all filesystem-enforced conversions
can be assumed already to have occured, so it's safer to treat the data
as binary, and not convert it any further.

Re: Little problem while converting from bash (quoting/splitting?)

[krasnal]

On Sun, 15 May 2005 18:48:25 +0000, Bart Schaefer wrote
> On May 15,  4:48pm, krasnal wrote:
> } Subject: Re: Little problem while converting from bash (quoting/splitting?
> }
> } On Thu, 12 May 2005 16:29:11 +0000, Bart Schaefer wrote
> } > Bash apparently reads stdin/stdout pipes in text mode, whereas 
> zsh } > does so in binary mode. } } I understand the difference but 
> I'm not sure I follow your arguments } here :)).  Why reading in 
> text mode would be problematic?
> 
> Suppose you have 7663 bytes of data containing 97 \r\n sequences.  
> You don't know whether it's text or binary data.

Sorry Bart for provoking such an elaborate response from you :).  I understand
that, I just had some strange mixture of expectations in my mind
(partially based on beliving in word splitting on white space characters).
I think I got it correct by now and there were no more problems during
transition so I'm now using zsh in all systems I have access to :)).

Best regards
Andrzej

-- 
    ____   _  ___ 
   /  | \_/ |/ _ \   Andrzej Marek Ostruszka 
  / _ |     | (_) |       MPSC (Cracow)
 /_/ L|_|V|_|\___/  (GnuPG key ID: 0x3D9C498A)