How to iterate over lines cross-platform

How to iterate over lines cross-platform

[Thorsten Kampe]

Hi,

I wrote a Zsh shell script which iterates over the output of an external 
tool ("slptool"). Each line of output is an item so I did this:

###
IFS=$'\n'

for scope in $(slptool findscopes); do
    for srvtype in $(slptool -s "$scope" findsrvtypes); do
        [inner loop]
    done
done
###

This works fine on Linux. Unfortunately the script does not works under 
Cygwin. It never enters the "for srvtype" loop (or it doesn't produce 
output). The reason is that the slptool on Windows is a native Win32 
application that outputs lines with "\r\n" endings (and not "\n").

So I simply set "IFS=$'\r\n'" and now the output is correct - but the 
scipt goes through an additional iteration for each loop so output is 
duplicated or results in an error at the end. I suspect that the shell 
does not split on "\r\n" but either on "\r" or "\n".

How can I iterate over lines of output when I don't know in advance 
whether the line endings will be DOS or Unix?!


Thorsten

Re: How to iterate over lines cross-platform

[Bart Schaefer]

On Apr 10,  7:36pm, Thorsten Kampe wrote:
}
} How can I iterate over lines of output when I don't know in advance 
} whether the line endings will be DOS or Unix?!

You might have better luck if you do not change IFS from the default,
and use this:

slptool findscopes |
while read scope; do
    slptool -s "$scope" findsrvtypes |
    while read srvtype; do
        [inner loop]
    done
done

It should be the case that "read" does the right thing with line ends.

} The reason is that the slptool on Windows is a native Win32 
} application that outputs lines with "\r\n" endings (and not "\n").

I believe this means that "$scope" has a trailing "\r" and therefore
slptool -s "$scope" doesn't find a matching scope.  If it turns out
that "read" also mishandles things, use ${scope%$'\r'} to trim.
 
} So I simply set "IFS=$'\r\n'" and now the output is correct - but the 
} scipt goes through an additional iteration for each loop

Yes, as you suspected, IFS=$'\r\n' means that each of "\r" and "\n" is
individually taken as a separator, so "foo\r\n" splits into "foo" and
(empty string).

Re: How to iterate over lines cross-platform

[Thorsten Kampe]

* Bart Schaefer (Sat, 10 Apr 2010 13:05:24 -0700)
> On Apr 10,  7:36pm, Thorsten Kampe wrote:
> } How can I iterate over lines of output when I don't know in advance 
> } whether the line endings will be DOS or Unix?!
> 
> You might have better luck if you do not change IFS from the default,
> and use this:
> 
> slptool findscopes |
> while read scope; do
>     slptool -s "$scope" findsrvtypes |
>     while read srvtype; do
>         [inner loop]
>     done
> done

Actually modifying IFS is not necessary (shame on me for the confusion).
 
> It should be the case that "read" does the right thing with line ends.
> 
> } The reason is that the slptool on Windows is a native Win32 
> } application that outputs lines with "\r\n" endings (and not "\n").
> 
> I believe this means that "$scope" has a trailing "\r" and therefore
> slptool -s "$scope" doesn't find a matching scope.  If it turns out
> that "read" also mishandles things, use ${scope%$'\r'} to trim.

I didn't try "read" but "%$'\r'" does the "trick".

Thanks, Thorsten