read options to slurp empty lines ?

read options to slurp empty lines ?

[Marc Chantreux]

hello people,

i'm very unhappy with the patch i made on uze.zsh today

https://github.com/zsh-uze/uze/commit/b695fff8d4b2b5dd91ca9033c8d98ac3c3badeef#diff-ea0555d620dfb73b2eda6e587c48d7b1L40

i use to have slurp function made like this

slurp     () IFS=$'\n' read -r -d ' -A $1

so i am able to write

    print -l foo bar bang | slurp them
    # got them=( foo bar bang )

i recently saw that

    print -l '' bar bang '' | slurp them
    # got them=( bar bang )
    # expected them=( '' bar bang '' )

i tried to patch it using some read options and

    slurp () {
        local it
        local n=${1:-them}
        set --
        while { IFS= read -r it }
        do
            set -- "$@" "$it"
        done
        set -A $n "$@"
    }


which is way much longer. did i miss something using read builtin ?

regards
marc

Re: read options to slurp empty lines ?

[Bart Schaefer]

On Fri, Jun 1, 2018 at 2:47 PM, Marc Chantreux <eiro@phear.org> wrote:
>
> i use to have slurp function made like this
>
> slurp     () IFS=$'\n' read -r -d ' -A $1

I presume that to be a typo or cut-and-paste error because you have an
unmatched single quote there.

> i recently saw that
>
>     print -l '' bar bang '' | slurp them
>     # got them=( bar bang )
>     # expected them=( '' bar bang '' )

For the confused or those with proportional fonts, '' above is two
single quotes, not a double quote, so there should be four lines
output by "print -l".

I get

% typeset -p them
typeset -a them=( bar bang '' )

The behavior of "read" is to collapse multiple consecutive occurrences
of $IFS characters into a single one.  Because an empty string
followed by a newline looks like nothing more than a newline, all
those lines collapse.  [The single empty element I got at the end is
there not because there were trailing empty lines, but because end of
input was reached without finding the delimiter (this is not standard
so probably ought to be controlled by POSIX_BUILTINS or something).]
So for example:

% print -l x '' '' '' bing bong '' '' '' y | slurp
% typeset -p reply
typeset -a reply=( x bing bong y '' )

Same thing happens in e.g. bash (except without the final empty
element, see above).

> i tried to patch it using some read options ... did i miss something using read builtin ?

Only that there's no simple way to make it do what you want. :-)

You're probably looking for this:

slurp() {
  local REPLY
  IFS= read -d '' &&
  set -A ${1:-reply} "${(@f)REPLY}"
}

Re: read options to slurp empty lines ?

[Marc Chantreux]

> > slurp     () IFS=$'\n' read -r -d ' -A $1
> I presume that to be a typo or cut-and-paste error because you have an
> unmatched single quote there.

indeed!

    slurp     () IFS=$'\n' read -r -d '' -A $1

> % typeset -p them
> typeset -a them=( bar bang '' )

typeset -p is awesome! thanks for this :)

> The behavior of "read" is to collapse multiple consecutive occurrences
> of $IFS characters into a single one.

and then i feel dump because i know it :(

> Only that there's no simple way to make it do what you want. :-)
> You're probably looking for this:

> slurp() {
>   local REPLY
>   IFS= read -d '' &&
>   set -A ${1:-reply} "${(@f)REPLY}"
> }

much more elegant that mine. thank you!
marc

Re: read options to slurp empty lines ?

[Marc Chantreux]

hello Bart,

> slurp() {
>   local REPLY
>   IFS= read -d '' &&
>   set -A ${1:-reply} "${(@f)REPLY}"
> }

i added this code in uze.zsh and spotted a bug thanks to my test
suite (https://github.com/zsh-uze/uze/blob/master/t/100_dump.t)
the thing is the first line doesn't survive to the (@f) modifier
so:

    print -l '' a '' b | read -r -d '' lines
    for it ( "${(@f)lines}" ) l "* ($it)"

got:

    * (a)
    * ()
    * (b)

expected:

    * ()
    * (a)
    * ()
    * (b)

so i reverted the code for the moment waiting for
the moment i can use a code closer than yours.

regards
marc