Field splitting: extra empty field with final non-whitespace IFS character

Field splitting: extra empty field with final non-whitespace IFS character

[Martijn Dekker]

% emulate sh
% IFS=':'
% x='a:b::'
% set -- $x
% echo $#
4

zsh counts 4 fields instead of 3, like most other shells do. In other
words, zsh does not treat the non-whitespace IFS character as a field
terminator.

POSIX:
http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_06_05
"The shell shall treat each character of the IFS as a delimiter and use
the delimiters as field TERMINATORS to split the results of parameter
expansion and command substitution into fields." (emphasis mine)

bash, ash, dash, ksh93 and mksh act according to POSIX.

pdksh (which is obsolete) and yash act like zsh.

I think this should be changed to match the behaviour specified by POSIX
in sh and ksh emulation modes.

Thanks,

- Martijn

Re: Field splitting: extra empty field with final non-whitespace IFS character

[Bart Schaefer]

On Jun 22,  5:06am, Martijn Dekker wrote:
}
} % emulate sh
} % IFS=':'
} % x='a:b::'
} % set -- $x
} % echo $#
} 4
} 
} zsh counts 4 fields instead of 3, like most other shells do.

I don't really like the following patch because it's testing SHWORDSPLIT
at such a low level.  I *think* spacesplit() is called only for field
splitting, for which SHWORDSPLIT explicitly applies, so it's probably
OK, but there might be a code path I haven't found where this could
leak over into to word splitting (where SHWORDSPLIT should NOT apply).

A similar change woule be needed in bin_read(), but I haven't made the
attempt yet to figure out where amidst all the multibyte handling and
metafication the final empty-string element is assigned.

diff --git a/Src/utils.c b/Src/utils.c
index 13fc96a..f0a92d2 100644
--- a/Src/utils.c
+++ b/Src/utils.c
@@ -3208,7 +3208,7 @@ spacesplit(char *s, int allownull, int heap, int quote)
 	    *ptr = (heap ? (char *) hcalloc((s - t) + 1) :
 		    (char *) zshcalloc((s - t) + 1));
 	    ztrncpy(*ptr++, t, s - t);
-	} else
+	} else if (*s || !isset(SHWORDSPLIT))
 	    *ptr++ = dup(nulstring);
 	t = s;
 	skipwsep(&s);

-- 
Barton E. Schaefer