zsh-workers
 help / color / mirror / Atom feed
* PATCH: Fix handling of NUL bytes in zexpandtabs multibyte version
@ 2021-03-12  2:51 Mikael Magnusson
  2021-03-12  3:17 ` Mikael Magnusson
  0 siblings, 1 reply; 8+ messages in thread
From: Mikael Magnusson @ 2021-03-12  2:51 UTC (permalink / raw)
  To: zsh-workers

Problem found by dg1727 on #zsh.

---

I checked other uses of zexpandtabs in the same file only and they all
seem to handle the 0 return correctly, and some of them even comment on
it, so fairly confident this is the right fix.

---
 Src/utils.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/Src/utils.c b/Src/utils.c
index 8fcea5362a..f8cbe65ba5 100644
--- a/Src/utils.c
+++ b/Src/utils.c
@@ -5936,8 +5936,11 @@ zexpandtabs(const char *s, int len, int width, int startpos, FILE *fout,
 		memset(&mbs, 0, sizeof(mbs));
 		s++;
 		len--;
-	    } else if (ret == MB_INCOMPLETE) {
+	    } else if (ret == MB_INCOMPLETE ||
 		/* incomplete at end --- assume likewise, best we've got */
+	               ret == 0) {
+		/* NUL character returns 0, which would loop infinitely, so advance
+		 * one byte in this case too */
 		s++;
 		len--;
 	    } else {
-- 
2.15.1



^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: PATCH: Fix handling of NUL bytes in zexpandtabs multibyte version
  2021-03-12  2:51 PATCH: Fix handling of NUL bytes in zexpandtabs multibyte version Mikael Magnusson
@ 2021-03-12  3:17 ` Mikael Magnusson
  2021-03-12 13:53   ` Daniel Shahaf
  0 siblings, 1 reply; 8+ messages in thread
From: Mikael Magnusson @ 2021-03-12  3:17 UTC (permalink / raw)
  To: zsh-workers

On 3/12/21, Mikael Magnusson <mikachu@gmail.com> wrote:
> Problem found by dg1727 on #zsh.

Forgot the actual problem part,

print -x2 '\0'

is the reduced code that causes it.


^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: PATCH: Fix handling of NUL bytes in zexpandtabs multibyte version
  2021-03-12  3:17 ` Mikael Magnusson
@ 2021-03-12 13:53   ` Daniel Shahaf
  2021-03-20  1:22     ` Lawrence Velázquez
  2021-03-20 15:15     ` PATCH v2: 48168: " Mikael Magnusson
  0 siblings, 2 replies; 8+ messages in thread
From: Daniel Shahaf @ 2021-03-12 13:53 UTC (permalink / raw)
  To: zsh-workers

Mikael Magnusson wrote on Fri, 12 Mar 2021 03:17 +00:00:
> On 3/12/21, Mikael Magnusson <mikachu@gmail.com> wrote:
> > Problem found by dg1727 on #zsh.
> 
> Forgot the actual problem part,
> 
> print -x2 '\0'
> 
> is the reduced code that causes it.

Regression test?



^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: PATCH: Fix handling of NUL bytes in zexpandtabs multibyte version
  2021-03-12 13:53   ` Daniel Shahaf
@ 2021-03-20  1:22     ` Lawrence Velázquez
  2021-03-20 15:15     ` PATCH v2: 48168: " Mikael Magnusson
  1 sibling, 0 replies; 8+ messages in thread
From: Lawrence Velázquez @ 2021-03-20  1:22 UTC (permalink / raw)
  To: Mikael Magnusson; +Cc: Daniel Shahaf, zsh-workers

On Fri, Mar 12, 2021, at 8:53 AM, Daniel Shahaf wrote:
> Mikael Magnusson wrote on Fri, 12 Mar 2021 03:17 +00:00:
> > On 3/12/21, Mikael Magnusson <mikachu@gmail.com> wrote:
> > > Problem found by dg1727 on #zsh.
> > 
> > Forgot the actual problem part,
> > 
> > print -x2 '\0'
> > 
> > is the reduced code that causes it.
> 
> Regression test?

ping

vq


^ permalink raw reply	[flat|nested] 8+ messages in thread

* PATCH v2: 48168: Fix handling of NUL bytes in zexpandtabs multibyte version
  2021-03-12 13:53   ` Daniel Shahaf
  2021-03-20  1:22     ` Lawrence Velázquez
@ 2021-03-20 15:15     ` Mikael Magnusson
  2021-03-27 16:11       ` Lawrence Velázquez
  1 sibling, 1 reply; 8+ messages in thread
From: Mikael Magnusson @ 2021-03-20 15:15 UTC (permalink / raw)
  To: zsh-workers

---

 Src/utils.c        | 5 ++++-
 Test/B03print.ztst | 9 +++++----
 2 files changed, 9 insertions(+), 5 deletions(-)

diff --git a/Src/utils.c b/Src/utils.c
index 8fcea5362a..f8cbe65ba5 100644
--- a/Src/utils.c
+++ b/Src/utils.c
@@ -5936,8 +5936,11 @@ zexpandtabs(const char *s, int len, int width, int startpos, FILE *fout,
 		memset(&mbs, 0, sizeof(mbs));
 		s++;
 		len--;
-	    } else if (ret == MB_INCOMPLETE) {
+	    } else if (ret == MB_INCOMPLETE ||
 		/* incomplete at end --- assume likewise, best we've got */
+	               ret == 0) {
+		/* NUL character returns 0, which would loop infinitely, so advance
+		 * one byte in this case too */
 		s++;
 		len--;
 	    } else {
diff --git a/Test/B03print.ztst b/Test/B03print.ztst
index 5634239346..d8618de7d4 100644
--- a/Test/B03print.ztst
+++ b/Test/B03print.ztst
@@ -304,15 +304,16 @@
  foo=$'one\ttwo\tthree\tfour\n'
  foo+=$'\tone\ttwo\tthree\tfour\n'
  foo+=$'\t\tone\t\ttwo\t\tthree\t\tfour'
- print -x4 $foo
- print -X4 $foo
+ foo+='\0' # regression test for multibyte tab expand
+ print -x4 $foo | sed 's/\x00/nul/' # avoid raw nul byte in expected output below
+ print -X4 $foo | sed 's/\x00/nul/'
 0:Tab expansion by print
 >one	two	three	four
 >    one	two	three	four
->        one		two		three		four
+>        one		two		three		fournul
 >one two three   four
 >    one two three   four
->        one     two     three       four
+>        one     two     three       fournul
 
  unset foo
  print -v foo once more
-- 
2.15.1



^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: PATCH v2: 48168: Fix handling of NUL bytes in zexpandtabs multibyte version
  2021-03-20 15:15     ` PATCH v2: 48168: " Mikael Magnusson
@ 2021-03-27 16:11       ` Lawrence Velázquez
  2021-04-01  5:21         ` Jun T
  0 siblings, 1 reply; 8+ messages in thread
From: Lawrence Velázquez @ 2021-03-27 16:11 UTC (permalink / raw)
  To: zsh-workers; +Cc: Mikael Magnusson

On Sat, Mar 20, 2021, at 11:15 AM, Mikael Magnusson wrote:
> ---
> 
>  Src/utils.c        | 5 ++++-
>  Test/B03print.ztst | 9 +++++----
>  2 files changed, 9 insertions(+), 5 deletions(-)

ping for review

vq


^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: PATCH v2: 48168: Fix handling of NUL bytes in zexpandtabs multibyte version
  2021-03-27 16:11       ` Lawrence Velázquez
@ 2021-04-01  5:21         ` Jun T
  2021-04-01  7:46           ` Mikael Magnusson
  0 siblings, 1 reply; 8+ messages in thread
From: Jun T @ 2021-04-01  5:21 UTC (permalink / raw)
  To: zsh-workers

The test in B03 fails on macOS and FreeBSD (and maybe on other BSDs)
because 'sed' can't handle the null byte.

It seems 'tr' is more portable.


diff --git a/Test/B03print.ztst b/Test/B03print.ztst
index 563423934..4d2cf9764 100644
--- a/Test/B03print.ztst
+++ b/Test/B03print.ztst
@@ -304,15 +304,16 @@
  foo=$'one\ttwo\tthree\tfour\n'
  foo+=$'\tone\ttwo\tthree\tfour\n'
  foo+=$'\t\tone\t\ttwo\t\tthree\t\tfour'
- print -x4 $foo
- print -X4 $foo
+ foo+='\0' # regression test for multibyte tab expand
+ print -x4 $foo | tr '\0' Z # avoid raw nul byte in expected output below
+ print -X4 $foo | tr '\0' Z
 0:Tab expansion by print
 >one	two	three	four
 >    one	two	three	four
->        one		two		three		four
+>        one		two		three		fourZ
 >one two three   four
 >    one two three   four
->        one     two     three       four
+>        one     two     three       fourZ
 
  unset foo
  print -v foo once more





^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: PATCH v2: 48168: Fix handling of NUL bytes in zexpandtabs multibyte version
  2021-04-01  5:21         ` Jun T
@ 2021-04-01  7:46           ` Mikael Magnusson
  0 siblings, 0 replies; 8+ messages in thread
From: Mikael Magnusson @ 2021-04-01  7:46 UTC (permalink / raw)
  To: Jun T; +Cc: zsh-workers

On 4/1/21, Jun T <takimoto-j@kba.biglobe.ne.jp> wrote:
> The test in B03 fails on macOS and FreeBSD (and maybe on other BSDs)
> because 'sed' can't handle the null byte.
>
> It seems 'tr' is more portable.
>
>
> diff --git a/Test/B03print.ztst b/Test/B03print.ztst
> index 563423934..4d2cf9764 100644
> --- a/Test/B03print.ztst
> +++ b/Test/B03print.ztst
> @@ -304,15 +304,16 @@
>   foo=$'one\ttwo\tthree\tfour\n'
>   foo+=$'\tone\ttwo\tthree\tfour\n'
>   foo+=$'\t\tone\t\ttwo\t\tthree\t\tfour'
> - print -x4 $foo
> - print -X4 $foo
> + foo+='\0' # regression test for multibyte tab expand
> + print -x4 $foo | tr '\0' Z # avoid raw nul byte in expected output below
> + print -X4 $foo | tr '\0' Z
>  0:Tab expansion by print
>  >one	two	three	four
>  >    one	two	three	four
> ->        one		two		three		four
> +>        one		two		three		fourZ
>  >one two three   four
>  >    one two three   four
> ->        one     two     three       four
> +>        one     two     three       fourZ
>
>   unset foo
>   print -v foo once more

Thanks for checking, I'll use this version instead.

-- 
Mikael Magnusson


^ permalink raw reply	[flat|nested] 8+ messages in thread

end of thread, other threads:[~2021-04-01  7:46 UTC | newest]

Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-03-12  2:51 PATCH: Fix handling of NUL bytes in zexpandtabs multibyte version Mikael Magnusson
2021-03-12  3:17 ` Mikael Magnusson
2021-03-12 13:53   ` Daniel Shahaf
2021-03-20  1:22     ` Lawrence Velázquez
2021-03-20 15:15     ` PATCH v2: 48168: " Mikael Magnusson
2021-03-27 16:11       ` Lawrence Velázquez
2021-04-01  5:21         ` Jun T
2021-04-01  7:46           ` Mikael Magnusson

zsh-workers

This inbox may be cloned and mirrored by anyone:

	git clone --mirror http://inbox.vuxu.org/zsh-workers

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V1 zsh-workers zsh-workers/ http://inbox.vuxu.org/zsh-workers \
		zsh-workers@zsh.org
	public-inbox-index zsh-workers

Example config snippet for mirrors.
Newsgroup available over NNTP:
	nntp://inbox.vuxu.org/vuxu.archive.zsh.workers


code repositories for the project(s) associated with this inbox:

	https://git.vuxu.org/mirror/zsh/

AGPL code for this site: git clone https://public-inbox.org/public-inbox.git