zsh-workers
 help / color / mirror / code / Atom feed
* Bug: Callback to widget set with `zle -Fw <widget>` shouldn't change $LASTWIDGET
@ 2022-11-16 15:06 Marlon Richert
  2022-11-16 18:24 ` Bart Schaefer
  2023-01-17  0:01 ` [PATCH] " Bart Schaefer
  0 siblings, 2 replies; 17+ messages in thread
From: Marlon Richert @ 2022-11-16 15:06 UTC (permalink / raw)
  To: Zsh hackers list

Currently, when the ZLE calls a widget set with `zle -Fw <widget>`,
this changes the value of $LASTWIDGET. This in turn breaks several of
the widgets listed in the manual under User Contributions. For
example:

% zsh -f
% autoload -Uz copy-earlier-word
% zle -N copy-earlier-word
% bindkey '\e,' copy-earlier-word
% # Pressing ^[, multiple times at this point successfully cycles
through the current words on the command line.
% handler() { local fd=$1; zle -F $fd; exec {fd}<&- }
% zle-line-pre-redraw() { local fd; exec {fd}< <( print ); zle -Fw $fd handler }
% zle -N handler
% zle -N zle-line-pre-redraw
% # Now ^[, still copies the last word on the line, but can no longer
cycle to previous words when pressed again.


^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: Bug: Callback to widget set with `zle -Fw <widget>` shouldn't change $LASTWIDGET
  2022-11-16 15:06 Bug: Callback to widget set with `zle -Fw <widget>` shouldn't change $LASTWIDGET Marlon Richert
@ 2022-11-16 18:24 ` Bart Schaefer
  2022-11-17 13:04   ` Marlon Richert
  2023-01-17  0:01 ` [PATCH] " Bart Schaefer
  1 sibling, 1 reply; 17+ messages in thread
From: Bart Schaefer @ 2022-11-16 18:24 UTC (permalink / raw)
  To: Marlon Richert; +Cc: Zsh hackers list

On Wed, Nov 16, 2022 at 7:08 AM Marlon Richert <marlon.richert@gmail.com> wrote:
>
> Currently, when the ZLE calls a widget set with `zle -Fw <widget>`,
> this changes the value of $LASTWIDGET.

Hrm, normally (running "zle WIDGET ..." explicitly) this would be
skipped (unless "zle WIDGET -w ...").  Too bad -w was overloaded in
this way.  It's not technically overloading because it's used in
different contexts, but mnemonically at least it could be confusing
that "zle -Fw FD WIDGET" and "zle WIDGET -w" would seem to have
inverted meanings of -w if we were to change the default behavior of
-F.

Of course it's also "-Fw" as a single "option", not "-F -w", so we
can't even do something like "zle -F +w" to indicate the desired
behavior.


^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: Bug: Callback to widget set with `zle -Fw <widget>` shouldn't change $LASTWIDGET
  2022-11-16 18:24 ` Bart Schaefer
@ 2022-11-17 13:04   ` Marlon Richert
  2022-11-17 13:28     ` Roman Perepelitsa
  2022-11-17 16:18     ` Bart Schaefer
  0 siblings, 2 replies; 17+ messages in thread
From: Marlon Richert @ 2022-11-17 13:04 UTC (permalink / raw)
  To: Bart Schaefer; +Cc: Zsh hackers list

The -w in `zle -Fw arg` is not the inverse of the -w in `zle -w arg`.
It has a different purpose:
* In `zle -F callback`, callback is a function and gets called as such.
* In `zle -Fw callback`, callback is a widget and gets called as such.

The latter form is necessary if you want to be able to call other ZLE
widgets from the callback or access other ZLE functionality. You
cannot do that with the first form.

It's not the "inverted meaning" here that is the bug. `zle -F` (with
or without -w) just shouldn't ever change $LASTWIDGET in the first
place. It breaks a lot of existing widget functions and there is no
possible workaround.

On Wed, Nov 16, 2022 at 8:25 PM Bart Schaefer <schaefer@brasslantern.com> wrote:
>
> On Wed, Nov 16, 2022 at 7:08 AM Marlon Richert <marlon.richert@gmail.com> wrote:
> >
> > Currently, when the ZLE calls a widget set with `zle -Fw <widget>`,
> > this changes the value of $LASTWIDGET.
>
> Hrm, normally (running "zle WIDGET ..." explicitly) this would be
> skipped (unless "zle WIDGET -w ...").  Too bad -w was overloaded in
> this way.  It's not technically overloading because it's used in
> different contexts, but mnemonically at least it could be confusing
> that "zle -Fw FD WIDGET" and "zle WIDGET -w" would seem to have
> inverted meanings of -w if we were to change the default behavior of
> -F.
>
> Of course it's also "-Fw" as a single "option", not "-F -w", so we
> can't even do something like "zle -F +w" to indicate the desired
> behavior.


^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: Bug: Callback to widget set with `zle -Fw <widget>` shouldn't change $LASTWIDGET
  2022-11-17 13:04   ` Marlon Richert
@ 2022-11-17 13:28     ` Roman Perepelitsa
  2022-11-17 16:18     ` Bart Schaefer
  1 sibling, 0 replies; 17+ messages in thread
From: Roman Perepelitsa @ 2022-11-17 13:28 UTC (permalink / raw)
  To: Marlon Richert; +Cc: Bart Schaefer, Zsh hackers list

On Thu, Nov 17, 2022 at 2:06 PM Marlon Richert <marlon.richert@gmail.com> wrote:
>
> It's not the "inverted meaning" here that is the bug. `zle -F` (with
> or without -w) just shouldn't ever change $LASTWIDGET in the first
> place. It breaks a lot of existing widget functions and there is no
> possible workaround.

Relevant: https://www.zsh.org/mla/workers/2019/msg00204.html

Especially this part:

> The same branch also adds -W option to zle widget command. This
> option instructs zle to keep LASTWIDGET unchanged.

This part of the patch wasn't applied and the code is long gone (I
didn't attach it to the email) but it's fairly straightforward to
implement.

Roman.


^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: Bug: Callback to widget set with `zle -Fw <widget>` shouldn't change $LASTWIDGET
  2022-11-17 13:04   ` Marlon Richert
  2022-11-17 13:28     ` Roman Perepelitsa
@ 2022-11-17 16:18     ` Bart Schaefer
  2023-01-11  7:45       ` Marlon Richert
  1 sibling, 1 reply; 17+ messages in thread
From: Bart Schaefer @ 2022-11-17 16:18 UTC (permalink / raw)
  To: Marlon Richert; +Cc: Zsh hackers list

On Thu, Nov 17, 2022 at 5:05 AM Marlon Richert <marlon.richert@gmail.com> wrote:
>
> The -w in `zle -Fw arg` is not the inverse of the -w in `zle -w arg`.
> It has a different purpose:

I KNOW that.  I was just pointing out potential future confusion.


^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: Bug: Callback to widget set with `zle -Fw <widget>` shouldn't change $LASTWIDGET
  2022-11-17 16:18     ` Bart Schaefer
@ 2023-01-11  7:45       ` Marlon Richert
  0 siblings, 0 replies; 17+ messages in thread
From: Marlon Richert @ 2023-01-11  7:45 UTC (permalink / raw)
  To: Bart Schaefer; +Cc: Zsh hackers list

On Thu, Nov 17, 2022 at 6:18 PM Bart Schaefer <schaefer@brasslantern.com> wrote:
>
> On Thu, Nov 17, 2022 at 5:05 AM Marlon Richert <marlon.richert@gmail.com> wrote:
> >
> > The -w in `zle -Fw arg` is not the inverse of the -w in `zle -w arg`.
> > It has a different purpose:
>
> I KNOW that.  I was just pointing out potential future confusion.

Aha. Sorry, that wasn't clear to me.


^ permalink raw reply	[flat|nested] 17+ messages in thread

* [PATCH] Re: Bug: Callback to widget set with `zle -Fw <widget>` shouldn't change $LASTWIDGET
  2022-11-16 15:06 Bug: Callback to widget set with `zle -Fw <widget>` shouldn't change $LASTWIDGET Marlon Richert
  2022-11-16 18:24 ` Bart Schaefer
@ 2023-01-17  0:01 ` Bart Schaefer
  2023-01-17  9:22   ` Peter Stephenson
  1 sibling, 1 reply; 17+ messages in thread
From: Bart Schaefer @ 2023-01-17  0:01 UTC (permalink / raw)
  To: Marlon Richert; +Cc: Zsh hackers list

Is this good enough, or is there a need to consider that the handler
might delete the previous last widget?  What does/should happen in
that case (or when a widget deletes itself, for that matter)?

diff --git a/Src/Zle/zle_main.c b/Src/Zle/zle_main.c
index 686c6f5b4..cfa0a739d 100644
--- a/Src/Zle/zle_main.c
+++ b/Src/Zle/zle_main.c
@@ -737,6 +737,7 @@ raw_getbyte(long do_keytmout, char *cptr, int full)
             ) {
             /* Handle the fd. */
             char *fdbuf;
+            Thingy save_lbindk = lbindk;
             {
                 char buf[BDIGBUFSIZE];
                 convbase(buf, lwatch_fd->fd, 10);
@@ -779,6 +780,7 @@ raw_getbyte(long do_keytmout, char *cptr, int full)
                  */
                 errtry = 1;
             }
+            lbindk = save_lbindk;
             }
         }
         /* Function may have invalidated the display. */


^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [PATCH] Re: Bug: Callback to widget set with `zle -Fw <widget>` shouldn't change $LASTWIDGET
  2023-01-17  0:01 ` [PATCH] " Bart Schaefer
@ 2023-01-17  9:22   ` Peter Stephenson
  2023-01-17 18:00     ` Bart Schaefer
  0 siblings, 1 reply; 17+ messages in thread
From: Peter Stephenson @ 2023-01-17  9:22 UTC (permalink / raw)
  To: Zsh hackers list

> On 17/01/2023 00:01 Bart Schaefer <schaefer@brasslantern.com> wrote:
> Is this good enough, or is there a need to consider that the handler
> might delete the previous last widget?  What does/should happen in
> that case (or when a widget deletes itself, for that matter)?

I can't think of a good way of handling that without a fair measure
of complexity both in the shell and for the user.  I think just keeping
the old name and letting the user deal with it further is good enough.

pws


^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [PATCH] Re: Bug: Callback to widget set with `zle -Fw <widget>` shouldn't change $LASTWIDGET
  2023-01-17  9:22   ` Peter Stephenson
@ 2023-01-17 18:00     ` Bart Schaefer
  2023-07-16 10:28       ` dana
  0 siblings, 1 reply; 17+ messages in thread
From: Bart Schaefer @ 2023-01-17 18:00 UTC (permalink / raw)
  To: Peter Stephenson; +Cc: Zsh hackers list

On Tue, Jan 17, 2023 at 1:23 AM Peter Stephenson
<p.w.stephenson@ntlworld.com> wrote:
>
> > On 17/01/2023 00:01 Bart Schaefer <schaefer@brasslantern.com> wrote:
> > Is this good enough, or is there a need to consider that the handler
> > might delete the previous last widget?
>
> I think just keeping
> the old name and letting the user deal with it further is good enough.

My concern is that the Thingy pointer becomes invalid.  If that won't
happen, nothing further is needed here.  If that COULD happen, then
the implementation of $LASTWIDGET needs to start storing the name
rather than the Thingy.


^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [PATCH] Re: Bug: Callback to widget set with `zle -Fw <widget>` shouldn't change $LASTWIDGET
  2023-01-17 18:00     ` Bart Schaefer
@ 2023-07-16 10:28       ` dana
  2023-07-17  8:42         ` Peter Stephenson
  2023-07-17 15:17         ` Bart Schaefer
  0 siblings, 2 replies; 17+ messages in thread
From: dana @ 2023-07-16 10:28 UTC (permalink / raw)
  To: Bart Schaefer; +Cc: Zsh hackers list, Peter Stephenson

On Tue 17 Jan 2023, at 12:00, Bart Schaefer wrote:
> My concern is that the Thingy pointer becomes invalid

I think this was justified

I've just tried running a new HEAD build for the first time in several months
and found that back-spacing causes the shell to crash. I'm afraid i don't
have a clean way to reproduce, as it seems to be triggered by something in my
profile, apparently an interaction between my (probably ancient) z-sy-h and
zsh-autosuggest installations, which produce sort of a web of interdependent
widgets -- but i've narrowed it down to this change

I assume whatever these functions are doing eventually results in us trying
to free the static name of the built-in widget? Trace below

Seems like the same issue as workers/51673 too

dana


...:~zsh % git checkout f93ad02b94bd18c96a0861506127e3a246fb8eec
...:~zsh % make clean && ./Util/preconfig && ./configure --enable-zsh-debug && make
...:~zsh % ZDOTDIR=$PWD lldb Src/zsh
(lldb) target create "Src/zsh"
Current executable set to '/Users/dana/Development/sf.net/zsh/zsh/Src/zsh' (arm64).
(lldb) run
Process 6843 launched: '/Users/dana/Development/sf.net/zsh/zsh/Src/zsh' (arm64)
[DEV] zsh % ZDOTDIR=$HOME; source ~/.zshrc
...:~zsh % echo foo bar.zsh(6843,0x1fac81e00) malloc: *** error for object 0x1005f0d78: pointer being freed was not allocated
zsh(6843,0x1fac81e00) malloc: *** set a breakpoint in malloc_error_break to debug
Process 6843 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = signal SIGABRT
    frame #0: 0x000000019fc70724 libsystem_kernel.dylib`__pthread_kill + 8
libsystem_kernel.dylib`:
->  0x19fc70724 <+8>:  b.lo   0x19fc70744               ; <+40>
    0x19fc70728 <+12>: pacibsp 
    0x19fc7072c <+16>: stp    x29, x30, [sp, #-0x10]!
    0x19fc70730 <+20>: mov    x29, sp
Target 0: (zsh) stopped.
(lldb) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = signal SIGABRT
  * frame #0: 0x000000019fc70724 libsystem_kernel.dylib`__pthread_kill + 8
    frame #1: 0x000000019fca7c28 libsystem_pthread.dylib`pthread_kill + 288
    frame #2: 0x000000019fbb5ae8 libsystem_c.dylib`abort + 180
    frame #3: 0x000000019fad6e28 libsystem_malloc.dylib`malloc_vreport + 908
    frame #4: 0x000000019fada6e8 libsystem_malloc.dylib`malloc_report + 64
    frame #5: 0x000000019fae6f20 libsystem_malloc.dylib`find_zone_and_free + 308
    frame #6: 0x000000010007a6f0 zsh`zsfree(p="backward-delete-char") at mem.c:1878:5
    frame #7: 0x00000001005d839c zle.so`freethingynode(hn=0x00000001005f8190) at zle_thingy.c:122:5
    frame #8: 0x00000001005d5688 zle.so`unrefthingy(th=0x00000001005f8190) at zle_thingy.c:150:2
    frame #9: 0x00000001005be834 zle.so`execzlefunc(func=0x00000001005f9fb8, args=0x00000001006ac300, set_bindk=1, set_lbindk=0) at zle_main.c:1560:2
    frame #10: 0x00000001005d7d38 zle.so`bin_zle_call(name="zle", args=0x00000001006ac300, ops=0x000000016fdf90e8, func='\0') at zle_thingy.c:806:11
    frame #11: 0x00000001005d5ed0 zle.so`bin_zle(name="zle", args=0x00000001006ac2f0, ops=0x000000016fdf90e8, func=0) at zle_thingy.c:388:12
    frame #12: 0x0000000100004068 zsh`execbuiltin(args=0x00000001006ac270, assigns=0x0000000000000000, bn=0x00000001005fe158) at builtin.c:506:13
    frame #13: 0x0000000100035d3c zsh`execcmd_exec(state=0x000000016fdf9b70, eparams=0x000000016fdf9818, input=0, output=0, how=2, last1=2, close_if_forked=-1) at exec.c:4187:17
    frame #14: 0x000000010003071c zsh`execpline2(state=0x000000016fdf9b70, pcode=195, how=2, input=0, output=0, last1=0) at exec.c:2003:2
    frame #15: 0x0000000100027c54 zsh`execpline(state=0x000000016fdf9b70, slcode=5154, how=2, last1=0) at exec.c:1728:5
    frame #16: 0x0000000100026a38 zsh`execlist(state=0x000000016fdf9b70, dont_change_job=1, exiting=0) at exec.c:1494:8
    frame #17: 0x0000000100026144 zsh`execode(p=0x0000600001736a80, dont_change_job=1, exiting=0, context="shfunc") at exec.c:1263:5
    frame #18: 0x000000010002e0e4 zsh`runshfunc(prog=0x0000600001736a80, wrap=0x0000000000000000, name="_zsh_highlight_call_widget") at exec.c:6113:5
    frame #19: 0x000000010002d81c zsh`doshfunc(shfunc=0x0000600001736a40, doshargs=0x0000000100678310, noreturnval=0) at exec.c:5960:2
    frame #20: 0x000000010002fd08 zsh`execshfunc(shf=0x0000600001736a40, args=0x0000000100678310) at exec.c:5529:5
    frame #21: 0x00000001000356b0 zsh`execcmd_exec(state=0x000000016fdfa800, eparams=0x000000016fdfa4a8, input=0, output=0, how=18, last1=2, close_if_forked=-1) at exec.c:4058:3
    frame #22: 0x000000010003071c zsh`execpline2(state=0x000000016fdfa800, pcode=67, how=18, input=0, output=0, last1=0) at exec.c:2003:2
    frame #23: 0x0000000100027c54 zsh`execpline(state=0x000000016fdfa800, slcode=6146, how=18, last1=0) at exec.c:1728:5
    frame #24: 0x00000001000269b4 zsh`execlist(state=0x000000016fdfa800, dont_change_job=1, exiting=0) at exec.c:1482:7
    frame #25: 0x0000000100026144 zsh`execode(p=0x0000600001736fc0, dont_change_job=1, exiting=0, context="shfunc") at exec.c:1263:5
    frame #26: 0x000000010002e0e4 zsh`runshfunc(prog=0x0000600001736fc0, wrap=0x0000000000000000, name="_zsh_highlight_widget_orig-s0.0000030000-r21548-backward-delete-char") at exec.c:6113:5
    frame #27: 0x000000010002d81c zsh`doshfunc(shfunc=0x0000600001737000, doshargs=0x0000000000000000, noreturnval=1) at exec.c:5960:2
    frame #28: 0x00000001005be6c0 zle.so`execzlefunc(func=0x0000600000c25bc0, args=0x00000001005ac640, set_bindk=0, set_lbindk=0) at zle_main.c:1533:12
    frame #29: 0x00000001005d7d38 zle.so`bin_zle_call(name="zle", args=0x00000001005ac640, ops=0x000000016fdfada8, func='\0') at zle_thingy.c:806:11
    frame #30: 0x00000001005d5ed0 zle.so`bin_zle(name="zle", args=0x00000001005ac630, ops=0x000000016fdfada8, func=0) at zle_thingy.c:388:12
    frame #31: 0x0000000100004068 zsh`execbuiltin(args=0x00000001005ac5d0, assigns=0x0000000000000000, bn=0x00000001005fe158) at builtin.c:506:13
    frame #32: 0x0000000100035d3c zsh`execcmd_exec(state=0x000000016fdfc240, eparams=0x000000016fdfb4d8, input=0, output=0, how=18, last1=2, close_if_forked=-1) at exec.c:4187:17
    frame #33: 0x000000010003071c zsh`execpline2(state=0x000000016fdfc240, pcode=643, how=18, input=0, output=0, last1=0) at exec.c:2003:2
    frame #34: 0x0000000100027c54 zsh`execpline(state=0x000000016fdfc240, slcode=6146, how=18, last1=0) at exec.c:1728:5
    frame #35: 0x00000001000269b4 zsh`execlist(state=0x000000016fdfc240, dont_change_job=1, exiting=0) at exec.c:1482:7
    frame #36: 0x00000001000720a0 zsh`execif(state=0x000000016fdfc240, do_exec=0) at loop.c:576:2
    frame #37: 0x00000001000353cc zsh`execcmd_exec(state=0x000000016fdfc240, eparams=0x000000016fdfbee8, input=0, output=0, how=18, last1=2, close_if_forked=-1) at exec.c:4007:13
    frame #38: 0x000000010003071c zsh`execpline2(state=0x000000016fdfc240, pcode=579, how=18, input=0, output=0, last1=0) at exec.c:2003:2
    frame #39: 0x0000000100027c54 zsh`execpline(state=0x000000016fdfc240, slcode=15362, how=18, last1=0) at exec.c:1728:5
    frame #40: 0x00000001000269b4 zsh`execlist(state=0x000000016fdfc240, dont_change_job=1, exiting=0) at exec.c:1482:7
    frame #41: 0x0000000100026144 zsh`execode(p=0x0000600001702080, dont_change_job=1, exiting=0, context="shfunc") at exec.c:1263:5
    frame #42: 0x000000010002e0e4 zsh`runshfunc(prog=0x0000600001702080, wrap=0x0000000000000000, name="_zsh_autosuggest_invoke_original_widget") at exec.c:6113:5
    frame #43: 0x000000010002d81c zsh`doshfunc(shfunc=0x00006000017020c0, doshargs=0x000000010059c720, noreturnval=0) at exec.c:5960:2
    frame #44: 0x000000010002fd08 zsh`execshfunc(shf=0x00006000017020c0, args=0x000000010059c720) at exec.c:5529:5
    frame #45: 0x00000001000356b0 zsh`execcmd_exec(state=0x000000016fdfced0, eparams=0x000000016fdfcb78, input=0, output=0, how=2, last1=2, close_if_forked=-1) at exec.c:4058:3
    frame #46: 0x000000010003071c zsh`execpline2(state=0x000000016fdfced0, pcode=1091, how=2, input=0, output=0, last1=0) at exec.c:2003:2
    frame #47: 0x0000000100027c54 zsh`execpline(state=0x000000016fdfced0, slcode=4098, how=2, last1=0) at exec.c:1728:5
    frame #48: 0x00000001000269b4 zsh`execlist(state=0x000000016fdfced0, dont_change_job=1, exiting=0) at exec.c:1482:7
    frame #49: 0x0000000100026144 zsh`execode(p=0x0000600001702400, dont_change_job=1, exiting=0, context="shfunc") at exec.c:1263:5
    frame #50: 0x000000010002e0e4 zsh`runshfunc(prog=0x0000600001702400, wrap=0x0000000000000000, name="_zsh_autosuggest_modify") at exec.c:6113:5
    frame #51: 0x000000010002d81c zsh`doshfunc(shfunc=0x0000600001702440, doshargs=0x00000001001e03c8, noreturnval=0) at exec.c:5960:2
    frame #52: 0x000000010002fd08 zsh`execshfunc(shf=0x0000600001702440, args=0x00000001001e03c8) at exec.c:5529:5
    frame #53: 0x00000001000356b0 zsh`execcmd_exec(state=0x000000016fdfdb60, eparams=0x000000016fdfd808, input=0, output=0, how=2, last1=2, close_if_forked=-1) at exec.c:4058:3
    frame #54: 0x000000010003071c zsh`execpline2(state=0x000000016fdfdb60, pcode=387, how=2, input=0, output=0, last1=0) at exec.c:2003:2
    frame #55: 0x0000000100027c54 zsh`execpline(state=0x000000016fdfdb60, slcode=4098, how=2, last1=0) at exec.c:1728:5
    frame #56: 0x00000001000269b4 zsh`execlist(state=0x000000016fdfdb60, dont_change_job=1, exiting=0) at exec.c:1482:7
    frame #57: 0x0000000100026144 zsh`execode(p=0x00006000017027c0, dont_change_job=1, exiting=0, context="shfunc") at exec.c:1263:5
    frame #58: 0x000000010002e0e4 zsh`runshfunc(prog=0x00006000017027c0, wrap=0x0000000000000000, name="_zsh_autosuggest_widget_modify") at exec.c:6113:5
    frame #59: 0x000000010002d81c zsh`doshfunc(shfunc=0x0000600001702800, doshargs=0x00000001001dc2b0, noreturnval=0) at exec.c:5960:2
    frame #60: 0x000000010002fd08 zsh`execshfunc(shf=0x0000600001702800, args=0x00000001001dc2b0) at exec.c:5529:5
    frame #61: 0x00000001000356b0 zsh`execcmd_exec(state=0x000000016fdfe7f0, eparams=0x000000016fdfe498, input=0, output=0, how=18, last1=2, close_if_forked=-1) at exec.c:4058:3
    frame #62: 0x000000010003071c zsh`execpline2(state=0x000000016fdfe7f0, pcode=131, how=18, input=0, output=0, last1=0) at exec.c:2003:2
    frame #63: 0x0000000100027c54 zsh`execpline(state=0x000000016fdfe7f0, slcode=5122, how=18, last1=0) at exec.c:1728:5
    frame #64: 0x00000001000269b4 zsh`execlist(state=0x000000016fdfe7f0, dont_change_job=1, exiting=0) at exec.c:1482:7
    frame #65: 0x0000000100026144 zsh`execode(p=0x0000600001740280, dont_change_job=1, exiting=0, context="shfunc") at exec.c:1263:5
    frame #66: 0x000000010002e0e4 zsh`runshfunc(prog=0x0000600001740280, wrap=0x0000000000000000, name="_zsh_autosuggest_bound_1_backward-delete-char") at exec.c:6113:5
    frame #67: 0x000000010002d81c zsh`doshfunc(shfunc=0x00006000017402c0, doshargs=0x0000000000000000, noreturnval=1) at exec.c:5960:2
    frame #68: 0x00000001005be6c0 zle.so`execzlefunc(func=0x00000001005f8190, args=0x00000001005fe710, set_bindk=0, set_lbindk=0) at zle_main.c:1533:12
    frame #69: 0x00000001005bec24 zle.so`zlecore at zle_main.c:1151:10
    frame #70: 0x00000001005bf7dc zle.so`zleread(lp=0x00000001000fad80, rp=0x00000001000fadc0, flags=3, context=0, init="zle-line-init", finish="zle-line-finish") at zle_main.c:1361:5
    frame #71: 0x00000001005c079c zle.so`zle_main_entry(cmd=1, ap="@\xa1\U0000000f") at zle_main.c:2137:9
    frame #72: 0x000000010005ba98 zsh`zleentry(cmd=1) at init.c:1648:8
    frame #73: 0x000000010005d848 zsh`inputline at input.c:424:15
    frame #74: 0x000000010005d330 zsh`ingetc at input.c:357:6
    frame #75: 0x000000010004c344 zsh`ihgetc at hist.c:415:13
    frame #76: 0x00000001000696b4 zsh`gettok at lex.c:622:12
    frame #77: 0x0000000100069328 zsh`zshlex at lex.c:275:8
    frame #78: 0x00000001000981ec zsh`parse_event(endtok=37) at parse.c:615:5
    frame #79: 0x00000001000572c0 zsh`loop(toplevel=1, justonce=0) at init.c:150:15
    frame #80: 0x000000010005bfc0 zsh`zsh_main(argc=1, argv=0x000000016fdff558) at init.c:1810:6
    frame #81: 0x000000010000306c zsh`main(argc=1, argv=0x000000016fdff558) at main.c:93:13
    frame #82: 0x000000019f94ff28 dyld`start + 2236


^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [PATCH] Re: Bug: Callback to widget set with `zle -Fw <widget>` shouldn't change $LASTWIDGET
  2023-07-16 10:28       ` dana
@ 2023-07-17  8:42         ` Peter Stephenson
  2023-07-17 15:17         ` Bart Schaefer
  1 sibling, 0 replies; 17+ messages in thread
From: Peter Stephenson @ 2023-07-17  8:42 UTC (permalink / raw)
  To: Zsh hackers list

> On 16/07/2023 11:28 dana <dana@dana.is> wrote:
> On Tue 17 Jan 2023, at 12:00, Bart Schaefer wrote:
> > My concern is that the Thingy pointer becomes invalid
> 
> I think this was justified
> 
> I've just tried running a new HEAD build for the first time in several months
> and found that back-spacing causes the shell to crash.

If this happens again, you might get some information by going back up the
stack in the debugger and see what was running in any runshfunc(s) or
doshfunc(s) there.  You probably just need the name, then you can retrieve
the contents of that function from a live shell.

Having said that, Bart's previous suggestion of saving the name instead of
the pointer might be the fix anyway.

pws


^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [PATCH] Re: Bug: Callback to widget set with `zle -Fw <widget>` shouldn't change $LASTWIDGET
  2023-07-16 10:28       ` dana
  2023-07-17  8:42         ` Peter Stephenson
@ 2023-07-17 15:17         ` Bart Schaefer
  2023-07-17 15:52           ` dana
  1 sibling, 1 reply; 17+ messages in thread
From: Bart Schaefer @ 2023-07-17 15:17 UTC (permalink / raw)
  To: dana; +Cc: Zsh hackers list, Peter Stephenson

On Sun, Jul 16, 2023 at 3:29 AM dana <dana@dana.is> wrote:
>
> On Tue 17 Jan 2023, at 12:00, Bart Schaefer wrote:
> > My concern is that the Thingy pointer becomes invalid
>
> I think this was justified
>
> I've just tried running a new HEAD build for the first time in several months
> and found that back-spacing causes the shell to crash.

The patch from the start of this thread has never been committed, so
is not directly related.  That is, if the Thingy is invalid, it's not
$LASTWIDGET that's making the bad reference here, so fixing
save/restore there probably will not resolve the problem.

In your backtrace I see
  _zsh_autosuggest_widget_modify
and
  _zsh_autosuggest_invoke_original_widget
followed by
  _zsh_highlight_widget_orig-s0.0000030000-r21548-backward-delete-char
and finally
  _zsh_highlight_call_widget
causes the actual crash below unrefthingy(), so I suspect this is a
reference-counting problem.


^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [PATCH] Re: Bug: Callback to widget set with `zle -Fw <widget>` shouldn't change $LASTWIDGET
  2023-07-17 15:17         ` Bart Schaefer
@ 2023-07-17 15:52           ` dana
  2023-07-17 15:57             ` Bart Schaefer
  0 siblings, 1 reply; 17+ messages in thread
From: dana @ 2023-07-17 15:52 UTC (permalink / raw)
  To: Bart Schaefer; +Cc: Zsh hackers list, Peter Stephenson

On Mon 17 Jul 2023, at 10:17, Bart Schaefer wrote:
> The patch from the start of this thread has never been committed, so
> is not directly related.  That is, if the Thingy is invalid, it's not
> $LASTWIDGET that's making the bad reference here, so fixing
> save/restore there probably will not resolve the problem.

Sorry, i'm not very familiar with the LASTWIDGET mechanism, but by the patch
at the start of this thread you mean workers/51310, right? It was committed
as f93ad02b94bd

I should have mentioned that i did a (manual) bisect to confirm that that is
when the problem started for me; the poster in workers/51673 found the same,
with what sounds like a very similar configuration (though they didn't
provide any other details). I can't get the problem to occur on a build off
the preceding commit or on the 5.9 release

I did examine the functions the trace references before; the
_zsh_highlight_widget_orig... one simply calls _zsh_highlight_call_widget,
and that one calls zle before calling another highlight function (but the
crash happens in zle so it's not getting to that). The autosuggest ones are a
little more elaborate. I can get back to it later

dana


^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [PATCH] Re: Bug: Callback to widget set with `zle -Fw <widget>` shouldn't change $LASTWIDGET
  2023-07-17 15:52           ` dana
@ 2023-07-17 15:57             ` Bart Schaefer
  2023-07-17 16:57               ` Bart Schaefer
  0 siblings, 1 reply; 17+ messages in thread
From: Bart Schaefer @ 2023-07-17 15:57 UTC (permalink / raw)
  To: dana; +Cc: Zsh hackers list, Peter Stephenson

On Mon, Jul 17, 2023 at 8:52 AM dana <dana@dana.is> wrote:
>
> Sorry, i'm not very familiar with the LASTWIDGET mechanism, but by the patch
> at the start of this thread you mean workers/51310, right? It was committed
> as f93ad02b94bd

(Facepalm)  Looked at the wrong source file.

> I should have mentioned that i did a (manual) bisect to confirm that that is
> when the problem started for me; the poster in workers/51673 found the same

OK, then it IS a reference counting problem.  Try this.

diff --git a/Src/Zle/zle_main.c b/Src/Zle/zle_main.c
index 4a6c02133..6e554f4b8 100644
--- a/Src/Zle/zle_main.c
+++ b/Src/Zle/zle_main.c
@@ -737,7 +737,7 @@ raw_getbyte(long do_keytmout, char *cptr, int full)
             ) {
             /* Handle the fd. */
             char *fdbuf;
-            Thingy save_lbindk = lbindk;
+            Thingy save_lbindk = refthingy(lbindk);
             {
                 char buf[BDIGBUFSIZE];
                 convbase(buf, lwatch_fd->fd, 10);
@@ -781,6 +781,7 @@ raw_getbyte(long do_keytmout, char *cptr, int full)
                 errtry = 1;
             }
             lbindk = save_lbindk;
+            unrefthingy(save_lbindk);
             }
         }
         /* Function may have invalidated the display. */


^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [PATCH] Re: Bug: Callback to widget set with `zle -Fw <widget>` shouldn't change $LASTWIDGET
  2023-07-17 15:57             ` Bart Schaefer
@ 2023-07-17 16:57               ` Bart Schaefer
  2023-07-20  4:01                 ` dana
  0 siblings, 1 reply; 17+ messages in thread
From: Bart Schaefer @ 2023-07-17 16:57 UTC (permalink / raw)
  To: dana; +Cc: Zsh hackers list, Peter Stephenson

On Mon, Jul 17, 2023 at 8:57 AM Bart Schaefer <schaefer@brasslantern.com> wrote:
>
>              lbindk = save_lbindk;
> +            unrefthingy(save_lbindk);

Hm, looking at redrawhook(), that perhaps should be

> +            unrefthingy(lbindk);
>              lbindk = save_lbindk;


^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [PATCH] Re: Bug: Callback to widget set with `zle -Fw <widget>` shouldn't change $LASTWIDGET
  2023-07-17 16:57               ` Bart Schaefer
@ 2023-07-20  4:01                 ` dana
  2023-07-20  4:25                   ` Bart Schaefer
  0 siblings, 1 reply; 17+ messages in thread
From: dana @ 2023-07-20  4:01 UTC (permalink / raw)
  To: Bart Schaefer; +Cc: Zsh hackers list, Peter Stephenson

On Mon 17 Jul 2023, at 11:57, Bart Schaefer wrote:
> Hm, looking at redrawhook(), that perhaps should be

That (the second one) seems to have fixed the crash, thank you

dana

PS: I don't know if it's just my client but your first patch had corrupt
white space for me


^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [PATCH] Re: Bug: Callback to widget set with `zle -Fw <widget>` shouldn't change $LASTWIDGET
  2023-07-20  4:01                 ` dana
@ 2023-07-20  4:25                   ` Bart Schaefer
  0 siblings, 0 replies; 17+ messages in thread
From: Bart Schaefer @ 2023-07-20  4:25 UTC (permalink / raw)
  To: dana; +Cc: Zsh hackers list, Peter Stephenson

On Wed, Jul 19, 2023 at 9:01 PM dana <dana@dana.is> wrote:
>
> That (the second one) seems to have fixed the crash, thank you

Thanks for confirming.

> PS: I don't know if it's just my client but your first patch had corrupt
> white space for me

Probably because I just pasted into gmail instead of attaching.  Same
thing might happen with the "piping NUL character" patch (which is
actually a read-with-delimiter patch).


^ permalink raw reply	[flat|nested] 17+ messages in thread

end of thread, other threads:[~2023-07-20  4:26 UTC | newest]

Thread overview: 17+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-11-16 15:06 Bug: Callback to widget set with `zle -Fw <widget>` shouldn't change $LASTWIDGET Marlon Richert
2022-11-16 18:24 ` Bart Schaefer
2022-11-17 13:04   ` Marlon Richert
2022-11-17 13:28     ` Roman Perepelitsa
2022-11-17 16:18     ` Bart Schaefer
2023-01-11  7:45       ` Marlon Richert
2023-01-17  0:01 ` [PATCH] " Bart Schaefer
2023-01-17  9:22   ` Peter Stephenson
2023-01-17 18:00     ` Bart Schaefer
2023-07-16 10:28       ` dana
2023-07-17  8:42         ` Peter Stephenson
2023-07-17 15:17         ` Bart Schaefer
2023-07-17 15:52           ` dana
2023-07-17 15:57             ` Bart Schaefer
2023-07-17 16:57               ` Bart Schaefer
2023-07-20  4:01                 ` dana
2023-07-20  4:25                   ` Bart Schaefer

Code repositories for project(s) associated with this public inbox

	https://git.vuxu.org/mirror/zsh/

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).