Bug: Callback to widget set with `zle -Fw <widget>` shouldn't change $LASTWIDGET

Bug: Callback to widget set with `zle -Fw <widget>` shouldn't change $LASTWIDGET

[Marlon Richert]

Currently, when the ZLE calls a widget set with `zle -Fw <widget>`,
this changes the value of $LASTWIDGET. This in turn breaks several of
the widgets listed in the manual under User Contributions. For
example:

% zsh -f
% autoload -Uz copy-earlier-word
% zle -N copy-earlier-word
% bindkey '\e,' copy-earlier-word
% # Pressing ^[, multiple times at this point successfully cycles
through the current words on the command line.
% handler() { local fd=$1; zle -F $fd; exec {fd}<&- }
% zle-line-pre-redraw() { local fd; exec {fd}< <( print ); zle -Fw $fd handler }
% zle -N handler
% zle -N zle-line-pre-redraw
% # Now ^[, still copies the last word on the line, but can no longer
cycle to previous words when pressed again.

Re: Bug: Callback to widget set with `zle -Fw <widget>` shouldn't change $LASTWIDGET

[Bart Schaefer]

On Wed, Nov 16, 2022 at 7:08 AM Marlon Richert <marlon.richert@gmail.com> wrote:
>
> Currently, when the ZLE calls a widget set with `zle -Fw <widget>`,
> this changes the value of $LASTWIDGET.

Hrm, normally (running "zle WIDGET ..." explicitly) this would be
skipped (unless "zle WIDGET -w ...").  Too bad -w was overloaded in
this way.  It's not technically overloading because it's used in
different contexts, but mnemonically at least it could be confusing
that "zle -Fw FD WIDGET" and "zle WIDGET -w" would seem to have
inverted meanings of -w if we were to change the default behavior of
-F.

Of course it's also "-Fw" as a single "option", not "-F -w", so we
can't even do something like "zle -F +w" to indicate the desired
behavior.

Re: Bug: Callback to widget set with `zle -Fw <widget>` shouldn't change $LASTWIDGET

[Marlon Richert]

The -w in `zle -Fw arg` is not the inverse of the -w in `zle -w arg`.
It has a different purpose:
* In `zle -F callback`, callback is a function and gets called as such.
* In `zle -Fw callback`, callback is a widget and gets called as such.

The latter form is necessary if you want to be able to call other ZLE
widgets from the callback or access other ZLE functionality. You
cannot do that with the first form.

It's not the "inverted meaning" here that is the bug. `zle -F` (with
or without -w) just shouldn't ever change $LASTWIDGET in the first
place. It breaks a lot of existing widget functions and there is no
possible workaround.

On Wed, Nov 16, 2022 at 8:25 PM Bart Schaefer <schaefer@brasslantern.com> wrote:
>
> On Wed, Nov 16, 2022 at 7:08 AM Marlon Richert <marlon.richert@gmail.com> wrote:
> >
> > Currently, when the ZLE calls a widget set with `zle -Fw <widget>`,
> > this changes the value of $LASTWIDGET.
>
> Hrm, normally (running "zle WIDGET ..." explicitly) this would be
> skipped (unless "zle WIDGET -w ...").  Too bad -w was overloaded in
> this way.  It's not technically overloading because it's used in
> different contexts, but mnemonically at least it could be confusing
> that "zle -Fw FD WIDGET" and "zle WIDGET -w" would seem to have
> inverted meanings of -w if we were to change the default behavior of
> -F.
>
> Of course it's also "-Fw" as a single "option", not "-F -w", so we
> can't even do something like "zle -F +w" to indicate the desired
> behavior.

Re: Bug: Callback to widget set with `zle -Fw <widget>` shouldn't change $LASTWIDGET

[Roman Perepelitsa]

On Thu, Nov 17, 2022 at 2:06 PM Marlon Richert <marlon.richert@gmail.com> wrote:
>
> It's not the "inverted meaning" here that is the bug. `zle -F` (with
> or without -w) just shouldn't ever change $LASTWIDGET in the first
> place. It breaks a lot of existing widget functions and there is no
> possible workaround.

Relevant: https://www.zsh.org/mla/workers/2019/msg00204.html

Especially this part:

> The same branch also adds -W option to zle widget command. This
> option instructs zle to keep LASTWIDGET unchanged.

This part of the patch wasn't applied and the code is long gone (I
didn't attach it to the email) but it's fairly straightforward to
implement.

Roman.

Re: Bug: Callback to widget set with `zle -Fw <widget>` shouldn't change $LASTWIDGET

[Bart Schaefer]

On Thu, Nov 17, 2022 at 5:05 AM Marlon Richert <marlon.richert@gmail.com> wrote:
>
> The -w in `zle -Fw arg` is not the inverse of the -w in `zle -w arg`.
> It has a different purpose:

I KNOW that.  I was just pointing out potential future confusion.

Re: Bug: Callback to widget set with `zle -Fw <widget>` shouldn't change $LASTWIDGET

[Marlon Richert]

On Thu, Nov 17, 2022 at 6:18 PM Bart Schaefer <schaefer@brasslantern.com> wrote:
>
> On Thu, Nov 17, 2022 at 5:05 AM Marlon Richert <marlon.richert@gmail.com> wrote:
> >
> > The -w in `zle -Fw arg` is not the inverse of the -w in `zle -w arg`.
> > It has a different purpose:
>
> I KNOW that.  I was just pointing out potential future confusion.

Aha. Sorry, that wasn't clear to me.

[PATCH] Re: Bug: Callback to widget set with `zle -Fw <widget>` shouldn't change $LASTWIDGET

[Bart Schaefer]

Is this good enough, or is there a need to consider that the handler
might delete the previous last widget?  What does/should happen in
that case (or when a widget deletes itself, for that matter)?

diff --git a/Src/Zle/zle_main.c b/Src/Zle/zle_main.c
index 686c6f5b4..cfa0a739d 100644
--- a/Src/Zle/zle_main.c
+++ b/Src/Zle/zle_main.c
@@ -737,6 +737,7 @@ raw_getbyte(long do_keytmout, char *cptr, int full)
             ) {
             /* Handle the fd. */
             char *fdbuf;
+            Thingy save_lbindk = lbindk;
             {
                 char buf[BDIGBUFSIZE];
                 convbase(buf, lwatch_fd->fd, 10);
@@ -779,6 +780,7 @@ raw_getbyte(long do_keytmout, char *cptr, int full)
                  */
                 errtry = 1;
             }
+            lbindk = save_lbindk;
             }
         }
         /* Function may have invalidated the display. */

Re: [PATCH] Re: Bug: Callback to widget set with `zle -Fw <widget>` shouldn't change $LASTWIDGET

[Peter Stephenson]

> On 17/01/2023 00:01 Bart Schaefer <schaefer@brasslantern.com> wrote:
> Is this good enough, or is there a need to consider that the handler
> might delete the previous last widget?  What does/should happen in
> that case (or when a widget deletes itself, for that matter)?

I can't think of a good way of handling that without a fair measure
of complexity both in the shell and for the user.  I think just keeping
the old name and letting the user deal with it further is good enough.

pws

Re: [PATCH] Re: Bug: Callback to widget set with `zle -Fw <widget>` shouldn't change $LASTWIDGET

[Bart Schaefer]

On Tue, Jan 17, 2023 at 1:23 AM Peter Stephenson
<p.w.stephenson@ntlworld.com> wrote:
>
> > On 17/01/2023 00:01 Bart Schaefer <schaefer@brasslantern.com> wrote:
> > Is this good enough, or is there a need to consider that the handler
> > might delete the previous last widget?
>
> I think just keeping
> the old name and letting the user deal with it further is good enough.

My concern is that the Thingy pointer becomes invalid.  If that won't
happen, nothing further is needed here.  If that COULD happen, then
the implementation of $LASTWIDGET needs to start storing the name
rather than the Thingy.

Re: [PATCH] Re: Bug: Callback to widget set with `zle -Fw <widget>` shouldn't change $LASTWIDGET

[dana]

On Tue 17 Jan 2023, at 12:00, Bart Schaefer wrote:
> My concern is that the Thingy pointer becomes invalid

I think this was justified

I've just tried running a new HEAD build for the first time in several months
and found that back-spacing causes the shell to crash. I'm afraid i don't
have a clean way to reproduce, as it seems to be triggered by something in my
profile, apparently an interaction between my (probably ancient) z-sy-h and
zsh-autosuggest installations, which produce sort of a web of interdependent
widgets -- but i've narrowed it down to this change

I assume whatever these functions are doing eventually results in us trying
to free the static name of the built-in widget? Trace below

Seems like the same issue as workers/51673 too

dana


...:~zsh % git checkout f93ad02b94bd18c96a0861506127e3a246fb8eec
...:~zsh % make clean && ./Util/preconfig && ./configure --enable-zsh-debug && make
...:~zsh % ZDOTDIR=$PWD lldb Src/zsh
(lldb) target create "Src/zsh"
Current executable set to '/Users/dana/Development/sf.net/zsh/zsh/Src/zsh' (arm64).
(lldb) run
Process 6843 launched: '/Users/dana/Development/sf.net/zsh/zsh/Src/zsh' (arm64)
[DEV] zsh % ZDOTDIR=$HOME; source ~/.zshrc
...:~zsh % echo foo bar.zsh(6843,0x1fac81e00) malloc: *** error for object 0x1005f0d78: pointer being freed was not allocated
zsh(6843,0x1fac81e00) malloc: *** set a breakpoint in malloc_error_break to debug
Process 6843 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = signal SIGABRT
    frame #0: 0x000000019fc70724 libsystem_kernel.dylib`__pthread_kill + 8
libsystem_kernel.dylib`:
->  0x19fc70724 <+8>:  b.lo   0x19fc70744               ; <+40>
    0x19fc70728 <+12>: pacibsp 
    0x19fc7072c <+16>: stp    x29, x30, [sp, #-0x10]!
    0x19fc70730 <+20>: mov    x29, sp
Target 0: (zsh) stopped.
(lldb) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = signal SIGABRT
  * frame #0: 0x000000019fc70724 libsystem_kernel.dylib`__pthread_kill + 8
    frame #1: 0x000000019fca7c28 libsystem_pthread.dylib`pthread_kill + 288
    frame #2: 0x000000019fbb5ae8 libsystem_c.dylib`abort + 180
    frame #3: 0x000000019fad6e28 libsystem_malloc.dylib`malloc_vreport + 908
    frame #4: 0x000000019fada6e8 libsystem_malloc.dylib`malloc_report + 64
    frame #5: 0x000000019fae6f20 libsystem_malloc.dylib`find_zone_and_free + 308
    frame #6: 0x000000010007a6f0 zsh`zsfree(p="backward-delete-char") at mem.c:1878:5
    frame #7: 0x00000001005d839c zle.so`freethingynode(hn=0x00000001005f8190) at zle_thingy.c:122:5
    frame #8: 0x00000001005d5688 zle.so`unrefthingy(th=0x00000001005f8190) at zle_thingy.c:150:2
    frame #9: 0x00000001005be834 zle.so`execzlefunc(func=0x00000001005f9fb8, args=0x00000001006ac300, set_bindk=1, set_lbindk=0) at zle_main.c:1560:2
    frame #10: 0x00000001005d7d38 zle.so`bin_zle_call(name="zle", args=0x00000001006ac300, ops=0x000000016fdf90e8, func='\0') at zle_thingy.c:806:11
    frame #11: 0x00000001005d5ed0 zle.so`bin_zle(name="zle", args=0x00000001006ac2f0, ops=0x000000016fdf90e8, func=0) at zle_thingy.c:388:12
    frame #12: 0x0000000100004068 zsh`execbuiltin(args=0x00000001006ac270, assigns=0x0000000000000000, bn=0x00000001005fe158) at builtin.c:506:13
    frame #13: 0x0000000100035d3c zsh`execcmd_exec(state=0x000000016fdf9b70, eparams=0x000000016fdf9818, input=0, output=0, how=2, last1=2, close_if_forked=-1) at exec.c:4187:17
    frame #14: 0x000000010003071c zsh`execpline2(state=0x000000016fdf9b70, pcode=195, how=2, input=0, output=0, last1=0) at exec.c:2003:2
    frame #15: 0x0000000100027c54 zsh`execpline(state=0x000000016fdf9b70, slcode=5154, how=2, last1=0) at exec.c:1728:5
    frame #16: 0x0000000100026a38 zsh`execlist(state=0x000000016fdf9b70, dont_change_job=1, exiting=0) at exec.c:1494:8
    frame #17: 0x0000000100026144 zsh`execode(p=0x0000600001736a80, dont_change_job=1, exiting=0, context="shfunc") at exec.c:1263:5
    frame #18: 0x000000010002e0e4 zsh`runshfunc(prog=0x0000600001736a80, wrap=0x0000000000000000, name="_zsh_highlight_call_widget") at exec.c:6113:5
    frame #19: 0x000000010002d81c zsh`doshfunc(shfunc=0x0000600001736a40, doshargs=0x0000000100678310, noreturnval=0) at exec.c:5960:2
    frame #20: 0x000000010002fd08 zsh`execshfunc(shf=0x0000600001736a40, args=0x0000000100678310) at exec.c:5529:5
    frame #21: 0x00000001000356b0 zsh`execcmd_exec(state=0x000000016fdfa800, eparams=0x000000016fdfa4a8, input=0, output=0, how=18, last1=2, close_if_forked=-1) at exec.c:4058:3
    frame #22: 0x000000010003071c zsh`execpline2(state=0x000000016fdfa800, pcode=67, how=18, input=0, output=0, last1=0) at exec.c:2003:2
    frame #23: 0x0000000100027c54 zsh`execpline(state=0x000000016fdfa800, slcode=6146, how=18, last1=0) at exec.c:1728:5
    frame #24: 0x00000001000269b4 zsh`execlist(state=0x000000016fdfa800, dont_change_job=1, exiting=0) at exec.c:1482:7
    frame #25: 0x0000000100026144 zsh`execode(p=0x0000600001736fc0, dont_change_job=1, exiting=0, context="shfunc") at exec.c:1263:5
    frame #26: 0x000000010002e0e4 zsh`runshfunc(prog=0x0000600001736fc0, wrap=0x0000000000000000, name="_zsh_highlight_widget_orig-s0.0000030000-r21548-backward-delete-char") at exec.c:6113:5
    frame #27: 0x000000010002d81c zsh`doshfunc(shfunc=0x0000600001737000, doshargs=0x0000000000000000, noreturnval=1) at exec.c:5960:2
    frame #28: 0x00000001005be6c0 zle.so`execzlefunc(func=0x0000600000c25bc0, args=0x00000001005ac640, set_bindk=0, set_lbindk=0) at zle_main.c:1533:12
    frame #29: 0x00000001005d7d38 zle.so`bin_zle_call(name="zle", args=0x00000001005ac640, ops=0x000000016fdfada8, func='\0') at zle_thingy.c:806:11
    frame #30: 0x00000001005d5ed0 zle.so`bin_zle(name="zle", args=0x00000001005ac630, ops=0x000000016fdfada8, func=0) at zle_thingy.c:388:12
    frame #31: 0x0000000100004068 zsh`execbuiltin(args=0x00000001005ac5d0, assigns=0x0000000000000000, bn=0x00000001005fe158) at builtin.c:506:13
    frame #32: 0x0000000100035d3c zsh`execcmd_exec(state=0x000000016fdfc240, eparams=0x000000016fdfb4d8, input=0, output=0, how=18, last1=2, close_if_forked=-1) at exec.c:4187:17
    frame #33: 0x000000010003071c zsh`execpline2(state=0x000000016fdfc240, pcode=643, how=18, input=0, output=0, last1=0) at exec.c:2003:2
    frame #34: 0x0000000100027c54 zsh`execpline(state=0x000000016fdfc240, slcode=6146, how=18, last1=0) at exec.c:1728:5
    frame #35: 0x00000001000269b4 zsh`execlist(state=0x000000016fdfc240, dont_change_job=1, exiting=0) at exec.c:1482:7
    frame #36: 0x00000001000720a0 zsh`execif(state=0x000000016fdfc240, do_exec=0) at loop.c:576:2
    frame #37: 0x00000001000353cc zsh`execcmd_exec(state=0x000000016fdfc240, eparams=0x000000016fdfbee8, input=0, output=0, how=18, last1=2, close_if_forked=-1) at exec.c:4007:13
    frame #38: 0x000000010003071c zsh`execpline2(state=0x000000016fdfc240, pcode=579, how=18, input=0, output=0, last1=0) at exec.c:2003:2
    frame #39: 0x0000000100027c54 zsh`execpline(state=0x000000016fdfc240, slcode=15362, how=18, last1=0) at exec.c:1728:5
    frame #40: 0x00000001000269b4 zsh`execlist(state=0x000000016fdfc240, dont_change_job=1, exiting=0) at exec.c:1482:7
    frame #41: 0x0000000100026144 zsh`execode(p=0x0000600001702080, dont_change_job=1, exiting=0, context="shfunc") at exec.c:1263:5
    frame #42: 0x000000010002e0e4 zsh`runshfunc(prog=0x0000600001702080, wrap=0x0000000000000000, name="_zsh_autosuggest_invoke_original_widget") at exec.c:6113:5
    frame #43: 0x000000010002d81c zsh`doshfunc(shfunc=0x00006000017020c0, doshargs=0x000000010059c720, noreturnval=0) at exec.c:5960:2
    frame #44: 0x000000010002fd08 zsh`execshfunc(shf=0x00006000017020c0, args=0x000000010059c720) at exec.c:5529:5
    frame #45: 0x00000001000356b0 zsh`execcmd_exec(state=0x000000016fdfced0, eparams=0x000000016fdfcb78, input=0, output=0, how=2, last1=2, close_if_forked=-1) at exec.c:4058:3
    frame #46: 0x000000010003071c zsh`execpline2(state=0x000000016fdfced0, pcode=1091, how=2, input=0, output=0, last1=0) at exec.c:2003:2
    frame #47: 0x0000000100027c54 zsh`execpline(state=0x000000016fdfced0, slcode=4098, how=2, last1=0) at exec.c:1728:5
    frame #48: 0x00000001000269b4 zsh`execlist(state=0x000000016fdfced0, dont_change_job=1, exiting=0) at exec.c:1482:7
    frame #49: 0x0000000100026144 zsh`execode(p=0x0000600001702400, dont_change_job=1, exiting=0, context="shfunc") at exec.c:1263:5
    frame #50: 0x000000010002e0e4 zsh`runshfunc(prog=0x0000600001702400, wrap=0x0000000000000000, name="_zsh_autosuggest_modify") at exec.c:6113:5
    frame #51: 0x000000010002d81c zsh`doshfunc(shfunc=0x0000600001702440, doshargs=0x00000001001e03c8, noreturnval=0) at exec.c:5960:2
    frame #52: 0x000000010002fd08 zsh`execshfunc(shf=0x0000600001702440, args=0x00000001001e03c8) at exec.c:5529:5
    frame #53: 0x00000001000356b0 zsh`execcmd_exec(state=0x000000016fdfdb60, eparams=0x000000016fdfd808, input=0, output=0, how=2, last1=2, close_if_forked=-1) at exec.c:4058:3
    frame #54: 0x000000010003071c zsh`execpline2(state=0x000000016fdfdb60, pcode=387, how=2, input=0, output=0, last1=0) at exec.c:2003:2
    frame #55: 0x0000000100027c54 zsh`execpline(state=0x000000016fdfdb60, slcode=4098, how=2, last1=0) at exec.c:1728:5
    frame #56: 0x00000001000269b4 zsh`execlist(state=0x000000016fdfdb60, dont_change_job=1, exiting=0) at exec.c:1482:7
    frame #57: 0x0000000100026144 zsh`execode(p=0x00006000017027c0, dont_change_job=1, exiting=0, context="shfunc") at exec.c:1263:5
    frame #58: 0x000000010002e0e4 zsh`runshfunc(prog=0x00006000017027c0, wrap=0x0000000000000000, name="_zsh_autosuggest_widget_modify") at exec.c:6113:5
    frame #59: 0x000000010002d81c zsh`doshfunc(shfunc=0x0000600001702800, doshargs=0x00000001001dc2b0, noreturnval=0) at exec.c:5960:2
    frame #60: 0x000000010002fd08 zsh`execshfunc(shf=0x0000600001702800, args=0x00000001001dc2b0) at exec.c:5529:5
    frame #61: 0x00000001000356b0 zsh`execcmd_exec(state=0x000000016fdfe7f0, eparams=0x000000016fdfe498, input=0, output=0, how=18, last1=2, close_if_forked=-1) at exec.c:4058:3
    frame #62: 0x000000010003071c zsh`execpline2(state=0x000000016fdfe7f0, pcode=131, how=18, input=0, output=0, last1=0) at exec.c:2003:2
    frame #63: 0x0000000100027c54 zsh`execpline(state=0x000000016fdfe7f0, slcode=5122, how=18, last1=0) at exec.c:1728:5
    frame #64: 0x00000001000269b4 zsh`execlist(state=0x000000016fdfe7f0, dont_change_job=1, exiting=0) at exec.c:1482:7
    frame #65: 0x0000000100026144 zsh`execode(p=0x0000600001740280, dont_change_job=1, exiting=0, context="shfunc") at exec.c:1263:5
    frame #66: 0x000000010002e0e4 zsh`runshfunc(prog=0x0000600001740280, wrap=0x0000000000000000, name="_zsh_autosuggest_bound_1_backward-delete-char") at exec.c:6113:5
    frame #67: 0x000000010002d81c zsh`doshfunc(shfunc=0x00006000017402c0, doshargs=0x0000000000000000, noreturnval=1) at exec.c:5960:2
    frame #68: 0x00000001005be6c0 zle.so`execzlefunc(func=0x00000001005f8190, args=0x00000001005fe710, set_bindk=0, set_lbindk=0) at zle_main.c:1533:12
    frame #69: 0x00000001005bec24 zle.so`zlecore at zle_main.c:1151:10
    frame #70: 0x00000001005bf7dc zle.so`zleread(lp=0x00000001000fad80, rp=0x00000001000fadc0, flags=3, context=0, init="zle-line-init", finish="zle-line-finish") at zle_main.c:1361:5
    frame #71: 0x00000001005c079c zle.so`zle_main_entry(cmd=1, ap="@\xa1\U0000000f") at zle_main.c:2137:9
    frame #72: 0x000000010005ba98 zsh`zleentry(cmd=1) at init.c:1648:8
    frame #73: 0x000000010005d848 zsh`inputline at input.c:424:15
    frame #74: 0x000000010005d330 zsh`ingetc at input.c:357:6
    frame #75: 0x000000010004c344 zsh`ihgetc at hist.c:415:13
    frame #76: 0x00000001000696b4 zsh`gettok at lex.c:622:12
    frame #77: 0x0000000100069328 zsh`zshlex at lex.c:275:8
    frame #78: 0x00000001000981ec zsh`parse_event(endtok=37) at parse.c:615:5
    frame #79: 0x00000001000572c0 zsh`loop(toplevel=1, justonce=0) at init.c:150:15
    frame #80: 0x000000010005bfc0 zsh`zsh_main(argc=1, argv=0x000000016fdff558) at init.c:1810:6
    frame #81: 0x000000010000306c zsh`main(argc=1, argv=0x000000016fdff558) at main.c:93:13
    frame #82: 0x000000019f94ff28 dyld`start + 2236

Re: [PATCH] Re: Bug: Callback to widget set with `zle -Fw <widget>` shouldn't change $LASTWIDGET

[Peter Stephenson]

> On 16/07/2023 11:28 dana <dana@dana.is> wrote:
> On Tue 17 Jan 2023, at 12:00, Bart Schaefer wrote:
> > My concern is that the Thingy pointer becomes invalid
> 
> I think this was justified
> 
> I've just tried running a new HEAD build for the first time in several months
> and found that back-spacing causes the shell to crash.

If this happens again, you might get some information by going back up the
stack in the debugger and see what was running in any runshfunc(s) or
doshfunc(s) there.  You probably just need the name, then you can retrieve
the contents of that function from a live shell.

Having said that, Bart's previous suggestion of saving the name instead of
the pointer might be the fix anyway.

pws

Re: [PATCH] Re: Bug: Callback to widget set with `zle -Fw <widget>` shouldn't change $LASTWIDGET

[Bart Schaefer]

On Sun, Jul 16, 2023 at 3:29 AM dana <dana@dana.is> wrote:
>
> On Tue 17 Jan 2023, at 12:00, Bart Schaefer wrote:
> > My concern is that the Thingy pointer becomes invalid
>
> I think this was justified
>
> I've just tried running a new HEAD build for the first time in several months
> and found that back-spacing causes the shell to crash.

The patch from the start of this thread has never been committed, so
is not directly related.  That is, if the Thingy is invalid, it's not
$LASTWIDGET that's making the bad reference here, so fixing
save/restore there probably will not resolve the problem.

In your backtrace I see
  _zsh_autosuggest_widget_modify
and
  _zsh_autosuggest_invoke_original_widget
followed by
  _zsh_highlight_widget_orig-s0.0000030000-r21548-backward-delete-char
and finally
  _zsh_highlight_call_widget
causes the actual crash below unrefthingy(), so I suspect this is a
reference-counting problem.

Re: [PATCH] Re: Bug: Callback to widget set with `zle -Fw <widget>` shouldn't change $LASTWIDGET

[dana]

On Mon 17 Jul 2023, at 10:17, Bart Schaefer wrote:
> The patch from the start of this thread has never been committed, so
> is not directly related.  That is, if the Thingy is invalid, it's not
> $LASTWIDGET that's making the bad reference here, so fixing
> save/restore there probably will not resolve the problem.

Sorry, i'm not very familiar with the LASTWIDGET mechanism, but by the patch
at the start of this thread you mean workers/51310, right? It was committed
as f93ad02b94bd

I should have mentioned that i did a (manual) bisect to confirm that that is
when the problem started for me; the poster in workers/51673 found the same,
with what sounds like a very similar configuration (though they didn't
provide any other details). I can't get the problem to occur on a build off
the preceding commit or on the 5.9 release

I did examine the functions the trace references before; the
_zsh_highlight_widget_orig... one simply calls _zsh_highlight_call_widget,
and that one calls zle before calling another highlight function (but the
crash happens in zle so it's not getting to that). The autosuggest ones are a
little more elaborate. I can get back to it later

dana

Re: [PATCH] Re: Bug: Callback to widget set with `zle -Fw <widget>` shouldn't change $LASTWIDGET

[Bart Schaefer]

On Mon, Jul 17, 2023 at 8:52 AM dana <dana@dana.is> wrote:
>
> Sorry, i'm not very familiar with the LASTWIDGET mechanism, but by the patch
> at the start of this thread you mean workers/51310, right? It was committed
> as f93ad02b94bd

(Facepalm)  Looked at the wrong source file.

> I should have mentioned that i did a (manual) bisect to confirm that that is
> when the problem started for me; the poster in workers/51673 found the same

OK, then it IS a reference counting problem.  Try this.

diff --git a/Src/Zle/zle_main.c b/Src/Zle/zle_main.c
index 4a6c02133..6e554f4b8 100644
--- a/Src/Zle/zle_main.c
+++ b/Src/Zle/zle_main.c
@@ -737,7 +737,7 @@ raw_getbyte(long do_keytmout, char *cptr, int full)
             ) {
             /* Handle the fd. */
             char *fdbuf;
-            Thingy save_lbindk = lbindk;
+            Thingy save_lbindk = refthingy(lbindk);
             {
                 char buf[BDIGBUFSIZE];
                 convbase(buf, lwatch_fd->fd, 10);
@@ -781,6 +781,7 @@ raw_getbyte(long do_keytmout, char *cptr, int full)
                 errtry = 1;
             }
             lbindk = save_lbindk;
+            unrefthingy(save_lbindk);
             }
         }
         /* Function may have invalidated the display. */

Re: [PATCH] Re: Bug: Callback to widget set with `zle -Fw <widget>` shouldn't change $LASTWIDGET

[Bart Schaefer]

On Mon, Jul 17, 2023 at 8:57 AM Bart Schaefer <schaefer@brasslantern.com> wrote:
>
>              lbindk = save_lbindk;
> +            unrefthingy(save_lbindk);

Hm, looking at redrawhook(), that perhaps should be

> +            unrefthingy(lbindk);
>              lbindk = save_lbindk;

Re: [PATCH] Re: Bug: Callback to widget set with `zle -Fw <widget>` shouldn't change $LASTWIDGET

[dana]

On Mon 17 Jul 2023, at 11:57, Bart Schaefer wrote:
> Hm, looking at redrawhook(), that perhaps should be

That (the second one) seems to have fixed the crash, thank you

dana

PS: I don't know if it's just my client but your first patch had corrupt
white space for me

Re: [PATCH] Re: Bug: Callback to widget set with `zle -Fw <widget>` shouldn't change $LASTWIDGET

[Bart Schaefer]

On Wed, Jul 19, 2023 at 9:01 PM dana <dana@dana.is> wrote:
>
> That (the second one) seems to have fixed the crash, thank you

Thanks for confirming.

> PS: I don't know if it's just my client but your first patch had corrupt
> white space for me

Probably because I just pasted into gmail instead of attaching.  Same
thing might happen with the "piping NUL character" patch (which is
actually a read-with-delimiter patch).