Bug: magic-space and expand-history delete part of line since 5.0.8

Bug: magic-space and expand-history delete part of line since 5.0.8

[Jacob Niehus]

Since zsh 5.0.8, using magic-space or expand-history can delete part of the
command line for no apparent reason. This does not occur in 5.0.7 or 5.0.5.

To reproduce:

    $ zsh -f
    $ bindkey ' ' magic-space

type this:

    $ vim =(ls) -c 'foo'

move the cursor between 's)' and type '|' then space, and it changes to this:

    $ vim =(ls| )

"-c 'foo'" has been deleted. Many times now I'll edit a command and the whole
right half will just disappear. The same problem happens when using
expand-history.

I git bisected this down to a single commit:
    https://github.com/zsh-users/zsh/commit/c0d01a6fe0c67911650730cf13a2b9a0db16e59b

The commit info/message:

>     commit c0d01a6fe0c67911650730cf13a2b9a0db16e59b
>     Author: Peter Stephenson <pws@zsh.org>
>     Date:   Tue Jan 6 17:05:17 2015 +0000
>
>         Fix command substitutions to parse contents as they are read in.
>
>         Do this by refactoring misnamed lexsave()/lexrestore() to allow
>         continuity of history and input.
>
>         Add test.

-Jacob Niehus

Re: Bug: magic-space and expand-history delete part of line since 5.0.8

[Bart Schaefer]

On Aug 31,  7:32pm, Jacob Niehus wrote:
}
}     $ vim =(ls) -c 'foo'
} 
} move the cursor between 's)' and type '|' then space, and it changes to this:
} 
}     $ vim =(ls| )
} 
} "-c 'foo'" has been deleted.

Confirmed this with zsh-5.1-3-g6e825e7 (recall my question about whether
fixing things like this will be 5.1.1 or ...?)

5.1.1?

[Peter Stephenson]

On Mon, 31 Aug 2015 22:27:22 -0700
Bart Schaefer <schaefer@brasslantern.com> wrote:
> Confirmed this with zsh-5.1-3-g6e825e7 (recall my question about whether
> fixing things like this will be 5.1.1 or ...?)

If the freeze Vincent found gets resolved, we might have enough tweaks
for a 5.1.1, i.e. 5.1 with a few fixes, in a week or two,

pws

Re: Bug: magic-space and expand-history delete part of line since 5.0.8

[Peter Stephenson]

On Mon, 31 Aug 2015 19:32:23 -0700
Jacob Niehus <jacob.niehus@gmail.com> wrote:
> Since zsh 5.0.8, using magic-space or expand-history can delete part of the
> command line for no apparent reason. This does not occur in 5.0.7 or 5.0.5.
> 
> To reproduce:
> 
>     $ zsh -f
>     $ bindkey ' ' magic-space
> 
> type this:
> 
>     $ vim =(ls) -c 'foo'
> 
> move the cursor between 's)' and type '|' then space, and it changes to this:
> 
>     $ vim =(ls| )
> 
> "-c 'foo'" has been deleted.

This function is low-level enough, and ctxtlex() already nasty enough,
that I hope the following is good enough.

It would be good to add a test for this, but I think this is way beyond
my interactive-test-fu.

pws

diff --git a/Src/Zle/zle_tricky.c b/Src/Zle/zle_tricky.c
index 97bdcc0..b1a6f9e 100644
--- a/Src/Zle/zle_tricky.c
+++ b/Src/Zle/zle_tricky.c
@@ -2812,6 +2812,8 @@ doexpandhist(void)
     do {
 	ctxtlex();
     } while (tok != ENDINPUT && tok != LEXERR);
+    if (tok == LEXERR)
+	lexstop = 0;
     while (!lexstop)
 	hgetc();
     /* We have to save errflags because it's reset in zcontext_restore. Since  *