completion

completion

[Ray Andrews]

Previous command recall issue now fixed:

    $ echo $(ls zsh)
    zsh-5.0.7-234-gaa503b5
    zsh

    $ echo $(ls zsh)
    zsh-5.0.7-234-gaa503b5
    zsh


However completion is still busted:

     $ ln -svf zsh-5.0.7-175   <TAB>

     $ ln -svf zsh-5.0.7-175-./Src/zsh: symbol lookup error: 
/usr/local/lib/zsh/5.0.7-dev-0/zsh/zle.so: undefined symbol: lexsave

... that was trying to complete this:

-rwxr-xr-x 1 1220492 2015-01-06-11:27:02 zsh-5.0.7-175-g7e74495

...  is there some missed change in zle.so perhaps?  I'm never sure 
what, besides the kernel, is ever supposed to change.


    $ l zle.so
    zsh-5.0.7-175-g7e74495

    Sorting All file types Backwards by Modification Time (zle.so):

    -rwxr-xr-x 1 273104 2014-11-27-16:54:23 zle.so* << latest?

Command substitution parsing issues (not really Re: completion)

[Bart Schaefer]

On Jan 16,  4:19pm, Ray Andrews wrote:
}
}      $ ln -svf zsh-5.0.7-175-./Src/zsh: symbol lookup error: 
} /usr/local/lib/zsh/5.0.7-dev-0/zsh/zle.so: undefined symbol: lexsave

You need to re-install the shared object files.  The symbols have in
fact changed, you're loading a .so from a previous build.

In fact if you're going to live on the bleeding edge of git pull, you
should have a separate shared-object directory for every build.


However, the latest change has NOT fixed things for me.  I still get:

schaefer<501> echo ÿÿÿÿÿÿÿÿ$(ls)
Completing all expansions
ÿÿÿÿÿÿÿÿ$(ls)
Completing original
$(ls)

And history recall screws up with aliases:

schaefer<501> echo $(ls)        
Config/ config.modules config.status* Etc/ Src/ config.h config.modules.local
core.28047 foo/ stamp-h config.log config.modules.sh Doc/ Makefile Test/
schaefer<502> echo $(lsls -CF)


Perhaps the alias is why I'm getting the ÿÿÿÿÿÿÿÿ text as well.  Let's
see ...  yes, that's the problem:

schaefer<502> unalias ls
schaefer<503> echo Config/
Completing expansions
Config                Test                  config.modules.sh   
Doc                   config.h              config.status       
Etc                   config.log            core.28047          
foo                   config.modules        Makefile            
Src                   config.modules.local  stamp-h             
Completing all expansions
Config config.h config.log config.modules config.modules.local config.modul ...
Completing original
$(ls)

Re: Command substitution parsing issues (not really Re: completion)

[Ray Andrews]

On 01/16/2015 06:57 PM, Bart Schaefer wrote:
> On Jan 16,  4:19pm, Ray Andrews wrote:
> }
> }      $ ln -svf zsh-5.0.7-175-./Src/zsh: symbol lookup error:
> } /usr/local/lib/zsh/5.0.7-dev-0/zsh/zle.so: undefined symbol: lexsave
>
> You need to re-install the shared object files.  The symbols have in
> fact changed, you're loading a .so from a previous build.

Ok, I thought that might be the case.  An unidentified symbol would not 
survive the developer's build of his own work.
>
> In fact if you're going to live on the bleeding edge of git pull, you
> should have a separate shared-object directory for every build.

Can you suggest a method?  It's easy enough to save old kernels, but the 
rest of zsh's various files here and there are still a mystery. I didn't 
even know there were library files like that.  Does 'git pull; make -s' 
even update those?  Seems not.  There are variables pointing to the 
locations of these support files, yes? If so, I can easily create 
numbered directories for the files and then ... well, whatever you 
recommend.

Re: Command substitution parsing issues (not really Re: completion)

[Vin Shelton]

On Fri, Jan 16, 2015 at 10:29 PM, Ray Andrews <rayandrews@eastlink.ca> wrote:
> Can you suggest a method?

Every time I build I configure with / install to a different prefix.
Usually I just:

configure ---prefix=/opt/zsh-$(date +'%Y-%m-%d')
make && make install

But if I'm making more than one time per day, I:

configure --prefix=/opt/zsh-$(date +'%Y-%m-%d-%H%M')
make && make install

i.e., I add -HHMM to the installation directory name of yyyy-mm-dd.

  - Vin

Re: Command substitution parsing issues (not really Re: completion)

[Peter Stephenson]

On Fri, 16 Jan 2015 19:29:42 -0800
Ray Andrews <rayandrews@eastlink.ca> wrote:
> > In fact if you're going to live on the bleeding edge of git pull, you
> > should have a separate shared-object directory for every build.
> 
> Can you suggest a method?

I find it good enough to be able to point the compiled executable in Src
at a special set of libraries by using an environment variable (you
could even avoid that if you detected where the shell was running from
early enough).

The basic idea (the actual implementation has got complicated) is
.zshenv contains

if [[ -n $TEST_MODULES ]]; then
  module_path=(~/src/zsh/modules .)
fi

and the module links were set up with some variant of the function
below.

pws


## create_module_links
local src=$1
local f

if [[ -z $src ]]; then
  print "Usage: $0 path-to-zsh-src-dir
Current directory is directory to create links (above zsh)."
fi

for f in $src/**/*.mdd~*/zsh.mdd; do
  lib=${f:r}.so
  [[ -f $lib ]] && print "Found $lib"
  name="${$(grep '^name=' $f)##name=}.so"
  print "Link for $lib is $name."
  mkdir -p ${name:h}
  ln -s $lib $name
done
##

Re: Command substitution parsing issues (not really Re: completion)

[Peter Stephenson]

On Mon, 19 Jan 2015 10:29:10 +0000
Peter Stephenson <p.stephenson@samsung.com> wrote:
> On Fri, 16 Jan 2015 19:29:42 -0800
> Ray Andrews <rayandrews@eastlink.ca> wrote:
> > > In fact if you're going to live on the bleeding edge of git pull, you
> > > should have a separate shared-object directory for every build.
> > 
> > Can you suggest a method?
>
> I find it good enough to be able to point the compiled executable in Src
> at a special set of libraries by using an environment variable (you
> could even avoid that if you detected where the shell was running from
> early enough).>
>
> The basic idea (the actual implementation has got complicated) is
> .zshenv contains
>
> if [[ -n $TEST_MODULES ]]; then
>   module_path=(~/src/zsh/modules .)
> fi
>
> and the module links were set up with some variant of the function
> below.

...and I see I haven't even bothered to explain the main point of this:
the module directory just contains links back into the compilation tree.
So you only ever do this once; after that you just remember to use

TEST_MODULES=1 ./zsh

when running in Src.

pws

make check problem

[Ray Andrews]

I'm getting this 'make check' problem:

     ./V07pcre.ztst: skipped (the zsh/pcre module is not available)

Studying the output of './configure':

    ...
    checking pcre.h usability... no
    checking pcre.h presence... no
    checking for pcre.h... no
    checking for pcre_compile... no
    checking for pcre_study... no
    checking for pcre_study... no
    ...


After installing 'libpcre3-dev' :

    checking pcre.h usability... yes
    checking pcre.h presence... yes
    checking for pcre.h... yes
    checking for pcre_compile... no
    checking for pcre_study... no
    checking for pcre_exec... no

All three 'pcre_*' functions exist in both:

/usr/lib/i386-linux-gnu/libpcre.a
/usr/lib/i386-linux-gnu/libpcre.so

... so I'm wondering why they are not linked.

This is under Debian  testing.

Re: make check problem

[Peter Stephenson]

[-- Attachment #1: Type: text/plain, Size: 848 bytes --]

On 22 January 2015 at 00:34, Ray Andrews <rayandrews@eastlink.ca> wrote:
> I'm getting this 'make check' problem:
>
>     ./V07pcre.ztst: skipped (the zsh/pcre module is not available)
>
> All three 'pcre_*' functions exist in both:
>
> /usr/lib/i386-linux-gnu/libpcre.a
> /usr/lib/i386-linux-gnu/libpcre.so
>
> ... so I'm wondering why they are not linked.
>
> This is under Debian  testing.

(On webmail, apologies if the format is a bit uncontrollable.)

This is quite a common problem if your Linux setup isn't
specifically installed for software development.

The check needs to be able to perform a compilation,
so it has to find the headers as well as the libraries.
Those are typically in some additional package.  You'll
probably find you have a package "pcre" installed and
need another one called "pcre-devel" or something similar.

pws

Re: make check problem

[Ray Andrews]

On 01/22/2015 12:49 AM, Peter Stephenson wrote:
> The check needs to be able to perform a compilation, so it has to find 
> the headers as well as the libraries. 

You mean besides /usr/include/pcre.h which is already found? 
'libpcre3-dev' is installed. Maybe it will take another Debian user to 
know what to do.  Or is this important at all? One doesn't like to see 
any defect in 'make check', but I know that some of the checks are not 
vital.
> Those are typically in some additional package. You'll probably find 
> you have a package "pcre" installed and need another one called 
> "pcre-devel" or something similar. pws 

Re: make check problem

[Jun T.]

2015/01/22 09:34, Ray Andrews <rayandrews@eastlink.ca> wrote:
>  checking for pcre_compile... no
>  checking for pcre_study... no
>  checking for pcre_exec... no

Did you use --enable-pcre with configure?

$ ./configure --enable-pcre ...other options...

Re: make check problem

[Ray Andrews]

On 01/22/2015 08:40 AM, Jun T. wrote:
> 2015/01/22 09:34, Ray Andrews <rayandrews@eastlink.ca> wrote:
>>   checking for pcre_compile... no
>>   checking for pcre_study... no
>>   checking for pcre_exec... no
> Did you use --enable-pcre with configure?
>
> $ ./configure --enable-pcre ...other options...
>
Nope.  I wonder why 'make check' indicates a fault when the test can't 
possibly succeed from the getgo without that option? Are there other 
modules that require that sort of special handling?  Why is it special? 
Do I need it anyway?

Anyway, I added that option, and now:


----------------------------------------------------
./V07pcre.ztst: starting.
Testing PCRE multibyte with locale en_US.UTF-8
*** /tmp/zsh.ztst.out.27929    2015-01-22 08:55:33.172945830 -0800
--- /tmp/zsh.ztst.tout.27929    2015-01-22 08:55:33.184945978 -0800
***************
*** 1,12 ****
! 0 o→b →b
! 1
! 1
! 0 o→† →†
! 0 o→b →b
! 1
! 0 o→b →b
! 0 o→b →b
! 0 xo→t →t
! 1
! 0 xo→t →t
! 0 Xo→t →t
--- 1,12 ----
! 127 bar 0 :pcre_compile interface testing: basic, anchored & 
case-insensitive
! 127
! 127
! 127
! 127
! 127
! 127
! 127
! 127
! 127
! 127
! 127
Test ./V07pcre.ztst failed: output differs from expected as shown above for:
   pcre_compile '.(→.)'
   pcre_match foo→bar
   print $? $MATCH $match ; unset MATCH match
   pcre_match foo.bar
   print $? $MATCH $match ; unset MATCH match
   pcre_match foo†bar
   print $? $MATCH $match ; unset MATCH match
   pcre_match foo→†ar
   print $? $MATCH $match ; unset MATCH match
   pcre_study
   pcre_match foo→bar
   print $? $MATCH $match ; unset MATCH match
   pcre_compile -a '.(→.)'
   pcre_match foo→bar
   print $? $MATCH $match ; unset MATCH match
   pcre_match o→bar
   print $? $MATCH $match ; unset MATCH match
   pcre_match o→b
   print $? $MATCH $match ; unset MATCH match
   pcre_compile 'x.(→.)'
   pcre_match xo→t
   print $? $MATCH $match ; unset MATCH match
   pcre_match Xo→t
   print $? $MATCH $match ; unset MATCH match
   pcre_compile -i 'x.(→.)'
   pcre_match xo→t
   print $? $MATCH $match ; unset MATCH match
   pcre_match Xo→t
   print $? $MATCH $match ; unset MATCH match
Error output:
(eval):1: command not found: pcre_compile
(eval):2: command not found: pcre_match
(eval):4: command not found: pcre_match
(eval):6: command not found: pcre_match
(eval):8: command not found: pcre_match
(eval):10: command not found: pcre_study
(eval):11: command not found: pcre_match
(eval):13: command not found: pcre_compile
(eval):14: command not found: pcre_match
(eval):16: command not found: pcre_match
(eval):18: command not found: pcre_match
(eval):20: command not found: pcre_compile
(eval):21: command not found: pcre_match
(eval):23: command not found: pcre_match
(eval):25: command not found: pcre_compile
(eval):26: command not found: pcre_match
(eval):28: command not found: pcre_match
Was testing: pcre_compile interface testing: basic, anchored & 
case-insensitive
./V07pcre.ztst: test failed.
-------------------------------------------------------------

... However, in config.log:

-----------------------------------------------------------
configure:8329: checking for pcre_compile
configure:8329: gcc -o conftest  -Wall -Wmissing-prototypes -O2 
conftest.c -lpcre -ldl -ltinfo -lrt -lm  -lc >&5
configure:8329: $? = 0
configure:8329: result: yes
configure:8329: checking for pcre_study
configure:8329: gcc -o conftest  -Wall -Wmissing-prototypes -O2 
conftest.c -lpcre -ldl -ltinfo -lrt -lm  -lc >&5
configure:8329: $? = 0
configure:8329: result: yes
configure:8329: checking for pcre_exec
configure:8329: gcc -o conftest  -Wall -Wmissing-prototypes -O2 
conftest.c -lpcre -ldl -ltinfo -lrt -lm  -lc >&5
configure:8329: $? = 0
configure:8329: result: yes
--------------------------------------------------------------------

... there are no failures shown anywhere.  It seems the functions are 
there but there's no matching commands available (if there are any such 
things).  However, I do have /usr/bin/pcre-config and perhaps that is 
not doing it's job? It is referred to in config.log.

Re: make check problem

[Bart Schaefer]

On Jan 22,  9:38am, Ray Andrews wrote:
}
} Nope.  I wonder why 'make check' indicates a fault when the test can't 
} possibly succeed from the getgo without that option?

I've said before that "not available" is not a "fault."  You're reading
way too much into "make check" skipping a test.

} Are there other 
} modules that require that sort of special handling?  Why is it special? 

Sorry I forgot about --enable-pcre before.  You shouldn't link=static
the zsh/pcre module without having used --enable-pcre in configure.

It's special because it's extremely non-standard and changes the meaning
of the expressions on the right-hand-side of the =~ comparison, so if
unknowingly enabled it might break scripts.

} Do I need it anyway?

You only need it if you want to use it.

} Error output:
} (eval):1: command not found: pcre_compile
} (eval):2: command not found: pcre_match
} (eval):10: command not found: pcre_study

Hmm, sorry, my patch in 34338 probably borked this.  Try the below:


diff --git a/Test/V07pcre.ztst b/Test/V07pcre.ztst
index 3c25be2..ddfd3f5 100644
--- a/Test/V07pcre.ztst
+++ b/Test/V07pcre.ztst
@@ -5,6 +5,8 @@
     ZTST_unimplemented="the zsh/pcre module is not available"
     return 0
   fi
+# Load the rest of the builtins
+  zmodload zsh/pcre
   setopt rematch_pcre
 # Find a UTF-8 locale.
   setopt multibyte

Re: make check problem

[Ray Andrews]

On 01/22/2015 09:56 AM, Bart Schaefer wrote:
> I've said before that "not available" is not a "fault."  You're reading
> way too much into "make check" skipping a test.

Sure.  But it's difficult to know what's important there and what is 
not.  I get the feeling that this pcre isn't important, still one might 
consider 'not available' to be something to worry about if you didn't 
know it wasn't, since one might ask why it's checked if it *isn't* 
important.  (Or checked at all if it is explicitly not available.)

> It's special because it's extremely non-standard and changes the meaning
> of the expressions on the right-hand-side of the =~ comparison, so if
> unknowingly enabled it might break scripts.

Ok thanks.  Looks  like I don't want it anyway.  At this point it's an 
exercise.

Patch: ...

       Testing PCRE multibyte with locale en_US.UTF-8

    ./V07pcre.ztst: all tests successful.
    ...
    **************************************
    43 successful test scripts, 0 failures, 0 skipped
    **************************************

Navy clean ;-)

Re: make check problem

[Bart Schaefer]

On Jan 22, 10:24am, Ray Andrews wrote:
}
} I get the feeling that this pcre isn't important, still one might 
} consider 'not available' to be something to worry about if you didn't 
} know it wasn't, since one might ask why it's checked if it *isn't* 
} important.  (Or checked at all if it is explicitly not available.)

The test suite doesn't know what's available and what's not until it
tries ... and being able to correctly determine that it's [not] there
(i.e., that zmodload does the right thing) is one of the things the
tests are supposed to cover.

It's checked for the obvious QA reason: If it *is* there, then you want
to know that it's working correctly.  The test suite has to be reasonbly
comprehensive, not depend on local configurations.

-- 
Barton E. Schaefer

Re: make check problem

[Ray Andrews]

On 01/22/2015 08:02 PM, Bart Schaefer wrote:
> It's checked for the obvious QA reason: If it *is* there, then you want
> to know that it's working correctly.  The test suite has to be reasonbly
> comprehensive, not depend on local configurations.
Sure, it is better to test too much than not enough.  I'm just excusing 
myself from persueing it for so long at your expence, only to find that 
I don't want it anyway, because it isn't obvious what is or is not 
important there.  One might speculate about some way to improve the test 
such that it knows that it's not even available (and very optional 
besides) and just bows out or says "optional feature pcre not 
available"  or something.  Anyway, your edits to the test seem just 
fine.  The next guy won't have this issue.

--enable-pcre and regex comparison (Was: make check problem)

[Lawrence Velázquez]

On Jan 22, 2015, at 12:56 PM, Bart Schaefer <schaefer@brasslantern.com> wrote:

> Sorry I forgot about --enable-pcre before.  You shouldn't link=static
> the zsh/pcre module without having used --enable-pcre in configure.
> 
> It's special because it's extremely non-standard and changes the meaning
> of the expressions on the right-hand-side of the =~ comparison, so if
> unknowingly enabled it might break scripts.

But doesn't that only take effect if RE_MATCH_PCRE is set?

vq

Re: --enable-pcre and regex comparison (Was: make check problem)

[Bart Schaefer]

On Jan 22,  2:04pm, Lawrence Velazquez wrote:
} Subject: --enable-pcre and regex comparison (Was: make check problem)
}
} On Jan 22, 2015, at 12:56 PM, Bart Schaefer <schaefer@brasslantern.com> wrote:
} 
} > It's special because it's extremely non-standard and changes the meaning
} > of the expressions on the right-hand-side of the =~ comparison, so if
} > unknowingly enabled it might break scripts.
} 
} But doesn't that only take effect if RE_MATCH_PCRE is set?

True; that setopt is more recent than the existence of the module, I think.
Still the point is that somebody might not want pcre automatically added
just because the headers+libraries can be found.

Re: Command substitution parsing issues (not really Re: completion)

[Peter Stephenson]

On Fri, 16 Jan 2015 18:57:32 -0800
Bart Schaefer <schaefer@brasslantern.com> wrote:
> And history recall screws up with aliases:
> 
> schaefer<501> echo $(ls)        
> Config/ config.modules config.status* Etc/ Src/ config.h config.modules.local
> core.28047 foo/ stamp-h config.log config.modules.sh Doc/ Makefile Test/
> schaefer<502> echo $(lsls -CF)

This fixes this but not the issue with completion when there's an alias
in the command substitution.

pws

diff --git a/Src/hist.c b/Src/hist.c
index e5c48db..11d9722 100644
--- a/Src/hist.c
+++ b/Src/hist.c
@@ -329,7 +329,16 @@ static void
 ihwaddc(int c)
 {
     /* Only if history line exists and lexing has not finished. */
-    if (chline && !(errflag || lexstop)) {
+    if (chline && !(errflag || lexstop) &&
+	/*
+	 * If we're reading inside a word for command substitution
+	 * we allow the lexer to expand aliases but don't deal
+	 * with them here.  Note matching code in ihungetc().
+	 * TBD: it might be neater to deal with all aliases in this
+	 * fashion as we never need the expansion in the history
+	 * line, only in the lexer and above.
+	 */
+	!((histactive & HA_INWORD) && (inbufflags & INP_ALIAS))) {
 	/* Quote un-expanded bangs in the history line. */
 	if (c == bangchar && stophist < 2 && qbang)
 	    /* If qbang is not set, we do not escape this bangchar as it's *
@@ -892,11 +901,16 @@ ihungetc(int c)
 	    zlemetall--;
 	    exlast++;
 	}
-	DPUTS(hptr <= chline, "BUG: hungetc attempted at buffer start");
-	hptr--;
-	DPUTS(*hptr != (char) c, "BUG: wrong character in hungetc() ");
-	qbang = (c == bangchar && stophist < 2 &&
-		 hptr > chline && hptr[-1] == '\\');
+	if (!(histactive & HA_INWORD) || !(inbufflags & INP_ALIAS)) {
+	    DPUTS(hptr <= chline, "BUG: hungetc attempted at buffer start");
+	    hptr--;
+	    DPUTS(*hptr != (char) c, "BUG: wrong character in hungetc() ");
+	    qbang = (c == bangchar && stophist < 2 &&
+		     hptr > chline && hptr[-1] == '\\');
+	} else {
+	    /* No active bangs in aliases */
+	    qbang = 0;
+	}
 	if (doit)
 	    inungetc(c);
 	if (!qbang)
diff --git a/Src/lex.c b/Src/lex.c
index 6d45c70..e4dfdfa 100644
--- a/Src/lex.c
+++ b/Src/lex.c
@@ -1674,7 +1674,7 @@ parse_subst_string(char *s)
 /* Called below to report word positions. */
 
 /**/
-mod_export void
+static void
 gotword(void)
 {
     we = zlemetall + 1 - inbufct + (addedx == 2 ? 1 : 0);

Re: Command substitution parsing issues (not really Re: completion)

[Peter Stephenson]

On Fri, 16 Jan 2015 18:57:32 -0800
Bart Schaefer <schaefer@brasslantern.com> wrote:
> I still get:
> 
> schaefer<501> echo ÿÿÿÿÿÿÿÿ$(ls)
> Completing all expansions
> ÿÿÿÿÿÿÿÿ$(ls)
> Completing original
> $(ls)

I've narrowed this down (note no completion in the following repro,
hurrah):

% alias foo='echo poop'
% echo ${(e):-'$(foo)'}
ÿÿÿÿÿÿÿÿ$(foo)

This comes from the call from subst_parse_str() to parsestr() and
parsestrnoerr(), which make unwarranted and undocumented assumptions
about the way the lexer works, in particular that if it passes in a
variable s then any parsing happens within that string --- a pretty
stupid assumption.  The interface needs fixing up, which
ought to be easy enough just by ensuring parsestrnoerr() passes back a
reference to tokstr --- however, I see parsestr is called all over
the place so I'm not going to have time to look now.

We also need to ensure and document that the string parsed in comes from
the heap --- I think that's already a requirement, just nobody got
around to writing it down.  In fact parsestrnoerr() appears to be doing
all sorts of hair-raising things without documentation --- why set the
input string and then immediately after set the lexical buffer to the
original string?

This is nothing fundamentally to do with the rest of the change.  If
anyone else has time, go ahead --- this is killing off my free time at
the moment.

pws

Re: Command substitution parsing issues (not really Re: completion)

[Peter Stephenson]

On Sun, 18 Jan 2015 17:31:57 +0000
Peter Stephenson <p.w.stephenson@ntlworld.com> wrote:
> % alias foo='echo poop'
> % echo ${(e):-'$(foo)'}
> ÿÿÿÿÿÿÿÿ$(foo)
> 
> This comes from the call from subst_parse_str() to parsestr() and
> parsestrnoerr(), which make unwarranted and undocumented assumptions
> about the way the lexer works, in particular that if it passes in a
> variable s then any parsing happens within that string --- a pretty
> stupid assumption.  The interface needs fixing up, which
> ought to be easy enough just by ensuring parsestrnoerr() passes back a
> reference to tokstr --- however, I see parsestr is called all over
> the place so I'm not going to have time to look now.

Changing the interface looks like it works OK.

> We also need to ensure and document that the string parsed in comes from
> the heap --- I think that's already a requirement, just nobody got
> around to writing it down.

And this was broken, too.  If the token string had needed resizing the
whole thing would have fallen over in a big heap.  The hope was that
would be avoided by using the input string as the output string ---
that's why it was duplicated to go into the input buffer.  However,
there was no contract with the lexer that it had to work this way.  I've
changed to use dupstring() where necessary and noted in the comment.

> In fact parsestrnoerr() appears to be doing
> all sorts of hair-raising things without documentation --- why set the
> input string and then immediately after set the lexical buffer to the
> original string?

Just explained above.

pws


diff --git a/Src/Zle/compctl.c b/Src/Zle/compctl.c
index 43dd4e2..189582d 100644
--- a/Src/Zle/compctl.c
+++ b/Src/Zle/compctl.c
@@ -3853,7 +3853,7 @@ makecomplistflags(Compctl cc, char *s, int incmd, int compadd)
 	    yaptr = get_user_var(uv);
 	if ((tt = cc->explain)) {
 	    tt = dupstring(tt);
-	    if ((cc->mask & CC_EXPANDEXPL) && !parsestr(tt)) {
+	    if ((cc->mask & CC_EXPANDEXPL) && !parsestr(&tt)) {
 		singsub(&tt);
 		untokenize(tt);
 	    }
@@ -3873,7 +3873,7 @@ makecomplistflags(Compctl cc, char *s, int incmd, int compadd)
 	}
     } else if ((tt = cc->explain)) {
 	tt = dupstring(tt);
-	if ((cc->mask & CC_EXPANDEXPL) && !parsestr(tt)) {
+	if ((cc->mask & CC_EXPANDEXPL) && !parsestr(&tt)) {
 	    singsub(&tt);
 	    untokenize(tt);
 	}
diff --git a/Src/Zle/compresult.c b/Src/Zle/compresult.c
index dbef7f8..9f383f4 100644
--- a/Src/Zle/compresult.c
+++ b/Src/Zle/compresult.c
@@ -1090,7 +1090,8 @@ do_single(Cmatch m)
 		    }
 		    if (tryit) {
 			noerrs = 1;
-			parsestr(p);
+			p = dupstring(p);
+			parsestr(&p);
 			singsub(&p);
 			errflag &= ~ERRFLAG_ERROR;
 			noerrs = ne;
diff --git a/Src/exec.c b/Src/exec.c
index 7b64951..f42fb2b 100644
--- a/Src/exec.c
+++ b/Src/exec.c
@@ -3772,19 +3772,20 @@ gethere(char **strp, int typ)
 	*bptr++ = '\n';
     }
     *t = '\0';
+    s = buf;
+    buf = dupstring(buf);
+    zfree(s, bsiz);
     if (!qt) {
 	int ef = errflag;
 
-	parsestr(buf);
+	parsestr(&buf);
 
 	if (!errflag) {
 	    /* Retain any user interrupt error */
 	    errflag = ef | (errflag & ERRFLAG_INT);
 	}
     }
-    s = dupstring(buf);
-    zfree(buf, bsiz);
-    return s;
+    return buf;
 }
 
 /* open here string fd */
diff --git a/Src/init.c b/Src/init.c
index e7d86fe..3e41fb9 100644
--- a/Src/init.c
+++ b/Src/init.c
@@ -1197,10 +1197,13 @@ run_init_scripts(void)
 	    if (islogin)
 		sourcehome(".profile");
 	    noerrs = 2;
-	    if (s && !parsestr(s)) {
-		singsub(&s);
-		noerrs = 0;
-		source(s);
+	    if (s) {
+		s = dupstring(s);
+		if (!parsestr(&s)) {
+		    singsub(&s);
+		    noerrs = 0;
+		    source(s);
+		}
 	    }
 	    noerrs = 0;
 	} else
diff --git a/Src/lex.c b/Src/lex.c
index e4dfdfa..0270ec9 100644
--- a/Src/lex.c
+++ b/Src/lex.c
@@ -1503,17 +1503,26 @@ dquote_parse(char endchar, int sub)
     return err;
 }
 
-/* Tokenize a string given in s. Parsing is done as in double *
- * quotes.  This is usually called before singsub().          */
+/*
+ * Tokenize a string given in s. Parsing is done as in double
+ * quotes.  This is usually called before singsub().
+ *
+ * parsestr() is noisier, reporting an error if the parse failed.
+ *
+ * On entry, *s must point to a string allocated from the stack of
+ * exactly the right length, i.e. strlen(*s), as the string
+ * is used as the lexical token string whose memory management
+ * demands this.
+ */
 
 /**/
 mod_export int
-parsestr(char *s)
+parsestr(char **s)
 {
     int err;
 
     if ((err = parsestrnoerr(s))) {
-	untokenize(s);
+	untokenize(*s);
 	if (err > 32 && err < 127)
 	    zerr("parse error near `%c'", err);
 	else
@@ -1524,18 +1533,20 @@ parsestr(char *s)
 
 /**/
 mod_export int
-parsestrnoerr(char *s)
+parsestrnoerr(char **s)
 {
-    int l = strlen(s), err;
+    int l = strlen(*s), err;
 
     zcontext_save();
-    untokenize(s);
-    inpush(dupstring(s), 0, NULL);
+    untokenize(*s);
+    inpush(dupstring(*s), 0, NULL);
     strinbeg(0);
     lexbuf.len = 0;
-    lexbuf.ptr = tokstr = s;
+    lexbuf.ptr = tokstr = *s;
     lexbuf.siz = l + 1;
     err = dquote_parse('\0', 1);
+    if (tokstr)
+	*s = tokstr;
     *lexbuf.ptr = '\0';
     strinend();
     inpop();
diff --git a/Src/params.c b/Src/params.c
index b8e0c42..64c78bd 100644
--- a/Src/params.c
+++ b/Src/params.c
@@ -1260,7 +1260,8 @@ getarg(char **str, int *inv, Value v, int a2, zlong *w,
     if (ishash && (keymatch || !rev))
 	remnulargs(s);
     if (needtok) {
-	if (parsestr(s))
+	s = dupstring(s);
+	if (parsestr(&s))
 	    return 0;
 	singsub(&s);
     } else if (rev)
diff --git a/Src/prompt.c b/Src/prompt.c
index 3552575..ffc1d0d 100644
--- a/Src/prompt.c
+++ b/Src/prompt.c
@@ -183,7 +183,7 @@ promptexpand(char *s, int ns, char *rs, char *Rs, unsigned int *txtchangep)
 	int oldval = lastval;
 
 	s = dupstring(s);
-	if (!parsestr(s))
+	if (!parsestr(&s))
 	    singsub(&s);
 	/*
 	 * We don't need the special Nularg hack here and we're
diff --git a/Src/subst.c b/Src/subst.c
index 5f993d6..a2bb648 100644
--- a/Src/subst.c
+++ b/Src/subst.c
@@ -1306,7 +1306,7 @@ get_intarg(char **s, int *delmatchp)
     p = dupstring(*s + arglen);
     *s = t + arglen;
     *t = sav;
-    if (parsestr(p))
+    if (parsestr(&p))
 	return -1;
     singsub(&p);
     if (errflag)
@@ -1329,7 +1329,8 @@ subst_parse_str(char **sp, int single, int err)
 
     *sp = s = dupstring(*sp);
 
-    if (!(err ? parsestr(s) : parsestrnoerr(s))) {
+    if (!(err ? parsestr(&s) : parsestrnoerr(&s))) {
+	*sp = s;
 	if (!single) {
             int qt = 0;
 
@@ -1439,7 +1440,8 @@ check_colon_subscript(char *str, char **endp)
     }
     sav = **endp;
     **endp = '\0';
-    if (parsestr(str = dupstring(str)))
+    str = dupstring(str);
+    if (parsestr(&str))
 	return NULL;
     singsub(&str);
     remnulargs(str);
diff --git a/Src/utils.c b/Src/utils.c
index f8d2394..4561b5e 100644
--- a/Src/utils.c
+++ b/Src/utils.c
@@ -1515,7 +1515,7 @@ checkmailpath(char **s)
 		    setunderscore(*s);
 
 		    u = dupstring(u);
-		    if (! parsestr(u)) {
+		    if (!parsestr(&u)) {
 			singsub(&u);
 			zputs(u, shout);
 			fputc('\n', shout);
diff --git a/Test/D04parameter.ztst b/Test/D04parameter.ztst
index 94d15f2..cf639fa 100644
--- a/Test/D04parameter.ztst
+++ b/Test/D04parameter.ztst
@@ -1663,3 +1663,10 @@
 0:SHLVL appears sensible when about to exit shell
 >2
 >2
+
+# The following tests the return behaviour of parsestr/parsestrnoerr
+  alias param-test-alias='print $'\''\x45xpanded in substitution'\' 
+  param='$(param-test-alias)'
+  print ${(e)param}
+0:Alias expansion in command substitution in parameter evaluation
+>Expanded in substitution

Re: completion

[Ray Andrews]

[-- Attachment #1: Type: text/plain, Size: 1255 bytes --]

On 01/07/2015 07:01 AM, Vincent Lefevre wrote:
> On 2014-12-11 11:34:20 -0800, Ray Andrews wrote:
>> Anyway, knowing that the star replaces the colons cracks it open.
> This is a bit old, but... When can an asterisk replace a colon?
> For instance, in:
> ...
> which have one or several missing colons. This is from the
> zshcompsys(1) man page.
>
> If an asterisk can match anything, there may be an ambiguity on
> which field some given word will match.
>
I'm glad I'm not the only guy who is puzzled by that.  I dunno
completion sorta works, and some people seem to understand
it, so I can't say much, but IMHO having a optional field
separator is madness.

first-name : last-name : job : fav. hockey team : city : vacation spot

Ray :*: Carpenter : Montreal
Ray : Andrews :*: Vancouver :*: Maui
:*: Andrews :*: Maui

... a bit hard to parse, but all 'true' if separators can be omitted.

Being just a mortal, this seems more tractable  to me:

:Ray::Carpenter:Montreal::Maui:
     I'm Ray, and Montreal is my hockey team

::Andrews:::Vancouver:Maui:
     I live in Vancouver

::Andrews:::::
     My last name is Andrews

:::Carpenter:::Maui:
     I'm a carpenter who vacations in Maui

:Montreal::
     Error! Not enough colons, there must be seven.