* completion @ 2015-01-17 0:19 Ray Andrews 2015-01-17 2:57 ` Command substitution parsing issues (not really Re: completion) Bart Schaefer 0 siblings, 1 reply; 20+ messages in thread From: Ray Andrews @ 2015-01-17 0:19 UTC (permalink / raw) To: Zsh hackers list Previous command recall issue now fixed: $ echo $(ls zsh) zsh-5.0.7-234-gaa503b5 zsh $ echo $(ls zsh) zsh-5.0.7-234-gaa503b5 zsh However completion is still busted: $ ln -svf zsh-5.0.7-175 <TAB> $ ln -svf zsh-5.0.7-175-./Src/zsh: symbol lookup error: /usr/local/lib/zsh/5.0.7-dev-0/zsh/zle.so: undefined symbol: lexsave ... that was trying to complete this: -rwxr-xr-x 1 1220492 2015-01-06-11:27:02 zsh-5.0.7-175-g7e74495 ... is there some missed change in zle.so perhaps? I'm never sure what, besides the kernel, is ever supposed to change. $ l zle.so zsh-5.0.7-175-g7e74495 Sorting All file types Backwards by Modification Time (zle.so): -rwxr-xr-x 1 273104 2014-11-27-16:54:23 zle.so* << latest? ^ permalink raw reply [flat|nested] 20+ messages in thread
* Command substitution parsing issues (not really Re: completion) 2015-01-17 0:19 completion Ray Andrews @ 2015-01-17 2:57 ` Bart Schaefer 2015-01-17 3:29 ` Ray Andrews ` (2 more replies) 0 siblings, 3 replies; 20+ messages in thread From: Bart Schaefer @ 2015-01-17 2:57 UTC (permalink / raw) To: Zsh hackers list On Jan 16, 4:19pm, Ray Andrews wrote: } } $ ln -svf zsh-5.0.7-175-./Src/zsh: symbol lookup error: } /usr/local/lib/zsh/5.0.7-dev-0/zsh/zle.so: undefined symbol: lexsave You need to re-install the shared object files. The symbols have in fact changed, you're loading a .so from a previous build. In fact if you're going to live on the bleeding edge of git pull, you should have a separate shared-object directory for every build. However, the latest change has NOT fixed things for me. I still get: schaefer<501> echo ÿÿÿÿÿÿÿÿ$(ls) Completing all expansions ÿÿÿÿÿÿÿÿ$(ls) Completing original $(ls) And history recall screws up with aliases: schaefer<501> echo $(ls) Config/ config.modules config.status* Etc/ Src/ config.h config.modules.local core.28047 foo/ stamp-h config.log config.modules.sh Doc/ Makefile Test/ schaefer<502> echo $(lsls -CF) Perhaps the alias is why I'm getting the ÿÿÿÿÿÿÿÿ text as well. Let's see ... yes, that's the problem: schaefer<502> unalias ls schaefer<503> echo Config/ Completing expansions Config Test config.modules.sh Doc config.h config.status Etc config.log core.28047 foo config.modules Makefile Src config.modules.local stamp-h Completing all expansions Config config.h config.log config.modules config.modules.local config.modul ... Completing original $(ls) ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: Command substitution parsing issues (not really Re: completion) 2015-01-17 2:57 ` Command substitution parsing issues (not really Re: completion) Bart Schaefer @ 2015-01-17 3:29 ` Ray Andrews 2015-01-17 13:25 ` Vin Shelton 2015-01-19 10:29 ` Peter Stephenson 2015-01-18 16:36 ` Command substitution parsing issues (not really Re: completion) Peter Stephenson 2015-01-18 17:31 ` Peter Stephenson 2 siblings, 2 replies; 20+ messages in thread From: Ray Andrews @ 2015-01-17 3:29 UTC (permalink / raw) To: zsh-workers On 01/16/2015 06:57 PM, Bart Schaefer wrote: > On Jan 16, 4:19pm, Ray Andrews wrote: > } > } $ ln -svf zsh-5.0.7-175-./Src/zsh: symbol lookup error: > } /usr/local/lib/zsh/5.0.7-dev-0/zsh/zle.so: undefined symbol: lexsave > > You need to re-install the shared object files. The symbols have in > fact changed, you're loading a .so from a previous build. Ok, I thought that might be the case. An unidentified symbol would not survive the developer's build of his own work. > > In fact if you're going to live on the bleeding edge of git pull, you > should have a separate shared-object directory for every build. Can you suggest a method? It's easy enough to save old kernels, but the rest of zsh's various files here and there are still a mystery. I didn't even know there were library files like that. Does 'git pull; make -s' even update those? Seems not. There are variables pointing to the locations of these support files, yes? If so, I can easily create numbered directories for the files and then ... well, whatever you recommend. ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: Command substitution parsing issues (not really Re: completion) 2015-01-17 3:29 ` Ray Andrews @ 2015-01-17 13:25 ` Vin Shelton 2015-01-19 10:29 ` Peter Stephenson 1 sibling, 0 replies; 20+ messages in thread From: Vin Shelton @ 2015-01-17 13:25 UTC (permalink / raw) To: Ray Andrews; +Cc: Zsh Hackers' List On Fri, Jan 16, 2015 at 10:29 PM, Ray Andrews <rayandrews@eastlink.ca> wrote: > Can you suggest a method? Every time I build I configure with / install to a different prefix. Usually I just: configure ---prefix=/opt/zsh-$(date +'%Y-%m-%d') make && make install But if I'm making more than one time per day, I: configure --prefix=/opt/zsh-$(date +'%Y-%m-%d-%H%M') make && make install i.e., I add -HHMM to the installation directory name of yyyy-mm-dd. - Vin ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: Command substitution parsing issues (not really Re: completion) 2015-01-17 3:29 ` Ray Andrews 2015-01-17 13:25 ` Vin Shelton @ 2015-01-19 10:29 ` Peter Stephenson 2015-01-19 10:39 ` Peter Stephenson 2015-01-22 0:34 ` make check problem Ray Andrews 1 sibling, 2 replies; 20+ messages in thread From: Peter Stephenson @ 2015-01-19 10:29 UTC (permalink / raw) To: zsh-workers On Fri, 16 Jan 2015 19:29:42 -0800 Ray Andrews <rayandrews@eastlink.ca> wrote: > > In fact if you're going to live on the bleeding edge of git pull, you > > should have a separate shared-object directory for every build. > > Can you suggest a method? I find it good enough to be able to point the compiled executable in Src at a special set of libraries by using an environment variable (you could even avoid that if you detected where the shell was running from early enough). The basic idea (the actual implementation has got complicated) is .zshenv contains if [[ -n $TEST_MODULES ]]; then module_path=(~/src/zsh/modules .) fi and the module links were set up with some variant of the function below. pws ## create_module_links local src=$1 local f if [[ -z $src ]]; then print "Usage: $0 path-to-zsh-src-dir Current directory is directory to create links (above zsh)." fi for f in $src/**/*.mdd~*/zsh.mdd; do lib=${f:r}.so [[ -f $lib ]] && print "Found $lib" name="${$(grep '^name=' $f)##name=}.so" print "Link for $lib is $name." mkdir -p ${name:h} ln -s $lib $name done ## ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: Command substitution parsing issues (not really Re: completion) 2015-01-19 10:29 ` Peter Stephenson @ 2015-01-19 10:39 ` Peter Stephenson 2015-01-22 0:34 ` make check problem Ray Andrews 1 sibling, 0 replies; 20+ messages in thread From: Peter Stephenson @ 2015-01-19 10:39 UTC (permalink / raw) To: zsh-workers On Mon, 19 Jan 2015 10:29:10 +0000 Peter Stephenson <p.stephenson@samsung.com> wrote: > On Fri, 16 Jan 2015 19:29:42 -0800 > Ray Andrews <rayandrews@eastlink.ca> wrote: > > > In fact if you're going to live on the bleeding edge of git pull, you > > > should have a separate shared-object directory for every build. > > > > Can you suggest a method? > > I find it good enough to be able to point the compiled executable in Src > at a special set of libraries by using an environment variable (you > could even avoid that if you detected where the shell was running from > early enough).> > > The basic idea (the actual implementation has got complicated) is > .zshenv contains > > if [[ -n $TEST_MODULES ]]; then > module_path=(~/src/zsh/modules .) > fi > > and the module links were set up with some variant of the function > below. ...and I see I haven't even bothered to explain the main point of this: the module directory just contains links back into the compilation tree. So you only ever do this once; after that you just remember to use TEST_MODULES=1 ./zsh when running in Src. pws ^ permalink raw reply [flat|nested] 20+ messages in thread
* make check problem 2015-01-19 10:29 ` Peter Stephenson 2015-01-19 10:39 ` Peter Stephenson @ 2015-01-22 0:34 ` Ray Andrews 2015-01-22 8:49 ` Peter Stephenson 2015-01-22 16:40 ` Jun T. 1 sibling, 2 replies; 20+ messages in thread From: Ray Andrews @ 2015-01-22 0:34 UTC (permalink / raw) To: zsh-workers I'm getting this 'make check' problem: ./V07pcre.ztst: skipped (the zsh/pcre module is not available) Studying the output of './configure': ... checking pcre.h usability... no checking pcre.h presence... no checking for pcre.h... no checking for pcre_compile... no checking for pcre_study... no checking for pcre_study... no ... After installing 'libpcre3-dev' : checking pcre.h usability... yes checking pcre.h presence... yes checking for pcre.h... yes checking for pcre_compile... no checking for pcre_study... no checking for pcre_exec... no All three 'pcre_*' functions exist in both: /usr/lib/i386-linux-gnu/libpcre.a /usr/lib/i386-linux-gnu/libpcre.so ... so I'm wondering why they are not linked. This is under Debian testing. ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: make check problem 2015-01-22 0:34 ` make check problem Ray Andrews @ 2015-01-22 8:49 ` Peter Stephenson 2015-01-22 16:00 ` Ray Andrews 2015-01-22 16:40 ` Jun T. 1 sibling, 1 reply; 20+ messages in thread From: Peter Stephenson @ 2015-01-22 8:49 UTC (permalink / raw) To: zsh workers [-- Attachment #1: Type: text/plain, Size: 848 bytes --] On 22 January 2015 at 00:34, Ray Andrews <rayandrews@eastlink.ca> wrote: > I'm getting this 'make check' problem: > > ./V07pcre.ztst: skipped (the zsh/pcre module is not available) > > All three 'pcre_*' functions exist in both: > > /usr/lib/i386-linux-gnu/libpcre.a > /usr/lib/i386-linux-gnu/libpcre.so > > ... so I'm wondering why they are not linked. > > This is under Debian testing. (On webmail, apologies if the format is a bit uncontrollable.) This is quite a common problem if your Linux setup isn't specifically installed for software development. The check needs to be able to perform a compilation, so it has to find the headers as well as the libraries. Those are typically in some additional package. You'll probably find you have a package "pcre" installed and need another one called "pcre-devel" or something similar. pws ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: make check problem 2015-01-22 8:49 ` Peter Stephenson @ 2015-01-22 16:00 ` Ray Andrews 0 siblings, 0 replies; 20+ messages in thread From: Ray Andrews @ 2015-01-22 16:00 UTC (permalink / raw) To: zsh-workers On 01/22/2015 12:49 AM, Peter Stephenson wrote: > The check needs to be able to perform a compilation, so it has to find > the headers as well as the libraries. You mean besides /usr/include/pcre.h which is already found? 'libpcre3-dev' is installed. Maybe it will take another Debian user to know what to do. Or is this important at all? One doesn't like to see any defect in 'make check', but I know that some of the checks are not vital. > Those are typically in some additional package. You'll probably find > you have a package "pcre" installed and need another one called > "pcre-devel" or something similar. pws ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: make check problem 2015-01-22 0:34 ` make check problem Ray Andrews 2015-01-22 8:49 ` Peter Stephenson @ 2015-01-22 16:40 ` Jun T. 2015-01-22 17:38 ` Ray Andrews 1 sibling, 1 reply; 20+ messages in thread From: Jun T. @ 2015-01-22 16:40 UTC (permalink / raw) To: zsh-workers 2015/01/22 09:34, Ray Andrews <rayandrews@eastlink.ca> wrote: > checking for pcre_compile... no > checking for pcre_study... no > checking for pcre_exec... no Did you use --enable-pcre with configure? $ ./configure --enable-pcre ...other options... ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: make check problem 2015-01-22 16:40 ` Jun T. @ 2015-01-22 17:38 ` Ray Andrews 2015-01-22 17:56 ` Bart Schaefer 0 siblings, 1 reply; 20+ messages in thread From: Ray Andrews @ 2015-01-22 17:38 UTC (permalink / raw) To: zsh-workers On 01/22/2015 08:40 AM, Jun T. wrote: > 2015/01/22 09:34, Ray Andrews <rayandrews@eastlink.ca> wrote: >> checking for pcre_compile... no >> checking for pcre_study... no >> checking for pcre_exec... no > Did you use --enable-pcre with configure? > > $ ./configure --enable-pcre ...other options... > Nope. I wonder why 'make check' indicates a fault when the test can't possibly succeed from the getgo without that option? Are there other modules that require that sort of special handling? Why is it special? Do I need it anyway? Anyway, I added that option, and now: ---------------------------------------------------- ./V07pcre.ztst: starting. Testing PCRE multibyte with locale en_US.UTF-8 *** /tmp/zsh.ztst.out.27929 2015-01-22 08:55:33.172945830 -0800 --- /tmp/zsh.ztst.tout.27929 2015-01-22 08:55:33.184945978 -0800 *************** *** 1,12 **** ! 0 o→b →b ! 1 ! 1 ! 0 o→† →† ! 0 o→b →b ! 1 ! 0 o→b →b ! 0 o→b →b ! 0 xo→t →t ! 1 ! 0 xo→t →t ! 0 Xo→t →t --- 1,12 ---- ! 127 bar 0 :pcre_compile interface testing: basic, anchored & case-insensitive ! 127 ! 127 ! 127 ! 127 ! 127 ! 127 ! 127 ! 127 ! 127 ! 127 ! 127 Test ./V07pcre.ztst failed: output differs from expected as shown above for: pcre_compile '.(→.)' pcre_match foo→bar print $? $MATCH $match ; unset MATCH match pcre_match foo.bar print $? $MATCH $match ; unset MATCH match pcre_match foo†bar print $? $MATCH $match ; unset MATCH match pcre_match foo→†ar print $? $MATCH $match ; unset MATCH match pcre_study pcre_match foo→bar print $? $MATCH $match ; unset MATCH match pcre_compile -a '.(→.)' pcre_match foo→bar print $? $MATCH $match ; unset MATCH match pcre_match o→bar print $? $MATCH $match ; unset MATCH match pcre_match o→b print $? $MATCH $match ; unset MATCH match pcre_compile 'x.(→.)' pcre_match xo→t print $? $MATCH $match ; unset MATCH match pcre_match Xo→t print $? $MATCH $match ; unset MATCH match pcre_compile -i 'x.(→.)' pcre_match xo→t print $? $MATCH $match ; unset MATCH match pcre_match Xo→t print $? $MATCH $match ; unset MATCH match Error output: (eval):1: command not found: pcre_compile (eval):2: command not found: pcre_match (eval):4: command not found: pcre_match (eval):6: command not found: pcre_match (eval):8: command not found: pcre_match (eval):10: command not found: pcre_study (eval):11: command not found: pcre_match (eval):13: command not found: pcre_compile (eval):14: command not found: pcre_match (eval):16: command not found: pcre_match (eval):18: command not found: pcre_match (eval):20: command not found: pcre_compile (eval):21: command not found: pcre_match (eval):23: command not found: pcre_match (eval):25: command not found: pcre_compile (eval):26: command not found: pcre_match (eval):28: command not found: pcre_match Was testing: pcre_compile interface testing: basic, anchored & case-insensitive ./V07pcre.ztst: test failed. ------------------------------------------------------------- ... However, in config.log: ----------------------------------------------------------- configure:8329: checking for pcre_compile configure:8329: gcc -o conftest -Wall -Wmissing-prototypes -O2 conftest.c -lpcre -ldl -ltinfo -lrt -lm -lc >&5 configure:8329: $? = 0 configure:8329: result: yes configure:8329: checking for pcre_study configure:8329: gcc -o conftest -Wall -Wmissing-prototypes -O2 conftest.c -lpcre -ldl -ltinfo -lrt -lm -lc >&5 configure:8329: $? = 0 configure:8329: result: yes configure:8329: checking for pcre_exec configure:8329: gcc -o conftest -Wall -Wmissing-prototypes -O2 conftest.c -lpcre -ldl -ltinfo -lrt -lm -lc >&5 configure:8329: $? = 0 configure:8329: result: yes -------------------------------------------------------------------- ... there are no failures shown anywhere. It seems the functions are there but there's no matching commands available (if there are any such things). However, I do have /usr/bin/pcre-config and perhaps that is not doing it's job? It is referred to in config.log. ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: make check problem 2015-01-22 17:38 ` Ray Andrews @ 2015-01-22 17:56 ` Bart Schaefer 2015-01-22 18:24 ` Ray Andrews 2015-01-22 19:04 ` --enable-pcre and regex comparison (Was: make check problem) Lawrence Velázquez 0 siblings, 2 replies; 20+ messages in thread From: Bart Schaefer @ 2015-01-22 17:56 UTC (permalink / raw) To: zsh-workers On Jan 22, 9:38am, Ray Andrews wrote: } } Nope. I wonder why 'make check' indicates a fault when the test can't } possibly succeed from the getgo without that option? I've said before that "not available" is not a "fault." You're reading way too much into "make check" skipping a test. } Are there other } modules that require that sort of special handling? Why is it special? Sorry I forgot about --enable-pcre before. You shouldn't link=static the zsh/pcre module without having used --enable-pcre in configure. It's special because it's extremely non-standard and changes the meaning of the expressions on the right-hand-side of the =~ comparison, so if unknowingly enabled it might break scripts. } Do I need it anyway? You only need it if you want to use it. } Error output: } (eval):1: command not found: pcre_compile } (eval):2: command not found: pcre_match } (eval):10: command not found: pcre_study Hmm, sorry, my patch in 34338 probably borked this. Try the below: diff --git a/Test/V07pcre.ztst b/Test/V07pcre.ztst index 3c25be2..ddfd3f5 100644 --- a/Test/V07pcre.ztst +++ b/Test/V07pcre.ztst @@ -5,6 +5,8 @@ ZTST_unimplemented="the zsh/pcre module is not available" return 0 fi +# Load the rest of the builtins + zmodload zsh/pcre setopt rematch_pcre # Find a UTF-8 locale. setopt multibyte ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: make check problem 2015-01-22 17:56 ` Bart Schaefer @ 2015-01-22 18:24 ` Ray Andrews 2015-01-23 4:02 ` Bart Schaefer 2015-01-22 19:04 ` --enable-pcre and regex comparison (Was: make check problem) Lawrence Velázquez 1 sibling, 1 reply; 20+ messages in thread From: Ray Andrews @ 2015-01-22 18:24 UTC (permalink / raw) To: zsh-workers On 01/22/2015 09:56 AM, Bart Schaefer wrote: > I've said before that "not available" is not a "fault." You're reading > way too much into "make check" skipping a test. Sure. But it's difficult to know what's important there and what is not. I get the feeling that this pcre isn't important, still one might consider 'not available' to be something to worry about if you didn't know it wasn't, since one might ask why it's checked if it *isn't* important. (Or checked at all if it is explicitly not available.) > It's special because it's extremely non-standard and changes the meaning > of the expressions on the right-hand-side of the =~ comparison, so if > unknowingly enabled it might break scripts. Ok thanks. Looks like I don't want it anyway. At this point it's an exercise. Patch: ... Testing PCRE multibyte with locale en_US.UTF-8 ./V07pcre.ztst: all tests successful. ... ************************************** 43 successful test scripts, 0 failures, 0 skipped ************************************** Navy clean ;-) ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: make check problem 2015-01-22 18:24 ` Ray Andrews @ 2015-01-23 4:02 ` Bart Schaefer 2015-01-23 4:48 ` Ray Andrews 0 siblings, 1 reply; 20+ messages in thread From: Bart Schaefer @ 2015-01-23 4:02 UTC (permalink / raw) To: zsh-workers On Jan 22, 10:24am, Ray Andrews wrote: } } I get the feeling that this pcre isn't important, still one might } consider 'not available' to be something to worry about if you didn't } know it wasn't, since one might ask why it's checked if it *isn't* } important. (Or checked at all if it is explicitly not available.) The test suite doesn't know what's available and what's not until it tries ... and being able to correctly determine that it's [not] there (i.e., that zmodload does the right thing) is one of the things the tests are supposed to cover. It's checked for the obvious QA reason: If it *is* there, then you want to know that it's working correctly. The test suite has to be reasonbly comprehensive, not depend on local configurations. -- Barton E. Schaefer ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: make check problem 2015-01-23 4:02 ` Bart Schaefer @ 2015-01-23 4:48 ` Ray Andrews 0 siblings, 0 replies; 20+ messages in thread From: Ray Andrews @ 2015-01-23 4:48 UTC (permalink / raw) To: zsh-workers On 01/22/2015 08:02 PM, Bart Schaefer wrote: > It's checked for the obvious QA reason: If it *is* there, then you want > to know that it's working correctly. The test suite has to be reasonbly > comprehensive, not depend on local configurations. Sure, it is better to test too much than not enough. I'm just excusing myself from persueing it for so long at your expence, only to find that I don't want it anyway, because it isn't obvious what is or is not important there. One might speculate about some way to improve the test such that it knows that it's not even available (and very optional besides) and just bows out or says "optional feature pcre not available" or something. Anyway, your edits to the test seem just fine. The next guy won't have this issue. ^ permalink raw reply [flat|nested] 20+ messages in thread
* --enable-pcre and regex comparison (Was: make check problem) 2015-01-22 17:56 ` Bart Schaefer 2015-01-22 18:24 ` Ray Andrews @ 2015-01-22 19:04 ` Lawrence Velázquez 2015-01-23 4:04 ` Bart Schaefer 1 sibling, 1 reply; 20+ messages in thread From: Lawrence Velázquez @ 2015-01-22 19:04 UTC (permalink / raw) To: Bart Schaefer; +Cc: zsh-workers On Jan 22, 2015, at 12:56 PM, Bart Schaefer <schaefer@brasslantern.com> wrote: > Sorry I forgot about --enable-pcre before. You shouldn't link=static > the zsh/pcre module without having used --enable-pcre in configure. > > It's special because it's extremely non-standard and changes the meaning > of the expressions on the right-hand-side of the =~ comparison, so if > unknowingly enabled it might break scripts. But doesn't that only take effect if RE_MATCH_PCRE is set? vq ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: --enable-pcre and regex comparison (Was: make check problem) 2015-01-22 19:04 ` --enable-pcre and regex comparison (Was: make check problem) Lawrence Velázquez @ 2015-01-23 4:04 ` Bart Schaefer 0 siblings, 0 replies; 20+ messages in thread From: Bart Schaefer @ 2015-01-23 4:04 UTC (permalink / raw) To: zsh-workers On Jan 22, 2:04pm, Lawrence Velazquez wrote: } Subject: --enable-pcre and regex comparison (Was: make check problem) } } On Jan 22, 2015, at 12:56 PM, Bart Schaefer <schaefer@brasslantern.com> wrote: } } > It's special because it's extremely non-standard and changes the meaning } > of the expressions on the right-hand-side of the =~ comparison, so if } > unknowingly enabled it might break scripts. } } But doesn't that only take effect if RE_MATCH_PCRE is set? True; that setopt is more recent than the existence of the module, I think. Still the point is that somebody might not want pcre automatically added just because the headers+libraries can be found. ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: Command substitution parsing issues (not really Re: completion) 2015-01-17 2:57 ` Command substitution parsing issues (not really Re: completion) Bart Schaefer 2015-01-17 3:29 ` Ray Andrews @ 2015-01-18 16:36 ` Peter Stephenson 2015-01-18 17:31 ` Peter Stephenson 2 siblings, 0 replies; 20+ messages in thread From: Peter Stephenson @ 2015-01-18 16:36 UTC (permalink / raw) To: Zsh hackers list On Fri, 16 Jan 2015 18:57:32 -0800 Bart Schaefer <schaefer@brasslantern.com> wrote: > And history recall screws up with aliases: > > schaefer<501> echo $(ls) > Config/ config.modules config.status* Etc/ Src/ config.h config.modules.local > core.28047 foo/ stamp-h config.log config.modules.sh Doc/ Makefile Test/ > schaefer<502> echo $(lsls -CF) This fixes this but not the issue with completion when there's an alias in the command substitution. pws diff --git a/Src/hist.c b/Src/hist.c index e5c48db..11d9722 100644 --- a/Src/hist.c +++ b/Src/hist.c @@ -329,7 +329,16 @@ static void ihwaddc(int c) { /* Only if history line exists and lexing has not finished. */ - if (chline && !(errflag || lexstop)) { + if (chline && !(errflag || lexstop) && + /* + * If we're reading inside a word for command substitution + * we allow the lexer to expand aliases but don't deal + * with them here. Note matching code in ihungetc(). + * TBD: it might be neater to deal with all aliases in this + * fashion as we never need the expansion in the history + * line, only in the lexer and above. + */ + !((histactive & HA_INWORD) && (inbufflags & INP_ALIAS))) { /* Quote un-expanded bangs in the history line. */ if (c == bangchar && stophist < 2 && qbang) /* If qbang is not set, we do not escape this bangchar as it's * @@ -892,11 +901,16 @@ ihungetc(int c) zlemetall--; exlast++; } - DPUTS(hptr <= chline, "BUG: hungetc attempted at buffer start"); - hptr--; - DPUTS(*hptr != (char) c, "BUG: wrong character in hungetc() "); - qbang = (c == bangchar && stophist < 2 && - hptr > chline && hptr[-1] == '\\'); + if (!(histactive & HA_INWORD) || !(inbufflags & INP_ALIAS)) { + DPUTS(hptr <= chline, "BUG: hungetc attempted at buffer start"); + hptr--; + DPUTS(*hptr != (char) c, "BUG: wrong character in hungetc() "); + qbang = (c == bangchar && stophist < 2 && + hptr > chline && hptr[-1] == '\\'); + } else { + /* No active bangs in aliases */ + qbang = 0; + } if (doit) inungetc(c); if (!qbang) diff --git a/Src/lex.c b/Src/lex.c index 6d45c70..e4dfdfa 100644 --- a/Src/lex.c +++ b/Src/lex.c @@ -1674,7 +1674,7 @@ parse_subst_string(char *s) /* Called below to report word positions. */ /**/ -mod_export void +static void gotword(void) { we = zlemetall + 1 - inbufct + (addedx == 2 ? 1 : 0); ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: Command substitution parsing issues (not really Re: completion) 2015-01-17 2:57 ` Command substitution parsing issues (not really Re: completion) Bart Schaefer 2015-01-17 3:29 ` Ray Andrews 2015-01-18 16:36 ` Command substitution parsing issues (not really Re: completion) Peter Stephenson @ 2015-01-18 17:31 ` Peter Stephenson 2015-01-18 22:34 ` Peter Stephenson 2 siblings, 1 reply; 20+ messages in thread From: Peter Stephenson @ 2015-01-18 17:31 UTC (permalink / raw) To: Zsh hackers list On Fri, 16 Jan 2015 18:57:32 -0800 Bart Schaefer <schaefer@brasslantern.com> wrote: > I still get: > > schaefer<501> echo ÿÿÿÿÿÿÿÿ$(ls) > Completing all expansions > ÿÿÿÿÿÿÿÿ$(ls) > Completing original > $(ls) I've narrowed this down (note no completion in the following repro, hurrah): % alias foo='echo poop' % echo ${(e):-'$(foo)'} ÿÿÿÿÿÿÿÿ$(foo) This comes from the call from subst_parse_str() to parsestr() and parsestrnoerr(), which make unwarranted and undocumented assumptions about the way the lexer works, in particular that if it passes in a variable s then any parsing happens within that string --- a pretty stupid assumption. The interface needs fixing up, which ought to be easy enough just by ensuring parsestrnoerr() passes back a reference to tokstr --- however, I see parsestr is called all over the place so I'm not going to have time to look now. We also need to ensure and document that the string parsed in comes from the heap --- I think that's already a requirement, just nobody got around to writing it down. In fact parsestrnoerr() appears to be doing all sorts of hair-raising things without documentation --- why set the input string and then immediately after set the lexical buffer to the original string? This is nothing fundamentally to do with the rest of the change. If anyone else has time, go ahead --- this is killing off my free time at the moment. pws ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: Command substitution parsing issues (not really Re: completion) 2015-01-18 17:31 ` Peter Stephenson @ 2015-01-18 22:34 ` Peter Stephenson 0 siblings, 0 replies; 20+ messages in thread From: Peter Stephenson @ 2015-01-18 22:34 UTC (permalink / raw) To: Zsh hackers list On Sun, 18 Jan 2015 17:31:57 +0000 Peter Stephenson <p.w.stephenson@ntlworld.com> wrote: > % alias foo='echo poop' > % echo ${(e):-'$(foo)'} > ÿÿÿÿÿÿÿÿ$(foo) > > This comes from the call from subst_parse_str() to parsestr() and > parsestrnoerr(), which make unwarranted and undocumented assumptions > about the way the lexer works, in particular that if it passes in a > variable s then any parsing happens within that string --- a pretty > stupid assumption. The interface needs fixing up, which > ought to be easy enough just by ensuring parsestrnoerr() passes back a > reference to tokstr --- however, I see parsestr is called all over > the place so I'm not going to have time to look now. Changing the interface looks like it works OK. > We also need to ensure and document that the string parsed in comes from > the heap --- I think that's already a requirement, just nobody got > around to writing it down. And this was broken, too. If the token string had needed resizing the whole thing would have fallen over in a big heap. The hope was that would be avoided by using the input string as the output string --- that's why it was duplicated to go into the input buffer. However, there was no contract with the lexer that it had to work this way. I've changed to use dupstring() where necessary and noted in the comment. > In fact parsestrnoerr() appears to be doing > all sorts of hair-raising things without documentation --- why set the > input string and then immediately after set the lexical buffer to the > original string? Just explained above. pws diff --git a/Src/Zle/compctl.c b/Src/Zle/compctl.c index 43dd4e2..189582d 100644 --- a/Src/Zle/compctl.c +++ b/Src/Zle/compctl.c @@ -3853,7 +3853,7 @@ makecomplistflags(Compctl cc, char *s, int incmd, int compadd) yaptr = get_user_var(uv); if ((tt = cc->explain)) { tt = dupstring(tt); - if ((cc->mask & CC_EXPANDEXPL) && !parsestr(tt)) { + if ((cc->mask & CC_EXPANDEXPL) && !parsestr(&tt)) { singsub(&tt); untokenize(tt); } @@ -3873,7 +3873,7 @@ makecomplistflags(Compctl cc, char *s, int incmd, int compadd) } } else if ((tt = cc->explain)) { tt = dupstring(tt); - if ((cc->mask & CC_EXPANDEXPL) && !parsestr(tt)) { + if ((cc->mask & CC_EXPANDEXPL) && !parsestr(&tt)) { singsub(&tt); untokenize(tt); } diff --git a/Src/Zle/compresult.c b/Src/Zle/compresult.c index dbef7f8..9f383f4 100644 --- a/Src/Zle/compresult.c +++ b/Src/Zle/compresult.c @@ -1090,7 +1090,8 @@ do_single(Cmatch m) } if (tryit) { noerrs = 1; - parsestr(p); + p = dupstring(p); + parsestr(&p); singsub(&p); errflag &= ~ERRFLAG_ERROR; noerrs = ne; diff --git a/Src/exec.c b/Src/exec.c index 7b64951..f42fb2b 100644 --- a/Src/exec.c +++ b/Src/exec.c @@ -3772,19 +3772,20 @@ gethere(char **strp, int typ) *bptr++ = '\n'; } *t = '\0'; + s = buf; + buf = dupstring(buf); + zfree(s, bsiz); if (!qt) { int ef = errflag; - parsestr(buf); + parsestr(&buf); if (!errflag) { /* Retain any user interrupt error */ errflag = ef | (errflag & ERRFLAG_INT); } } - s = dupstring(buf); - zfree(buf, bsiz); - return s; + return buf; } /* open here string fd */ diff --git a/Src/init.c b/Src/init.c index e7d86fe..3e41fb9 100644 --- a/Src/init.c +++ b/Src/init.c @@ -1197,10 +1197,13 @@ run_init_scripts(void) if (islogin) sourcehome(".profile"); noerrs = 2; - if (s && !parsestr(s)) { - singsub(&s); - noerrs = 0; - source(s); + if (s) { + s = dupstring(s); + if (!parsestr(&s)) { + singsub(&s); + noerrs = 0; + source(s); + } } noerrs = 0; } else diff --git a/Src/lex.c b/Src/lex.c index e4dfdfa..0270ec9 100644 --- a/Src/lex.c +++ b/Src/lex.c @@ -1503,17 +1503,26 @@ dquote_parse(char endchar, int sub) return err; } -/* Tokenize a string given in s. Parsing is done as in double * - * quotes. This is usually called before singsub(). */ +/* + * Tokenize a string given in s. Parsing is done as in double + * quotes. This is usually called before singsub(). + * + * parsestr() is noisier, reporting an error if the parse failed. + * + * On entry, *s must point to a string allocated from the stack of + * exactly the right length, i.e. strlen(*s), as the string + * is used as the lexical token string whose memory management + * demands this. + */ /**/ mod_export int -parsestr(char *s) +parsestr(char **s) { int err; if ((err = parsestrnoerr(s))) { - untokenize(s); + untokenize(*s); if (err > 32 && err < 127) zerr("parse error near `%c'", err); else @@ -1524,18 +1533,20 @@ parsestr(char *s) /**/ mod_export int -parsestrnoerr(char *s) +parsestrnoerr(char **s) { - int l = strlen(s), err; + int l = strlen(*s), err; zcontext_save(); - untokenize(s); - inpush(dupstring(s), 0, NULL); + untokenize(*s); + inpush(dupstring(*s), 0, NULL); strinbeg(0); lexbuf.len = 0; - lexbuf.ptr = tokstr = s; + lexbuf.ptr = tokstr = *s; lexbuf.siz = l + 1; err = dquote_parse('\0', 1); + if (tokstr) + *s = tokstr; *lexbuf.ptr = '\0'; strinend(); inpop(); diff --git a/Src/params.c b/Src/params.c index b8e0c42..64c78bd 100644 --- a/Src/params.c +++ b/Src/params.c @@ -1260,7 +1260,8 @@ getarg(char **str, int *inv, Value v, int a2, zlong *w, if (ishash && (keymatch || !rev)) remnulargs(s); if (needtok) { - if (parsestr(s)) + s = dupstring(s); + if (parsestr(&s)) return 0; singsub(&s); } else if (rev) diff --git a/Src/prompt.c b/Src/prompt.c index 3552575..ffc1d0d 100644 --- a/Src/prompt.c +++ b/Src/prompt.c @@ -183,7 +183,7 @@ promptexpand(char *s, int ns, char *rs, char *Rs, unsigned int *txtchangep) int oldval = lastval; s = dupstring(s); - if (!parsestr(s)) + if (!parsestr(&s)) singsub(&s); /* * We don't need the special Nularg hack here and we're diff --git a/Src/subst.c b/Src/subst.c index 5f993d6..a2bb648 100644 --- a/Src/subst.c +++ b/Src/subst.c @@ -1306,7 +1306,7 @@ get_intarg(char **s, int *delmatchp) p = dupstring(*s + arglen); *s = t + arglen; *t = sav; - if (parsestr(p)) + if (parsestr(&p)) return -1; singsub(&p); if (errflag) @@ -1329,7 +1329,8 @@ subst_parse_str(char **sp, int single, int err) *sp = s = dupstring(*sp); - if (!(err ? parsestr(s) : parsestrnoerr(s))) { + if (!(err ? parsestr(&s) : parsestrnoerr(&s))) { + *sp = s; if (!single) { int qt = 0; @@ -1439,7 +1440,8 @@ check_colon_subscript(char *str, char **endp) } sav = **endp; **endp = '\0'; - if (parsestr(str = dupstring(str))) + str = dupstring(str); + if (parsestr(&str)) return NULL; singsub(&str); remnulargs(str); diff --git a/Src/utils.c b/Src/utils.c index f8d2394..4561b5e 100644 --- a/Src/utils.c +++ b/Src/utils.c @@ -1515,7 +1515,7 @@ checkmailpath(char **s) setunderscore(*s); u = dupstring(u); - if (! parsestr(u)) { + if (!parsestr(&u)) { singsub(&u); zputs(u, shout); fputc('\n', shout); diff --git a/Test/D04parameter.ztst b/Test/D04parameter.ztst index 94d15f2..cf639fa 100644 --- a/Test/D04parameter.ztst +++ b/Test/D04parameter.ztst @@ -1663,3 +1663,10 @@ 0:SHLVL appears sensible when about to exit shell >2 >2 + +# The following tests the return behaviour of parsestr/parsestrnoerr + alias param-test-alias='print $'\''\x45xpanded in substitution'\' + param='$(param-test-alias)' + print ${(e)param} +0:Alias expansion in command substitution in parameter evaluation +>Expanded in substitution ^ permalink raw reply [flat|nested] 20+ messages in thread
end of thread, other threads:[~2015-01-23 4:48 UTC | newest] Thread overview: 20+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2015-01-17 0:19 completion Ray Andrews 2015-01-17 2:57 ` Command substitution parsing issues (not really Re: completion) Bart Schaefer 2015-01-17 3:29 ` Ray Andrews 2015-01-17 13:25 ` Vin Shelton 2015-01-19 10:29 ` Peter Stephenson 2015-01-19 10:39 ` Peter Stephenson 2015-01-22 0:34 ` make check problem Ray Andrews 2015-01-22 8:49 ` Peter Stephenson 2015-01-22 16:00 ` Ray Andrews 2015-01-22 16:40 ` Jun T. 2015-01-22 17:38 ` Ray Andrews 2015-01-22 17:56 ` Bart Schaefer 2015-01-22 18:24 ` Ray Andrews 2015-01-23 4:02 ` Bart Schaefer 2015-01-23 4:48 ` Ray Andrews 2015-01-22 19:04 ` --enable-pcre and regex comparison (Was: make check problem) Lawrence Velázquez 2015-01-23 4:04 ` Bart Schaefer 2015-01-18 16:36 ` Command substitution parsing issues (not really Re: completion) Peter Stephenson 2015-01-18 17:31 ` Peter Stephenson 2015-01-18 22:34 ` Peter Stephenson
Code repositories for project(s) associated with this public inbox https://git.vuxu.org/mirror/zsh/ This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).