A week ago I identified the problem [attached mail: lex.c add() extends tokstr=calloc() by a non-zeroing hrealloc]. The bug is (rarely) triggered by : AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA="" The bug is presumed to still be present in latest zsh (based on valgrind instrumentation and test). I believe to have been clear on my expectations : << was expecting HP to ask/pay Redhat to ask/pay the zsh developers to solve issues like this one ... A simple solution is to set *bptr=0 at the end of function add, but I am not sure this has no other consequences, to be checked with zsh developers. >> The suggested workaround was delivered today and Ludovic immediately identified a regression, which I now understand is the direct consequence of the suggested workaround. As I find no trace of requests for help to zsh developers in http://www.zsh.org/mla/workers/2011/index.html I have put them in Cc to make sure they are aware of the issue and can provide help if they can. The problem with the suggested workaround is that the add() in lex.c is also used when zsh re-parses, during execution, the already parsed left-hand side of the assignment V[0] in order to extract the subscript : parse_subscript(char *s, int sub) { ... bptr = tokstr = s; #0 add (c=48) at lex.c:559 -> c=48 is '0' in "[0]", *bptr=0 will zero the ']' after the '0'. #1 0x0000000000455b1a in dquote_parse (endchar=93 ']', sub=1) at lex.c:1514 #2 0x0000000000455e04 in parse_subscript (s=0x2aaaaaac9122 "0", sub=1) at lex.c:1600 #3 0x0000000000469d3a in isident (s=0x2aaaaaac9120 "Q[0") at params.c:985 #4 0x000000000046e834 in assignsparam (s=0x2aaaaaac9120 "Q[0", val=0x6f0100 "1", flags=0) at params.c:2529 #5 0x000000000042c49d in addvars (state=0x7fffffffb9a0, pc=0x2aaaaaac90e8, addflags=0) at exec.c:2143 #6 0x00000000004290f0 in execsimple (state=0x7fffffffb9a0) at exec.c:1014 So the suggested workaround truncates the V[0] to V[0, which causes the "invalid subscript" regression. A more conservative workaround would be to only set the memory extension to zero immediately after the hrealloc call, getting something like a hrecalloc effect, this is much less likely to trigger other side-effects : After: bptr = len + (tokstr = (char *)hrealloc(tokstr, bsiz, newbsiz)); Add: memset (bptr, 0, newbsiz - bsiz); /* len == bsiz, bptr points at first re-allocated byte, newbsiz - bsiz is size added */ I could not find the zsh-4.2.6-5.el5.src.rpm on the web (RedHat hiding re-packaged GPL software ???), so I did some amount of testing applying this minimal fix in zsh-4.2.7. This shows the original problem is resolved, the regression is avoided, and I did not found other regressions so far. High priority : - HP : Produce a new patch, validate/test, verify the new proposed fix with zsh developers. - MAS/WLS : If no fast progress, please grant me permission to patch /usr/bin/ksh on dhws029. - HP : Provide full source rpm/tgz for zsh-4.2.6-5.el5.src Medium priority : - Fix in latest zsh version (if confirmed still to be a problem). - Add/use valgrind instrumentation (see attached mail) to avoid we get such defects again in the future. - Factorize the workaround in a hrecalloc and check for other patterns of this defect (realloc to extend calloc). - Clarify if RedHat engineers asked for help from zsh-workers, and if not, explain why not. - Clarify what HP/RedHat software support really means. We seem to repeatedly get no added value in terms of problem identification/resolution/validation, while we suffer delays of approx. 1 year based on unjustified hope and lagging behind zsh version approx. 4 years. Low priority : - Clarify why RedHat does not publish src.rpm of re-packaged GPL software. -----Original Message----- From: Genot, Harry [mailto:harry.genot@hp.com] Sent: Friday 2 December 2011 15:26 To: BOVEN Tom; LORANG Geert; VAN DE VOORDE Bart; 'Bart van den Heuvel' Cc: CFMU HP Verbeke K; Godts, Jeroen; VAN VLIERBERGHE Stef Subject: RE: zsh test package (PBI110) and openmotif fixes (INC14171 and INC8550) Hello all, Here is a quick status update after the phone conference we had with Red Hat. The test package that we received yesterday was based on zsh-4.2.6-6 which is part of the future RHEL5.8. The test-rpm just contained 1 small change to fix the original problem. However, as you observed, it introduced an other (worse) problem. The plan is now to build a new test package based on the zsh version that you are using (version 4.2.6-5) with the fix for the original problem. We will get this rpm before 16:00 today and we hope this rpm can be tested quickly. If that package works fine, then Red Hat will investigate the problem that was introduced with the 4.2.6-6 test package. Best regards, Harry -----Original Message----- From: Godts, Jeroen Sent: vrijdag 2 december 2011 14:43 To: stef.van-vlierberghe@eurocontrol.int Cc: Genot, Harry; Verbeke, Koen Subject: FW: zsh test package (PBI110) and openmotif fixes (INC14171 and INC8550) Hi Stef, We reported this problem towards Red Hat and they are already working on it, We have a follow up call at 15h CET and Harry Genot will report the status update Towards HPMS so they can add this in the Remedy system. Harry, Can you include Stef in the mailing for feedback Thx Jeroen GODTS MCP Account Support Manager Hewlett-Packard Belgium BVBA/SPRL   +32 2 729 73 09 / Tel +32 477 299 282 / Mobile Godts@hp.com / Email Hermes Laan 1A, B-1831 Diegem Enterprise number 0402.220.594 - RPM/RPR Brussels     Please consider the environment before printing this email. -----Original Message----- From: Ludovic Brenta [mailto:ludovic.brenta@eurocontrol.int] Sent: Friday 2 December 2011 12:36 To: VAN VLIERBERGHE Stef Cc: LORANG Geert; MEERSMAN Koen; BOVEN Tom; WAROQUIERS Philippe Subject: Re: zsh test package (PBI110) and openmotif fixes (INC14171 and INC8550) "VAN VLIERBERGHE Stef" writes: > ASAP and please also dhws018 (WAO). > > -----Original Message----- > From: LORANG Geert > Sent: Thursday 1 December 2011 19:47 > To: VAN VLIERBERGHE Stef; BRENTA Ludovic; MEERSMAN Koen > Cc: BOVEN Tom > Subject: zsh test package (PBI110) and openmotif fixes > (INC14171 and INC8550) > > Stef, Koen, Ludovic, > > Red Hat provided us a test package for the zsh problem; A=B > command not found - PBI160, many thanks for your investigations > by the way Stef! > They also provided us a new openmotif package that should > contain the fixes for INC14171 and INC8550 (memory leaks and > seg. fault on tear off) > > We suggest to install the packages on your workstations > (dhws029, dhws009 and dhws027) so you can validate it. > > Could you please let us know when we can do it? It can be done > online but please expect an interruption of a few seconds while > we replace the binaries. *PLEASE ROLLBACK IMMEDIATELY* I assume that the version of the package in question is zsh-4.2.6-6.el5.test.1? It is installed on my machine and I think it caused my Tbuild_view to fail with a REALLY STUPID bug. In each subsystem that I built, the Csystem_build.log contains: ================================================== CC_CHECKPOINT: link_adalib starting at Fri Dec 2 12:07:37 CET 2011 ================================================== /cm/ot/CMA/CM_TOOL!2.1.264/build_G!82.IP.L5/exe/adamake_bind:59: invalid subscript ================================================== CC_CHECKPOINT: link_adalib finished at Fri Dec 2 12:07:37 CET 2011 ================================================== This error is not present in the baseline build. The line that triggers this bug, in adamake_bind, is: ( PROGRAMS[0]="" <- line 59 PROGRAMS[1]="" Consequence of this bug: the build produces no executables at all, despite the fact that all Ada units compiled properly; they are "just" not linked. In fact the problem is worse: $ grep -n 'invalid subscript' /cm/ot/ELD/*!LBR.5*/build_default_Linux/Csystem_build.log /cm/ot/TACT/*!LBR.5*/build_default_Linux/Csystem_build.log /cm/ot/ELD/GEO_ENV!LBR.57663_ENTRY_TIME_IN_TV_ELD/build_default_Linux/Cs ystem_build.log:5972:/cm/ot/CMA/CM_TOOL!2.1.264/build_G!82.IP.L5/exe/ada make_bind:59: invalid subscript /cm/ot/ELD/GEO_FLIGHT!LBR.57663_ENTRY_TIME_IN_TV_ELD/build_default_Linux /Csystem_build.log:2010:/cm/ot/CMA/CM_TOOL!2.1.264/build_G!82.IP.L5/exe/ adamake_bind:59: invalid subscript /cm/ot/ELD/GEO_MMI!LBR.57663_ENTRY_TIME_IN_TV_ELD/build_default_Linux/Cs ystem_build.log:2451:/cm/ot/CMA/CM_TOOL!2.1.264/build_G!82.IP.L5/exe/ada make_bind:59: invalid subscript ____   This message and any files transmitted with it are legally privileged and intended for the sole use of the individual(s) or entity to whom they are addressed. If you are not the intended recipient, please notify the sender by reply and delete the message and any attachments from your system. Any unauthorised use or disclosure of the content of this message is strictly prohibited and may be unlawful.   Nothing in this e-mail message amounts to a contractual or legal commitment on the part of EUROCONTROL, unless it is confirmed by appropriately signed hard copy.   Any views expressed in this message are those of the sender.