zsh-4.2.6-5.el5 rhel5.5 accesses uninitialized memory in an assignment statement using a variable name of 31 or more characters.

zsh-4.2.6-5.el5 rhel5.5 accesses uninitialized memory in an assignment statement using a variable name of 31 or more characters.

[VAN VLIERBERGHE Stef]

[-- Attachment #1: Type: text/plain, Size: 9434 bytes --]

A week ago I identified the problem [attached mail: lex.c add() extends tokstr=calloc() by a non-zeroing hrealloc].

The bug is (rarely) triggered by : AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=""

The bug is presumed to still be present in latest zsh (based on valgrind instrumentation and test).

I believe to have been clear on my expectations :
<<
was expecting HP to ask/pay Redhat to ask/pay the zsh developers to solve issues like this one
...
A simple solution is to set *bptr=0 at the end of function add, but I am not sure
this has no other consequences, to be checked with zsh developers.
>>

The suggested workaround was delivered today and Ludovic immediately identified a regression, which I now
understand is the direct consequence of the suggested workaround.

As I find no trace of requests for help to zsh developers in http://www.zsh.org/mla/workers/2011/index.html
I have put them in Cc to make sure they are aware of the issue and can provide help if they can.

The problem with the suggested workaround is that the add() in lex.c is also used when zsh re-parses,
during execution, the already parsed left-hand side of the assignment V[0] in order to extract the subscript :

parse_subscript(char *s, int sub)
{
...
    bptr = tokstr = s;

#0  add (c=48) at lex.c:559  -> c=48 is '0' in "[0]", *bptr=0 will zero the ']' after the '0'.

#1  0x0000000000455b1a in dquote_parse (endchar=93 ']', sub=1) at lex.c:1514
#2  0x0000000000455e04 in parse_subscript (s=0x2aaaaaac9122 "0", sub=1) at lex.c:1600
#3  0x0000000000469d3a in isident (s=0x2aaaaaac9120 "Q[0") at params.c:985
#4  0x000000000046e834 in assignsparam (s=0x2aaaaaac9120 "Q[0", val=0x6f0100 "1", flags=0) at params.c:2529
#5  0x000000000042c49d in addvars (state=0x7fffffffb9a0, pc=0x2aaaaaac90e8, addflags=0) at exec.c:2143
#6  0x00000000004290f0 in execsimple (state=0x7fffffffb9a0) at exec.c:1014

So the suggested workaround truncates the V[0] to V[0, which causes the "invalid subscript" regression.

A more conservative workaround would be to only set the memory extension to zero immediately after the hrealloc
call, getting something like a hrecalloc effect, this is much less likely to trigger other side-effects :

After:
	bptr = len + (tokstr = (char *)hrealloc(tokstr, bsiz, newbsiz));
Add:
      memset (bptr, 0, newbsiz - bsiz); /* len == bsiz, bptr points at first re-allocated byte, newbsiz - bsiz is size added */

I could not find the zsh-4.2.6-5.el5.src.rpm on the web (RedHat hiding re-packaged GPL software ???), so I did some
amount of testing applying this minimal fix in zsh-4.2.7.

This shows the original problem is resolved, the regression is avoided, and I did not found other regressions so far.

High priority :
- HP : Produce a new patch, validate/test, verify the new proposed fix with zsh developers.
- MAS/WLS : If no fast progress, please grant me permission to patch /usr/bin/ksh on dhws029.
- HP : Provide full source rpm/tgz for zsh-4.2.6-5.el5.src

Medium priority :
- Fix in latest zsh version (if confirmed still to be a problem).
- Add/use valgrind instrumentation (see attached mail) to avoid we get such defects again in the future.
- Factorize the workaround in a hrecalloc and check for other patterns of this defect (realloc to extend calloc).
- Clarify if RedHat engineers asked for help from zsh-workers, and if not, explain why not.
- Clarify what HP/RedHat software support really means. We seem to repeatedly get no added value in terms of problem
  identification/resolution/validation, while we suffer delays of approx. 1 year based on unjustified hope and
  lagging behind zsh version approx. 4 years.

Low priority :
- Clarify why RedHat does not publish src.rpm of re-packaged GPL software.

-----Original Message-----
From: Genot, Harry [mailto:harry.genot@hp.com]
Sent: Friday 2 December 2011 15:26
To: BOVEN Tom; LORANG Geert; VAN DE VOORDE Bart; 'Bart van den Heuvel'
Cc: CFMU HP Verbeke K; Godts, Jeroen; VAN VLIERBERGHE Stef
Subject: RE: zsh test package (PBI110) and openmotif fixes
(INC14171 and INC8550)

Hello all,

Here is a quick status update after the phone conference we had
with Red Hat.

The test package that we received yesterday was based on
zsh-4.2.6-6 which is part of the future RHEL5.8.  The test-rpm
just contained 1 small change to fix the original problem.
However, as you observed, it introduced an other (worse) problem.

The plan is now to build a new test package based on the zsh
version that you are using (version 4.2.6-5) with the fix for
the original problem.  We will get this rpm before 16:00 today
and we hope this rpm can be tested quickly.  If that package
works fine, then Red Hat will investigate the problem that was
introduced with the 4.2.6-6 test package.

Best regards,
Harry

-----Original Message-----
From: Godts, Jeroen
Sent: vrijdag 2 december 2011 14:43
To: stef.van-vlierberghe@eurocontrol.int
Cc: Genot, Harry; Verbeke, Koen
Subject: FW: zsh test package (PBI110) and openmotif fixes
(INC14171 and INC8550)

Hi Stef,

We reported this problem towards Red Hat and they are already
working on it,
We have a follow up call at 15h CET and Harry Genot will report
the status update
Towards HPMS so they can add this in the Remedy system.

Harry,
Can you include Stef in the mailing for feedback




Thx


Jeroen GODTS
MCP Account Support Manager
Hewlett-Packard Belgium BVBA/SPRL
 
+32 2 729 73 09  / Tel
+32 477 299 282 / Mobile
Godts@hp.com       / Email
Hermes Laan 1A,
B-1831 Diegem
Enterprise number 0402.220.594 - RPM/RPR Brussels
 

 
Please consider the environment before printing this email.



-----Original Message-----
From: Ludovic Brenta [mailto:ludovic.brenta@eurocontrol.int]
Sent: Friday 2 December 2011 12:36
To: VAN VLIERBERGHE Stef
Cc: LORANG Geert; MEERSMAN Koen; BOVEN Tom; WAROQUIERS Philippe
Subject: Re: zsh test package (PBI110) and openmotif fixes (INC14171 and
INC8550)

"VAN VLIERBERGHE Stef" <stef.van-vlierberghe@eurocontrol.int> writes:
> ASAP and please also dhws018 (WAO).
>
> -----Original Message-----
> From: LORANG Geert
> Sent: Thursday 1 December 2011 19:47
> To: VAN VLIERBERGHE Stef; BRENTA Ludovic; MEERSMAN Koen
> Cc: BOVEN Tom
> Subject: zsh test package (PBI110) and openmotif fixes
> (INC14171 and INC8550)
>
> Stef, Koen, Ludovic,
>
> Red Hat provided us a test package for the zsh problem; A=B
> command not found - PBI160, many thanks for your investigations
> by the way Stef!
> They also provided us a new openmotif package that should
> contain the fixes for INC14171 and INC8550 (memory leaks and
> seg. fault on tear off)
>
> We suggest to install the packages on your workstations
> (dhws029, dhws009 and dhws027) so you can validate it.
>
> Could you please let us know when we can do it? It can be done
> online but please expect an interruption of a few seconds while
> we replace the binaries.

*PLEASE ROLLBACK IMMEDIATELY*

I assume that the version of the package in question is
zsh-4.2.6-6.el5.test.1?  It is installed on my machine and I think it
caused my Tbuild_view to fail with a REALLY STUPID bug.  In each
subsystem
that I built, the Csystem_build.log contains:

==================================================
CC_CHECKPOINT: link_adalib starting at Fri Dec  2 12:07:37 CET 2011
==================================================
/cm/ot/CMA/CM_TOOL!2.1.264/build_G!82.IP.L5/exe/adamake_bind:59: invalid
subscript
==================================================
CC_CHECKPOINT: link_adalib finished at Fri Dec  2 12:07:37 CET 2011
==================================================

This error is not present in the baseline build.  The line that triggers
this bug, in adamake_bind, is:

  (
  PROGRAMS[0]=""    <- line 59
  PROGRAMS[1]=""



Consequence of this bug: the build produces no executables at all,
despite
the fact that all Ada units compiled properly; they are "just" not
linked.

In fact the problem is worse:

$ grep -n 'invalid subscript'
/cm/ot/ELD/*!LBR.5*/build_default_Linux/Csystem_build.log
/cm/ot/TACT/*!LBR.5*/build_default_Linux/Csystem_build.log
/cm/ot/ELD/GEO_ENV!LBR.57663_ENTRY_TIME_IN_TV_ELD/build_default_Linux/Cs
ystem_build.log:5972:/cm/ot/CMA/CM_TOOL!2.1.264/build_G!82.IP.L5/exe/ada
make_bind:59: invalid subscript
/cm/ot/ELD/GEO_FLIGHT!LBR.57663_ENTRY_TIME_IN_TV_ELD/build_default_Linux
/Csystem_build.log:2010:/cm/ot/CMA/CM_TOOL!2.1.264/build_G!82.IP.L5/exe/
adamake_bind:59: invalid subscript
/cm/ot/ELD/GEO_MMI!LBR.57663_ENTRY_TIME_IN_TV_ELD/build_default_Linux/Cs
ystem_build.log:2451:/cm/ot/CMA/CM_TOOL!2.1.264/build_G!82.IP.L5/exe/ada
make_bind:59: invalid subscript

____
 
This message and any files transmitted with it are legally privileged and intended for the sole use of the individual(s) or entity to whom they are addressed. If you are not the intended recipient, please notify the sender by reply and delete the message and any attachments from your system. Any unauthorised use or disclosure of the content of this message is strictly prohibited and may be unlawful.
 
Nothing in this e-mail message amounts to a contractual or legal commitment on the part of EUROCONTROL, unless it is confirmed by appropriately signed hard copy.
 
Any views expressed in this message are those of the sender.

[-- Attachment #2: Type: message/rfc822, Size: 33995 bytes --]

From: "VAN VLIERBERGHE Stef" <stef.van-vlierberghe@eurocontrol.int>
To: "MEERSMAN Koen" <koen.meersman@eurocontrol.int>, "LORANG Geert" <geert.lorang@eurocontrol.int>
Cc: "CFMU HP MS" <cfmu.hp.ms@eurocontrol.int>, "CFMU DEVD TC DE" <cfmu.devsup@eurocontrol.int>, "CFMU ENGD NeOS" <CFMU.ENGD.NeOS@eurocontrol.int>, "MAES Stefan" <stefan.maes@eurocontrol.int>, "WILLEMS Eric" <eric.willems@eurocontrol.int>
Subject: INC000000017487: RE: Check build closure - TACT.TACT_CONFIG.16.0.100/G!97.OP.L5
Date: Mon, 28 Nov 2011 02:42:39 +0100
Message-ID: <1B2B2EF98D55CB41BD16F13B18B9B008134CC128@FFBRUE001.cfmu.corp.eurocontrol.int>

There is a bug in mem.c, accessing uninitialized memory after a token.

Details :

The variable name of 32 chars or longer triggers the bug, but the
presumed rehash "culprit" is unrelated.

In gettokstr(int c, int sub)
the search for a new token starts with the allocation of a tokstr
containing all zeros :

	bptr = tokstr = (char *) hcalloc(bsiz = 32);

When in the same function the act=LX_EQUALS actions tries to recognize
the left hand side
of the assignment, tokstr contains this left-hand-side, and bptr is
pointing to the next
byte after the "added" characters in tokstr so far.

After checking for a token starting with a digit

		char *t = tokstr;
		if (idigit(*t))
		    while (++t < bptr && idigit(*t));
		else {

a call is made to itype_end to determine the length of an identifier.
What is strange is
that temporarily the byte after the last added is set to zero, such that
tokstr will
temporarily be zero-terminated, but in the else part (if there are no
parentheses)
this local repair is undone by setting *bptr back to sav.

		    int sav = *bptr;
		    *bptr = '\0';
		    t = itype_end(t, IIDENT, 0);
		    if (t < bptr) {
			skipparens(Inbrack, Outbrack, &t);
		    } else {
			*bptr = sav;
		    }
		}

One may wonder why there is possibly no zero byte after tokstr, as the
initial
allocation was made via hcalloc, which suggests that the idea is to
always have
the unintialized part of tokstr zero-filled.

However, in add() tokstr gets extended from length 32 to length 64
using:

	bptr = len + (tokstr = (char *)hrealloc(tokstr, bsiz, newbsiz));

using hrealloc, which does not set bytes in the extension to zero.

So, this local repair is necessary, but the root cause, the unreliable
design,
is not removed.

The next statement leads to the problem : The construct below reads the
uninitialized
value and increments t :

		if (*t == '+')
                    t++;

So this occurs if after a more than 32-char left-hand-side token without
brackets is 
recognized, but as the uninitialized memory is very rarely a '+' (in
much less than
1/256 of the cases, zero bytes are far mor probable), the bug is hard to
reproduce
and anybody not having understood the issue would have to resist jumping
to the
conclusion to have solved it.

We need asap a fix and in medium term a solution removing the root cause
and some
way to avoid that hairy issues have to be resolved by me after waiting
in vain
for about a periods of approx. 1 year (mallinfo, zsh hangs, and now
this).
Whatever the KPIs are having as an effect, they clearly do not force
people paid
for support to sink their teeth in less tasteful objects (I obviously do
not mean Geert,
but was expecting HP to ask/pay Redhat to ask/pay the zsh developers to
solve issues
like this one).

The bug is still in the latest zsh version (assuming there is no hidden
undocumented zero-ing
effect in the hrealloc() call, which appears to be the case because it
contains              
a construct :


#ifdef ZSH_MEM_DEBUG
            memset(p, 0xff, old);
#endif                           

which suggests that bytes are only zero-d to simplify  (or in the case
deteriorate) heap debugging).

Adding before each hrealloc return a call to signal valgrind that the
bytes in the extension
(p+old, new-old) or (ptr+old, new-old) (depending on the value returned)
are undefined :                  

#include "valgrind/memcheck.h"
#include "valgrind/valgrind.h"

      VALGRIND_MAKE_MEM_UNDEFINED(p+old, new-old);
            unqueue_signals();                    

detects this error systematically, using a simple test :

vvl@hsim001> ls -ltr
ls -ltr             
total 4364          
lrwxrwxrwx. 1 vvl vvl       3 Nov 26 20:57 ksh -> zsh
-rwxr-xr-x. 1 vvl vvl   92872 Nov 27 22:14 vgdb      
-rwxr-xr-x. 1 vvl vvl   42455 Nov 27 22:14 valgrind  
-rwxr-xr-x. 1 vvl vvl   65500 Nov 27 22:15 cg_merge  
-rwxr-xr-x. 1 vvl vvl   10417 Nov 27 22:15 cg_diff   
-rwxr-xr-x. 1 vvl vvl   32169 Nov 27 22:15 cg_annotate
-rwxr-xr-x. 1 vvl vvl   11522 Nov 27 22:15 callgrind_control
-rwxr-xr-x. 1 vvl vvl   40873 Nov 27 22:15 callgrind_annotate
-rwxr-xr-x. 1 vvl vvl   24211 Nov 27 22:15 ms_print          
-rwxr-xr-x. 1 vvl vvl   23734 Nov 27 22:15 valgrind-listener         
-rwxr-xr-x. 2 vvl vvl 1362233 Nov 28 00:47 zsh-4.3.12        
-rwxr-xr-x. 2 vvl vvl 1362233 Nov 28 00:47 zsh
-rw-r--r--. 1 vvl vvl      37 Nov 28 00:51 test.ksh
vvl@hsim001> cat test.ksh
cat test.ksh
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=""

vvl@hsim001> ./valgrind --trace-children=yes  ./ksh ./test.ksh
./valgrind --trace-children=yes  ./ksh ./test.ksh
==3222== Memcheck, a memory error detector
==3222== Copyright (C) 2002-2011, and GNU GPL'd, by Julian Seward et al.
==3222== Using Valgrind-3.7.0 and LibVEX; rerun with -h for copyright
info
==3222== Command: ./ksh ./test.ksh
==3222==
==3222== Conditional jump or move depends on uninitialised value(s)
==3222==    at 0x4561C4: gettokstr (lex.c:1299) (lex.c modified, 1299 is
"if (*t == '+')" in my version) 
==3222==    by 0x455990: gettok (lex.c:1001)
==3222==    by 0x4546C7: zshlex (lex.c:395)
==3222==    by 0x4765C2: parse_event (parse.c:451)
==3222==    by 0x4479AC: loop (init.c:132)
==3222==    by 0x44AD81: zsh_main (init.c:1528)
==3222==    by 0x40DD33: main (main.c:93)
==3222==

A simple solution is to set *bptr=0 at the end of function add, but I am
not sure
this has no other consequences, to be checked with zsh developers.
Adding *bptr='+'
forces the bug to show up on each execution.

Ideally a fix is deployed on OPS before CCAMS is activated (in Feb 2012
I believe),
please give high priority. We could of course go over all scripts and
try to rename
all variable names everywhere to less than 32 chars, but that is
obviously not the
most reliable solution.

I would also suggest to add the valgrind support with a configuration
option to latest
zsh, similar to ZSH_MEM_DEBUG, such that zsh heap management can be
tested more thoroughly.

-----Original Message-----
From: VAN VLIERBERGHE Stef 
Sent: 19 November, 2011 03:20
To: LORANG Geert; MEERSMAN Koen
Cc: WAROQUIERS Philippe
Subject: RE: Check build closure - TACT.TACT_CONFIG.16.0.100/G!97.OP.L5

Nice progress, thanks.

Koen, I wanted to try it but could not find ./mea_test.

Geert, the length of variables is probably only affecting the
circumstances where the bug triggers, if we add
one line of code before that it will most likely fail with different
lengths.

I suggest next steps :
- Get zsh built from sources
- Use valgrind to check for uninitialized memory read.

-----Original Message-----
From: LORANG Geert 
Sent: Friday 18 November 2011 00:33
To: MEERSMAN Koen; VAN VLIERBERGHE Stef
Subject: RE: Check build closure - TACT.TACT_CONFIG.16.0.100/G!97.OP.L5

Another update on this issue: 

I captured all ENV variables and unset them one by one while 
reproducing the error: 

log@dhinfd01: for i in `cat env_keys`; do echo $i ; unset $i; 
for i in 1 2 3 4 5; do ksh meatest; sleep 1; done ; done
TERM
perform:1: command not found: 
OTHER_UNIT_REACHABLE_AND_CCAMS_ACTIVE=FALSE
perform:1: command not found: 
OTHER_UNIT_REACHABLE_AND_CCAMS_ACTIVE=FALSE
perform:1: command not found: 
OTHER_UNIT_REACHABLE_AND_CCAMS_ACTIVE=FALSE
HOME
perform:1: command not found: 
OTHER_UNIT_REACHABLE_AND_CCAMS_ACTIVE=FALSE
SHELL
perform:1: command not found: 
OTHER_UNIT_REACHABLE_AND_CCAMS_ACTIVE=FALSE
perform:1: command not found: 
OTHER_UNIT_REACHABLE_AND_CCAMS_ACTIVE=FALSE
perform:1: command not found: 
OTHER_UNIT_REACHABLE_AND_CCAMS_ACTIVE=FALSE

...


OPUSDIR
perform:1: command not found: 
OTHER_UNIT_REACHABLE_AND_CCAMS_ACTIVE=FALSE
perform:1: command not found: 
OTHER_UNIT_REACHABLE_AND_CCAMS_ACTIVE=FALSE
perform:1: command not found: 
OTHER_UNIT_REACHABLE_AND_CCAMS_ACTIVE=FALSE
perform:1: command not found: 
OTHER_UNIT_REACHABLE_AND_CCAMS_ACTIVE=FALSE
CFMU_OS
perform:1: command not found: 
OTHER_UNIT_REACHABLE_AND_CCAMS_ACTIVE=FALSE
http_proxy
perform:1: command not found: 
OTHER_UNIT_REACHABLE_AND_CCAMS_ACTIVE=FALSE
perform:1: command not found: 
OTHER_UNIT_REACHABLE_AND_CCAMS_ACTIVE=FALSE
perform:1: command not found: 
OTHER_UNIT_REACHABLE_AND_CCAMS_ACTIVE=FALSE
ENV
LD_LIBRARY_PATH
CDPATH
MANPATH
BROWSER
ORACLE_SQLNET
HISTFILE

Once ENV is unset it's not reproducible anymore. If you reset 
ENV to /etc/kshrc it's reproducible again.

The culprit in /etc/kshrc is the "hash -r" command (that was 
added for the 'command-not-found' INC / 'INC2976 - nohashcmds 
doesn't seem to work in some cases '). Once "hash -r" from 
/etc/kshrc is removed it's also not reproducible anymore.

I'll keep you posted.

Regards,
Geert

-----Original Message-----
From: LORANG Geert 
Sent: 17 November 2011 20:09
To: MEERSMAN Koen
Cc: VAN VLIERBERGHE Stef
Subject: RE: Check build closure - TACT.TACT_CONFIG.16.0.100/G!97.OP.L5

Koen, 

Many thanks for this, this is very useful info. I don't have a 
solution yet, but it might help you:

I did some tests and it looks like everything that is between 
34 and 37 characters long is prone to this bug.
Especially the length of 37 characters seems the most vulnerable.

The two reported errors "OPTION_IGNORE_NOT_EXISTING_REGULATION" 
and "OTHER_UNIT_REACHABLE_AND_CCAMS_ACTIVE" are both exactly 37 
characters long.

Short summary of tests (complete list below): 

- 10000 tests of 32 characters : 0 failed
- 10000 tests of 33 characters : 0 failed
- 1000 tests of 34 characters : 7 failed
- 1000 tests of 35 characters : 5 failed
- 1000 tests of 36 characters : 3 failed
- 1000 tests of 37 characters : 658-673 failed
- 10000 tests of 38 characters : 0 failed

As a temporary workaround, would it be possible to shorten the 
variable? 

Thanks.
Geert






log@dhinfd01: export i=0; while (( $i < 10000 )) ; do ksh 
meatest32 2>&1 ; let i=$i+1; echo $i ; done |grep "not found" -B1
log@dhinfd01:


log@dhinfd01: export i=0; while (( $i < 10000 )) ; do ksh 
meatest33 2>&1 ; let i=$i+1; echo $i ; done |grep "not found" -B1
log@dhinfd01:



log@dhinfd01: export i=0; while (( $i < 1000 )) ; do ksh 
meatest34 2>&1 ; let i=$i+1; echo $i ; done |grep "not found" -B1
496
a: command not found: SOMETHING_THAT_IS_34_CHARACTERS_LO=TRUE
--
545
a: command not found: SOMETHING_THAT_IS_34_CHARACTERS_LO=TRUE
--
619
a: command not found: SOMETHING_THAT_IS_34_CHARACTERS_LO=TRUE
--
625
a: command not found: SOMETHING_THAT_IS_34_CHARACTERS_LO=TRUE
--
691
a: command not found: SOMETHING_THAT_IS_34_CHARACTERS_LO=TRUE
--
946
a: command not found: SOMETHING_THAT_IS_34_CHARACTERS_LO=TRUE
--
961
a: command not found: SOMETHING_THAT_IS_34_CHARACTERS_LO=TRUE






log@dhinfd01: export i=0; while (( $i < 1000 )) ; do ksh 
meatest35 2>&1 ; let i=$i+1; echo $i ; done |grep "not found" -B1
350
a: command not found: SOMETHING_THAT_IS_35_CHARACTERS_LON=TRUE
--
365
a: command not found: SOMETHING_THAT_IS_35_CHARACTERS_LON=TRUE
--
375
a: command not found: SOMETHING_THAT_IS_35_CHARACTERS_LON=TRUE
--
405
a: command not found: SOMETHING_THAT_IS_35_CHARACTERS_LON=TRUE
--
411
a: command not found: SOMETHING_THAT_IS_35_CHARACTERS_LON=TRUE






log@dhinfd01: export i=0; while (( $i < 1000 )) ; do ksh 
meatest36 2>&1 ; let i=$i+1; echo $i ; done |grep "not found" -B1
67
a: command not found: SOMETHING_THAT_IS_36_CHARACTERS_LONG=TRUE
--
248
a: command not found: SOMETHING_THAT_IS_36_CHARACTERS_LONG=TRUE
--
906
a: command not found: SOMETHING_THAT_IS_36_CHARACTERS_LONG=TRUE



log@dhinfd01: export i=0; while (( $i < 1000 )) ; do ksh 
meatest37 2>&1 ; let i=$i+1; echo $i ; done |grep "not found" |wc -l
658
log@dhinfd01: export i=0; while (( $i < 1000 )) ; do ksh 
meatest37 2>&1 ; let i=$i+1; echo $i ; done |grep "not found" |wc -l
658
log@dhinfd01: export i=0; while (( $i < 1000 )) ; do ksh 
meatest37 2>&1 ; let i=$i+1; echo $i ; done |grep "not found" |wc -l
673
log@dhinfd01: export i=0; while (( $i < 1000 )) ; do ksh 
meatest37 2>&1 ; let i=$i+1; echo $i ; done |grep "not found" |wc -l
654



log@dhinfd01: export i=0; while (( $i < 10000 )) ; do ksh 
meatest38 2>&1 ; let i=$i+1; echo $i ; done |grep "not found" -B1
log@dhinfd01:


log@dhinfd01: tail meatest3*
==> meatest32 <==
#!/usr/bin/ksh
function a { SOMETHING_THAT_IS_32_CHARACTERS_=TRUE }
a

==> meatest33 <==
#!/usr/bin/ksh
function a { SOMETHING_THAT_IS_33_CHARACTERS_L=TRUE }
a

==> meatest34 <==
#!/usr/bin/ksh
function a { SOMETHING_THAT_IS_34_CHARACTERS_LO=TRUE }
a

==> meatest35 <==
#!/usr/bin/ksh
function a { SOMETHING_THAT_IS_35_CHARACTERS_LON=TRUE }
a

==> meatest36 <==
#!/usr/bin/ksh
function a { SOMETHING_THAT_IS_36_CHARACTERS_LONG=TRUE }
a

==> meatest37 <==
#!/usr/bin/ksh
function a { SOMETHING_THAT_IS_37_CHARACTERS_LOONG=TRUE }
a

==> meatest38 <==
#!/usr/bin/ksh
function a { SOMETHING_THAT_IS_38_CHARACTERS_LOOONG=TRUE }
a

 

-----Original Message-----
From: Koen Meersman [mailto:koen.meersman@eurocontrol.int] 
Sent: 17 November, 2011 13:37
To: LORANG Geert
Cc: VAN VLIERBERGHE Stef; CFMU HP MS; CFMU DEVD TC DE; CFMU ENGD NeOS;
MAES Stefan; WILLEMS Eric
Subject: RE: Check build closure - TACT.TACT_CONFIG.16.0.100/G!97.OP.L5

Try by doing: 

TAC_DVH: ./mea_test 
./mea_test 
perform:1: command not found:
OTHER_UNIT_REACHABLE_AND_CCAMS_ACTIVE=FALSE
TAC_DVH: cat mea_test
cat mea_test
#!/usr/bin/ksh

perform () {
  OTHER_UNIT_REACHABLE_AND_CCAMS_ACTIVE="FALSE"
}

perform 
TAC_DVH: hostname
hostname
bell

On Fri, 2011-11-11 at 00:09 +0100, LORANG Geert wrote:
> Stef,
>  
> As discussed earlier today: We are already aware it is not fixed on
> RHEL 5.5 (CFMU Release 5.5.1.5.1); At the time you logged
> INC000000017487:
>  
>  
> INC000000007593 AGAIN
> The problem occurred again, so INC 7593 was wrongly considered
> resolved.
> I made a reproducer, and executed it both as vvl and as cmadm (to make
> sure there is no influence from my account).
> Will attach reprod.tgz (tar -cz reprod) :
> vvl@dhws029: ssh cmadm@localhost
> Last login: Thu Jul 7 16:16:16 2011 from localhost
> System has CFMU Release 5.5.1.5.1 installed
> cmadm@dhws029: cd ~vvl/reprod
> cmadm@dhws029: ./restore_env_dmp env.dmp
> TAC_DVH: ksh ./compare_ft_data_simul ft_data_OYBUD_simul.out
> ft_data_OYBUD_live.out -IGNORE_IFPS -IGNORE_NOT_EXISTING_REGULATION
> ./compare_ft_data_simul:29: command not found:
> OPTION_IGNORE_NOT_EXISTING_REGULATION=TRUE
> TAC_DVH: ksh ./compare_ft_data_simul ft_data_OYBUD_simul.out
> ft_data_OYBUD_live.out -IGNORE_IFPS -IGNORE_NOT_EXISTING_REGULATION
> ./compare_ft_data_simul:29: command not found:
> OPTION_IGNORE_NOT_EXISTING_REGULATION=TRUE
> TAC_DVH: ksh ./compare_ft_data_simul ft_data_OYBUD_simul.out
> ft_data_OYBUD_live.out -IGNORE_IFPS -IGNORE_NOT_EXISTING_REGULATION
> TAC_DVH: ksh ./compare_ft_data_simul ft_data_OYBUD_simul.out
> ft_data_OYBUD_live.out -IGNORE_IFPS -IGNORE_NOT_EXISTING_REGULATION
> ./compare_ft_data_simul:29: command not found:
> OPTION_IGNORE_NOT_EXISTING_REGULATION=TRUE
> TAC_DVH: ksh ./compare_ft_data_simul ft_data_OYBUD_simul.out
> ft_data_OYBUD_live.out -IGNORE_IFPS -IGNORE_NOT_EXISTING_REGULATION
> So you can see that the problem occurs not systematically, looks like
> uninitialized memory or race conditions.
>  
> Suddenly this was not reproducable anymore and we're a bit stuck on
> this call. We'll try to reproduce it again and keep you posted. 
> In case you find a way to reproduce it again, please let us know.
>  
> I promoted the INC to a problem in Remedy : PBI000000000160 and linked
> all INCs to the problem.
>  
> On a side node: Thierry left our team already a few months. In case
> you want HP management attention, direct your emails to Koen Verbeke.
>  
> Thanks.
> Geert
>  
> 
> 
> ______________________________________________________________________
> From: VAN VLIERBERGHE Stef 
> Sent: 08 November 2011 19:40
> To: CFMU HP MS
> Cc: CFMU DEVD TC DE; CFMU ENGD NeOS; MAES Stefan; WILLEMS Eric;
> Thierry Bruyere
> Subject: RE: Check build closure - TACT.TACT_CONFIG.16.0.100/G!
> 97.OP.L5 
> 
> 
> 
> In the attached mail I said that the bug with the signature below
> corresponded to an old zsh bug
> (INC000000002959 zsh/ksh giving error command not found: A=B when
> encountering assignments)
> that was supposed to be fixed in Linux 5.5 :
> 
> run_test_programs.log contains 'does not exist' error messages
> perform:32: command not found:
> OTHER_UNIT_REACHABLE_AND_CCAMS_ACTIVE=FALSE
> perform:32: command not found:
> OTHER_UNIT_REACHABLE_AND_CCAMS_ACTIVE=FALSE
> perform:32: command not found:
> OTHER_UNIT_REACHABLE_AND_CCAMS_ACTIVE=FALSE
> run_test_programs.log contains 'command not found' messages.
> Build missing: .finished_test_programs flag
> 
> Unfortunately, the build report below from todays build on dhbldd13,
> still shows the problem, and the message on that host
> shows it runs 15.5.... so it looks like the bug is NOT fixed !
> 
> vvl@dhbldd03: ssh dhbldd13
> Last login: Tue Nov  8 20:06:37 2011 from
> dhws029.cfmu.corp.eurocontrol.int
>
************************************************************************
*****************************************************
> * !!! WARNING !!!
> *
> *
> *
> * This system has the next CFMU Release installed (CFMU Release
> 5.5.1.5.1). Use this machine ONLY for CFMU 15.5 development *
> *
> *
> * !!! WARNING !!!
> *
>
************************************************************************
*****************************************************
> 
> HP MS please check if the fix for this INC has been deployed on
> dhbldd13, and if not, please re-analyze the INC (and re-open the INC
> or make a new one).
> 
> P.S. The top priority problem is still the NFS I/O errors and
> timeouts, this INC can wait until this issue gets resolved.
> 
> -----Original Message-----
> From: CM Administrator [mailto:cfmu.devsup@eurocontrol.int]
> Sent: Tuesday 8 November 2011 18:57
> To: CFMU DEVD TACT
> Subject: Check build closure - TACT.TACT_CONFIG.16.0.100/G!97.OP.L5
> 
> 
> ===== Checking for errors in
> /cm/build4/cm/ot/TOOL/POSIXLIB!00.0001/build_G!97.OP.L5/Csystem_
> build.log
> ===== No errors found in
> /cm/build4/cm/ot/TOOL/POSIXLIB!00.0001/build_G!97.OP.L5/Csystem_
> build.log
> 
> ===== Checking for errors in
> /cm/build4/cm/ot/TOOL/GNU!16.0.13/build_G!97.OP.L5/Csystem_build.log
> ===== No errors found in
> /cm/build4/cm/ot/TOOL/GNU!16.0.13/build_G!97.OP.L5/Csystem_build.log
> ===== Checking for errors in
> /cm/build4/cm/ot/TOOL/GNU!16.0.13/build_G!
> 97.OP.L5/run_test_programs.log
> ===== No errors found in
> /cm/build4/cm/ot/TOOL/GNU!16.0.13/build_G!
> 97.OP.L5/run_test_programs.log
> 
> ===== Checking for errors in
> /cm/build4/cm/ot/TOOL/AWS!16.0.13/build_G!97.OP.L5/Csystem_build.log
> ===== No errors found in
> /cm/build4/cm/ot/TOOL/AWS!16.0.13/build_G!97.OP.L5/Csystem_build.log
> 
> ===== Checking for errors in
> /cm/build12/cm/ot/TOOL/ADA_PRIM!16.0.33/build_G!97.OP.L5/Csystem
> _build.log
> Csystem_build.log contains 'Compiler' error messages.
> contains (non-fatal) 'warnings' error messages:
> ==========================================
> ===== Checking for errors in
> /cm/build12/cm/ot/TOOL/ADA_PRIM!16.0.33/build_G!97.OP.L5/run_tes
> t_programs.log
> ===== No errors found in
> /cm/build12/cm/ot/TOOL/ADA_PRIM!16.0.33/build_G!97.OP.L5/run_tes
> t_programs.log
> 
> ===== Checking for errors in
> /cm/build1/cm/ot/ORA/DB_TOOLS!15.5.59/build_G!
> 97.OP.L5/Csystem_build.log
> ===== No errors found in
> /cm/build1/cm/ot/ORA/DB_TOOLS!15.5.59/build_G!
> 97.OP.L5/Csystem_build.log
> 
> ===== Checking for errors in
> /cm/build12/cm/ot/TOOL/ORACLE!16.0.44/build_G!
> 97.OP.L5/Csystem_build.log
> contains (non-fatal) 'warnings' error messages:
> ===== No errors found in
> /cm/build12/cm/ot/TOOL/ORACLE!16.0.44/build_G!
> 97.OP.L5/Csystem_build.log
> ===== Checking for errors in
> /cm/build12/cm/ot/TOOL/ORACLE!16.0.44/build_G!97.OP.L5/run_test_
> programs.log
> ===== No errors found in
> /cm/build12/cm/ot/TOOL/ORACLE!16.0.44/build_G!97.OP.L5/run_test_
> programs.log
> 
> ===== Checking for errors in
> /cm/build12/cm/ot/TOOL/ADA_BASIC!16.0.45/build_G!97.OP.L5/Csyste
> m_build.log
> contains (non-fatal) 'warnings' error messages:
> ===== No errors found in
> /cm/build12/cm/ot/TOOL/ADA_BASIC!16.0.45/build_G!97.OP.L5/Csyste
> m_build.log
> ===== Checking for errors in
> /cm/build12/cm/ot/TOOL/ADA_BASIC!16.0.45/build_G!97.OP.L5/run_te
> st_programs.log
> ===== No errors found in
> /cm/build12/cm/ot/TOOL/ADA_BASIC!16.0.45/build_G!97.OP.L5/run_te
> st_programs.log
> 
> ===== Checking for errors in
> /cm/build12/cm/ot/TOOL/BOOCH!16.0.33/build_G!
> 97.OP.L5/Csystem_build.log
> ===== No errors found in
> /cm/build12/cm/ot/TOOL/BOOCH!16.0.33/build_G!
> 97.OP.L5/Csystem_build.log
> 
> ===== Checking for errors in
> /cm/build12/cm/ot/TOOL/COMM!16.0.45/build_G!97.OP.L5/Csystem_build.log
> contains (non-fatal) 'warnings' error messages:
> ===== No errors found in
> /cm/build12/cm/ot/TOOL/COMM!16.0.45/build_G!97.OP.L5/Csystem_build.log
> ===== Checking for errors in
> /cm/build12/cm/ot/TOOL/COMM!16.0.45/build_G!97.OP.L5/run_test_pr
> ograms.log
> ===== No errors found in
> /cm/build12/cm/ot/TOOL/COMM!16.0.45/build_G!97.OP.L5/run_test_pr
> ograms.log
> 
> ===== Checking for errors in
> /cm/build4/cm/ot/TOOL/XML!16.0.13/build_G!97.OP.L5/Csystem_build.log
> ===== No errors found in
> /cm/build4/cm/ot/TOOL/XML!16.0.13/build_G!97.OP.L5/Csystem_build.log
> 
> ===== Checking for errors in
> /cm/build6/cm/ot/ELD/DOM!16.0.20/build_G!97.OP.L5/Csystem_build.log
> ===== No errors found in
> /cm/build6/cm/ot/ELD/DOM!16.0.20/build_G!97.OP.L5/Csystem_build.log
> 
> ===== Checking for errors in
> /cm/build12/cm/ot/ELD/GEO_ENV!16.0.76/build_G!
> 97.OP.L5/Csystem_build.log
> contains (non-fatal) 'warnings' error messages:
> ===== No errors found in
> /cm/build12/cm/ot/ELD/GEO_ENV!16.0.76/build_G!
> 97.OP.L5/Csystem_build.log
> ===== Checking for errors in
> /cm/build12/cm/ot/ELD/GEO_ENV!16.0.76/build_G!97.OP.L5/run_test_
> programs.log
> ===== No errors found in
> /cm/build12/cm/ot/ELD/GEO_ENV!16.0.76/build_G!97.OP.L5/run_test_
> programs.log
> 
> ===== Checking for errors in
> /cm/build12/cm/ot/ELD/GEO_FLIGHT!16.0.82/build_G!97.OP.L5/Csyste
> m_build.log
> contains (non-fatal) 'warnings' error messages:
> ===== No errors found in
> /cm/build12/cm/ot/ELD/GEO_FLIGHT!16.0.82/build_G!97.OP.L5/Csyste
> m_build.log
> ===== Checking for errors in
> /cm/build12/cm/ot/ELD/GEO_FLIGHT!16.0.82/build_G!97.OP.L5/run_te
> st_programs.log
> ===== No errors found in
> /cm/build12/cm/ot/ELD/GEO_FLIGHT!16.0.82/build_G!97.OP.L5/run_te
> st_programs.log
> 
> ===== Checking for errors in
> /cm/build4/cm/ot/TOOL/GTK!16.0.14/build_G!97.OP.L5/Csystem_build.log
> ===== No errors found in
> /cm/build4/cm/ot/TOOL/GTK!16.0.14/build_G!97.OP.L5/Csystem_build.log
> 
> ===== Checking for errors in
> /cm/build12/cm/ot/TOOL/MOTIF!16.0.47/build_G!
> 97.OP.L5/Csystem_build.log
> contains (non-fatal) 'warnings' error messages:
> ===== No errors found in
> /cm/build12/cm/ot/TOOL/MOTIF!16.0.47/build_G!
> 97.OP.L5/Csystem_build.log
> ===== Checking for errors in
> /cm/build12/cm/ot/TOOL/MOTIF!16.0.47/build_G!97.OP.L5/run_test_p
> rograms.log
> ===== No errors found in
> /cm/build12/cm/ot/TOOL/MOTIF!16.0.47/build_G!97.OP.L5/run_test_p
> rograms.log
> 
> ===== Checking for errors in
> /cm/build12/cm/ot/ELD/GEO_MMI!16.0.84/build_G!
> 97.OP.L5/Csystem_build.log
> contains (non-fatal) 'warnings' error messages:
> ===== No errors found in
> /cm/build12/cm/ot/ELD/GEO_MMI!16.0.84/build_G!
> 97.OP.L5/Csystem_build.log
> ===== Checking for errors in
> /cm/build12/cm/ot/ELD/GEO_MMI!16.0.84/build_G!97.OP.L5/run_test_
> programs.log
> ===== No errors found in
> /cm/build12/cm/ot/ELD/GEO_MMI!16.0.84/build_G!97.OP.L5/run_test_
> programs.log
> 
> ===== Checking for errors in
> /cm/build6/cm/ot/TOOL/OPAPI!16.0.0/build_G!97.OP.L5/Csystem_build.log
> ===== No errors found in
> /cm/build6/cm/ot/TOOL/OPAPI!16.0.0/build_G!97.OP.L5/Csystem_build.log
> 
> ===== Checking for errors in
> /cm/build4/cm/ot/TOOL/PERL!16.0.1/build_G!97.OP.L5/Csystem_build.log
> contains (non-fatal) 'warnings' error messages:
> ===== No errors found in
> /cm/build4/cm/ot/TOOL/PERL!16.0.1/build_G!97.OP.L5/Csystem_build.log
> 
> ===== Checking for errors in
> /cm/build6/cm/ot/TOOL/PHYLOS!16.0.11/build_G!
> 97.OP.L5/Csystem_build.log
> ===== No errors found in
> /cm/build6/cm/ot/TOOL/PHYLOS!16.0.11/build_G!
> 97.OP.L5/Csystem_build.log
> 
> ===== Checking for errors in
> /cm/build12/cm/ot/TOOL/COMMON_UTILS!16.0.37/build_G!97.OP.L5/Csy
> stem_build.log
> ===== No errors found in
> /cm/build12/cm/ot/TOOL/COMMON_UTILS!16.0.37/build_G!97.OP.L5/Csy
> stem_build.log
> ===== Checking for errors in
> /cm/build12/cm/ot/TOOL/COMMON_UTILS!16.0.37/build_G!97.OP.L5/run
> _test_programs.log
> ===== No errors found in
> /cm/build12/cm/ot/TOOL/COMMON_UTILS!16.0.37/build_G!97.OP.L5/run
> _test_programs.log
> 
> ===== Checking for errors in
> /cm/build12/cm/ot/TOOL/COMMON_CONFIG!16.0.61/build_G!97.OP.L5/Cs
> ystem_build.log
> 111108.064035: Error: Check glan4 in the host_definition table
> because it is an invalid host
> Error: MMI LOCATION NOT DEFINED AS A PACKAGE TAC_VP1 tacot11  
> mmivp1      ptacvp1         TACOT
> Csystem_build.log contains error messages.
> 111108.064035: Error: Check glan4 in the host_definition table
> because it is an invalid host
> Csystem_build.log contains error messages.
> ==========================================
> ===== Checking for errors in
> /cm/build12/cm/ot/TOOL/COMMON_CONFIG!16.0.61/build_G!97.OP.L5/ru
> n_test_programs.log
> ===== No errors found in
> /cm/build12/cm/ot/TOOL/COMMON_CONFIG!16.0.61/build_G!97.OP.L5/ru
> n_test_programs.log
> 
> ===== Checking for errors in
> /cm/build1/cm/ot/GLANCE/GLANCE_CONFIG!16.0.93/build_G!97.OP.L5/C
> system_build.log
> ===== No errors found in
> /cm/build1/cm/ot/GLANCE/GLANCE_CONFIG!16.0.93/build_G!97.OP.L5/C
> system_build.log
> 
> ===== Checking for errors in
> /cm/build12/cm/ot/ELD/ELD_CONFIG!16.0.93/build_G!97.OP.L5/Csyste
> m_build.log
> ===== No errors found in
> /cm/build12/cm/ot/ELD/ELD_CONFIG!16.0.93/build_G!97.OP.L5/Csyste
> m_build.log
> ===== Checking for errors in
> /cm/build12/cm/ot/ELD/ELD_CONFIG!16.0.93/build_G!97.OP.L5/run_te
> st_programs.log
> Build missing: .finished_test_programs flag
> ==========================================
> 
> ===== Checking for errors in
> /cm/build8/cm/ot/TACT/CONFIG!16.0.19/build_G!
> 97.OP.L5/Csystem_build.log
> ===== No errors found in
> /cm/build8/cm/ot/TACT/CONFIG!16.0.19/build_G!
> 97.OP.L5/Csystem_build.log
> 
> ===== Checking for errors in
> /cm/build1/cm/ot/TACT/INSTALL!16.0.7/build_G!
> 97.OP.L5/Csystem_build.log
> ===== No errors found in
> /cm/build1/cm/ot/TACT/INSTALL!16.0.7/build_G!
> 97.OP.L5/Csystem_build.log
> 
> ===== Checking for errors in
> /cm/build8/cm/ot/TACT/SVISION!16.0.35/build_G!
> 97.OP.L5/Csystem_build.log
> ===== No errors found in
> /cm/build8/cm/ot/TACT/SVISION!16.0.35/build_G!
> 97.OP.L5/Csystem_build.log
> 
> ===== Checking for errors in
> /cm/build12/cm/ot/TACT/BASIC!16.0.48/build_G!
> 97.OP.L5/Csystem_build.log
> ===== No errors found in
> /cm/build12/cm/ot/TACT/BASIC!16.0.48/build_G!
> 97.OP.L5/Csystem_build.log
> 
> ===== Checking for errors in
> /cm/build12/cm/ot/TACT/UNAS!16.0.43/build_G!97.OP.L5/Csystem_build.log
> contains (non-fatal) 'warnings' error messages:
> ===== No errors found in
> /cm/build12/cm/ot/TACT/UNAS!16.0.43/build_G!97.OP.L5/Csystem_build.log
> 
> ===== Checking for errors in
> /cm/build12/cm/ot/TACT/TACT_UNAS!16.0.79/build_G!97.OP.L5/Csyste
> m_build.log
> contains (non-fatal) 'warnings' error messages:
> ===== No errors found in
> /cm/build12/cm/ot/TACT/TACT_UNAS!16.0.79/build_G!97.OP.L5/Csyste
> m_build.log
> 
> ===== Checking for errors in
> /cm/build12/cm/ot/TACT/ENV!16.0.82/build_G!97.OP.L5/Csystem_build.log
> contains (non-fatal) 'warnings' error messages:
> ===== No errors found in
> /cm/build12/cm/ot/TACT/ENV!16.0.82/build_G!97.OP.L5/Csystem_build.log
> 
> ===== Checking for errors in
> /cm/build12/cm/ot/TACT/CORE!16.0.92/build_G!97.OP.L5/Csystem_build.log
> contains (non-fatal) 'warnings' error messages:
> ===== No errors found in
> /cm/build12/cm/ot/TACT/CORE!16.0.92/build_G!97.OP.L5/Csystem_build.log
> 
> ===== Checking for errors in
> /cm/build12/cm/ot/TACT/UIF!16.0.93/build_G!97.OP.L5/Csystem_build.log
> contains (non-fatal) 'warnings' error messages:
> ===== No errors found in
> /cm/build12/cm/ot/TACT/UIF!16.0.93/build_G!97.OP.L5/Csystem_build.log
> 
> ===== Checking for errors in
> /cm/build12/cm/ot/TACT/TACOT!16.0.93/build_G!
> 97.OP.L5/Csystem_build.log
> contains (non-fatal) 'warnings' error messages:
> ===== No errors found in
> /cm/build12/cm/ot/TACT/TACOT!16.0.93/build_G!
> 97.OP.L5/Csystem_build.log
> 
> ===== Checking for errors in
> /cm/build12/cm/ot/TACT/TACOT_SCR!16.0.80/build_G!97.OP.L5/Csyste
> m_build.log
> ===== No errors found in
> /cm/build12/cm/ot/TACT/TACOT_SCR!16.0.80/build_G!97.OP.L5/Csyste
> m_build.log
> 
> ===== Checking for errors in
> /cm/build8/cm/ot/TACT/TOOLS!16.0.0/build_G!97.OP.L5/Csystem_build.log
> ===== No errors found in
> /cm/build8/cm/ot/TACT/TOOLS!16.0.0/build_G!97.OP.L5/Csystem_build.log
> 
> ===== Checking for errors in
> /cm/build1/cm/ot/TACT/UNAS_SYS!16.0.43/build_G!97.OP.L5/Csystem_
> build.log
> contains (non-fatal) 'warnings' error messages:
> ===== No errors found in
> /cm/build1/cm/ot/TACT/UNAS_SYS!16.0.43/build_G!97.OP.L5/Csystem_
> build.log
> 
> ===== Checking for errors in
> /cm/build1/cm/ot/TACT/MASTER!16.0.94/build_G!
> 97.OP.L5/Csystem_build.log
> ===== No errors found in
> /cm/build1/cm/ot/TACT/MASTER!16.0.94/build_G!
> 97.OP.L5/Csystem_build.log
> 
> ===== Checking for errors in
> /cm/build1/cm/ot/TACT/TACT_CONFIG!16.0.100/build_G!97.OP.L5/Csys
> tem_build.log
> ===== No errors found in
> /cm/build1/cm/ot/TACT/TACT_CONFIG!16.0.100/build_G!97.OP.L5/Csys
> tem_build.log
> ===== Checking for errors in
> /cm/build1/cm/ot/TACT/TACT_CONFIG!16.0.100/build_G!97.OP.L5/run_
> test_programs.log
> error: sequence TACT_ID_SEQ not correct
> run_test_programs.log contains error messages.
> contains (non-fatal) 'warnings' error messages:
> ORA-00942: table or view does not exist
> ORA-00942: table or view does not exist
> ORA-00942: table or view does not exist
> ORA-00942: table or view does not exist
> ORA-00942: table or view does not exist
> ORA-00942: table or view does not exist
> ORA-00942: table or view does not exist
> ORA-00942: table or view does not exist
> run_test_programs.log contains 'does not exist' error messages
> perform:32: command not found:
> OTHER_UNIT_REACHABLE_AND_CCAMS_ACTIVE=FALSE
> perform:32: command not found:
> OTHER_UNIT_REACHABLE_AND_CCAMS_ACTIVE=FALSE
> perform:32: command not found:
> OTHER_UNIT_REACHABLE_AND_CCAMS_ACTIVE=FALSE
> run_test_programs.log contains 'command not found' messages.
> Build missing: .finished_test_programs flag
> ==========================================
> 
> 

Re: zsh-4.2.6-5.el5 rhel5.5 accesses uninitialized memory in an assignment statement using a variable name of 31 or more characters.

[Peter Stephenson]

On Fri, 2 Dec 2011 22:54:34 +0100
"VAN VLIERBERGHE Stef" <stef.van-vlierberghe@eurocontrol.int> wrote:
> A more conservative workaround would be to only set the memory
> extension to zero immediately after the hrealloc call, getting
> something like a hrecalloc effect, this is much less likely to trigger
> other side-effects :
> 
> After:
> 	bptr = len + (tokstr = (char *)hrealloc(tokstr, bsiz, newbsiz));
> Add:
>       memset (bptr, 0, newbsiz - bsiz); /* len == bsiz, bptr points at first re-allocated byte, newbsiz - bsiz is size added */

This looks a safe and robust fix, and isn't obviously inefficient given
that the original allocation is doing the equivalent clearing of memory.
I'll apply the following patch to the current head (post 4.3.13) (it
also deletes some cruft there's no need to propagate).

Thanks.

Index: Src/lex.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/lex.c,v
retrieving revision 1.68
diff -p -u -r1.68 lex.c
--- Src/lex.c	15 Sep 2011 14:04:51 -0000	1.68
+++ Src/lex.c	3 Dec 2011 17:09:58 -0000
@@ -567,22 +567,14 @@ add(int c)
 {
     *bptr++ = c;
     if (bsiz == ++len) {
-#if 0
-	int newbsiz;
-
-	newbsiz = bsiz * 8;
-	while (newbsiz < inbufct)
-	    newbsiz *= 2;
-	bptr = len + (tokstr = (char *)hrealloc(tokstr, bsiz, newbsiz));
-	bsiz = newbsiz;
-#endif
-
 	int newbsiz = bsiz * 2;
 
 	if (newbsiz > inbufct && inbufct > bsiz)
 	    newbsiz = inbufct;
 
 	bptr = len + (tokstr = (char *)hrealloc(tokstr, bsiz, newbsiz));
+	/* len == bsiz, so bptr is at the start of newly allocated memory */
+	memset(bptr, 0, newbsiz - bsiz);
 	bsiz = newbsiz;
     }
 }


-- 
Peter Stephenson <p.w.stephenson@ntlworld.com>
Web page now at http://homepage.ntlworld.com/p.w.stephenson/

Re: uninitialized memory using a variable name of 31 or more characters

[Bart Schaefer]

[Starting a new thread per Geoff's suggestion.]

On Dec 2, 10:54pm, VAN VLIERBERGHE Stef wrote:
} Subject: zsh-4.2.6-5.el5 rhel5.5 accesses uninitialized memory in an assig
}
} A week ago I identified the problem [attached mail: lex.c add() extends tokstr=calloc() by a non-zeroing hrealloc].
} 
} The bug is (rarely) triggered by : AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=""

> ...
> A simple solution is to set *bptr=0 at the end of function add, but I am not sure
> this has no other consequences, to be checked with zsh developers.

It's amazing to me that RedHat made this change without discovering that
it causes other/worse problems.  Zsh's own test suite fails if that change
is made:

Test/A01grammar.ztst: starting.
ZTST_getsect:14: invalid subscript
[repeat for all other tests]

One problem, I suppose, is that this inability to run the tests doesn't end
up causing the suite itself to report a failure:

**************************************
41 successful test scripts, 0 failures, 0 skipped
**************************************

} After:
} 	bptr = len + (tokstr = (char *)hrealloc(tokstr, bsiz, newbsiz));
} Add:
}       memset (bptr, 0, newbsiz - bsiz); /* len == bsiz, bptr points at first re-allocated byte, newbsiz - bsiz is size added */

This seems to work fine, the full test suite runs and passes.

Index: Src/lex.c
--- ../zsh-forge/current/Src/lex.c      2011-09-19 08:26:12.000000000 -0700
+++ ./Src/lex.c 2011-12-03 08:59:39.000000000 -0800
@@ -583,6 +583,7 @@
            newbsiz = inbufct;
 
        bptr = len + (tokstr = (char *)hrealloc(tokstr, bsiz, newbsiz));
+       memset(bptr, 0, newbsiz - bsiz);  /* tokstr came from calloc() */
        bsiz = newbsiz;
     }
 }

RE: uninitialized memory using a variable name of 31 or more characters

[VAN VLIERBERGHE Stef]

Thank you both.

Initially I thought it was safer to use the *bptr=0 fix because I was
not sure that the tokstr
buffer was always freshly allocated, if somewhere bptr is set back to
the beginning (tokstr)
then the problem would still exist (but be triggered even more rarely).

If you are sure that bptr can only advance, then it seems to me that you
can also remove
the temporary zero byte insertion further down in lex.c :

		    int sav = *bptr;
		    *bptr = '\0';
		    t = itype_end(t, IIDENT, 0);
		    if (t < bptr) {
			skipparens(Inbrack, Outbrack, &t);
		    } else {
			*bptr = sav;
		    }

I already asked Bart if RedHat would be willing to put in the valgrind
instrumentation needed
to catch errors like these systematically, and he was wondering if you
would agree to such a
change. I believe it would be a good idea, but I understand it would be
silly for RedHat to
spend the effort if you would not accept it.

Bart also confirmed that RedHat did run the test suite, so I guess they
expected the suite to
report the error.

Best regards.

-----Original Message-----
From: Bart Schaefer [mailto:schaefer@brasslantern.com]
Sent: Saturday 3 December 2011 22:22
To: zsh-workers@zsh.org
Cc: Godts, Jeroen; Bart van den Heuvel; Genot, Harry; VAN
VLIERBERGHE Stef
Subject: Re: uninitialized memory using a variable name of 31
or more characters

[Starting a new thread per Geoff's suggestion.]

On Dec 2, 10:54pm, VAN VLIERBERGHE Stef wrote:
} Subject: zsh-4.2.6-5.el5 rhel5.5 accesses uninitialized
memory in an assig
}
} A week ago I identified the problem [attached mail: lex.c
add() extends tokstr=calloc() by a non-zeroing hrealloc].
}
} The bug is (rarely) triggered by : AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=""

> ...
> A simple solution is to set *bptr=0 at the end of function
add, but I am not sure
> this has no other consequences, to be checked with zsh developers.

It's amazing to me that RedHat made this change without discovering that
it causes other/worse problems.  Zsh's own test suite fails if
that change
is made:

Test/A01grammar.ztst: starting.
ZTST_getsect:14: invalid subscript
[repeat for all other tests]

One problem, I suppose, is that this inability to run the tests
doesn't end
up causing the suite itself to report a failure:

**************************************
41 successful test scripts, 0 failures, 0 skipped
**************************************

} After:
} 	bptr = len + (tokstr = (char *)hrealloc(tokstr, bsiz, newbsiz));
} Add:
}       memset (bptr, 0, newbsiz - bsiz); /* len == bsiz, bptr
points at first re-allocated byte, newbsiz - bsiz is size added */

This seems to work fine, the full test suite runs and passes.

Index: Src/lex.c
--- ../zsh-forge/current/Src/lex.c      2011-09-19
08:26:12.000000000 -0700
+++ ./Src/lex.c 2011-12-03 08:59:39.000000000 -0800
@@ -583,6 +583,7 @@
            newbsiz = inbufct;

        bptr = len + (tokstr = (char *)hrealloc(tokstr, bsiz, newbsiz));
+       memset(bptr, 0, newbsiz - bsiz);  /* tokstr came from
calloc() */
        bsiz = newbsiz;
     }
 }

____
 
This message and any files transmitted with it are legally privileged and intended for the sole use of the individual(s) or entity to whom they are addressed. If you are not the intended recipient, please notify the sender by reply and delete the message and any attachments from your system. Any unauthorised use or disclosure of the content of this message is strictly prohibited and may be unlawful.
 
Nothing in this e-mail message amounts to a contractual or legal commitment on the part of EUROCONTROL, unless it is confirmed by appropriately signed hard copy.
 
Any views expressed in this message are those of the sender.

Re: uninitialized memory using a variable name of 31 or more characters

[Peter Stephenson]

On Mon, 5 Dec 2011 21:17:36 +0100
VAN VLIERBERGHE Stef <stef.van-vlierberghe@eurocontrol.int> wrote:
> I already asked Bart if RedHat would be willing to put in the valgrind
> instrumentation needed to catch errors like these systematically, and
> he was wondering if you would agree to such a change. I believe it
> would be a good idea, but I understand it would be silly for RedHat to
> spend the effort if you would not accept it.

That would be fine, with suitable #ifdef's.  I run it under valgrind
myself occasionally.

-- 
Peter Stephenson <pws@csr.com>            Software Engineer
Tel: +44 (0)1223 692070                   Cambridge Silicon Radio Limited
Churchill House, Cambridge Business Park, Cowley Road, Cambridge, CB4 0WZ, UK


Member of the CSR plc group of companies. CSR plc registered in England and Wales, registered number 4187346, registered office Churchill House, Cambridge Business Park, Cowley Road, Cambridge, CB4 0WZ, United Kingdom
More information can be found at www.csr.com. Follow CSR on Twitter at http://twitter.com/CSR_PLC and read our blog at www.csr.com/blog