Re: Strange coredump in new zsh-3.0.1 on Sunos4.1.3 (FIXED)

[Zoltan Hidvegi <hzoli@cs.elte.hu>]

> On Nov 3,  4:37pm, C. v. Stuckrad wrote:
> } Subject: Re: Strange coredump in new zsh-3.0.1 on Sunos4.1.3 (FIXED)
> }
> } OK, setting OPEN_MAX to 64 (and applying Zoltan's fix too)
> } did get rid of the bug. 
> 
> I'd be curious to know if you tried the two fixes separately.  I don't
> see how decrementing max_zsh_fd below zero in that loop could possibly
> change it to 16777234 ( == (1<<24)+18, which is a lot of decrements);
> but taking the bytes in 16777234 as an array of chars (char fdtable[])
> yields "\01\0\0\021" which looks suspiciously like what you'd get if
> you started with max_zsh_fd == 18 and stuffed 1 into the first byte.

I guess that the problem is that due to the incorrect OPEN_MAX value an
fdtable[OPEN_MAX] = 1 assignment happens somewhere.  max_zsh_fd comes right
after the fdtable array so this assignment overwrites the first byte of
max_zsh_fd.  On big endian machines like the sparc this causes exactly the
described problem.  My bugfix has nothing to do with that.  That bug I
fixed probably can never cause any real problems (it's been there for a
while now and noone noticed that) since max_zsh_fd decrements stop at the
first non-zero byte and max_zsh_fd is never used for writing.

Zoltan