BUG: doinsert() in NextStep/OpenStep 4.2

BUG: doinsert() in NextStep/OpenStep 4.2

[Brian Boonstra]

Hi folks

	I just did a compile of the bart7 patchlevel on an OpenStep/Intel  
version 4.2 machine.  Building on OpenStep requires specifying a different  
compiler than the ancient gcc they give you, so I used

  % CC=/bin/cc ./configure

edited the linker commands in the makefile, and then ran Src/zsh.   I typed  
"ls", got a listing, and then zsh hung.  I investigated this bug, and to the  
best of my limited ability, here is a characterization:

First, I rebuilt with debugging symbols:

  % CC=/bin/cc ./configure --enable-zsh-debug

then I ran zsh.


	In another window, I opened gdb, and attached.  I (c)ontinued, then  
went to the bart7 window, and typed the single letter "l".  This is what I  
got from gdb:

(gdb) c
Reading in symbols for zle_misc.c...done.
Program generated(1): Memory access exception on address 0x0 (protection failure).
0x8b63f in doinsert (str=0xbfffeea8 "l") at zle_misc.c:54
54                  line[cs++] = *s == Meta ? *++s ^ 32 : *s;


I then set a breakpoint so I could get a better idea of what is going on,  
and here is what I saw:

(gdb) att 17401
Dynamic Linkeditor at 0x12000000 offset 0x0
Executable at 0x2000 offset 0x0
/NextLibrary/Frameworks/System.framework/Versions/A/System at 0x5000000 offset 0x0
0x507a4c8 in _read ()
(gdb) b zle_misc.c:54
(gdb) c

Breakpoint 1, doinsert (str=0xbfffeea8 "l") at zle_misc.c:54
54                  line[cs++] = *s == Meta ? *++s ^ 32 : *s;
(gdb) l
49              spaceinline(m * len);
50          else if(cs + m * len > ll)
51              spaceinline(cs + m * len - ll);
52          while(m--)
53              for(s = str; *s; s++)
54                  line[cs++] = *s == Meta ? *++s ^ 32 : *s;
55          if(neg)
56              cs += zmult * len;
57      }
58
(gdb) p line
Reading in symbols for lex.c...done.
$1 = (unsigned char *) 0xd0f94 "L"
(gdb) p s
$2 = 0xbfffeea8 "l"
(gdb) p len
$3 = 1
(gdb) p Meta
No symbol "Meta" in current context.
(gdb) p c1
$4 = 108
(gdb) p m
$5 = 0
(gdb) p cs
No symbol "cs" in current context.
(gdb)


History of zsh version 3 on OpenStep 4.2 (as I know it)

3.0.x does not compile
3.1.x compiles only after some patches, but has completion bugs
bart7 compiles with almost no trouble (some compiler and linker problems)
	but sees this bug



	That's all I know to do for now.  Does anyone have further advice on  
debugging this?  I'm a decent programmer, but no expert on shells or zsh.


			- Brian

Re: BUG: doinsert() in NextStep/OpenStep 4.2

[Bart Schaefer]

On Oct 21, 10:31am, Brian Boonstra wrote:
} Subject: BUG: doinsert() in NextStep/OpenStep 4.2
}
} I typed "ls", got a listing, and then zsh hung.
} 
} First, I rebuilt with debugging symbols:
} 
}   % CC=/bin/cc ./configure --enable-zsh-debug
} 
} then I ran zsh.
} 
} 	In another window, I opened gdb, and attached.

So did you attach after zsh was hung (as in your first example)?  Or did
you attach first and then try to get zsh to hang?  And did it hang, or
did you get a crash of some kind?  I can't be sure from your gdb trace.

BTW, here's an update of my "gdbterm" script / autoloaded function,
which is specifically meant for running gdb on interactive programs like
shells and editors:

------
#! /bin/sh
xterm -title GDBterm -e sh -c 'exec xterm -e gdb --tty=`tty` '"$*"
------

} (gdb) p Meta
} No symbol "Meta" in current context.
} (gdb) p cs
} No symbol "cs" in current context.

"Meta" and "cs" are #define'd.  "Meta" is always 0x83, but "cs" expands
to "zshcs" everywhere except in the spot where "zshcs" is defined.  This
is a hack to get around some system libraries (curses?) that also define
a symbol "cs".

} Breakpoint 1, doinsert (str=0xbfffeea8 "l") at zle_misc.c:54
} 54                  line[cs++] = *s == Meta ? *++s ^ 32 : *s;
} (gdb) l
} 49              spaceinline(m * len);
} 50          else if(cs + m * len > ll)
} 51              spaceinline(cs + m * len - ll);
} 52          while(m--)
} 53              for(s = str; *s; s++)
} 54                  line[cs++] = *s == Meta ? *++s ^ 32 : *s;
} 55          if(neg)
} 56              cs += zmult * len;
} 57      }
} 58
} (gdb) p s
} $2 = 0xbfffeea8 "l"
} (gdb) p m
} $5 = 0

And if you continue from this point, what happens?

Since m == 0 and s[1] == 0 it looks as if it should exit the loop.  I
guess we need to see the value of "zshcs" (aka "cs") to see if it is -1
or some other bad index into line[].

} 3.0.x does not compile

Not even 3.0.7?  What's the failure, again?

-- 
Bart Schaefer                                 Brass Lantern Enterprises
http://www.well.com/user/barts              http://www.brasslantern.com

Re: BUG: doinsert() in NextStep/OpenStep 4.2

[Brian Boonstra]

Bart wrote:
> So did you attach after zsh was hung (as in your first example)?  Or did
> you attach first and then try to get zsh to hang?  And did it hang, or
> did you get a crash of some kind?  I can't be sure from your gdb trace.

Sorry if I was unclear.  Perhaps the following snippet will be more  
informative (I'm still trying to work out what is relevant).   I attached  
before any kind of hang, continued, then typed the letter "l", which from  
previous experience gave a memory exception at line 54:

(gdb) att 22701
Dynamic Linkeditor at 0x12000000 offset 0x0
Executable at 0x2000 offset 0x0
/NextLibrary/Frameworks/System.framework/Versions/A/System at 0x5000000 offset 0x0
0x507a4c8 in _read ()
(gdb) c

Breakpoint 1, doinsert (str=0xbfffeea8 "l") at zle_misc.c:54
54                  line[cs++] = *s == Meta ? *++s ^ 32 : *s;
(gdb) p s
$4 = 0xbfffeea8 "l"
(gdb) p m
$5 = 0
(gdb) p zshcs
$6 = 0
(gdb) p line
$7 = (unsigned char *) 0xd0fbc "|"
(gdb) p *s
$8 = 108 'l'
(gdb) s
Program generated(1): Memory access exception on address 0x0 (protection failure).
0x8b517 in doinsert (str=0xbfffeea8 "l") at zle_misc.c:54
54                  line[cs++] = *s == Meta ? *++s ^ 32 : *s;



I have to say that I can't see why this line is failing.  Since zshcs==0,  
and *s!=Meta, this should just be setting line[0]="l" with no problem.  Is  
there something unusual I misunderstand about attaching debuggers to shells?

Lacking xterm, I did not try to use your shell code

> xterm -title GDBterm -e sh -c 'exec xterm -e gdb --tty=`tty` '"$*"

especially because I don't think my gdb accepts a --tty argument.

> } 3.0.x does not compile
>
> Not even 3.0.7?  What's the failure, again?

I hadn't tested 3.0.7 before, actually.  I just took a look at it, and it  
does compile, but gives a different kind of problem.  Any command (even  
"exit") ends up in the mini-editor until you Ctrl-C out of it, like this:

wo1203 % ls                                                /Cache/zsh/zsh-3.0.7
>
>
wo1203 % exit                                              /Cache/zsh/zsh-3.0.7
>
>
wo1203 %                                                   /Cache/zsh/zsh-3.0.7



It doesn't crash, though...and gdb given no indication of errors.  For  
grins, I tried the same breakpoint as for bart7, but (and this is kind of  
interesting) I never hit it at all.


Incidentally, for this email, both builds were configured with


CC=/bin/cc ./configure --enable-zsh-debug --enable-zsh-mem-debug  
--enable-zsh-mem-warning --enable-zsh-secure-free --enable-zsh-hash-debug



I'm ready to check out anything else about what's going on here -- just let  
me know what to look at!


	Best Regards,

		Brian

Re: BUG: doinsert() in NextStep/OpenStep 4.2

[Bart Schaefer]

On Oct 21,  2:28pm, Brian Boonstra wrote:
> Subject: Re: BUG: doinsert() in NextStep/OpenStep 4.2
> 
> (gdb) c
> 
> Breakpoint 1, doinsert (str=0xbfffeea8 "l") at zle_misc.c:54
> 54                  line[cs++] = *s == Meta ? *++s ^ 32 : *s;
> (gdb) s
> Program generated(1): Memory access exception on address 0x0 (protection failure).
> 0x8b517 in doinsert (str=0xbfffeea8 "l") at zle_misc.c:54
> 54                  line[cs++] = *s == Meta ? *++s ^ 32 : *s;

That sure looks like the debugger thinks a null pointer deref happened.

> I have to say that I can't see why this line is failing.  Since zshcs==0,  
> and *s!=Meta, this should just be setting line[0]="l" with no problem.  Is  
> there something unusual I misunderstand about attaching debuggers to shells?

Not that I can see ... but it may be that the debugger is confused.  Do you
get a crash at that point when running without the debugger attached?

> Lacking xterm, I did not try to use your shell code
> especially because I don't think my gdb accepts a --tty argument.

You can also redirect the I/O at the (gdb) prompt like this:

(gdb) run < /dev/ttypq 2>&1 > /dev/ttypq

but then you don't get keyboard signals from the new tty, so you can't debug
things that require interrupts.

> I hadn't tested 3.0.7 before, actually.  I just took a look at it, and it  
> does compile, but gives a different kind of problem.  Any command (even  
> "exit") ends up in the mini-editor until you Ctrl-C out of it, like this:
> 
> wo1203 % ls                                                /Cache/zsh/zsh-3.0.7
> >
> >
> wo1203 % exit                                              /Cache/zsh/zsh-3.0.7
> >
> >
> wo1203 %                                                   /Cache/zsh/zsh-3.0.7

That indicates that your RETURN key is not sending the character that zsh
has bound to the ZLE accept-line action.  Zsh is supposed to map both CR
and LF to the same thing on input so that this does not happen.  What if
you type Ctrl-J or Ctrl-M explicitly?

If one of those works, what is the output you get from "stty -a" ?

It may be that zsh is selecting the wrong terminal-manipulation package.
grep TERM config.h	might tell something.

You should try running both bart-7 and 3.0.7 as "zsh +Z" to turn off the
line editor, to see if the problems are related to terminal handling.

Re: BUG: doinsert() in NextStep/OpenStep 4.2

[Brian Boonstra]

You wrote:
> > I have to say that I can't see why this line is failing. Since zshcs==0,
> > and *s!=Meta, this should just be setting line[0]="l" with no problem. Is
> > there something unusual I misunderstand about attaching debuggers to
> > shells?
>
> Not that I can see ... but it may be that the debugger is confused.  Do you
> get a crash at that point when running without the debugger attached?

	Yes, I did.  But only on the config with --enable-zsh-debug.  When I  
config normally, I get a hang after a command has finished, like this:

wo1203 % Src/zsh +Z
wo1203 % ls
ChangeLog       INSTALL         StartupFiles/   config.h        install-sh*
ChangeLog.3.0   META-FAQ        Util/           config.h.in     mkinstalldirs*
Completion/     Makefile        acconfig.h      config.log      stamp-h
Config/         Makefile.in     aclocal.m4      config.status*  stamp-h.in
Doc/            Misc/           aczsh.m4        config.sub*
Etc/            README          config.cache    configure*
Functions/      Src/            config.guess*   configure.in

** hung here (killed from another terminal window)

> > [problem with remaining in mini-editor in 3.0.7]

>  That indicates that your RETURN key is not sending the
>  character that zsh has bound to the ZLE accept-line
>  action.  Zsh is supposed to map both CR and LF to the
>  same thing on input so that this does not happen.  What
>  if you type Ctrl-J or Ctrl-M explicitly?

It still stays in the mini-editor -- never getting the accept-line, I guess.  
 Is it possible zsh isn't binding any such character?


>  If one of those works, what is the output you get from
>  "stty -a" ?

Neither worked, but I ran "stty all" in each.  For both 3.0.7 and bart-7 I got:

wo1203 % stty all                            /Cache/zsh/zsh-3.1.6-bart-7
new tty, speed 38400 baud, 55 rows, 94 columns; -tabs
crt pass8
pass8out
erase  kill   werase rprnt  flush  lnext  susp   intr   quit   stop   eof
^?     ^U     ^W     ^R     ^O     ^V     ^Z/^Y  ^C     ^\     ^S/^Q  ^D     



>  It may be that zsh is selecting the wrong
>  terminal-manipulation package.  grep TERM config.h
>  might tell something.

wo1203 % grep TERM config.h            /Cache/zsh/zsh-3.1.6-bart-7
/* #undef HAVE_TERMCAP_H */
/* #undef HAVE_TERMIO_H */
#define HAVE_TERMIOS_H 1

	I took a look at that header, and there was some NeXT-specific stuff  
in it (according to whether _NEXT_SOURCE is set) that--for all I know--is  
causing the problem.  I've included it below.  I tried a new build of bart-7  
having added -D_NEXT_SOURCE to the compiler flags, but got the same hanging  
behavior:

wo1203 % ls                             /Cache/zsh/zsh-3.1.6-bart-7
ChangeLog       INSTALL         StartupFiles/   config.h        install-sh*
ChangeLog.3.0   META-FAQ        Util/           config.h.in     mkinstalldirs*
Completion/     Makefile        acconfig.h      config.log      stamp-h
Config/         Makefile.in     aclocal.m4      config.status*  stamp-h.in
Doc/            Misc/           aczsh.m4        config.sub*
Etc/            README          config.cache    configure*
Functions/      Src/            config.guess*   configure.in

** (hung here)


>  You should try running both bart-7 and 3.0.7 as "zsh +Z"
>  to turn off the line editor, to see if the problems are
>  related to terminal handling.

	They must be, at least in the case of 3.0.7.  When I ran 3.0.7 as  
"zsh +Z", it seemed to work just fine.   Nice deductive work, there!

	On the other hand, running bart-7 this way did not solve the hanging  
problem.  I would like to check a backtrace in gdb to see where it is  
hanging, but of course the debug build exhibits that immediate crash on the  
first character typed, rather than the hang after the command finishes  
executing.  Any other reasonable avenues of investigation?


             Best Regards,


		Brian




---------------  /usr/include/bsd/sys/termios.h ---------------------


/*
 *  termios structure
 */
#ifndef _TERMIOS_H_
#define _TERMIOS_H_

/*
 * Special Control Characters
 *
 * Index into c_cc[] character array.
 *
 *	Name	     Subscript	Enabled by
 */
#define	VEOF		0				/* ICANON */
#define	VEOL		1				/* ICANON */
#define	VERASE		2				/* ICANON */
#define VKILL		3				/* ICANON */
#define VINTR		4				/* ISIG */
#define VQUIT		5				/* ISIG */
#define VSUSP		6				/* ISIG */
#define VSTART		7				/* IXON, IXOFF */
#define VSTOP		8				/* IXON, IXOFF */
#define VMIN		9				/* !ICANON */
#define VTIME		10				/* !ICANON */

#if defined(_NEXT_SOURCE)
#define VWERASE 	11				/* ICANON */
#define VREPRINT 	12				/* ICANON */
#define VLNEXT		13				/* IEXTEN */
#define VDISCARD	14				/* IEXTEN */
#define VDSUSP		15				/* ISIG */
#define VQUOTE		16				/* ICANON */
#endif 	/* _NEXT_SOURCE */

#define	NCCS		17

/*
 * Input flags - software input processing
 */
#define	IGNBRK		0x00000001	/* ignore BREAK condition */
#define	BRKINT		0x00000002	/* map BREAK to SIGINTR */
#define	IGNPAR		0x00000004	/* ignore (discard) parity errors */
#define	PARMRK		0x00000008	/* mark parity and framing errors */
#define	INPCK		0x00000010	/* enable checking of parity errors */
#define	ISTRIP		0x00000020	/* strip 8th bit off chars */
#define	INLCR		0x00000040	/* map NL into CR */
#define	IGNCR		0x00000080	/* ignore CR */
#define	ICRNL		0x00000100	/* map CR to NL (ala CRMOD) */
#define	IXON		0x00000200	/* enable output flow control */
#define	IXOFF		0x00000400	/* enable input flow control */

#if defined(_NEXT_SOURCE)
#define	IXANY		0x00000800	/* any char will restart after stop */
#define IMAXBEL		0x00002000	/* ring bell on input queue full */
#endif	 /* _NEXT_SOURCE */

/*
 * Output flags - software output processing
 */
#define	OPOST		0x00000001	/* enable following output processing */
#if defined(_NEXT_SOURCE)
#define ONLCR		0x00000002	/* map NL to CR-NL (ala CRMOD) */
	/* use the same bits as old delay flags */
#define	NLDELAY		0x00000300	/* \n delay */
#define		NL0	0x00000000
#define		NL1	0x00000100	/* tty 37 */
#define		NL2	0x00000200	/* vt05 */
#define		NL3	0x00000300
#define	TBDELAY		0x00000c00	/* horizontal tab delay */
#define		TAB0	0x00000000
#define		TAB1	0x00000400	/* tty 37 */
#define		TAB2	0x00000800
#define	XTABS		0x00000c00	/* expand tabs on output */
#define	CRDELAY		0x00003000	/* \r delay */
#define		CR0	0x00000000
#define		CR1	0x00001000	/* tn 300 */
#define		CR2	0x00002000	/* tty 37 */
#define		CR3	0x00003000	/* concept 100 */
#define	VTDELAY		0x00004000	/* vertical tab delay */
#define		FF0	0x00000000
#define		FF1	0x00004000	/* tty 37 */
#define	BSDELAY		0x00008000	/* \b delay */
#define		BS0	0x00000000
#define		BS1	0x00008000
#define	ALLDELAY	(NLDELAY|TBDELAY|CRDELAY|VTDELAY|BSDELAY)
#endif	/* _NEXT_SOURCE */

/*
 * Control flags - hardware control of terminal
 */
#if defined(_NEXT_SOURCE)
#define	CIGNORE		0x00000001	/* ignore control flags */
#endif	/* _NEXT_SOURCE */

#define CSIZE		0x00000300	/* character size mask */
#define     CS5		    0x00000000	    /* 5 bits (pseudo) */
#define     CS6		    0x00000100	    /* 6 bits */
#define     CS7		    0x00000200	    /* 7 bits */
#define     CS8		    0x00000300	    /* 8 bits */
#define CSTOPB		0x00000400	/* send 2 stop bits */
#define CREAD		0x00000800	/* enable receiver */
#define PARENB		0x00001000	/* parity enable */
#define PARODD		0x00002000	/* odd parity, else even */
#define HUPCL		0x00004000	/* hang up on last close */
#define CLOCAL		0x00008000	/* ignore modem status lines */

#if defined(_NEXT_SOURCE)
#define CSTOPB110	0x00010000
#define PAR0		0x00020000	/* Space parity */
#define PAR1		0x00040000	/* Mark parity */
#endif	/* _NEXT_SOURCE */


/*
 * "Local" flags - dumping ground for other state
 *
 * Warning: some flags in this structure begin with
 * the letter "I" and look like they belong in the
 * input flag.
 */

#if defined(_NEXT_SOURCE)
#define	ECHOKE		0x00000001	/* visual erase for line kill */
#endif	/* _NEXT_SOURCE */

#define	ECHOE		0x00000002	/* visually erase chars */
#define	ECHOK		0x00000004	/* echo NL after line kill */
#define ECHO		0x00000008	/* enable echoing */
#define	ECHONL		0x00000010	/* echo NL even if ECHO is off */
#define	ICANON		0x00000020	/* canonicalize input lines */
#define	ISIG		0x00000040	/* enable signals INTR, QUIT, [D]SUSP */
#define	IEXTEN		0x00000080	/* enable DISCARD and LNEXT */

#if defined(_NEXT_SOURCE)
#define	ECHOCRT		0x00000100	/* visual erase mode for crt */
#define	ECHOPRT		0x00000200	/* visual erase mode for hardcopy */
#define ECHOCTL  	0x00000400	/* echo control chars as ^(Char) */
#define ALTWERASE	0x00000800	/* use alternate WERASE algorithm */
#define	MDMBUF		0x00100000	/* flow control output via Carrier */
#endif	/* _NEXT_SOURCE */

#define TOSTOP		0x00400000	/* stop background jobs from output */

#if defined(_NEXT_SOURCE)
#define XLCASE		0x04000000
#define XEUCBKSP	0x08000000
#endif	/* _NEXT_SOURCE */

#define	NOFLSH		0x80000000	/* don't flush after interrupt */

typedef unsigned long	tcflag_t;
typedef unsigned char	cc_t;
typedef char		speed_t;

struct termios {
	tcflag_t	c_iflag;	/* input flags */
	tcflag_t	c_oflag;	/* output flags */
	tcflag_t	c_cflag;	/* control flags */
	tcflag_t	c_lflag;	/* local flags */
	cc_t		c_cc[NCCS];	/* control chars */
	speed_t		c_ispeed;	/* input speed */
	speed_t		c_ospeed;	/* output speed */
};

/*
 * Commands passed to tcsetattr() for setting the termios structure.
 */
#define	TCSANOW		0		/* make change immediate */
#define	TCSADRAIN	1		/* drain output, then change */
#define	TCSAFLUSH	2		/* drain output, flush input */

#if defined(_NEXT_SOURCE)
#define TCSASOFT	0x10		/* flag - don't alter h.w. state */
#endif	/* _NEXT_SOURCE */

/*
 * Standard speeds
 */
#import <sys/ttydev.h>

#ifndef KERNEL

#ifdef __STDC__
speed_t	cfgetispeed(const struct termios *);
speed_t	cfgetospeed(const struct termios *);
int	cfsetispeed(struct termios *, int);
int	cfsetospeed(struct termios *, int);
int	tcdrain(int);
int	tcflow(int, int);
int	tcflush(int, int);
int	tcgetattr(int, struct termios *);
int	tcsendbreak(int, int);
int	tcsetattr(int, int, const struct termios *);

#if defined(_NEXT_SOURCE)
void	cfmakeraw(struct termios *);
void	cfsetspeed(struct termios *, int);
#endif	/* _NEXT_SOURCE */
#endif /* __STDC__ */

#define	TCIFLUSH	1
#define	TCOFLUSH	2
#define TCIOFLUSH	3
#define	TCOOFF		1
#define	TCOON		2
#define TCIOFF		3
#define TCION		4

#endif /* !KERNEL */

/*
 * END OF PROTECTED INCLUDE.
 */
#endif /* !_TERMIOS_H_ */