Re: Zsh 3.1.6 still hangs on for loops utilizing lots of stdout

Re: Zsh 3.1.6 still hangs on for loops utilizing lots of stdout

[Sven Wischnowsky]

Clint Olsen wrote:

> I went ahead and ran with set -x.  It is failing in the middle of one of
> the cats (no meow):
> 
> ...
> +/afs/pdx/proj/otools/bin/AIX/zsh:8> cat x_0m0fm00pm_d_m4_250.char_summary
> +/afs/pdx/proj/otools/bin/AIX/zsh:6> read file
> +/afs/pdx/proj/otools/bin/AIX/zsh:8> cat x_0m0fm00pm_d_m4_350.char_summary
> +/afs/pdx/proj/otools/bin/AIX/zsh:6> read file
> +/afs/pdx/proj/otools/bin/AIX/zsh:8> cat x_0m0fm01p1_d_m2_150.char_summary
> +/afs/pdx/proj/otools/bin/AIX/zsh:6> read file
> +/afs/pdx/proj/otools/bin/AIX/zsh:8> cat x_0m0fm01p1_d_m4_250.char_summary
> 
> The shell hangs.  ^Z nor ^C cause it to return a prompt.

I presume that a simple cat without a loop on this files works...

Could you tell us the how-many'th (heck, how does one say `wievielte'
in English?) cat this is? The 24'th or 48'th, perchance?

And, to repeat Bart's question: in which states are the cat and zsh?
(I.e. what does ps say, with as much information as possible, please,
even signal masks if your ps can give them).

Bye
 Sven

P.S.: And it's a slightly different loop, right?

--
Sven Wischnowsky                         wischnow@informatik.hu-berlin.de

Re: Zsh 3.1.6 still hangs on for loops utilizing lots of stdout

[Bart Schaefer]

On Feb 17, 10:20am, Sven Wischnowsky wrote:
} Subject: Re: Zsh 3.1.6 still hangs on for loops utilizing lots of stdout
}
} Could you tell us the how-many'th (heck, how does one say `wievielte'
} in English?) cat this is? The 24'th or 48'th, perchance?

You can't say "wievielte" in English, or not any better than you did.

} And, to repeat Bart's question: in which states are the cat and zsh?

Clint showed me "ps" output in private mail that indicates that the cat
processes have all exited (so it isn't "in the middle of a cat", I guess)
and that the OS believes zsh to be in a running state.

-- 
Bart Schaefer                                 Brass Lantern Enterprises
http://www.well.com/user/barts              http://www.brasslantern.com

Re: Zsh 3.1.6 still hangs on for loops utilizing lots of stdout

[Sven Wischnowsky]

Bart Schaefer wrote:

> On Feb 17, 10:20am, Sven Wischnowsky wrote:
> } Subject: Re: Zsh 3.1.6 still hangs on for loops utilizing lots of stdout
> }
> } Could you tell us the how-many'th (heck, how does one say `wievielte'
> } in English?) cat this is? The 24'th or 48'th, perchance?
> 
> You can't say "wievielte" in English, or not any better than you did.
> 
> } And, to repeat Bart's question: in which states are the cat and zsh?
> 
> Clint showed me "ps" output in private mail that indicates that the cat
> processes have all exited (so it isn't "in the middle of a cat", I guess)
> and that the OS believes zsh to be in a running state.

So it's a different problem than the one reported by Alexandre.

The obvious question: in which function(s) is the zsh at that time?

Bye
 Sven


--
Sven Wischnowsky                         wischnow@informatik.hu-berlin.de

Re: Zsh 3.1.6 still hangs on for loops utilizing lots of stdout

[Clint Olsen]

I went ahead and ran with set -x.  It is failing in the middle of one of
the cats (no meow):

...
+/afs/pdx/proj/otools/bin/AIX/zsh:8> cat x_0m0fm00pm_d_m4_250.char_summary
+/afs/pdx/proj/otools/bin/AIX/zsh:6> read file
+/afs/pdx/proj/otools/bin/AIX/zsh:8> cat x_0m0fm00pm_d_m4_350.char_summary
+/afs/pdx/proj/otools/bin/AIX/zsh:6> read file
+/afs/pdx/proj/otools/bin/AIX/zsh:8> cat x_0m0fm01p1_d_m2_150.char_summary
+/afs/pdx/proj/otools/bin/AIX/zsh:6> read file
+/afs/pdx/proj/otools/bin/AIX/zsh:8> cat x_0m0fm01p1_d_m4_250.char_summary

The shell hangs.  ^Z nor ^C cause it to return a prompt.

-Clint

On Feb 03, Bart Schaefer wrote:
> On Feb 3, 10:45am, Clint Olsen wrote:
> > Subject: Zsh 3.1.6 still hangs on for loops utilizing lots of stdout
> >
> > for file in *.something
> > do
> >   cat $file
> > done > /tmp/concat
> 
> Have you tried doing this with "set -x" (aka "setopt xtrace") in effect?
> You should then get one line written to standard error for each "cat"
> that executes.  This may help narrow down where the failure is.
> 
> You're certain that none of the *.something files is a named pipe or
> other device special file?  What does "ps" say the state of the process
> is at the time that it hangs?  (I.e., you're sure the process isn't in
> state D, disk wait?)

Re: Zsh 3.1.6 still hangs on for loops utilizing lots of stdout

[Sven Wischnowsky]

[ No idea about the main problem yet, but... ]

Alexandre Duret-Lutz wrote:

> ...
> /goinfre/swiss-2 % for i in *.wav                                    Err 139 #6
> do
>   8hz-mp3 $i $i:r.mp3
> done | od
> 
> [...lot's of output...]
> 
> zsh: exit 141            for i in *.wav; do; 8hz-mp3 $i $i:r.mp3; done | od | 
> zsh: segmentation fault  od                                               
> 
> How fun! now, that's od which is segfaulting. BTW where does this last pipe
> (after `od' on the exit report line) come from?.

At least this can be fixed. Thinko in execpline2().

Bye
 Sven

diff -ru ../z.old/Src/exec.c Src/exec.c
--- ../z.old/Src/exec.c	Fri Feb  4 16:07:26 2000
+++ Src/exec.c	Mon Feb  7 11:11:29 2000
@@ -1112,7 +1112,10 @@
 
     if (pline_level == 1) {
 	if ((how & Z_ASYNC) || (!sfcontext && !sourcelevel))
-	    strcpy(list_pipe_text, getjobtext(state->prog, state->pc - 1));
+	    strcpy(list_pipe_text,
+		   getjobtext(state->prog,
+			      state->pc + (WC_PIPE_TYPE(pcode) == WC_PIPE_END ?
+					   0 : 1)));
 	else
 	    list_pipe_text[0] = '\0';
     }
@@ -1142,7 +1145,7 @@
 	    } else if (pid) {
 		char dummy, *text;
 
-		text = getjobtext(state->prog, state->pc - 2);
+		text = getjobtext(state->prog, state->pc);
 		addproc(pid, text);
 		close(synch[1]);
 		read(synch[0], &dummy, 1);

--
Sven Wischnowsky                         wischnow@informatik.hu-berlin.de

Re: Zsh 3.1.6 still hangs on for loops utilizing lots of stdout

[Alexandre Duret-Lutz]

>>> "Sven" == Sven Wischnowsky <wischnow@informatik.hu-berlin.de> writes:

 Sven> Alexandre Duret-Lutz wrote:

[...]

 >> for i in *.wav
 >> do
 >> 8hz-mp3 $i $i:r.mp3
 >> done

[...]

 Sven> Well, we had some problems with IO and AIX but this stack trace looks
 Sven> completely fine -- saying only that it's waiting for the 8hz-mp3.

Indeed.  In fact zsh was still running and the loop finished: 8hz's work
was done on each file. 

 Sven> In both cases: does it happen with other shells, too?

Yes!  Maybe there is something wrong,  

In the two cases below, I run the loop in the launched rxvt.  
And that rxvt which is segfaulting.

/goinfre/swiss-1 % gdbterm zsh                                        11:27 #14
/home/prolob/pollux/mbin/gdbterm: line 2:  6751 Segmentation fault      rxvt +ls +sb -title GDBterm -e sh -c 'exec rxvt +ls +sb -e gdb --tty=`tty` '"$*"

/goinfre/swiss-1 % rxvt -e bash                                     Err 139 #15
zsh: segmentation fault  rxvt -e bash

with another terminal emulator (xterm), this behave differently: 
after a few iterations in the for loop, it's like something is going mad
and print newlines in loop.  I tried to run the whole loop
through `od' to see if these where realy newlines or something else :

/goinfre/swiss-2 % for i in *.wav                                    Err 139 #6
do
  8hz-mp3 $i $i:r.mp3
done | od

[...lot's of output...]

zsh: exit 141            for i in *.wav; do; 8hz-mp3 $i $i:r.mp3; done | od | 
zsh: segmentation fault  od                                               

How fun! now, that's od which is segfaulting. BTW where does this last pipe
(after `od' on the exit report line) come from?.

I will try to see if I can reproduce this without 8hz.
-- 
Alexandre Duret-Lutz

Re: Zsh 3.1.6 still hangs on for loops utilizing lots of stdout

[Sven Wischnowsky]

Alexandre Duret-Lutz wrote:

> >>> "Clint" == Clint Olsen <olsenc@ichips.intel.com> writes:
> 
> [...]
> 
>  Clint> for file in *.something
>  Clint> do
>  Clint>   cat $file
>  Clint> done > /tmp/concat
> 
>  Clint> The number of files matched were probably in the thousand or so
>  Clint> range, each with about 10 lines of data.  Zsh hangs on large jobs
>  Clint> like this to the point of having to kill the window entirely (^C
>  Clint> doesn't work, neither does ^Z).

I've tried it now under DU and Linux but can't reproduce it. I can
understand that it is hard (even seemingly impossible) to stop such a
loop with ^Z because you have to hit one of the cats (and then those
animal-rights people will get you and... er, no), which can be
difficult if they finish very quickly. However, ^C should work even
then -- and it does for me.

We had some patches for the execution, jobs and signal code but most
of them were before the 3.1.6 release and I don't think any of those
that came afterwards should have changed this.

Hm, don't have an idea how this can happen.

> I have seen this on linux too (and attributed to
> my kernel, wich is making `Oops' sometimes...) 
> when making a loop of the form :
> 
> for i in *.wav
> do
>   8hz-mp3 $i $i:r.mp3
> done
> 
> *.wav did match only about 20 files, and the xterm disapeared
> after the second or third file processed.  Although the xterm 
> is dead, zsh and 8hz are still alive (but waiting) :
> 
>   PID  PPID WCHAN  COMMAND
>  3533  6548 signal zsh
>  3587  3533 down_f 8hz-mp3 audio_12.wav audio_12.mp3
> 
> (I never saw that `down_f' flag before)
> 
> If I attach to 3533 to see where it wait I see
> 
> (gdb) bt
> #0  0x400ad8fa in sigsuspend () from /lib/libc.so.6
> ...
> 
> Don't know if it can help.
> 
> (And since you are guessing it maybe related to stdout output, I must
> state that 8hz is quite verbose too).

Well, we had some problems with IO and AIX but this stack trace looks
completely fine -- saying only that it's waiting for the 8hz-mp3.

In both cases: does it happen with other shells, too?

Bye
 Sven


--
Sven Wischnowsky                         wischnow@informatik.hu-berlin.de

Re: Zsh 3.1.6 still hangs on for loops utilizing lots of stdout

[Alexandre Duret-Lutz]

>>> "Clint" == Clint Olsen <olsenc@ichips.intel.com> writes:

[...]

 Clint> for file in *.something
 Clint> do
 Clint>   cat $file
 Clint> done > /tmp/concat

 Clint> The number of files matched were probably in the thousand or so
 Clint> range, each with about 10 lines of data.  Zsh hangs on large jobs
 Clint> like this to the point of having to kill the window entirely (^C
 Clint> doesn't work, neither does ^Z).

I have seen this on linux too (and attributed to
my kernel, wich is making `Oops' sometimes...) 
when making a loop of the form :

for i in *.wav
do
  8hz-mp3 $i $i:r.mp3
done

*.wav did match only about 20 files, and the xterm disapeared
after the second or third file processed.  Although the xterm 
is dead, zsh and 8hz are still alive (but waiting) :

  PID  PPID WCHAN  COMMAND
 3533  6548 signal zsh
 3587  3533 down_f 8hz-mp3 audio_12.wav audio_12.mp3

(I never saw that `down_f' flag before)

If I attach to 3533 to see where it wait I see

(gdb) bt
#0  0x400ad8fa in sigsuspend () from /lib/libc.so.6
#1  0x80991bc in signal_suspend (sig=17, sig2=0)
    at ../../last/Src/signals.c:343
#2  0x8077e03 in waitjob (job=2, sig=0) at ../../last/Src/jobs.c:886
#3  0x8077f57 in waitjobs () at ../../last/Src/jobs.c:916
#4  0x805eabe in execpline (state=0xbffff6d0, slcode=2562, how=18, last1=0)
    at ../../last/Src/exec.c:986
#5  0x805e0b9 in execlist (state=0xbffff6d0, dont_change_job=1, exiting=0)
    at ../../last/Src/exec.c:766
#6  0x807dd1e in execfor (state=0xbffff6d0, do_exec=0)
    at ../../last/Src/loop.c:134
#7  0x806240e in execcmd (state=0xbffff6d0, input=0, output=0, how=18, last1=2)
    at ../../last/Src/exec.c:2109
#8  0x805f39b in execpline2 (state=0xbffff6d0, pcode=131, how=18, input=0, 
    output=0, last1=0) at ../../last/Src/exec.c:1119
#9  0x805e682 in execpline (state=0xbffff6d0, slcode=6146, how=18, last1=0)
    at ../../last/Src/exec.c:915
#10 0x805e0b9 in execlist (state=0xbffff6d0, dont_change_job=0, exiting=0)
    at ../../last/Src/exec.c:766
#11 0x805df66 in execode (p=0x40015278, dont_change_job=0, exiting=0)
    at ../../last/Src/exec.c:712
#12 0x8072a3d in loop (toplevel=1, justonce=0) at ../../last/Src/init.c:150
#13 0x805138c in main (argc=1, argv=0xbffff7c4) at ../../last/Src/main.c:89

Don't know if it can help.

(And since you are guessing it maybe related to stdout output, I must
state that 8hz is quite verbose too).

[...]

-- 
Alexandre Duret-Lutz

Zsh 3.1.6 still hangs on for loops utilizing lots of stdout

[Clint Olsen]

I'm currently on AIX 4.1.X, and I had this problem with 3.0.5 which made me
try to upgrade to 3.1.6.  Everything seemed to be working great until I did
the following:

for file in *.something
do
  cat $file
done > /tmp/concat

The number of files matched were probably in the thousand or so range, each
with about 10 lines of data.  Zsh hangs on large jobs like this to the
point of having to kill the window entirely (^C doesn't work, neither does
^Z).

Does anyone have any idea how they could possibly reproduce this or
determine if it's an AIX-specific issue?

Thanks,

-Clint

P.S. I'm not on the list, so please Cc me if you want me to provide any
     further info or make suggestions.