Freezing

Freezing

[David Baelde]

Hi list,

This could be a bug report but I have only a very bad description of
it, and it may also be nothing but an unfortunate feature...

I have a program streaming audio, and occasionally downloading files,
using ocaml-smbclient wrapped in ocaml-fetch. I want to avoid too long
(or frozen) downloads. To do so, here is essentially the code I use:

<<<
flush_all () ;
let pid = Unix.fork () in
if pid = 0 then begin
  ignore (Unix.alarm 20) ;
  Fetch.cp there here ;
  exit 0
end else
  Unix.waitpid [] pid
>>>

It's that simple (modulo the mess inside the Fetch.cp) and it works as
it should.. until some download process gets stuck, never returns. At
the end of this message is the gdb trace I got from such a frozen
process. It looks like the process wants to flush IO (could it be part
of the exit procedure ?) but cannot because some other thread is
blocking.

The main process is threaded but the child isn't. Isn't it weird that
it gets blocked in such a way ? Gdb doesn't show me any other thread
in the child process.

Another question: assuming this kind of hangup is unavoidable if I try
to flush in a threaded process, is there a way to terminate a caml
process without any extra possibly-blocking stuff ?

I'm sorry to experience and report such a mess..

(gdb) bt
#0  0xffffe410 in __kernel_vsyscall ()
#1  0xb7bc556e in __lll_mutex_lock_wait ()
   from /lib/tls/i686/cmov/libpthread.so.0
#2  0xb7bc323b in pthread_cond_signal@@GLIBC_2.3.2 ()
   from /lib/tls/i686/cmov/libpthread.so.0
#3  0x08066ad5 in caml_thread_enter_blocking_section ()
#4  0x0806c006 in caml_enter_blocking_section ()
#5  0x08066c72 in caml_io_mutex_lock ()
#6  0x0807025d in caml_ml_flush ()
#7  0x08079f8d in caml_interprete ()
#8  0x08075eb0 in caml_callbackN_exn ()
#9  0x08075efa in caml_callback_exn ()
#10 0x080670b8 in caml_thread_start ()
#11 0xb7bc0e60 in start_thread () from /lib/tls/i686/cmov/libpthread.so.0
#12 0xb7afe8ee in clone () from /lib/tls/i686/cmov/libc.so.6

Re: [Caml-list] Freezing

[skaller]

On Tue, 2006-05-23 at 02:04 +0200, David Baelde wrote:

> <<<
> flush_all () ;
> let pid = Unix.fork () in
> if pid = 0 then begin
>   ignore (Unix.alarm 20) ;
>   Fetch.cp there here ;
>   exit 0
> end else
>   Unix.waitpid [] pid
> >>>

> The main process is threaded but the child isn't. Isn't it weird that
> it gets blocked in such a way ? 

When you fork() a process the whole memory space of the parent 
is cloned, including locks, file handles, buffers, etc -- including
those being used by threads other than the one doing the fork().

So now you can have this: some thread of the parent
is doing I/O and acquires a lock, you fork, then the
buffer is written, pointers into it adjusted,
and the lock is released.

When the forked() process tries to terminate ..
the lock is locked because it is a copy of the original
lock. And the buffer pointers say the buffer is full.
So the process hangs waiting for the lock to be released
and the buffer to be emptied by a thread that has
already done so .. but the state saying that
is in the original address space, not that of the
process.

The moral is -- use threads or processes but not both
together, Unix can't handle it. 

-- 
John Skaller <skaller at users dot sf dot net>
Felix, successor to C++: http://felix.sf.net

Re: [Caml-list] Freezing

[Eric Cooper]

On Tue, May 23, 2006 at 01:03:51PM +1000, skaller wrote:
> On Tue, 2006-05-23 at 02:04 +0200, David Baelde wrote:
> > flush_all () ;
> > let pid = Unix.fork () in
> > if pid = 0 then begin
> >   ignore (Unix.alarm 20) ;
> >   Fetch.cp there here ;
> >   exit 0
> > end else
> >   Unix.waitpid [] pid
> [...]
> The moral is -- use threads or processes but not both
> together, Unix can't handle it. 

Or at least, don't use fork without an almost-immediate exec.
In this particular case, the OP might consider using exec to run a
standalone program in the child -- perhaps cp or wget -- if there's
not a lot of state shared with the parent.

-- 
Eric Cooper             e c c @ c m u . e d u

Re: [Caml-list] Freezing

[David Baelde]

Thanks for the answer.

That is a simple moral, and I'm happy with it. Even if one could ask
if it's really impossible that OCaml manages to deal in a nicer way
with its internal locks, leaving to the user the task to manage his
locks. It's probably not worth working that issue, since people should
avoid doing that.. then why not issuing a warning ?

I said I was happy with the moral. Moving to a fork()+exec() was easy
and even made more sense. We didn't experience frozen downloads since
then. We get memory corruption, but that's another story :p

Cheers.
-- 
David