windows, threads and sockets

windows, threads and sockets

[Christoph Bauer]

Hi,

I'm using OCaml 3.11.0. My question is about my program,
which used to work fine under windows and linux. The program
is a daemon, which waits for socket connections and respondse
to queries.

Now I added an extension with uses threads. Everything is
find on linux, but I have a very strange behaviour on windows.
For the troubles it is sufficient to link agains threads and compile the
main program with -thread. Without it everything is fine again.

I'm not quite sure what is going on, but my best bet is this: the main
loop waits for the sockets with

Unix.select listOfSockets [] [] timeout

One socket is the listener socket on which new connections 
are made. Unix.select returns a list of sockets. Initially
it must be the listener socket. This is checked with a compare

  socket = listenSocket 

and this seems to be suddenly wrong (just in the thread case).

Any ideas?

Thanks,

Christoph Bauer

RE: [Caml-list] windows, threads and sockets

[David Allsopp]

Christoph Bauer wrote:
<snip>
> I'm not quite sure what is going on, but my best bet is this: the main
> loop waits for the sockets with
> 
> Unix.select listOfSockets [] [] timeout
> 
> One socket is the listener socket on which new connections
> are made. Unix.select returns a list of sockets. Initially
> it must be the listener socket. This is checked with a compare
> 
>   socket = listenSocket
> 
> and this seems to be suddenly wrong (just in the thread case).
> 
> Any ideas?

I can't tell you explicitly why it has failed, but Unix.select was
completely rewritten for OCaml 3.11.0 based on a big contribution from
Sylvain Le Gall (see otherlibs/win32unix/select.c). The principal aim was to
make the semantics of Unix.select the same between *NIX and Windows so if it
works on Linux then it sounds like you've hit a bug...


David

RE: [Caml-list] windows, threads and sockets

[Christoph Bauer]

> > Any ideas?
> 
> I can't tell you explicitly why it has failed, but 
> Unix.select was completely rewritten for OCaml 3.11.0 based 
> on a big contribution from Sylvain Le Gall (see 
> otherlibs/win32unix/select.c). The principal aim was to make 
> the semantics of Unix.select the same between *NIX and 
> Windows so if it works on Linux then it sounds like you've 
> hit a bug...
> 
Good to know, I missed that change. So the new Unix.select should
now work also on pipes? I guess it from the function named
read_pipe_poll()
in select.c. This is good news, because I can throw away some
workarounds.

But maybe the problem with thread is, that not
otherlibs/win32unix/select.c
is used. There is also otherlibs/threads/unix.ml with seems to overwrite
Unix.select
and there are select() calls in otherlibs/threads/scheduler.c. I'm just
guessing
here.

So maybe the simples solution for me (I have to stick to 3.11.0)
would be to create my thread in plain C, link against is and omit the
dreaded threads library ;-) The task of my thread is very simple...

Thanks for help,
Christoph Bauer

Re: windows, threads and sockets

[Sylvain Le Gall]

Hello,

On 07-08-2009, Christoph Bauer <christoph.bauer@lmsintl.com> wrote:
>> > Any ideas?
>> 
>> I can't tell you explicitly why it has failed, but 
>> Unix.select was completely rewritten for OCaml 3.11.0 based 
>> on a big contribution from Sylvain Le Gall (see 
>> otherlibs/win32unix/select.c). The principal aim was to make 
>> the semantics of Unix.select the same between *NIX and 
>> Windows so if it works on Linux then it sounds like you've 
>> hit a bug...
>> 
> Good to know, I missed that change. So the new Unix.select should
> now work also on pipes? I guess it from the function named
> read_pipe_poll()
> in select.c. This is good news, because I can throw away some
> workarounds.

In no-thread context, it works great... (with pipe and console et al)

>
> But maybe the problem with thread is, that not
> otherlibs/win32unix/select.c
> is used. There is also otherlibs/threads/unix.ml with seems to overwrite
> Unix.select
> and there are select() calls in otherlibs/threads/scheduler.c. I'm just
> guessing
> here.

Indeed, I miss this case because I don't use thread in ocaml code.
However I don't see interaction between otherlibs/win32unix/select.c and
otherlibs/threads/unix.ml, so I am not sure where the bug comes from.

It also show that pipe/console is not available when you use select with
thread :( 

Something that can change is that we now use WinSock 2.0...

>
> So maybe the simples solution for me (I have to stick to 3.11.0)
> would be to create my thread in plain C, link against is and omit the
> dreaded threads library ;-) The task of my thread is very simple...
>

Regards,
Sylvain Le Gall

Re: windows, threads and sockets

[Sylvain Le Gall]

On 07-08-2009, Christoph Bauer <christoph.bauer@lmsintl.com> wrote:
> Hi,
>
> I'm using OCaml 3.11.0. My question is about my program,
> which used to work fine under windows and linux. The program
> is a daemon, which waits for socket connections and respondse
> to queries.
>
> Now I added an extension with uses threads. Everything is
> find on linux, but I have a very strange behaviour on windows.
> For the troubles it is sufficient to link agains threads and compile the
> main program with -thread. Without it everything is fine again.
>
> I'm not quite sure what is going on, but my best bet is this: the main
> loop waits for the sockets with
>
> Unix.select listOfSockets [] [] timeout
>
> One socket is the listener socket on which new connections 
> are made. Unix.select returns a list of sockets. Initially
> it must be the listener socket. This is checked with a compare
>
>   socket = listenSocket 
>
> and this seems to be suddenly wrong (just in the thread case).
>

Do you mean that the following code is wrong with 3.11.1 and not 3.10.2:

let res_listener = Unix.select listOfSockets [] [] timeout
in
(List.nth res_listener 0) = listenSocket

Can you find listenSocket in the result elsewhere ?

Regards,
Sylvain Le Gall

RE: [Caml-list] Re: windows, threads and sockets

[Christoph Bauer]

> Do you mean that the following code is wrong with 3.11.1 and 
> not 3.10.2:
> 
> let res_listener = Unix.select listOfSockets [] [] timeout in 
> (List.nth res_listener 0) = listenSocket
> 
> Can you find listenSocket in the result elsewhere ?

No! I try to clarify: 

my check is then 

  match Unix.select sockets [] [] timeout with
     | (l,_,_) ->
          List.iter (fun sock -> printf "%b,%b\n" (List.mem sock
sockets) (List.memq sock sockets)) l

I only use OCaml 3.11.0. It only happens on windows.

If I compile and link with -thread and link with ocamlfind ... -package
threads
then the result is

  false,false

But without the thread package the result is

  true,true


BTW, I use systhreads, but the files (scheduler.c, unix.ml) I mentioned
before are used in the vmthread case.
In otherlibs\systhreads\threadUnix.ml I found

let select = Unix.select

Regards,
Christoph Bauer

	     

Re: windows, threads and sockets

[Sylvain Le Gall]

Hello,

On 07-08-2009, Christoph Bauer <christoph.bauer@lmsintl.com> wrote:
>
>> Do you mean that the following code is wrong with 3.11.1 and 
>> not 3.10.2:
>> 
>> let res_listener = Unix.select listOfSockets [] [] timeout in 
>> (List.nth res_listener 0) = listenSocket
>> 
>> Can you find listenSocket in the result elsewhere ?
>
> No! I try to clarify: 
>
> my check is then 
>
>   match Unix.select sockets [] [] timeout with
>      | (l,_,_) ->
>           List.iter (fun sock -> printf "%b,%b\n" (List.mem sock
> sockets) (List.memq sock sockets)) l
>
> I only use OCaml 3.11.0. It only happens on windows.
>
> If I compile and link with -thread and link with ocamlfind ... -package
> threads
> then the result is
>
>   false,false
>
> But without the thread package the result is
>
>   true,true
>

Maybe, I understand the bug. I use "lpOrig = (void *)fd;" where fd is an
OCaml value (in select.c). Without thread and during the
enter/leave_blocking_section(), there is less chance to trigger the GC
and to modify "fd" value. This is the reason why it seems correct
without thread and fall into a bug with thread.

The obvious solution is to replace "lpOrig = (void *)fd" by a non-OCaml
value (e.g. EMode + nth in ocaml list).

Could you reduce your bug to the smallest possible program and submit
the bug on OCaml bug tracking ? 

>
> BTW, I use systhreads, but the files (scheduler.c, unix.ml) I mentioned
> before are used in the vmthread case.
> In otherlibs\systhreads\threadUnix.ml I found
>
> let select = Unix.select
>

This remains another bug (less important) since we cannot use
-vmthreads, Unix.select and pipes on windows. 

Regards,
Sylvain Le Gall

RE: [Caml-list] Re: windows, threads and sockets

[Christoph Bauer]

Hello,

> Could you reduce your bug to the smallest possible program 
> and submit the bug on OCaml bug tracking ? 

here it is:

http://caml.inria.fr/mantis/view.php?id=4844

If you run the program, please press some keys, because of the
Unix.select on Unix.stdin.

Regards,
Christoph Bauer