Unix.send blocks

Unix.send blocks

[Paul Steckler]

I've written a wee Web server in OCaml that's compiled using the ocamlopt from the
Fedora MinGW distribution of ocaml.  I'm running the server in Windows 7.

Sometimes after receiving several requests, the Unix.send call that sends a response
back to a Web client just blocks.  The send buffer is pretty large (64k), and the data
to be sent is always much less than that.

While it's easy to reproduce the error (a certain pattern of requests from the browser),
I can't tell what particular conditions cause the blocking behavior.

Any help appreciated.

-- Paul
--
Paul Steckler
National ICT Australia
paul DOT steckler AT nicta.com.au

The information in this e-mail may be confidential and subject to legal professional privilege and/or copyright. National ICT Australia Limited accepts no liability for any damage caused by this email or its attachments.

Re: Unix.send blocks

[Sylvain Le Gall]

On 16-06-2010, Paul Steckler <Paul.Steckler@nicta.com.au> wrote:
> I've written a wee Web server in OCaml that's compiled using the ocamlopt from the
> Fedora MinGW distribution of ocaml.  I'm running the server in Windows 7.
>
> Sometimes after receiving several requests, the Unix.send call that sends a response
> back to a Web client just blocks.  The send buffer is pretty large (64k), and the data
> to be sent is always much less than that.
>
> While it's easy to reproduce the error (a certain pattern of requests from the browser),
> I can't tell what particular conditions cause the blocking behavior.
>
> Any help appreciated.
>


I don't have a clue but maybe you can tell us more about "the pattern of
requests from the browser".

Regards,
Sylvain Le Gall

Re: [Caml-list] Unix.send blocks

[Török Edwin]

On 06/16/2010 10:32 AM, Paul Steckler wrote:
> I've written a wee Web server in OCaml that's compiled using the ocamlopt from the
> Fedora MinGW distribution of ocaml.  I'm running the server in Windows 7.
> 
> Sometimes after receiving several requests, the Unix.send call that sends a response
> back to a Web client just blocks.

Why is a blocking send() a problem? Isn't your application multithreaded?

>  The send buffer is pretty large (64k), and the data
> to be sent is always much less than that.

>From the manpage of send:
"If space is not available at the sending socket to hold the message to
be transmitted, and the socket file descriptor does not have O_NONBLOCK
set, send() shall block until space is available."

I think this occurs if the client has a slow/high latency connection,
and it doesn't acknowledge the TCP packets in time.
The packets you send are queued up in the kernel's TCP stack until they
are acknowledged or time out. There is only a limited amount of data
that can be buffered up like this per connection, when that is hit
send() blocks.

You could set the socket to nonblocking mode (and check with 'select'
whether you can send), but according to the manual that doesn't work on
the native windows port of OCaml.

Best regards,
--Edwin

RE: [Caml-list] Re: Unix.send blocks

[Christoph Bauer]

 
> While it's easy to reproduce the error (a certain pattern of requests 
> from the browser), I can't tell what particular conditions cause the blocking behavior.
>
> Any help appreciated.

I guess you are using Unix.select. There is (was) a bug in the windows
select implementation. It had problems with the GC.

http://caml.inria.fr/mantis/view.php?id=4844

Just an idea.

Christoph Bauer

Re: Unix.send blocks

[Sylvain Le Gall]

On 16-06-2010, Christoph Bauer <christoph.bauer@lmsintl.com> wrote:
>  
>> While it's easy to reproduce the error (a certain pattern of requests 
>> from the browser), I can't tell what particular conditions cause the blocking behavior.
>>
>> Any help appreciated.
>
> I guess you are using Unix.select. There is (was) a bug in the windows
> select implementation. It had problems with the GC.
>
> http://caml.inria.fr/mantis/view.php?id=4844

"There was" when OCaml 3.12 will be out ;-)

However, I think it should have crash if it falls into this bug.

Regards,
Sylvain Le Gall

Re: Unix.send blocks

[Sylvain Le Gall]

On 16-06-2010, Török Edwin <edwintorok@gmail.com> wrote:
> On 06/16/2010 10:32 AM, Paul Steckler wrote:
>
> You could set the socket to nonblocking mode (and check with 'select'
> whether you can send), but according to the manual that doesn't work on
> the native windows port of OCaml.
>

select works on Windows with latest version of OCaml. But there were a
couple of bugs corrected in version 3.12 (esp. O_NONBLOCK + select).
Next version will be safer for the use of select on Windows.

Regards,
Sylvain Le Gall

RE: [Caml-list] Unix.send blocks

[David Allsopp]

k/Paul Steckler wrote:
> I've written a wee Web server in OCaml that's compiled using the ocamlopt
> from the Fedora MinGW distribution of ocaml.  I'm running the server in
> Windows 7.
> 
> Sometimes after receiving several requests, the Unix.send call that sends
> a response back to a Web client just blocks.

Sounds like terminology may be getting in the way of a solution - do you
mean that the call *blocks* for a longer time than you expect then carries
on working (the usual meaning of "block" - i.e. the call to send doesn't
return until it's actually finished) or do you mean *freezes* - i.e. the
call seemingly *never* returns and so your application hangs. When this
happens on a given connection, does that prevent further connections from
working - i.e. is the entire server dead or just that one connection? 

Additionally, just applying the usual debugging principle of "simplify,
simplify and then simplify some more", why not eliminate the cross-compiler
and instead use ocamlopt directly on Windows?


David

PS Although it's not your question, why are you writing a web server?
CGI+IIS (if Windows is your target) is dead easy and if you do strictly want
your own webserver then there's one in ocamlnet (although I'm not sure if
that part works under Windows - my default compiled version of ocamlnet
2.2.9 doesn't seem to have it but that may be because by default it's not
compiled rather than that it's not possible to compile it). Given that
you've hit problems, reinventing the wheel seems potentially questionable.

[Caml-list] Re: Unix.send blocks

[Paul Steckler]

I wrote:

> Sometimes after receiving several requests, the Unix.send call that sends a response
> back to a Web client just blocks.  The send buffer is pretty large (64k), and the data
> to be sent is always much less than that.

I've found a solution.  When receiving data from the browser, the socket only ever
made one call to Unix.recv, even if there was more data available, because in this
application, that was always enough.  That strategy must have left some internal
buffers filled in the socket implementation.  When I allow multiple calls to Unix.recv,
until all data has been received, all calls to Unix.send proceed.

Re the issue of multi-threadedness, I'm unable to build a multi-threaded OCaml
application using the Fedora distribution of the MingGW build, for reasons I
mentioned in an earlier message to this list.  A non-blocking socket wouldn't
have helped in this situation, anyway, because the blocking status of the socket
wasn't the issue.

-- Paul
--
Paul Steckler
National ICT Australia
paul DOT steckler AT nicta.com.au

The information in this e-mail may be confidential and subject to legal professional privilege and/or copyright. National ICT Australia Limited accepts no liability for any damage caused by this email or its attachments.