From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Delivered-To: caml-list@yquem.inria.fr Received: from concorde.inria.fr (concorde.inria.fr [192.93.2.39]) by yquem.inria.fr (Postfix) with ESMTP id A9BF0BC48 for ; Tue, 19 Apr 2005 23:12:17 +0200 (CEST) Received: from pauillac.inria.fr (pauillac.inria.fr [128.93.11.35]) by concorde.inria.fr (8.13.0/8.13.0) with ESMTP id j3JLCHGB028899 for ; Tue, 19 Apr 2005 23:12:17 +0200 Received: from nez-perce.inria.fr (nez-perce.inria.fr [192.93.2.78]) by pauillac.inria.fr (8.7.6/8.7.3) with ESMTP id XAA27969 for ; Tue, 19 Apr 2005 23:12:16 +0200 (MET DST) Received: from plover.csun.edu (plover.csun.edu [130.166.1.24]) by nez-perce.inria.fr (8.13.0/8.13.0) with ESMTP id j3JLCFlq004599 for ; Tue, 19 Apr 2005 23:12:16 +0200 Received: from puffin.csun.edu (puffin.csun.edu [130.166.1.21]) by plover.csun.edu (MOS 3.5.4-GR) with ESMTP id BOD53249; Tue, 19 Apr 2005 14:11:45 -0700 (PDT) Received: from [130.166.10.117] (s010n117.csun.edu [130.166.10.117]) by puffin.csun.edu (MOS 3.5.4-GR) with ESMTP id DEI54530 (AUTH eric); Tue, 19 Apr 2005 14:11:44 -0700 (PDT) In-Reply-To: <1113933973.6248.76.camel@localhost.localdomain> References: <810e04dc0f6bb601fb828db8d18def6c@fas.harvard.edu> <426361DB.10704@rftp.com> <4263BAD0.5020901@barettadeit.com> <1113834691.6248.42.camel@localhost.localdomain> <4263E01D.5090705@barettadeit.com> <426479A2.1020404@fas.harvard.edu> <42647A4B.3050507@fas.harvard.edu> <1113902296.6248.47.camel@localhost.localdomain> <42652387.4030006@fas.harvard.edu> <1113933973.6248.76.camel@localhost.localdomain> Mime-Version: 1.0 (Apple Message framework v622) Content-Type: text/plain; charset=US-ASCII; format=flowed Message-Id: <03900e2a40e57683649cd7d76261b427@csun.edu> Content-Transfer-Encoding: 7bit Cc: caml-list@inria.fr, Mike Hamburg From: Eric Stokes Subject: Re: [Caml-list] CamlGI question [doh] Date: Tue, 19 Apr 2005 14:11:43 -0700 To: Gerd Stolpmann X-Mailer: Apple Mail (2.622) X-Miltered: at concorde with ID 42657431.000 by Joe's j-chkmail (http://j-chkmail.ensmp.fr)! X-Miltered: at nez-perce with ID 4265742F.000 by Joe's j-chkmail (http://j-chkmail.ensmp.fr)! X-Spam: no; 0.00; caml-list:01 slap:01 invocation:01 chunks:01 threads:01 sockets:01 ocamlnet:01 handler:01 buffered:01 gerd:01 stolpmann:01 bug:01 ocamlnet:01 buffered:01 o'caml:01 X-Spam-Checker-Version: SpamAssassin 3.0.2 (2004-11-16) on yquem.inria.fr X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=disabled version=3.0.2 X-Spam-Level: I've looked into this a bit more, and I have two things to say. 1. Unfortunately, it may not get any better than write. Once I slap on the fastcgi header telling the server how many bytes I'm going to write, I've committed to write that many bytes, If I don't its an error. That means I'd end up having to do the same thing as write if I used write_single. I'd have to check the return value to be sure that I wrote the correct number of bytes and if I didn't then call write_single again. This means that if there is an error on the third invocation, I'd have a protocol error, because some data has already been written. I might mitigate this by writing data in smaller chunks, but this can effect performance. I'll do some thinking about this. 2. We are not specific in the documentation about how you actually write threaded apps with Netcgi_fastcgi. This is very likely the cause of your problem. You need to call a new serv function for each thread that you want to work on web stuff. The idea is that each thread calls accept on the listening socket, and clients are distributed to threads in this manner. If your thread is never going to send data back via the fastcgi sockets then it is fine if it does not call serv, however any thread which communicates with the web server must do so only through serv. eg. (* this is how I run threaded web applications via Ocamlnet *) let worker_thread () = Netcgi_fcgi.serv req_handler buffered_transactional_optype;; for i = 0 to numthreads do Thread.create worker_thread () done On Apr 19, 2005, at 11:06 AM, Gerd Stolpmann wrote: > Am Dienstag, den 19.04.2005, 11:28 -0400 schrieb Mike Hamburg: >> I'll try to help work out this bug in either Apache/FastCGI or Netcgi, >> and if that doesn't work, I'll go to AJP. I compiled >> mod_fastcgi-2.4.0 >> on Apache 2.0.53. It successfully serves FastCGI pages written with >> CamlGI, except that it sometimes pauses after 8k of data, and >> sometimes >> breaks the pipe to the CGI. > > Just to make sure Netcgi works also for multi-threaded programs, I made > a test for myself. I added a second thread to the example > add_fastcgi.ml > (of Ocamlnet), such that it reads now > > let a = ref ["Z"; "X"; "A"; "P"; "B" ];; > > Thread.create > (fun () -> > while true do > a := List.sort compare !a > done > ) > () > ;; > > (* start the fastcgi server *) > serv process buffered_transactional_optype;; > > at the end. So there is now a thread sorting this list over and over > again. > > I tried it with Apache 2.0.50 and mod_fastcgi 2.4.2 (those versions > coming with Ubuntu Warty). The first test failed, I had to comment out > the FastCgiWrapper which is part of the default configuration. After > that, the program just worked. > > I also looked at the sources. There might be problems because the > O'Caml > thread implementation uses signals to wake up threads in some > situations. This may cause that the Unix.read/write (one should better > use Unix.single_write, btw.) fail with EINTR. This is not caught. > >> Under Netcgi 1.0, the CGI version works but the FastCGI version does >> not >> work at all. The difference between the CGI and FastCGI versions is >> only: >> >> let () = Netcgi_fcgi.serv main (`Direct "

"); >> (* let cgi = new Netcgi.std_activation () >> let () = main cgi *) >> >> The application reads files from the outside world, and may launch >> processes with fork+execv to thumbnail images. It is multithreaded, >> with at least 4 threads at all times: the main thread, a thumbnail >> cache >> manager, a thread which handles signals (this cannot be done in a >> signal >> handler, as it may block, so I wake up the handler thread instead), >> and >> a thread which rereads the configuration files periodically, so that >> the >> user does not have to send HUP. >> >> The application usually raises Unix.Unix_error (EPIPE, "write", "") on >> the line Netcgi_fcgi.serv main (`Direct "

") or rather, >> inside Netcgi_fcgi.serv at some point; I believe this occurs before >> main >> is called. > > You can figure this out exactly when you byte-compile the program with > -g, and set OCAMLRUNPARAM=b=1, e.g. using a wrapper script and > redirecting stderr to a file. > > The EPIPE error is quite surprising, because Netcgi wraps all its > exceptions into FCGI_Error. > >> The webserver also receives EPIPE and returns error 500. >> Sometimes, I get other errors: sometimes main gets called, and I get >> Failure("send_output_header"). > > This is very strange, too. Failure("send_output_header") can only > happen > when the HTTP header is printed at the wrong moment (i.e. too late). > >> If this happens, then rollback_output >> successfully prints "

" but no more (i.e. no further >> diagnostics). Sometimes, the webserver times out waiting for the >> first >> read, in which case the application still raises EPIPE. >> >> Is there any more information that would help to debug the problem? > > I suspect a part of the program prints something to stdout at the wrong > moment, maybe because of a race condition. This is only my intuition, > but this is the first thing I would try to look for. This would explain > why the fcgi protocol is violated, and EPIPE is the logical consequence > (the web server sees wrong data and shuts down the socket). > > Maybe an strace of the process can show what is really going on. > > Gerd > >> >> Mike >> >> Alex Baretta wrote: >> >>> If at all possible, rather than AJP I'd stick to FastCGI. The code >>> in >>> Ocamlnet reportedly works fine, but I don't have experience with it. >>> Are you sure that the broken pipe is caused by a bug in Ocamlnet and >>> not in the web server? >>> >>> Alex >> >> >> >> Gerd Stolpmann wrote: >> >>> Am Montag, den 18.04.2005, 23:26 -0400 schrieb Mike Hamburg: >>> >>> >>>> I'm obviously too tired to be programming. That should, of course, >>>> read >>>> (1) fix a broken pipe error in Netcgi_fast or >>>> >>>> >>> >>> Well, we would need more details to help you. >>> >>> >>> >>>> (2) port the application to (and configure the webserver for) AJP >>>> >>>> >>> >>> Porting the application is quite simple, just follow the examples >>> coming >>> with Ocamlnet (in examples/jserv). For the webserver, you need mod_jk >>> (jk=Jakarta). The configuration reference is here: >>> >>> http://jakarta.apache.org/tomcat/connectors-doc/config/apache.html >>> >>> Note that Ocamlnet only supports AJP-1.2, not 1.3 which is the >>> current >>> default for Tomcat. >>> >>> Gerd >>> >>> >> >> > -- > ------------------------------------------------------------ > Gerd Stolpmann * Viktoriastr. 45 * 64293 Darmstadt * Germany > gerd@gerd-stolpmann.de http://www.gerd-stolpmann.de > ------------------------------------------------------------ > >