Re: [Caml-list] Re: Looking for stubs for sendmsg/recvmsg

[Goswin von Brederlow <goswin-v-b@web.de>]

Sylvain Le Gall <sylvain@le-gall.net> writes:

> Hello,
>
> On 19-11-2010, Goswin von Brederlow <goswin-v-b@web.de> wrote:
>> Sylvain Le Gall <sylvain@le-gall.net> writes:
>>
>>> On 18-11-2010, Goswin von Brederlow <goswin-v-b@web.de> wrote:
>>>> Hi,
>>>>
>>>> I'm looking for stubs for
>>>>
>>>>        ssize_t sendmsg(int sockfd, const struct msghdr *msg, int flags);
>>>>        ssize_t recvmsg(int sockfd, struct msghdr *msg, int flags);
>>>>
>>>> Specifically I need those to send (among normal messages) an
>>>> Unix.file_descr over a Unix Domain Socket.
>>>>
>>>> Does anyone know of a module that has them?
>>>>
>>>
>>> If you don't find one and plan to write it yourself, this would be a
>>> good addition to extunix:
>>> http://extunix.forge.ocamlcore.org
>>>
>>> Regards,
>>> Sylvain Le Gall
>>
>> I'm thinking of changing Unix.file_descr from int to a custom block
>> (containing the FD) with finalizer. Unix.close would set the FD to -1
>> and the finalizer gives an error if FD != -1 and closes it.
>>
>> Actually I want that tunable with 3 possible behaviours:
>>
>> type fd_leak_mode = Silent | Complain | Fail
>> val set_leak_mode : fd_leak_mode -> unit = <fun>
>>
>> Silent just closes the FD if it is still open, Complain (default)
>> outputs to stderr and closes it and Fail aborts.
>>
>> That would change most of the Unix module and mean a complete fork of it.
>
> Not that much, if you proceed in another way. I think what you are
> looking for is a fd leak detector?
>
> Here is a small modules that I have written for this purpose:
>
> File UnixExt.ml:
> (** Count open/close call *)
> IFNDEF NDBUG THEN
> let fd_opened =
>   Hashtbl.create 13
> ;;
>
> let fd_once_opened =
>   Hashtbl.create 13
> ;;
>
> let fd_open fd fn out =
>   dbug_print
>     (fun () -> 
>        Printf.sprintf "%s '%s'" 
>          (if out then "open-out" else "open-in")
>          fn);
>   Hashtbl.add fd_opened fd (fn, out)
> ;;
>
> let fd_close fd =
>   try 
>     let (fn, out) as data = 
>       Hashtbl.find fd_opened fd 
>     in
>       dbug_print
>         (fun () ->
>            Printf.sprintf "%s '%s'" 
>              (if out then "close-out" else "close-in")
>              fn);
>       Hashtbl.add fd_once_opened fd data;
>       Hashtbl.remove fd_opened fd;
>   with Not_found ->
>     begin 
>       dbug_print
>         (fun () -> 
>            let fn =
>              try 
>                fst (Hashtbl.find fd_once_opened fd)
>              with Not_found ->
>                "unknown"
>            in
>              Printf.sprintf "Trying to close %s again" fn)
>     end
> ;;
>
> let () = 
>   at_exit
>     (fun () ->
>        let exit_error =
>          ref false
>        in
>          Hashtbl.iter 
>            (fun fd (fn, out) ->
>               if fd <> Unix.stdin && fd <> Unix.stdout && fd <> Unix.stderr then
>                 begin
>                   Printf.eprintf "Not closed '%s' (out: %b)\n" fn out;
>                   exit_error := true
>                 end)
>            fd_opened;
>          Hashtbl.clear fd_opened;
>          if !exit_error then 
>            exit 3
>     )
> ;;
>
> let opened_files () = 
>   let lst = 
>     ref []
>   in
>     Hashtbl.iter 
>       (fun _ e -> lst := e :: !lst)
>       fd_opened;
>     List.sort compare !lst;
>
> ELSE
> let fd_open _ _ _ = 
>   ()
> ;;
>
> let fd_close _ = 
>   ()
> ;;
>
> let opened_files () = 
>   []
> ;;
> ENDIF
>
> (** See UnixExt.mli *)
> let to_file_descr_in fd = 
>   fd_open fd "<converted>" false;
>   fd
> ;;
>
> (** See UnixExt.mli *)
> let to_file_descr_out fd = 
>   fd_open fd "<converted>" true;
>   fd
> ;;
>
> (** See UnixExt.mli *)
> let close_in fd =
>   Unix.close fd;
>   fd_close fd
> ;;
>
> (** See UnixExt.mli *)
> let stdout =
>   fd_open Unix.stdout "<stdout>" true;
>   Unix.stdout
> ;;
>
> [...override other functions that open/close fd...]
>
> Then in the modules using this features, you just have to open UnixExt
> after Unix...
>
> You can even probably design a library that will transparently hide Unix
> with a custom Unix module providing this feature.

Much less usefull.

Using a custom block with finalizer means that the FD will be closed
relative close to where/when it was leaked. Makes it easier to find
where it was leaked and adding GC.compact calls at strategic locations
can narrow it down even more. Leaking FDs also becomes much less
serious. The GC will clean them up and close them. So you can use an app
that leaks FDs just fine.

MfG
        Goswin