Re: [Caml-list] Netclient

[Gerd Stolpmann <info@gerd-stolpmann.de>]

Am Montag, den 13.03.2006, 00:03 -0800 schrieb Jean-David:
> Hello there,
> I was wondering about the best way to create a stream
> from a remote file for which I have the URL
> 
> open Http_client
> open Stream
> 
> let body = Stream.of_string Convenience.http_get
> "http://dfl.sjsu.edu";;
> 
> Is the content of body downloaded completely before
> being streamed(which defeats the purpose of streams)
> or not? and if not how can I get the content of an URL
> gradually (without getting the whole body into
> memory)?

Yes, Convenience.http_get always downloads the whole body into a string.

For a stream-like download, you must use the pipeline interface. For
example, to store the body into a file do:

let call = new get "http://dfl.sjsu.edu" in
call # set_response_body_storage (`File (fun () -> "filename"));
let pipeline = new pipeline in
pipeline # set_proxy_from_environment();
pipeline # run()

Now check call#status whether the HTTP request was successfully
responded.

If you want to postprocess the body while it is downloaded, things will
get more complicated. In particular, it is possible to store the body in
an object that fulfils the Netmime.mime_body class type. Netclient
invokes the open_value_wr method and outputs into this netchannel. Using
this mechanism to define a callback-style interface (i.e. a function is
called whenever a chunk of data arrives) can be implemented in a
straight-forward manner.

If you really need a "string Stream.t": This is possible because
Netclient is programmed in an event-driven way, but is very difficult,
because you must arrange a so-called control inversion. This is
absolutely non-trivial but implementable in less than 100 lines of code.

Hope this helps,

Gerd

> 
> Thank you,
> JD
> 
> __________________________________________________
> Do You Yahoo!?
> Tired of spam?  Yahoo! Mail has the best spam protection around 
> http://mail.yahoo.com 
> 
> _______________________________________________
> Caml-list mailing list. Subscription management:
> http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
> Archives: http://caml.inria.fr
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs
> 
-- 
------------------------------------------------------------
Gerd Stolpmann * Viktoriastr. 45 * 64293 Darmstadt * Germany 
gerd@gerd-stolpmann.de          http://www.gerd-stolpmann.de
Phone: +49-6151-153855                  Fax: +49-6151-997714
------------------------------------------------------------