OCaml IDE (OCaml and stdout/stderr)

OCaml IDE (OCaml and stdout/stderr)

[Nathaniel J. Gaylinn]

My program is having a hard time coping with how OCaml splits its output
between stdout and stderr. The order in which a line of output is received
from OCaml's stdout and a warning from its stderr seems to be completely
random and therefore my program's output is also random. I'm pretty sure
this has to do with how OCaml buffers stdout, but I'm not sure how to
change this.

To make my program more portable, I'm writing it using the QT toolkit
which has a nice abstraction of inter-process communication in a class
called QProcess. This is what I'm using to communicate with OCaml.

>From your experience, do you think the buffering is taking place on
OCaml's end or on Qt's? Some research seems to suggest that how output is
buffered from a program can be controlled from within that program (since
there is direct access to the various output streams) but not from
outside. If that's true, then the buffering problem must be inside OCaml.

Is there any nice way to force OCaml to flush its output buffers more
often? Is there any way to customize what data OCaml sends to stdout and
what it sends to stderr (this would be a nice feature to have!)? Do any
other ideas come to mind?

Thanks,

  -- Nate Gaylinn

Re: [Caml-list] OCaml IDE (OCaml and stdout/stderr)

[Jacques Garrigue]

From: "Nathaniel J. Gaylinn" <ngaylinn@cs.brown.edu>

> My program is having a hard time coping with how OCaml splits its output
> between stdout and stderr. The order in which a line of output is received
> from OCaml's stdout and a warning from its stderr seems to be completely
> random and therefore my program's output is also random. I'm pretty sure
> this has to do with how OCaml buffers stdout, but I'm not sure how to
> change this.
> 
> To make my program more portable, I'm writing it using the QT toolkit
> which has a nice abstraction of inter-process communication in a class
> called QProcess. This is what I'm using to communicate with OCaml.
> 
> From your experience, do you think the buffering is taking place on
> OCaml's end or on Qt's? Some research seems to suggest that how output is
> buffered from a program can be controlled from within that program (since
> there is direct access to the various output streams) but not from
> outside. If that's true, then the buffering problem must be inside OCaml.
> 
> Is there any nice way to force OCaml to flush its output buffers more
> often? Is there any way to customize what data OCaml sends to stdout and
> what it sends to stderr (this would be a nice feature to have!)? Do any
> other ideas come to mind?

I believe the problem is on QT's side. OCaml is flushing output just
as much as necessary. Otherwise you would have incorrect output even
in a terminal!
The difficulty with GUIs is that they may add their own buffering.
This is also the case with ocamlbrowser on windows for instance,
because input has to be handled by separate threads. Then the main GUI
thread ends up polling buffers filled by these threads. I remember I
used to have problems with this solution, and solved it by having both
the stdout and stderr reader threads write to the same buffer. But I
don't know if you can get that level of access to QT internals...

By the way, do you plan to integrate your IDE in kdevelop (or something
else)?
If it is to be standalone, why not write it in ocaml?

Cheers,

Jacques Garrigue

Re: [Caml-list] OCaml IDE (OCaml and stdout/stderr)

[Nathaniel J. Gaylinn]

On Tue, 21 Jun 2005, Jacques Garrigue wrote:

> By the way, do you plan to integrate your IDE in kdevelop (or something
> else)?
> If it is to be standalone, why not write it in ocaml?

I'm writing my own little IDE, based vaguely on DrScheme, just so that the
students have an easier time transitioning to it. One problem in this
course was that DrScheme was so friendly to program in and then suddenly
we dump them into ocaml using Emacs and Tuareg mode; suddenly they're
confronted with baffling error messages and an interface that is
unintuitive and hard for them to use (Emacs is great! Very powerful, too!
But try teaching a busy college student with almost no computer background
how to use it while having him develop a major project  ;)  Doesn't work
all that well.). My program is meant to be friendlier, more helpful, and
something they can just start using without any training period or having
to memorize key combinations. Hopefully that will make a difference.

As for writing in OCaml, part of me really wishes I was. The problem is, I
don't have extensive experience with OCaml and am not familiar with the
object system or any GUI toolkit interfaces it has. In short, it's much,
much faster for me to write it in C++ using QT (systems I already know
inside and out) than to teach myself the depths of OCaml as I am
writing. Since I'm trying to produce a fully functional and relatively
stable program by the end of the summer, speed is an issue so I chose the
C++ route. I definitely believe that it would be far nicer (at least
conceptually) to be writing in OCaml, though.


  -- Nate

Re: [Caml-list] OCaml IDE (OCaml and stdout/stderr)

[Nathaniel J. Gaylinn]

I've resolved this issue with stdout/stderr. The problem with buffering
was indeed within the QProcess class, and there is a simple solution that
I've already put into use! Thanks for the input, though. I was looking in
the wrong place!

  -- Nate

Re: [Caml-list] OCaml IDE (OCaml and stdout/stderr)

[Brian Hurt]

On Mon, 20 Jun 2005, Nathaniel J. Gaylinn wrote:

>
> My program is having a hard time coping with how OCaml splits its output
> between stdout and stderr. The order in which a line of output is received
> from OCaml's stdout and a warning from its stderr seems to be completely
> random and therefore my program's output is also random. I'm pretty sure
> this has to do with how OCaml buffers stdout, but I'm not sure how to
> change this.

Welcome to programming on Unix.  The function you want is flush (in 
pervasives), with a signature  out_channel -> unit.  So to flush stdout, 
you just do flush stdout.  Same with stderr.  If the order of output is 
important, flush after every I/O.  Note that this a performance hit.

Note that this isn't an Ocaml problem- you can hit the same problem with 
C, C++, Java, Pascal, etc.

Note that being able to seperate out errors from everything else is nice 
when you're shell scripting.  This is why windows is iffy at best about 
really splitting the two different streams.

>> From your experience, do you think the buffering is taking place on
> OCaml's end or on Qt's?

Actually, I beleive it's the C stdio.h layer that doing the buffering. 
All the Ocaml flush function does is call the fflush function in C.

> Is there any nice way to force OCaml to flush its output buffers more
> often? Is there any way to customize what data OCaml sends to stdout and
> what it sends to stderr (this would be a nice feature to have!)? Do any
> other ideas come to mind?

output_string (and _int and _char and etc.) take an out_channel argument, 
allowing you to do:
    output_string stderr "Hello, world!\n"
to write something to stderr instead of stdout.

Brian