Re: [Caml-list] What does Jane Street use/want for an IDE? What about you?

[Kuba Ober <ober.14@osu.edu>]

On Tuesday 04 November 2008, Jon Harrop wrote:
> On Tuesday 04 November 2008 18:35:45 Kuba Ober wrote:
> > On Monday 03 November 2008, Jon Harrop wrote:
> > > On Monday 03 November 2008 14:15:38 Kuba Ober wrote:
> > > > I could port Camelia to OCaml if
> > > > someone would first develop Qt bindings for OCaml that would allow me
> > > > to do in OCaml what I'm doing in C++ so far ;)
> > >
> > > That may already be possible if you go via more mainstream dynamic
> > > languages like Python. However, python-qt4-dev has fewer installs on
> > > Debian and Ubuntu than OCaml does, so you may well find that the Python
> > > bindings are inadequate as well.
> >
> > Python Qt bindings are pretty much feature-complete, but C++'s class
> > system maps perhaps better to Python/LISP than to OCaml?
>
> You'll just be invoking autogenerated Python code using OCaml so OCaml's
> class system is only relevant if you want to do some fancy statically-typed
> shim. I'd forget about that and just focus on making the whole of Qt4
> available safely from OCaml in any form first. Even this is an unsolved
> problem in the OCaml world!

Python bindings for Qt extensively use Python's API. You'd need OCaml to
provide said API first.

> > Since Camelia uses so much of Qt's functionality, this is a task on par
> > with reimplementing Qt. Good luck with that ;)
>
> From Qt4's architecture diagram:
>
>  
> http://trolltech.com/images/template/product-architecture-diagram-collapsed
>
>   Core, GUI, database, scripting, network, OpenGL, XML, Multimedia, Font
> Engine and Webkit
>
> Of those, only core and GUI might be relevant here but the core lacks
> first-class functions for events and callbacks 

Maybe they are not first-class, but the way they've done it is via 
preprocessing using moc and things behave like they were first-class.

In Qt, if you have a user interface dialog box (a form) called MyForm
with an element named say ExitButton, you'd have this:

class MyForm : public QDialog {
Q_OBJECT
	Ui::MyForm ui;

	MyForm::MyForm(QWidget * parent) : QDialog(parent) {
	  ui.setupUi(this); // calls code autogenerated by UI compiler
        }

public slots:
        void on_ExitButton_clicked() {
           ui.ExitButton->hide()
        }
};

The code is hopefully easy to understand and as you can see you don't even
have to connect signals and slots manually if you don't want to. All the
magic happens behind the scenes. It's very easy to use.

> and the GUI is unable to 
> express integrated graphics (e.g. OpenGL layers).

You're talking like everyone was after integrated graphics. You are and
I understand your viewpoint, but say I'm not :)

> So the amount of Qt that is objectively beneficial is tiny and the cost of
> using it is very high.

I really doubt that.

> > > > Qt 4 is leaps and bounds above anything gtk provides, in
> > > > terms of integrated functionality. This is not meant as a flamebait,
> > > > I believe it to be an accurate statement of fact.
> > >
> > > And WPF is leaps and bounds above anything Qt 4 provides, in terms of
> > > functionality, integration and performance. Combined with the fact that
> > > using Qt4 from a decent language is very hard and basically completely
> > > pointless by design anyway, I think there is a strong argument for
> > > starting from scratch.
> >
> > WPF is just another thing, and it's not really portable in any decent
> > sense of the word.
>
> Sure. My point is that you're building on antique technology when you could
> be building modern technology instead.

With Qt there's a real chance of machine-translating the code to say LISP
or OCaml or what have you. With WPF, you're stuck with .Net and that's the end
of it.

> > Camelia could be run on a 486 system with perhaps 64MB of
> > memory when compiled using Qt/Embedded. While one can question whether
> > it's useful or not, Qt has the benefit of targeting all major computing
> > platforms out there.
>
> That sales hype for Qt is just a reflection of the fact that Qt was built
> on sand using a dying technology (C++). In practice, the world moved on to
> garbage collected virtual machines for GUI programming years ago. Qt was
> left behind and is desperately trying to catch up by migrating to the JVM
> but the JVM is now dying...

Qt has not migrated to JVM. Relevant Qt classes implement parent-child
ownership, implicit copy-on-write and reference counting, and are used
just like you would any random collection of things in a garbage-collected
language. You're also free to use a garbage collector with any C++ project,
this has nothing to do with Qt.

> > Using Qt4 in a decent language will be easy once there is a C++ parser in
> > said decent language.
>
> You'll be Greenspunning managed C++ from the .NET world. That's a fine idea
> if you regard Qt as useful legacy code but I don't see the point myself. Qt
> will always have a stone-age API and reimplementing managed C++ is a huge
> undertaking.

This is not about greenspunning anything. Most of Qt code is written in such 
way that the C++-isms are completely abstracted out at the level of core
classes such as QObject and collections (QList, QVector, etc). By translating
the use of basic Qt primitives (really QObject and collections) to whatever
OCaml/Lisp/Python/blabla provides, you can very cleanly port Qt codebase
to another language, and it will look pretty much as if it were written in
said language.

> In contrast, you can implement a GUI toolkit in OCaml that far exceeds the
> relevant limitations of Qt4 with quite easily.

Qt has very few language-inherent limitations that are not abstracted out to
the core. Whatever other limitations Qt has are design decisions that have
little to do with choice of the language.

> > I don't know if I mentioned it, but some time ago I
> > did a proof-of-concept of machine-translating QObject.cpp and friends to
> > LISP and it not only worked, but required no moc ;) So I'd say that as
> > soon as there's a good-enough C++ parser written in OCaml, and given
> > availability of a basic AI package (to implement a KB and searches), it
> > should be perfectly doable to translate Qt4 into human-readable,
> > not-too-bad OCaml.
>
> Then the question is simply whether it is easier to write a C++ parser in
> OCaml along with the translation code and a sane shim over the whole of Qt4
> or just replace it wholesale.

Qt has to cover a lot of platform-specific ground. Reimplementing it from
scratch implies having to fix most of the bugs that Qt folks have already
fixed. Qt's code base as a fine reference for a lot of platform-specific
borkedness that Qt's users have run into, and the trolls have fixed.

> > > > I wish I could use gtk
> > > > since it's easier to bind with, but I'd end up having to either
> > > > reimplement parts of Qt, or have to deal with lots of 3rd party
> > > > libraries, each from a different author who had a different API
> > > > design mindset. Qt takes care of those integration issues internally
> > > > and provides a relatively
> > > > self-consistent API -- this saves a metric crapton of time (I use the
> > > > darn thing).
> > >
> > > Swings and roundabouts. The metric crapton of time you save with Qt4's
> > > functionality is paid by having to use an awful language.
> >
> > Let's not flame about it,
>
> I would not regard that as flame bait here. I'm sure if OCaml had decent Qt
> bindings they would be very heavily used indeed (the OCaml language would
> excel at GUI programming, which is precisely why so many people here are
> yearning for such bindings) but Qt4 is nowhere near useful enough to make
> me want to go back to C++ or any language that does have Qt bindings.
>
> From what I've heard, a lot of OCaml programming use OpenGL for GUI
> programming instead of GUI toolkits.

Show me a generic, reuseable OCaml-written, OpenGL-based common widgets such
as model-view tables, buttons, editors, tree views, ... Then we'll talk.

> > > > > . Graphical throwback of the documentation related to what is under
> > > > > the mouse pointer.
> > > >
> > > > Easy to do -- goes on my to-do list.
> > >
> > > How would you implement it?
> >
> > 1. Everything has type annotations.
> > 2. I know scope of symbols from my simple parser.
> >
> > It's easy enough to look things up by name+context and type, right?
>
> You require the programmer to write type annotations everywhere?

OCaml does produce type annotations for you :) Camelia uses those.

> > > > > . Graphical throwback of errors and, in the case of type errors,
> > > > > optional highlighting of previous unification points.
> > > >
> > > > Camelia does that.
> > >
> > > Cool. AFAIK, OCaml does not export information about unification
> > > points. So how do you get hold of that data in Camelia?
> >
> > I was probably wrong about unification points. What are they?
>
> The points in the program where the type checker unified two or more types
> in order to check type correctness, e.g. the two branches of an "if"
> expression.
>
> When the programmer makes a mistake, the type checker can be misled early
> in the code but report an actual type error later in the code. So it can be
> helpful (particularly for newbies) to see where the type checker unified
> because the error will almost certainly have been at one of those points.
>
> For example, a function to sum a list of floats:
>
>   let rec sum = function
>
>     | [] -> 0
>     | h::t -> h +. sum t
>
> The type error is reported at the "+." operator because the type was
> incorrectly inferred to be "int" when the type checker came across the "0"
> in the first clause. Highlighting the unification points of that type
> immediately leads the programmer to the point of error. Without this, the
> programmer has no idea where the error really is.

So, OCaml compiler itself doesn't produce that? It'd be easier to add it to
OCaml, than to redo it from scratch.

> > > > > . Parallel so seven of my cores aren't idle while I'm waiting.
> > > >
> > > > Everything is done in one thread right now, but over time it can be
> > > > spread out. Implementing this properly requires refectoring of the
> > > > code first: right now Camelia is in no shape to attempt that. After
> > > > 2.0 release there will be time to attack that.
> > >
> > > It is trivial if you can pull the sources of the OCaml compiler into
> > > your IDE and if OCaml has a parallel GC. ;-)
> >
> > Doing even that requires a lot of refactoring. Camelia 1.1 had its OCaml-
> > awareness spread about everywhere, so before anything major happens I had
> > to get all language-specific stuff abstracted out anyway. I'm in the
> > process of doing that.
>
> If you want your IDE to support multiple languages, yes. My preference
> would be to write an IDE specifically for OCaml first.

Sure, but that should not imply that the code is a mess like it is right now.

> > > It is a logical progression from building your own GUI toolkit. You
> > > represent graphics using scene graphs. Everything in an interactive
> > > session is converted into scene graphs for rendering.
> > >
> > > So:
> > >
> > >   val pretty_printer : Expr.t -> string
> > >
> > > becomes:
> > >
> > >   val pretty_printer : Expr.t -> Scene.t
> > >
> > > and the resulting Scene.t is fed through a renderer like Smoke.
> >
> > 1. How exactly do you see the pretty_printer rendering things?
>
> Easiest for me to answer by just posting our code. Here is the code from
> the current version of Smoke that typesets paragraphs of text by
> overflowing words onto new lines:
>
> let rec charsmap = function
>
>     | 'f' :: 'f' :: 'i' :: t -> (`CMR, 177) :: charsmap t
>     | 'f' :: 'f' :: 'l' :: t -> (`CMR, 178) :: charsmap t
>     | 'f' :: 'f' :: t -> (`CMR, 174) :: charsmap t
>     | 'f' :: 'i' :: t -> (`CMR, 175) :: charsmap t
>     | 'f' :: 'l' :: t -> (`CMR, 176) :: charsmap t
>     | c :: t -> charmap c :: charsmap t
>     | [] -> []
>
> let word string =
>   let string = charsmap(list_of_string string) in
>   let aux x (font1, c1) (font2, c2) =
>     let contours = Cmfont.glyph font1 c1 in
>     let contours = List.map Contour.make contours in
>     let g = List.map (Contour.trans (translate x 0.)) contours in
>     let advance =
>       if font1=font2 then Cmfont.advance font1 c1 c2 else
>         Cmfont.advance `CMR 32 32 in
>     x +. advance, g in
>   let rec aux2 x = function
>
>     | [] -> []
>     | h1::t ->
>
>         let h2 = match t with h2::_ -> h2 | _ -> `CMR, 32 in
>         let x, g = aux x h1 h2 in
>         g :: aux2 x t in
>   aux2 0. string |> List.concat |> ContourGeometry.of_contours
>
> let word = memoize (fun _ -> word)
>
> let paragraph ~skip ~justify ~width ~string =
>   let words = List.map word (String.split ' ' string) in

LOL. If you seriously think it's worth reimplementing the wheel for
a millionth time, go for it. But I don't think this would be the
best way to spend my time just to get text rendered using OpenGL.

Qt already can do rendering in OpenGL, so if one wants to do something
that actually really needs OpenGL, it can be done.

But to just render a paragraph of text? There's code for it in Qt ;)

As for showing more complex flat views of data, one can use QGraphicsView
& friends.

Have you ran recent Qt demos distributed with Qt? I'd say they look pretty
cool in my book.

>
> > 2. How easy it is to replace pretty_printer built into OCaml?
>
> Already trivial:
>
> $ ocaml
>         Objective Caml version 3.10.2
>
> # 3.4;;
> - : float = 3.4
> # open Format;;
> # let print_float ff x = fprintf ff "%0.20f" x;;
> val print_float : Format.formatter -> float -> unit = <fun>
> # #install_printer print_float;;
> # 3.4;;
> - : float = 3.39999999999999991118
>
> However, I'd rather OCaml carried run-time type information and the pretty
> printer associated run-time types with typesetting functions.
>
> > I presume that the running OCaml instance (toplevel) would
> > dump scene graphs, and then a separate OCaml instance would
> > take those and render them?
>
> I would just replace the backend of the current top-level to generate
> graphics instead of text.
>
> > > > Processing toplevel output is an issue nicely orthogonal to editor
> > > > and knowledgebase, so this doesn't block on the major refactoring
> > > > that has to happen on the editor end.
> > >
> > > Yes and no. I assume your editor is completely hard-wired from the
> > > ground up for editing plain text and maybe even unicode but is
> > > completely incapable of rendering arbitrary vector graphics. If so, it
> > > will need to be completely gutted.
> >
> > Well, the thing about toplevel is that it's completely separate from the
> > actual source editor.
>
> In the current design, yes.
>
> > So I can actually replace toplevel at any time by simply substituting a
> > different widget for it. The toplevel widget talks to a wrapper around
> > the real OCaml process, which would be re-used.
>
> You really want to preserve information (e.g. types) in the graphical
> output for throwback to work so I would go for tighter integration of the
> top-level and IDE.

All of that can certainly be done in the future.

Kuba