Re: [Caml-list] What does Jane Street use/want for an IDE? What about you?

[Kuba Ober <ober.14@osu.edu>]

On Monday 03 November 2008, Jon Harrop wrote:
> On Monday 03 November 2008 14:15:38 Kuba Ober wrote:
> > On Friday 31 October 2008, Jon Harrop wrote:
> > > . Written in OCaml using OCaml's own lexer and parser to save effort
> > > and make it possible for others to help develop it without losing their
> > > hair.
> >
> > That's perhaps possible in the longer run by linking some OCaml code in.
> > Right now Camelia has its own parser.
>
> You reimplemented the whole OCaml parser in C++? Does it handle Camlp4
> syntax extensions?

It's only good enough for simple things, like looking up symbols and figuring 
out in which context does a symbol appear. Heavy lifting like type inference 
etc. is left to OCaml, and its type annotations are used extensively. Even
though I see no problem with having a real OCaml parser. Type inference
notwithstanding, of course.

> > I could port Camelia to OCaml if
> > someone would first develop Qt bindings for OCaml that would allow me to
> > do in OCaml what I'm doing in C++ so far ;)
>
> That may already be possible if you go via more mainstream dynamic
> languages like Python. However, python-qt4-dev has fewer installs on Debian
> and Ubuntu than OCaml does, so you may well find that the Python bindings
> are inadequate as well.

Python Qt bindings are pretty much feature-complete, but C++'s class system
maps perhaps better to Python/LISP than to OCaml?

> > That's an undertaking bigger than Camelia itself. I will not develop for
> > gtk-anything as lost feature parity with Qt right around the time when Qt
> > 3 came around, IIRC.
>
> I can well believe that. Another option is to create a new GUI toolkit from
> scratch in OCaml.

Since Camelia uses so much of Qt's functionality, this is a task on par with
reimplementing Qt. Good luck with that ;)

> > Qt 4 is leaps and bounds above anything gtk provides, in
> > terms of integrated functionality. This is not meant as a flamebait, I
> > believe it to be an accurate statement of fact.
>
> And WPF is leaps and bounds above anything Qt 4 provides, in terms of
> functionality, integration and performance. Combined with the fact that
> using Qt4 from a decent language is very hard and basically completely
> pointless by design anyway, I think there is a strong argument for starting
> from scratch.

WPF is just another thing, and it's not really portable in any decent sense
of the word. Camelia could be run on a 486 system with perhaps 64MB of memory
when compiled using Qt/Embedded. While one can question whether it's useful
or not, Qt has the benefit of targeting all major computing platforms out 
there.

Using Qt4 in a decent language will be easy once there is a C++ parser in said
decent language. I don't know if I mentioned it, but some time ago I did
a proof-of-concept of machine-translating QObject.cpp and friends to LISP and
it not only worked, but required no moc ;) So I'd say that as soon as there's
a good-enough C++ parser written in OCaml, and given availability of a basic
AI package (to implement a KB and searches), it should be perfectly doable to
translate Qt4 into human-readable, not-too-bad OCaml.

> > I wish I could use gtk
> > since it's easier to bind with, but I'd end up having to either
> > reimplement parts of Qt, or have to deal with lots of 3rd party
> > libraries, each from a different author who had a different API design
> > mindset. Qt takes care of those integration issues internally and
> > provides a relatively
> > self-consistent API -- this saves a metric crapton of time (I use the
> > darn thing).
>
> Swings and roundabouts. The metric crapton of time you save with Qt4's
> functionality is paid by having to use an awful language.

Let's not flame about it, but as a pure exercise I have implemented everything
(as in all presented algorithms/approacheds) for three "major" courses from
www.osu.edu (MATH707/727, CSE630 and CSE680), in LISP, OCaml and C++.
The code in each case has similar readability and uses similarly high-level
concepts. I'd say that if you think in OCaml or LISP and implement in C++,
you can still get pretty good code ;) Of course some LISP/OCaml-isms are
unavailable in C++, but then some other ones can be implemented with relative
cleanliness using available template metaprogramming.

> > > . Graphical throwback of the documentation related to what is under the
> > > mouse pointer.
> >
> > Easy to do -- goes on my to-do list.
>
> How would you implement it?

1. Everything has type annotations.
2. I know scope of symbols from my simple parser.

It's easy enough to look things up by name+context and type, right?

> > > . Graphical throwback of errors and, in the case of type errors,
> > > optional highlighting of previous unification points.
> >
> > Camelia does that.
>
> Cool. AFAIK, OCaml does not export information about unification points. So
> how do you get hold of that data in Camelia?

I was probably wrong about unification points. What are they?

> > > . Autocompletion that handles structural types in LablGTK code and
> > > operators, i.e. when the user types "+" present options for int "+",
> > > float "+.", num "+/" and any other operators that begin with "+".
> >
> > Version 2.2 ;)
>
> I believe that kind of IDE support would allow a language like OCaml (i.e.
> without overloading) to handle arithmetic over many types gracefully.

I agree.

> > > . Represent an OCaml project as a tree of modules that happen to have
> > > the first level stored as files.
> >
> > Level 1 is easy to do, but what's on the second and deeper levels in your
> > idea?
>
> The first level is defined by the filename and all subsequent levels are
> defined as nested modules within a file. I would not mind if the IDE
> abstracted away the concept of files completely: they have no place in
> modern programming IMHO.

OK, got it. Since I parse things already, I know all about modules so it
should be easy to do. I don't know if it'd be v2.0 or 2.1, though.

> > > . Performant enough to handle projects with hundreds of thousands of
> > > lines of code.
> >
> > Shouldn't be a problem, perhaps as long as you don't put all the lines in
> > one file ;)
> >
> > In the long run I may switch to using QCodeEdit from edyuk, which is
> > supposedly better performing than QPlainTextEdit.
>
> Until you get a significant user base I wouldn't worry about it.

That's what I was thinking, too.

> > > . Parallel so seven of my cores aren't idle while I'm waiting.
> >
> > Everything is done in one thread right now, but over time it can be
> > spread out. Implementing this properly requires refectoring of the code
> > first: right now Camelia is in no shape to attempt that. After 2.0
> > release there will be time to attack that.
>
> It is trivial if you can pull the sources of the OCaml compiler into your
> IDE and if OCaml has a parallel GC. ;-)

Doing even that requires a lot of refactoring. Camelia 1.1 had its OCaml-
awareness spread about everywhere, so before anything major happens I had
to get all language-specific stuff abstracted out anyway. I'm in the process
of doing that.

> > > . The ability to hide the tail of +., +/ etc. operators to make
> > > numerical code more readable.
> >
> > Easy to do, can go in for 2.0.
>
> You need type throwback on operators as well, of course.

> > > Only one: an interactive top-level where output is presented via
> > > graphical elements (e.g. a scene graph) and is no longer limited to
> > > just ASCII text. This would give OCaml the graphical capabilities of
> > > Mathematica's awesome "notebook" front-end.
> >
> > How would you like that to work?
>
> It is a logical progression from building your own GUI toolkit. You
> represent graphics using scene graphs. Everything in an interactive session
> is converted into scene graphs for rendering.
>
> So:
>
>   val pretty_printer : Expr.t -> string
>
> becomes:
>
>   val pretty_printer : Expr.t -> Scene.t
>
> and the resulting Scene.t is fed through a renderer like Smoke.

1. How exactly do you see the pretty_printer rendering things?
2. How easy it is to replace pretty_printer built into OCaml?

I presume that the running OCaml instance (toplevel) would
dump scene graphs, and then a separate OCaml instance would
take those and render them?

> Look at Mathematica (and not ddd) for inspiration.

We have it at school, so I'll give it a look.

> > Processing toplevel output is an issue nicely orthogonal to editor and
> > knowledgebase, so this doesn't block on the major refactoring that has
> > to happen on the editor end.
>
> Yes and no. I assume your editor is completely hard-wired from the ground
> up for editing plain text and maybe even unicode but is completely
> incapable of rendering arbitrary vector graphics. If so, it will need to be
> completely gutted.

Well, the thing about toplevel is that it's completely separate from the
actual source editor.

So I can actually replace toplevel at any time by simply substituting a 
different widget for it. The toplevel widget talks to a wrapper around the
real OCaml process, which would be re-used.

Cheers, Kuba