[Caml-list] a simple example of compiler-libs.toplevel?

[Caml-list] a simple example of compiler-libs.toplevel?

[Kenichi Asai]

Is there a simple (but complete) example that uses
compiler-libs.toplevel, like creating an OCaml toplevel that uses a
different character for a prompt?  When I tried:

- copy toplevel/topstart.ml to my_topstart.ml and change Topmain in it
  to My_topmain
- copy toplevel/topmain.ml to my_topmain.ml and change Toploop in it
  to My_toploop
- copy toplevel/toploop.ml to my_toploop.ml and change the prompt
  character in it
- optionally, copy toplevel/topdirs.ml to my_topdirs.ml and change
  Toploop, etc., to My_toploop (this does not affect the story below)

I ran into the following error:

The ocamltoplevel.cma library from compiler-libs cannot be loaded
inside the OCaml toplevel

When I disable it (by removing the expression that raises this error
in my_toploop.ml), I can launch the new toplevel, but I cannot define
a new name:

let a = 3

results in "Fatal error: a unbound at toplevel".  Is it possible to do
this kind of creating a new OCaml toplevel using compiler-libs.toplevel?
Or is the compiler-libs.toplevel intended for something else?

A bit of background.  I am developing an OCaml type debugger:

http://pllab.is.ocha.ac.jp/~asai/TypeDebugger/

and am currently trying to port it to a newer version of OCaml.  For
the current type debugger, I inserted code for type debugging into the
main loop in toploop.ml.  However, inserting code directly into the
OCaml source requires me to recompile whole (or at least the core of)
the OCaml.  If I could do the same with compiler-libs, that would be
great.

Any help is welcome.  (I also welcome the negative information saying
that it is impossible to do this kind of thing.  Then, I would fall
back to the old way of compiling whole the OCaml.)

Thank you in advance.

Sincerely,

-- 
Kenichi Asai

Re: [Caml-list] a simple example of compiler-libs.toplevel?

[Jeremie Dimino]

[-- Attachment #1: Type: text/plain, Size: 3915 bytes --]

Hi Kenichi,

It is possible to do this kind of thing. If you only want to change the
character for the prompt, you can set the Toploop.read_interactive_input
function. Then to create a custom toplevel, you need to call [Topmain.main]
and link your program with the compiler-libs.toplevel library. Here is a
complete example where I copied and adapted the default
[read_interactive_input] function from the compiler libraries, and used
dune to build a custom toplevel:

$ cat mytop.ml
let read_input prompt buffer len =
  let prompt =
    if prompt <> "" && prompt.[0] = '#' then
      String.mapi (fun i c -> if i = 0 then '%' else c) prompt
    else
      prompt
  in
  output_string stdout prompt; flush stdout;
  let i = ref 0 in
  try
    while true do
      if !i >= len then raise Exit;
      let c = input_char stdin in
      Bytes.set buffer !i c;
      Option.iter (fun b -> Buffer.add_char b c)
!Location.input_phrase_buffer;
      incr i;
      if c = '\n' then raise Exit;
    done;
    (!i, false)
  with
  | End_of_file ->
      (!i, true)
  | Exit ->
      (!i, false)

let () =
  Toploop.read_interactive_input := read_input;
  Topmain.main ()
$ cat dune
(executable
 (name mytop)
 (libraries compiler-libs.toplevel)
 (link_flags :standard -linkall)
 (modes byte))
$ dune exec ./mytop.exe
        OCaml version 4.11.1

% 1 + 1;;
- : int = 2

FTR, the compiler invocation to build myutop.exe is:
$ ocamlc -linkall -I +compiler-libs ocamlcommon.cma ocamlbytecomp.cma
ocamltoplevel.cma mytop.ml -o mytop.exe
$ ./mytop.exe
        OCaml version 4.11.1

% 1 + 1;;
- : int = 2

If you want to go further, there are few projects that add line edition
support to the toplevel, colors etc, using similar methods:
- utop: https://github.com/ocaml-community/utop
- down: https://erratique.ch/software/down

Finally, if you'd like to improve the toplevel API so that doing this kind
of thing is easier in the future, a PR is always welcome :)

Happy hacking,

Jeremie



On Fri, Mar 5, 2021 at 5:05 AM Kenichi Asai <asai@is.ocha.ac.jp> wrote:

> Is there a simple (but complete) example that uses
> compiler-libs.toplevel, like creating an OCaml toplevel that uses a
> different character for a prompt?  When I tried:
>
> - copy toplevel/topstart.ml to my_topstart.ml and change Topmain in it
>   to My_topmain
> - copy toplevel/topmain.ml to my_topmain.ml and change Toploop in it
>   to My_toploop
> - copy toplevel/toploop.ml to my_toploop.ml and change the prompt
>   character in it
> - optionally, copy toplevel/topdirs.ml to my_topdirs.ml and change
>   Toploop, etc., to My_toploop (this does not affect the story below)
>
> I ran into the following error:
>
> The ocamltoplevel.cma library from compiler-libs cannot be loaded
> inside the OCaml toplevel
>
> When I disable it (by removing the expression that raises this error
> in my_toploop.ml), I can launch the new toplevel, but I cannot define
> a new name:
>
> let a = 3
>
> results in "Fatal error: a unbound at toplevel".  Is it possible to do
> this kind of creating a new OCaml toplevel using compiler-libs.toplevel?
> Or is the compiler-libs.toplevel intended for something else?
>
> A bit of background.  I am developing an OCaml type debugger:
>
> http://pllab.is.ocha.ac.jp/~asai/TypeDebugger/
>
> and am currently trying to port it to a newer version of OCaml.  For
> the current type debugger, I inserted code for type debugging into the
> main loop in toploop.ml.  However, inserting code directly into the
> OCaml source requires me to recompile whole (or at least the core of)
> the OCaml.  If I could do the same with compiler-libs, that would be
> great.
>
> Any help is welcome.  (I also welcome the negative information saying
> that it is impossible to do this kind of thing.  Then, I would fall
> back to the old way of compiling whole the OCaml.)
>
> Thank you in advance.
>
> Sincerely,
>
> --
> Kenichi Asai
>


-- 
Jeremie

[-- Attachment #2: Type: text/html, Size: 7107 bytes --]

Re: [Caml-list] a simple example of compiler-libs.toplevel?

[Kenichi Asai]

Thanks for the e-mail, Jeremie!

Your example first modifies Toploop.read_interactive_input and then
calls the original Topmain.main.  Because toploop.ml had a hook for
reading interactive input, we could change the character for the
prompt.

A side question.  How can I find what hooks are available?  By looking
at the source and finding what functions are defined as refs?  (I find
that parse_toplevel_phrase, parse_use_file, and toplevel_startup_hook
are defined as refs in toploop.ml, so they can be replaced?)

What if there is no such hook?  For the type debugger, I want to call
a specific function whenever a type error occurs in the second
execute_phrase (right after "with exn ->") in toploop.ml.  But there
is no hook there.  Since the execute_phrase function is not defined as
a ref, I cannot replace it, either.  What can I do then?

-- 
Kenichi Asai

Re: [Caml-list] a simple example of compiler-libs.toplevel?

[Jeremie Dimino]

[-- Attachment #1: Type: text/plain, Size: 954 bytes --]

On Fri, Mar 5, 2021 at 2:04 PM Kenichi Asai <asai@is.ocha.ac.jp> wrote:

> A side question.  How can I find what hooks are available?  By looking
> at the source and finding what functions are defined as refs?  (I find
> that parse_toplevel_phrase, parse_use_file, and toplevel_startup_hook
> are defined as refs in toploop.ml, so they can be replaced?)
>

That's the idea. Only the refs exposed in the mli can be set from the
outside though.


> What if there is no such hook?  For the type debugger, I want to call
> a specific function whenever a type error occurs in the second
> execute_phrase (right after "with exn ->") in toploop.ml.  But there
> is no hook there.  Since the execute_phrase function is not defined as
> a ref, I cannot replace it, either.  What can I do then?
>

If there is no hook, I'm afraid there is no good solution at the moment;
you need to patch the toplevel library, or copy some of its code and modify
it.

-- 
Jeremie

[-- Attachment #2: Type: text/html, Size: 2121 bytes --]