[Caml-list] Literate (sort-of) programming in OCaml

[Caml-list] Literate (sort-of) programming in OCaml

[Jonathan Protzenko]

Hi,

I am currently writing a big, mostly textual document in format blah 
(blah being of a course a meta-variable). Are there any tools that 
would allow me to interleave OCaml code with the contents of my 
document?

More on my use-case. Creating this document requires me to perform 
various subtasks, such as:
- write a code snippet in the document,
- put the code snippet in a file,
- call an external program on the file,
- paste the output into the document.

Naturally, I wish to automate this. I could roll my own set of 
commands, and parse them with OCaml, but I would be re-creating a 
scripting language, and it seems to me that interleaving OCaml code 
within my document would be better. Here's what I have in mind:


blah blah blah blah blah blah blah blah blah blah blah blah blah blah 
blah blah
blah blah blah blah blah blah blah blah blah blah blah blah blah blah 
blah blah
blah ... we thus write the following code ... :

{%
  let code = "<sample code>" in
  output code
%}

blah blah blah ... after running the command blah ... the output is as 
follows ... blah

{%
  let f = write_into_temp_file code in
  let s = run_and_read "myprogram" [f] in
  output s
%}

blah blah blah blah blah blah blah blah blah blah blah blah blah blah 
blah blah ...


Are you aware of any tool that would allow me to achieve this?

Thanks,

~ jonathan

Re: [Caml-list] Literate (sort-of) programming in OCaml

[Alain Frisch]

On 09/02/2013 05:39 PM, Jonathan Protzenko wrote:
> blah blah blah blah blah blah blah blah blah blah blah blah blah blah
> blah blah
> blah blah blah blah blah blah blah blah blah blah blah blah blah blah
> blah blah
> blah ... we thus write the following code ... :
>
> {%
>    let code = "<sample code>" in
>    output code
> %}
>
> blah blah blah ... after running the command blah ... the output is as
> follows ... blah
>
> {%
>    let f = write_into_temp_file code in
>    let s = run_and_read "myprogram" [f] in
>    output s
> %}
>
> blah blah blah blah blah blah blah blah blah blah blah blah blah blah
> blah blah ...

I'm sure you can use sed to (I use '' for meta quotes):

  - Add ''print_string"'' in front of the file and ''";;'' at the end.
  - Replace ''{%'' with ''";;'' and ''%}'' with ''";; print_string"''

If you apply this transformation and call the toplevel on the result, 
you should get what you want, no?

-- Alain

Re: [Caml-list] Literate (sort-of) programming in OCaml

[Török Edwin]

On 09/02/2013 06:39 PM, Jonathan Protzenko wrote:
> Hi,
> 
> I am currently writing a big, mostly textual document in format blah 
> (blah being of a course a meta-variable). Are there any tools that 
> would allow me to interleave OCaml code with the contents of my 
> document?
> 
> More on my use-case. Creating this document requires me to perform 
> various subtasks, such as:
> - write a code snippet in the document,
> - put the code snippet in a file,
> - call an external program on the file,
> - paste the output into the document.
> 
> Naturally, I wish to automate this. I could roll my own set of 
> commands, and parse them with OCaml, but I would be re-creating a 
> scripting language, and it seems to me that interleaving OCaml code 
> within my document would be better. Here's what I have in mind:
> 
> 
> blah blah blah blah blah blah blah blah blah blah blah blah blah blah 
> blah blah
> blah blah blah blah blah blah blah blah blah blah blah blah blah blah 
> blah blah
> blah ... we thus write the following code ... :
> 
> {%
>   let code = "<sample code>" in
>   output code
> %}
> 
> blah blah blah ... after running the command blah ... the output is as 
> follows ... blah
> 
> {%
>   let f = write_into_temp_file code in
>   let s = run_and_read "myprogram" [f] in
>   output s
> %}
> 
> blah blah blah blah blah blah blah blah blah blah blah blah blah blah 
> blah blah ...
> 
> 
> Are you aware of any tool that would allow me to achieve this?

http://zoggy.github.io/stog/posts/ocaml-sessions.html
https://github.com/realworldocaml/scripts

There are probably others.

Best regards,
--Edwin

Re: [Caml-list] Literate (sort-of) programming in OCaml

[Roberto Di Cosmo]

Hi Jonathan,
   for writing course notes, many of us used in the old times
a script (written in ... cough, cough .. Perl) that sort of
answered your needs.

I honestly do not remember who from the old Formel team wrote
it for CamlLight in the first place, and who adapted it to OCaml,
but you can see the old code in a message from Xavier in 1996 :-)

http://caml.inria.fr/pub/old_caml_site/caml-list-ar/0651.html

Here is the version I still use to produce course notes with a recent
OCaml and lstlistings environments (that knows how to highlight OCaml syntax).

One writes a source LaTeX file with environments like

\begin{caml_example}
let x = "this code will be executed, and printed out with toplevel output"
\end{caml_example}


\begin{caml_example*}
let x = "this code will be executed, and printed out without toplevel output"
\end{caml_example*}

\begin{caml_eval}
let x = "this code will be executed, but not printed out"
\end{caml_eval}

and passes it through ocaml-tex to get the LaTeX file with the results
properly inlined.

Ah, by the way, beware, this old code doew not do a very good job at identifying
phrases, so you need to use ;; all too often in the code.

Having a better (OCaml?) version with proper handling of all this would
certainly be nice.

--
Roberto


#!/usr/bin/perl
# ocaml-tex

$camllight = "TERM=dumb ocaml";
$camlbegin = "\\begin{lstlisting}[escapechar=§]\n";
$camlend = "\\end{lstlisting}\n";
$camlin = "";
$camloutstart = "§\\camloutbol §";
$camloutstop = "§\\camlouteol §";
$camlblank = "\n";

$linelen = 72;
$output = "";
$cut_at_blanks = 0;

while ($#ARGV >= 0) {
  $_ = $ARGV[0];
  last unless (/^-/);
  $linelen = $ARGV[1], shift, shift, next    if (/^-n$/);
  $output  = $ARGV[1], shift, shift, next    if (/^-o$/);
  $camllight = $ARGV[1], shift, shift, next    if (/^-caml$/);
  $cut_at_blanks = 1, shift, next            if (/^-w$/);
  printf STDERR ("Unknown option '%s', ignored\n", $_);
  shift;
}

# First pass: extract the Caml phrases to evaluate

open(ML, "> .input.ml") || die("Cannot create .input.ml : $!");

foreach $infile (@ARGV) {
  open(IN, $infile) || die("Cannot open $infile : $!");
  while(<IN>) {
    if (m/^\\begin{caml_(example|example\*|eval)}\s*$/) {
      while(<IN>) {
        last if m/^\\end{caml_(example|example\*|eval)}\s*$/;
        print ML $_;
      }
    }
  }
  close(IN);
}

close(ML);

# Feed the phrases to a Caml toplevel

open(TOPLEVEL, "$camllight 2>&1 < .input.ml |") ||
       die("Cannot start camllight : $!");

<TOPLEVEL>; <TOPLEVEL>;		# skip the banner
$lastread = <TOPLEVEL>;
$lastread =~ s/^# //;

# Second pass: shuffle the TeX source and the output of the toplevel

if ($output) {
  if ($output eq "-") {
    open(OUT, ">&STDOUT");
  } else {
    open(OUT, ">$output") || die("Cannot create $output: $!");
  }
}

foreach $infile (@ARGV) {
  open(IN, $infile) || die("Cannot open $infile: $!");
  if (! $output) {
    $outfile = $infile;
    $outfile =~ s/\.tex$//;
    open(OUT, "> $outfile.ml.tex") || die("Cannot create $outfile.ml.tex: $!");
  }
  while(<IN>) {
    if (m/^\\begin{caml_example(\*?)}\s*$/) {
      $omit_answer = $1;     # true if caml_example*, false if caml_example
      print OUT $camlbegin;
      $severalphrases = 0;
      while(<IN>) {
        last if m/\\end{caml_example\*?}\s*$/;
        print OUT $camlblank if ($severalphrases);
        while(1) {
          s/\\/\\\\/g;
          print OUT $camlin, $_;
          last if m/;; *$/;
          $_ = <IN>;
        }
	while ($lastread =~ s/^  //) { }
        while($lastread) {
          last if $lastread =~ s/^# //;
          print STDERR $lastread;
          if (! $omit_answer) {
            while (length($lastread) > $linelen) {
              if ($cut_at_blanks) {
                $cutpos = rindex($lastread, ' ', $linelen);
                if ($cutpos == -1) { $cutpos = $linelen; } else { $cutpos++; }
              } else {
                $cutpos = $linelen;
              }
              $line = substr($lastread, 0, $cutpos);
              $line =~ s/\\/\\\\/g;
              print OUT $camloutstart, $line, $camloutstop, "\n";
              $lastread = substr($lastread, $cutpos,
                                 length($lastread) - $cutpos);
            }
            $lastread =~ s/\\/\\\\/g;
            print OUT $camloutstart, $lastread, $camloutstop;
	  }
          $lastread = <TOPLEVEL>;
        }
        $severalphrases = 1;
      }
      print OUT $camlend;
    }
    elsif (m/^\\begin{caml_eval}\s*$/) {
      while(<IN>) {
        last if m/^\\end{caml_eval}\s*$/;
        if (m/;; *$/) {
	  while ($lastread =~ s/^#//) { }
	  while($lastread) {
	    last if $lastread =~ s/^#//;
	    print $lastread;
	    $lastread = <TOPLEVEL>;
	  }
        }
      }
    }
    else {
      print OUT $_;
    }
  }
  close(IN);
}

close(TOPLEVEL);




On Mon, Sep 02, 2013 at 05:39:35PM +0200, Jonathan Protzenko wrote:
> Hi,
> 
> I am currently writing a big, mostly textual document in format blah 
> (blah being of a course a meta-variable). Are there any tools that 
> would allow me to interleave OCaml code with the contents of my 
> document?
> 
> More on my use-case. Creating this document requires me to perform 
> various subtasks, such as:
> - write a code snippet in the document,
> - put the code snippet in a file,
> - call an external program on the file,
> - paste the output into the document.
> 
> Naturally, I wish to automate this. I could roll my own set of 
> commands, and parse them with OCaml, but I would be re-creating a 
> scripting language, and it seems to me that interleaving OCaml code 
> within my document would be better. Here's what I have in mind:
> 
> 
> blah blah blah blah blah blah blah blah blah blah blah blah blah blah 
> blah blah
> blah blah blah blah blah blah blah blah blah blah blah blah blah blah 
> blah blah
> blah ... we thus write the following code ... :
> 
> {%
>   let code = "<sample code>" in
>   output code
> %}
> 
> blah blah blah ... after running the command blah ... the output is as 
> follows ... blah
> 
> {%
>   let f = write_into_temp_file code in
>   let s = run_and_read "myprogram" [f] in
>   output s
> %}
> 
> blah blah blah blah blah blah blah blah blah blah blah blah blah blah 
> blah blah ...
> 
> 
> Are you aware of any tool that would allow me to achieve this?
> 
> Thanks,
> 
> ~ jonathan
> 
> -- 
> Caml-list mailing list.  Subscription management and archives:
> https://sympa.inria.fr/sympa/arc/caml-list
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs

-- 
Roberto Di Cosmo
 
------------------------------------------------------------------
Professeur               En delegation a l'INRIA
PPS                      E-mail: roberto@dicosmo.org
Universite Paris Diderot WWW  : http://www.dicosmo.org
Case 7014                Tel  : ++33-(0)1-57 27 92 20
5, Rue Thomas Mann       
F-75205 Paris Cedex 13   Identica: http://identi.ca/rdicosmo
FRANCE.                  Twitter: http://twitter.com/rdicosmo
------------------------------------------------------------------
Attachments:
MIME accepted, Word deprecated
      http://www.gnu.org/philosophy/no-word-attachments.html
------------------------------------------------------------------
Office location:
 
Bureau 3020 (3rd floor)
Batiment Sophie Germain
Avenue de France
Metro Bibliotheque Francois Mitterrand, ligne 14/RER C
-----------------------------------------------------------------
GPG fingerprint 2931 20CE 3A5A 5390 98EC 8BFC FCCA C3BE 39CB 12D3                        

Re: [Caml-list] Literate (sort-of) programming in OCaml

[Ashish Agarwal]

[-- Attachment #1: Type: text/plain, Size: 2294 bytes --]

OCaml Labs is developing MPP (Meta Preprocessor) [1]. It will be used in a
new implementation of ocaml.org [2], which includes pages like 99 problems
[3] that are generated by running ocaml code through the toploop. In case
blah = Markdown, please also note the new Markdown library [4].

[1] https://github.com/pw374/MPP-language-blender
[2] http://lists.ocaml.org/pipermail/infrastructure/2013-July/000211.html
[3] http://ocaml.org/tutorials/99problems.html
[4] http://lists.ocaml.org/pipermail/infrastructure/2013-July/000223.html



On Mon, Sep 2, 2013 at 11:39 AM, Jonathan Protzenko <
jonathan.protzenko@gmail.com> wrote:

> Hi,
>
> I am currently writing a big, mostly textual document in format blah
> (blah being of a course a meta-variable). Are there any tools that
> would allow me to interleave OCaml code with the contents of my
> document?
>
> More on my use-case. Creating this document requires me to perform
> various subtasks, such as:
> - write a code snippet in the document,
> - put the code snippet in a file,
> - call an external program on the file,
> - paste the output into the document.
>
> Naturally, I wish to automate this. I could roll my own set of
> commands, and parse them with OCaml, but I would be re-creating a
> scripting language, and it seems to me that interleaving OCaml code
> within my document would be better. Here's what I have in mind:
>
>
> blah blah blah blah blah blah blah blah blah blah blah blah blah blah
> blah blah
> blah blah blah blah blah blah blah blah blah blah blah blah blah blah
> blah blah
> blah ... we thus write the following code ... :
>
> {%
>   let code = "<sample code>" in
>   output code
> %}
>
> blah blah blah ... after running the command blah ... the output is as
> follows ... blah
>
> {%
>   let f = write_into_temp_file code in
>   let s = run_and_read "myprogram" [f] in
>   output s
> %}
>
> blah blah blah blah blah blah blah blah blah blah blah blah blah blah
> blah blah ...
>
>
> Are you aware of any tool that would allow me to achieve this?
>
> Thanks,
>
> ~ jonathan
>
> --
> Caml-list mailing list.  Subscription management and archives:
> https://sympa.inria.fr/sympa/arc/caml-list
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs
>

[-- Attachment #2: Type: text/html, Size: 3540 bytes --]

Re: [Caml-list] Literate (sort-of) programming in OCaml

[Raphaël Proust]

On Mon, Sep 2, 2013 at 5:39 PM, Jonathan Protzenko
<jonathan.protzenko@gmail.com> wrote:
> I am currently writing a big, mostly textual document in format blah
> (blah being of a course a meta-variable). Are there any tools that
> would allow me to interleave OCaml code with the contents of my
> document?
>
> More on my use-case. Creating this document requires me to perform
> various subtasks, such as:
> - write a code snippet in the document,
> - put the code snippet in a file,
> - call an external program on the file,
> - paste the output into the document.

Current workflow I use involves:
- mkfile: a rule file for mk (Plan9's take on make, it has simple
escaping and a readable man page) to drive the whole thing,
- bar: a compiler for the language bar that includes a Latex pretty
printing mode
- inc/: a directory where I place all my code samples from language bar
- main.pdc: a pandoc file holding the source.

Extracts from the mkfile:
~~~~~~~
TARGET=main.pdf
SOURCE=${TARGET:%.pdf=%.pdc}

INCLUDES=`{cat $SOURCE | grep '\\input{.*\.tex}' | sed
's/^.*\\input{([^\$\\}]*\.tex)}.*$/\1/'}

$TARGET: $SOURCE $INCLUDES
        pandoc -o $TARGET $SOURCE

%.pp.tex: %.bar bar.byte
        OCAMLRUNPARAM=$OCAMLRUNOPT ./bar.byte -latex $stem.bar > $target

%.pp.annot.tex: %.bar bar.byte
        OCAMLRUNPARAM=$OCAMLRUNOPT ./bar.byte -latex -annot $stem.bar > $target
~~~~~~~

Extracts from bar.mli:
~~~~~~~
module type PRINTER = sig
  val t: annotations:bool -> t -> string
end
module LatexPP : PRINTER
~~~~~~~

Extracts from the main.pdc
~~~~~~~
The annotation process, blahblah. Thus the program $\input{inc/foo.pp.tex}$
is annotated in the following way: $\input{inc/foo.pp.annot.tex}$
~~~~~~~


Known bugs and shortcomings:
- sed works line-wise so the mkfile INCLUDES variable is not set
properly when there are several \input on the same line.
- the passing of argument (using two rules with and two file
extensions) is hackish and would not scale.


Less painful than copy-pasting things and easier to keep everything in
sync. It's also nice to be able to do things such as:
\begin{align*}
<some math>\input{inc/function-application.pp.tex}<more math>\\
<more math>\input{inc/function-declaration.pp.tex}<more math>\\
<more math>\input{inc/if-then-else.pp.tex}<more math>\\
<etc.>
\end{align*}

-- 
______________
Raphaël Proust

Re: [Caml-list] Literate (sort-of) programming in OCaml

[Simon Cruanes]

[-- Attachment #1: Type: text/plain, Size: 2838 bytes --]

You may also try https://www.lri.fr/~filliatr/ocamlweb/ .

Cheers


"Raphaël Proust" <raphlalou@gmail.com> a écrit :
>On Mon, Sep 2, 2013 at 5:39 PM, Jonathan Protzenko
><jonathan.protzenko@gmail.com> wrote:
>> I am currently writing a big, mostly textual document in format blah
>> (blah being of a course a meta-variable). Are there any tools that
>> would allow me to interleave OCaml code with the contents of my
>> document?
>>
>> More on my use-case. Creating this document requires me to perform
>> various subtasks, such as:
>> - write a code snippet in the document,
>> - put the code snippet in a file,
>> - call an external program on the file,
>> - paste the output into the document.
>
>Current workflow I use involves:
>- mkfile: a rule file for mk (Plan9's take on make, it has simple
>escaping and a readable man page) to drive the whole thing,
>- bar: a compiler for the language bar that includes a Latex pretty
>printing mode
>- inc/: a directory where I place all my code samples from language bar
>- main.pdc: a pandoc file holding the source.
>
>Extracts from the mkfile:
>~~~~~~~
>TARGET=main.pdf
>SOURCE=${TARGET:%.pdf=%.pdc}
>
>INCLUDES=`{cat $SOURCE | grep '\\input{.*\.tex}' | sed
>'s/^.*\\input{([^\$\\}]*\.tex)}.*$/\1/'}
>
>$TARGET: $SOURCE $INCLUDES
>        pandoc -o $TARGET $SOURCE
>
>%.pp.tex: %.bar bar.byte
>       OCAMLRUNPARAM=$OCAMLRUNOPT ./bar.byte -latex $stem.bar > $target
>
>%.pp.annot.tex: %.bar bar.byte
>OCAMLRUNPARAM=$OCAMLRUNOPT ./bar.byte -latex -annot $stem.bar > $target
>~~~~~~~
>
>Extracts from bar.mli:
>~~~~~~~
>module type PRINTER = sig
>  val t: annotations:bool -> t -> string
>end
>module LatexPP : PRINTER
>~~~~~~~
>
>Extracts from the main.pdc
>~~~~~~~
>The annotation process, blahblah. Thus the program
>$\input{inc/foo.pp.tex}$
>is annotated in the following way: $\input{inc/foo.pp.annot.tex}$
>~~~~~~~
>
>
>Known bugs and shortcomings:
>- sed works line-wise so the mkfile INCLUDES variable is not set
>properly when there are several \input on the same line.
>- the passing of argument (using two rules with and two file
>extensions) is hackish and would not scale.
>
>
>Less painful than copy-pasting things and easier to keep everything in
>sync. It's also nice to be able to do things such as:
>\begin{align*}
><some math>\input{inc/function-application.pp.tex}<more math>\\
><more math>\input{inc/function-declaration.pp.tex}<more math>\\
><more math>\input{inc/if-then-else.pp.tex}<more math>\\
><etc.>
>\end{align*}
>
>-- 
>______________
>Raphaël Proust
>
>-- 
>Caml-list mailing list.  Subscription management and archives:
>https://sympa.inria.fr/sympa/arc/caml-list
>Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
>Bug reports: http://caml.inria.fr/bin/caml-bugs

-- 
Envoyé de mon téléphone Android avec K-9 Mail. Excusez la brièveté.

[-- Attachment #2: Type: text/html, Size: 3330 bytes --]

Re: [Caml-list] Literate (sort-of) programming in OCaml

[Maxence Guesdon]

On Mon, 02 Sep 2013 18:59:33 +0300
Török Edwin <edwin+ml-ocaml@etorok.net> wrote:

> On 09/02/2013 06:39 PM, Jonathan Protzenko wrote:
> > Hi,
> > 
> > I am currently writing a big, mostly textual document in format blah 
> > (blah being of a course a meta-variable). Are there any tools that 
> > would allow me to interleave OCaml code with the contents of my 
> > document?
> > 
> > More on my use-case. Creating this document requires me to perform 
> > various subtasks, such as:
> > - write a code snippet in the document,
> > - put the code snippet in a file,
> > - call an external program on the file,
> > - paste the output into the document.
> > 
> > Naturally, I wish to automate this. I could roll my own set of 
> > commands, and parse them with OCaml, but I would be re-creating a 
> > scripting language, and it seems to me that interleaving OCaml code 
> > within my document would be better. Here's what I have in mind:
> > 
> > 
> > blah blah blah blah blah blah blah blah blah blah blah blah blah blah 
> > blah blah
> > blah blah blah blah blah blah blah blah blah blah blah blah blah blah 
> > blah blah
> > blah ... we thus write the following code ... :
> > 
> > {%
> >   let code = "<sample code>" in
> >   output code
> > %}
> > 
> > blah blah blah ... after running the command blah ... the output is as 
> > follows ... blah
> > 
> > {%
> >   let f = write_into_temp_file code in
> >   let s = run_and_read "myprogram" [f] in
> >   output s
> > %}
> > 
> > blah blah blah blah blah blah blah blah blah blah blah blah blah blah 
> > blah blah ...
> > 
> > 
> > Are you aware of any tool that would allow me to achieve this?
> 
> http://zoggy.github.io/stog/posts/ocaml-sessions.html

You can have a look at a huge usage of this in my ocaml introduction:
  http://form-ocaml.forge.ocamlcore.org/
The document:
  http://form-ocaml.forge.ocamlcore.org/intro_ocaml.html
The source code:
  http://forge.ocamlcore.org/scm/?group_id=228

You will need stog and stog-writing:
  http://zoggy.github.io/stog/
  http://zoggy.github.io/stog/stog-writing.html

Hope this helps,

Maxence

Re: [Caml-list] Literate (sort-of) programming in OCaml

[Jonathan Protzenko]

[-- Attachment #1: Type: text/plain, Size: 4084 bytes --]

Hi everyone,

Thanks for the numerous replies. There are a lot of good solutions in
there, and the ones that looked closest to what I had in mind seem to be
MPP and the rwo-run-toplevel script. Some questions about these two tools:
- MPP seems to just have the built-in language. I couldn't find any
example in the examples/ directory that used another language, such as
OCaml. I guess that's because it's still in development?
- rwo-run-toplevel seems closer to what I have in mind; if I could send
all the ocaml parts to a top-level session, and put in my document what
gets sent to stdout (not the toplevel reply, that is, not the "val f:
int -> int" parts), that would be pretty much all I need.
rwo-run-toplevel seems to perform a lot of work, though: what are all
these rewrite rules?

Other solutions seemed interesting. The Perl script scared me a little
bit; being no Perl hacker, I would have a hard time using it. The
Markdown library does not seem to be an exact fit (I want the output of
arbitrary OCaml commands to be put in the middle of my markdown.) Stog
seems to be oriented towards reproducing an interactive toploop session,
while I'm more interested in executing arbitrary commands. Alain's sed
trick seems to be exactly what I need, I'm just a bit afraid if I ever
need to change something, it'll be break. But in essence, yes, that's
just all I need. Other proposals seemed a little bit too far away from
what I had in mind.

Thanks again for the replies!

~ jonathan

On 09/02/2013 06:29 PM, Ashish Agarwal wrote:
> OCaml Labs is developing MPP (Meta Preprocessor) [1]. It will be used
> in a new implementation of ocaml.org <http://ocaml.org> [2], which
> includes pages like 99 problems [3] that are generated by running
> ocaml code through the toploop. In case blah = Markdown, please also
> note the new Markdown library [4].
>
> [1] https://github.com/pw374/MPP-language-blender
> [2] http://lists.ocaml.org/pipermail/infrastructure/2013-July/000211.html
> [3] http://ocaml.org/tutorials/99problems.html
> [4] http://lists.ocaml.org/pipermail/infrastructure/2013-July/000223.html
>
>
>
> On Mon, Sep 2, 2013 at 11:39 AM, Jonathan Protzenko
> <jonathan.protzenko@gmail.com <mailto:jonathan.protzenko@gmail.com>>
> wrote:
>
>     Hi,
>
>     I am currently writing a big, mostly textual document in format blah
>     (blah being of a course a meta-variable). Are there any tools that
>     would allow me to interleave OCaml code with the contents of my
>     document?
>
>     More on my use-case. Creating this document requires me to perform
>     various subtasks, such as:
>     - write a code snippet in the document,
>     - put the code snippet in a file,
>     - call an external program on the file,
>     - paste the output into the document.
>
>     Naturally, I wish to automate this. I could roll my own set of
>     commands, and parse them with OCaml, but I would be re-creating a
>     scripting language, and it seems to me that interleaving OCaml code
>     within my document would be better. Here's what I have in mind:
>
>
>     blah blah blah blah blah blah blah blah blah blah blah blah blah blah
>     blah blah
>     blah blah blah blah blah blah blah blah blah blah blah blah blah blah
>     blah blah
>     blah ... we thus write the following code ... :
>
>     {%
>       let code = "<sample code>" in
>       output code
>     %}
>
>     blah blah blah ... after running the command blah ... the output is as
>     follows ... blah
>
>     {%
>       let f = write_into_temp_file code in
>       let s = run_and_read "myprogram" [f] in
>       output s
>     %}
>
>     blah blah blah blah blah blah blah blah blah blah blah blah blah blah
>     blah blah ...
>
>
>     Are you aware of any tool that would allow me to achieve this?
>
>     Thanks,
>
>     ~ jonathan
>
>     --
>     Caml-list mailing list.  Subscription management and archives:
>     https://sympa.inria.fr/sympa/arc/caml-list
>     Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
>     Bug reports: http://caml.inria.fr/bin/caml-bugs
>
>


[-- Attachment #2: Type: text/html, Size: 7082 bytes --]

Re: [Caml-list] Literate (sort-of) programming in OCaml

[oliver]

Hi,

On Mon, Sep 02, 2013 at 05:39:35PM +0200, Jonathan Protzenko wrote:
> Hi,
> 
> I am currently writing a big, mostly textual document in format blah 
> (blah being of a course a meta-variable). Are there any tools that 
> would allow me to interleave OCaml code with the contents of my 
> document?
> 
> More on my use-case. Creating this document requires me to perform 
> various subtasks, such as:
> - write a code snippet in the document,
> - put the code snippet in a file,
> - call an external program on the file,
> - paste the output into the document.
> 
> Naturally, I wish to automate this. I could roll my own set of 
> commands, and parse them with OCaml, but I would be re-creating a 
> scripting language, and it seems to me that interleaving OCaml code 
> within my document would be better. Here's what I have in mind:
> 
> 
> blah blah blah blah blah blah blah blah blah blah blah blah blah blah 
> blah blah
> blah blah blah blah blah blah blah blah blah blah blah blah blah blah 
> blah blah
> blah ... we thus write the following code ... :
> 
> {%
>   let code = "<sample code>" in
>   output code
> %}
> 
> blah blah blah ... after running the command blah ... the output is as 
> follows ... blah
> 
> {%
>   let f = write_into_temp_file code in
>   let s = run_and_read "myprogram" [f] in
>   output s
> %}
> 
> blah blah blah blah blah blah blah blah blah blah blah blah blah blah 
> blah blah ...
> 
> 
> Are you aware of any tool that would allow me to achieve this?
[...]


Hmhhh, long, long ago I explored some of the good-old / classical
literate programming tools (well, to be detailed, I did not tried Web,
so it was not THE classical one, but later developments, influenced by
Knuth's Web.)

Some of these tools were limited to certain languages, but as far as I remember
at least one was able to work with any (???) language.
Or at least some tools were very flexible to work with many languages.

I have forgotten most of the names of the tools I used, possibly I tried
noweb and some others.

All had their advantages and also their limitations.

There was one (or two?) that looked so promising that I had planned to try it
(them) too, but I stopped with Literate Programming (LP) before I had the
motivation to try it/them.
  (Trying so many tools at some time has exhausted and demotivated me.
   Those I tried were not powerful/flexible enough, nevertheless needed their
   time to look at them. Tools should help in the work and help saving time,
   not eat the time...)

As far as I remember, and I'm not sure, FWEB and/or FunnelWeb was/were the
remaining tools that I had selected as the most prommising ones (but did not
tried them, as mentioned above, so I can't really vote for them).

(In/with R you can use Sweave, which allows to embed code and results from code
 like tables, and even to embedd created graphics into the document!)


An overview with links to some of the LP-tools can be found here:

  http://www.literateprogramming.com/tools.html


I remember that there were a lot more free tools, than mentioned there.

  I remember one really nice tool only for C-dialects, which did not have
  big functionality, but allowed to pretty print C-Code with real TeX-Comments
  inserted. And the syntax was very simple.
  But it was not allowing insertion of the results of the program,
  at least not directly. But via TeX/LaTeX-include/input--commands I think it was possible
  indirectly.

  But you look for more functionality, and want to pass back the results into
  your document. Something like Sweave.

Possibly FWEB or FunnelWEB are the tools, you should explore first.
(But be prepared to learn their syntax....)

Let me know, which tool you selected, and how your experience was with it.


Ciao,
   Oliver


P.S.: In a documenting-/reverse-engineering project I programmed my own
      literate programming tool(s), so that I could insert Text and
      graphviz-commands into the code-comments. The Perl-Tools did extract the
      text and the graphviz code, invoked graphviz and created a LaTeX-document
      that contained the Code, the documenting text and the Graphviz-graphics
      (e.g. for Flowcharts), and created a nice pdf-document, including the
      nice references, to open certain parts of the doc by clicking on the
      entries in the table of contents. :-)
      By organizing the document via includes/inputs, the document structure could be
      organized the way I wanted. (The laTeX-code was also created by the
      Perl-script(s).


P.P.S.: When using troff it's possible to write C code and it's documentation
        in troff (including PIC-macros :-)) in a way, that it can be C-compiled
        directly, without tangling-process, and the document can be set with
        troff without weaving-process. So, it's possible to have one C-file and
        pass it into C-compiler and troff without changes. :-) Very funny, to
        make this. But changing the order of sections is not possible. Real
        LP-tools IMHO should offer reordering of sections, but most of them
        don't.

Re: [Caml-list] Literate (sort-of) programming in OCaml

[Alain Frisch]

On 09/02/2013 10:16 PM, Jonathan Protzenko wrote:
> Alain's sed
> trick seems to be exactly what I need, I'm just a bit afraid if I ever
> need to change something, it'll be break. But in essence, yes, that's
> just all I need.

Be careful with OCaml lexical convention though.  If you use e.g. " or \ 
in your text, you'll need also to escape them with sed.  Alternatively, 
you could use the new syntax for string literals ({foo|....|foo}) now 
available in trunk.

-- Alain

Re: [Caml-list] Literate (sort-of) programming in OCaml

[Alan Schmitt]

Hi Jonathan,

jonathan.protzenko@gmail.com writes:

> Hi,
>
> I am currently writing a big, mostly textual document in format blah 
> (blah being of a course a meta-variable). Are there any tools that 
> would allow me to interleave OCaml code with the contents of my 
> document?

...

> Are you aware of any tool that would allow me to achieve this?

I'm using org-mode for this (http://orgmode.org/). The support for caml
uses a session, so when you evaluate / export code the previous code is
taken into account. I've used it for courses slides (exported to beamer)
and for caml exams (where the type of the expected answers is computed
from the correction, which is of course not exported).

Alan

Re: [Caml-list] Literate (sort-of) programming in OCaml

[Philippe Wang]

On Mon, Sep 2, 2013 at 9:16 PM, Jonathan Protzenko
<jonathan.protzenko@gmail.com> wrote:
> Hi everyone,
>
> Thanks for the numerous replies. There are a lot of good solutions in there,
> and the ones that looked closest to what I had in mind seem to be MPP and
> the rwo-run-toplevel script. Some questions about these two tools:
> - MPP seems to just have the built-in language. I couldn't find any example
> in the examples/ directory that used another language, such as OCaml. I
> guess that's because it's still in development?

Hi,

MPP currently has 2 things: the built-in language and its commands,
and the ability to bring OCaml as a pre-processor language.

With mpp -its (-its is an option to be less strict on trailing spaces
in commands) :

((x cmd ocamlopt foo.ml x))
will execute "ocamlopt foo.ml" in a shell and input its output.
Note that "x" is the name of the block. It could be FOO or 23NUI'è!çé
instead, so that one can write ((x cmd echo $((42+42)) x)) for
instance without being bothered by escaping stuff.
-> This is an example of the builtin commands.
To have the list of built-in commands: mpp -b

One may want to nest mpp commands. In this case, another pair of
tokens is used: {{ and }}.
so ((x cmd ocamlopt foo.ml x)) could be written {{x cmd ocamlopt
foo.ml x}} instead. In this precise case, there is absolutely no
difference. However, one can also write {{x cmd ocamlopt {{y get
filename y}} x}} or {{x cmd ocamlopt ((y get filename y)) x}}. But if
one write ((x ocamlopt ((y get filename y)) x)) then a shell will try
to run "ocamlopt ((y get filename y))", which is probably not what's
intended...

All tokens of the language provided by mpp are customisable on the
command line and live in mpp as well. mpp -so 42 will use "42" instead
of "((" to open blocks.
(( so TOTO)) will use TOTO to open blocks after this one, meaning that
to set the open token back to what it was, one could write TOTO so
(())

Well, it's up to the user to imagine how powerful it is.

Then, there is the ability to bring ocaml as a pre-processor language.
This is actually the main motivation for mpp.
Sometimes, you write something using a specific language and you just
wish you could easily use OCaml (or another language...) to preprocess
your file.
With mpp, the default tokens are {< and >} to embed ocaml.
This means that in any file, one can write ocaml and have its output.

Small example in markdown:

If you write this in the file e.md
```
# Number of the day
{< let foo = 42 >}
The number of the day is {< let () = Printf.printf "%d" foo >}.
```

And you run mpp -l ocaml over e.md (note the -l option), then you obtain
```
 let _ = print_string "# Number of the day\n\n"
 let foo = 42  let _ = print_string "\nThe number of the day is \n"
 let () = Printf.printf "%d" foo  let _ = print_string ".\n\n"
```
which is an ocaml programme that you can run to have your final document:

```
# Number of the day


The number of the day is
42.
```

You can see that the environment is shared. If you want different
environments, it's possible to run mpp several times with different
opening and closing tokens.

Of course, you may use ocaml as a preprocessor language in an ocaml
programme (instead of a markdown document), or any textual file
actually.

For the moment, ocaml is the only option supported by "mpp -l" but
I'll provide more language soon. (To provide a language X, one has to
write a "description" of how to convert a text file into an X
programme.)


I've not written much documentation for MPP yet because I've been
mainly working on OMD, a Markdown->HTML converted and Markdown parser
library in OCaml. Now I believe OMD is nearly done, so I'm getting
back to MPP soon...

-- 
Philippe Wang
   mail@philippewang.info