Re: [Caml-list] New release of Menhir (20141215)

[oleg@okmij.org]

Regarding incremental parsing of protocols like IMAP: I have
successfully (as in successfully deployed in production and being used
around the clock, since about 2010 or so) used iteratees for
incremental parsing of full XML, including CDATA, parsed entities and
namespaces. The full XML is actually quite difficult to parse: for
example, parsed entity references like & are not recognized within
CDATA blocks; the content of attributes has its own whitespace
handling rules. The parser is used for handling sometimes quite large
XML documents. The parser is incremental and so can work in constant
memory.

        http://okmij.org/ftp/Streams.html#xml

I have also used iteratees to parse HTTP Log files, also
incrementally. The log files have an (unintended, I hope) complication:
the user-agent string (quoted in the log) may, according to RFC,
itself contain quotes. Since the embedded quotes are not escaped
(again, according to RFC), we may end up with quoted strings
containing unescaped quote characters. Parsing will require unbounded
look-ahead then. Iteratees can handle that -- and report errors
precisely and recover.

        http://okmij.org/ftp/Streams.html#good-error
        http://okmij.org/ftp/Streams.html#fork

Incidentally, there are quite many iteratee libraries. Some, like
pipes, emphasize apparent simplicity and do no input buffering. The
performance is indeed pretty bad then. 

        I should also mention that a parser with a call-back interface
and the absence of visible side-effects can _automatically_ be made
incremental. The following web page describes incrementalization of
stdlib's Genlex lexer.
        http://okmij.org/ftp/continuations/differentiating-parsers.html