[announce] XStream: streaming XML transformation

[announce] XStream: streaming XML transformation

[Alain Frisch]

Dear caml-list,

This is to announce the first public release of XStream, a simple
programming language for XML. Transformations written in XStream are
compiled into efficient XML stream processors: the output is computed
and produced while the input is being parsed, which makes it possible to
run some transformations on very big XML documents which could not even
fit in memory.

Though XStream is mostly intended as a back-end for higher-level
languages, it is also possible to use it directly. The language
features ML-like pattern matching and higher-order functions, but no types.


Here's how XStream relates to OCaml: the compiler is written in OCaml
and uses OCaml as a back-end to produce efficient native code. Moreover,
XStream programs can embed OCaml code and manipulate OCaml values
(useful to deal with non-XML data).


Web-site: http://yquem.inria.fr/~frisch/xstream


  Alain

Re: [Caml-list] [announce] XStream: streaming XML transformation

[David MENTRE]

Hello Alain,

2006/6/28, Alain Frisch <Alain.Frisch@inria.fr>:
> Here's how XStream relates to OCaml: the compiler is written in OCaml
> and uses OCaml as a back-end to produce efficient native code. Moreover,
> XStream programs can embed OCaml code and manipulate OCaml values
> (useful to deal with non-XML data).

It seems obvious but I want to be sure: can XStream generated code be
embedded in OCaml programs (native and bytecode)?

How XStream compares to OCamlDuce? From what you said, XStream can
only do XML->XML or XML->OCaml transformation, no OCaml->XML
transformation. So XStream cannot be used as a replacement to
OCamlDuce. Is it correct?

Best wishes,
d.

Re: [Caml-list] [announce] XStream: streaming XML transformation

[Alain Frisch]

Hello David,

David MENTRE wrote:
> It seems obvious but I want to be sure: can XStream generated code be
> embedded in OCaml programs (native and bytecode)?

Currently, XStream generates only a fully functional executable
transformaion which reads its input from an XML file and send its output
to the standard output.

It would be quite simple to have it generate an OCaml module which
exports a ``filter'' function (a transformer of XML event streams) if
there is some demand for such a feature.

> How XStream compares to OCamlDuce?

OCamlDuce is intended to write large OCaml applications which need to
manipulate XML data. XStream is really meant as an XML transformation
tool. In particular, to achieve efficient streaming, XStream keeps
control over the evaluation of the transformation as a whole. OCamlDuce,
instead, relies on the regular OCaml evaluation mechanism.

> So XStream cannot be used as a replacement to
> OCamlDuce. Is it correct?

Yes it is.


Regards,

Alain

Re: [Caml-list] [announce] XStream: streaming XML transformation

[Richard Jones]

On Wed, Jun 28, 2006 at 05:22:57PM +0200, Alain Frisch wrote:
> This is to announce the first public release of XStream, a simple
> programming language for XML. Transformations written in XStream are
> compiled into efficient XML stream processors: the output is computed
> and produced while the input is being parsed, which makes it possible to
> run some transformations on very big XML documents which could not even
> fit in memory.

It seems interesting, but ...  I downloaded the tarball and looked at
the examples and I can't tell: can XStream transform an XML file into
something else?

In particular we have a requirement to convert a huge XML file into
tabular CSV data on the fly (while the XML is being downloaded).  The
XML file just consists of a very large number of <row>...</row>
records.

Rich.

-- 
Richard Jones, CTO Merjis Ltd.
Merjis - web marketing and technology - http://merjis.com
Team Notepad - intranets and extranets for business - http://team-notepad.com

Re: [Caml-list] [announce] XStream: streaming XML transformation

[Alain Frisch]

Richard Jones wrote:
> I downloaded the tarball and looked at
> the examples and I can't tell: can XStream transform an XML file into
> something else?
> 
> In particular we have a requirement to convert a huge XML file into
> tabular CSV data on the fly (while the XML is being downloaded).  The
> XML file just consists of a very large number of <row>...</row>
> records.

That's possible with XStream: text is just a special case of XML.
For instance, the following script extracts the stream of text from the
input XML stream.

main(_[x1] x2) -> concat(main(x1),main(x2))
main(%s x) -> %s main(x)
main(()) -> ()

-- Alain