[Caml-list] Parse crazy HTML, output XML

[Caml-list] Parse crazy HTML, output XML

[Richard Jones]

I have a bunch of HTML documents from an external source which I do
not control.  They aren't valid XML, by any means.  I need to read
them in, do a "best effort" to build a DOM, do various manipulations
over the DOM (such as removing <script> tags, replace <B> with
<strong>, etc.), and output an XHTML fragment.  I also need to do this
from an OCaml program.

Example input:

---
This is <B>the sort of document</B> which I have to parse.<br>
<br>
<br>
---

Desired output:

---
<p>
This is <strong>the sort of document</strong> which I have to parse.
</p>
---

The problem is the parsing phase.  Both PXP and XmlLight will only
parse valid XML (as far as I can see).  Is there any simple pure OCaml
library for parsing HTML and producing a DOM?

Rich.

-- 
Richard Jones. http://www.annexia.org/ http://www.j-london.com/
Merjis Ltd. http://www.merjis.com/ - improving website return on investment
PTHRLIB is a library for writing small, efficient and fast servers in C.
HTTP, CGI, DBI, lightweight threads: http://www.annexia.org/freeware/pthrlib/

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Parse crazy HTML, output XML

[Alain Frisch]

On Mon, 21 Jun 2004, Richard Jones wrote:

> The problem is the parsing phase.  Both PXP and XmlLight will only
> parse valid XML (as far as I can see).  Is there any simple pure OCaml
> library for parsing HTML and producing a DOM?

There is an html parser in the ocamlnet library.

-- Alain

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Parse crazy HTML, output XML

[Gerd Stolpmann]

Am Mon, 2004-06-21 um 18.03 schrieb Richard Jones:
> I have a bunch of HTML documents from an external source which I do
> not control.  They aren't valid XML, by any means.  I need to read
> them in, do a "best effort" to build a DOM, do various manipulations
> over the DOM (such as removing <script> tags, replace <B> with
> <strong>, etc.), and output an XHTML fragment.  I also need to do this
> from an OCaml program.
> 
> Example input:
> 
> ---
> This is <B>the sort of document</B> which I have to parse.<br>
> <br>
> <br>
> ---
> 
> Desired output:
> 
> ---
> <p>
> This is <strong>the sort of document</strong> which I have to parse.
> </p>
> ---
> 
> The problem is the parsing phase.  Both PXP and XmlLight will only
> parse valid XML (as far as I can see).  Is there any simple pure OCaml
> library for parsing HTML and producing a DOM?

The Nethtml of ocamlnet can do this (ocamlnet.sourceforge.net).

Gerd
-- 
------------------------------------------------------------
Gerd Stolpmann * Viktoriastr. 45 * 64293 Darmstadt * Germany 
gerd@gerd-stolpmann.de          http://www.gerd-stolpmann.de
------------------------------------------------------------

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Parse crazy HTML, output XML

[James Leifer]

Alain Frisch <Alain.Frisch@ens.fr> writes:

> On Mon, 21 Jun 2004, Richard Jones wrote:
>
>> The problem is the parsing phase.  Both PXP and XmlLight will only
>> parse valid XML (as far as I can see).  Is there any simple pure OCaml
>> library for parsing HTML and producing a DOM?

If you've got really broken documents then perhaps "tidy" is your
friend. Yes, I know it may be outside the scope of your request
because it's an external program but apparently it can do wonders for
syntactically dubious tag-soup html.  I believe that it can produce
pure xhtml, for example.

-J

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Parse crazy HTML, output XML

[Richard Jones]

Thanks.  The Nethtml module from ocamlnet looks ideal.  I even had it
installed on my system without knowing about it :-)

Rich.

-- 
Richard Jones. http://www.annexia.org/ http://www.j-london.com/
Merjis Ltd. http://www.merjis.com/ - improving website return on investment
Perl4Caml lets you use any Perl library in your type-safe Objective
CAML programs. http://www.merjis.com/developers/perl4caml/

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Parse crazy HTML, output XML

[Shawn Wagner]

On Mon, Jun 21, 2004 at 05:03:28PM +0100, Richard Jones wrote:
> 
> The problem is the parsing phase.  Both PXP and XmlLight will only
> parse valid XML (as far as I can see).  Is there any simple pure OCaml
> library for parsing HTML and producing a DOM?
> 

There's a html parser in the ocamlnet library. 

-- 
Shawn Wagner
shawnw@speakeasy.org

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Parse crazy HTML, output XML

[Paul Snively]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


On Jun 21, 2004, at 9:19 AM, Shawn Wagner wrote:

> On Mon, Jun 21, 2004 at 05:03:28PM +0100, Richard Jones wrote:
>>
>> The problem is the parsing phase.  Both PXP and XmlLight will only
>> parse valid XML (as far as I can see).  Is there any simple pure OCaml
>> library for parsing HTML and producing a DOM?
>>
>
> There's a html parser in the ocamlnet library.
>
I've recently found the OCamlNet HTML parser also. Does anyone know if 
Alain Frisch's XPath implementation, which is modularized and 
functorized, has been/can be used on the resulting tree from Nethtml?

> -- 
> Shawn Wagner
> shawnw@speakeasy.org
>
Many thanks and best regards,
Paul Snively

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (Darwin)

iEYEARECAAYFAkDb0WIACgkQbot1wzHBQBUd7QCcDkrzAX1diwMisH31VUDR2aeV
S3MAoLatoYjH1lmpKSaOxhAm4VmYKfCc
=Skxm
-----END PGP SIGNATURE-----

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners