From mboxrd@z Thu Jan 1 00:00:00 1970 Received: (from majordomo@localhost) by pauillac.inria.fr (8.7.6/8.7.3) id LAA00271; Tue, 21 Sep 2004 11:25:21 +0200 (MET DST) X-Authentication-Warning: pauillac.inria.fr: majordomo set sender to owner-caml-list@pauillac.inria.fr using -f Received: from nez-perce.inria.fr (nez-perce.inria.fr [192.93.2.78]) by pauillac.inria.fr (8.7.6/8.7.3) with ESMTP id LAA00850 for ; Tue, 21 Sep 2004 11:25:20 +0200 (MET DST) Received: from smtp1.adl2.internode.on.net (smtp1.adl2.internode.on.net [203.16.214.181]) by nez-perce.inria.fr (8.13.0/8.13.0) with ESMTP id i8L9PHwl021129; Tue, 21 Sep 2004 11:25:19 +0200 Received: from [192.168.1.200] (ppp202-133.lns1.syd3.internode.on.net [203.122.202.133]) by smtp1.adl2.internode.on.net (8.12.9/8.12.9) with ESMTP id i8L9P24Y061653; Tue, 21 Sep 2004 18:55:13 +0930 (CST) Subject: Re: [Caml-list] Lexing.lexeme_start_p broken? From: skaller Reply-To: skaller@users.sourceforge.net To: Damien Doligez Cc: caml-list In-Reply-To: References: <414B602F.2020400@clemson.edu> <16718.41346.210917.885719@gargle.gargle.HOWL> <1095691464.2580.554.camel@pelican.wigram> Content-Type: text/plain Message-Id: <1095758700.2580.857.camel@pelican.wigram> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.2.2 (1.2.2-4) Date: 21 Sep 2004 19:25:02 +1000 Content-Transfer-Encoding: 7bit X-Miltered: at nez-perce with ID 414FF37D.000 by Joe's j-chkmail (http://j-chkmail.ensmp.fr)! X-Loop: caml-list@inria.fr X-Spam: no; 0.00; caml-list:01 sourceforge:01 2004:99 damien:01 44,:01 char:01 char:01 fetches:01 interfere:01 bootstrap:01 decoupling:01 passing:01 decoupling:01 retaining:01 wrappers:01 Sender: owner-caml-list@pauillac.inria.fr Precedence: bulk On Tue, 2004-09-21 at 18:26, Damien Doligez wrote: > On Sep 20, 2004, at 16:44, skaller wrote: > The token supplying function is supposed to _update_ the lexbuf, if > you want the parser to report the correct locations. ocamllex does > some of the work by updating the char count, Not in my case it doesn't. The lexing function isn't an ocamllex lexer. So I'd have to update the char count too. I actually am using ocamllex, but I drive it manually to collect a token list, then feed the list to the parser. Hmmm.. OK, how is this for an idea: Suppose we add to the lexbuf a mutable field of type: lexbuf -> loc which returns the location information the parser needs given a lexbuf. The parser then fetches the location information by calling this function on the lexbuf from which it was obtained. I can then provide a function which accepts my own state object and curry it. This way, I don't have to keep updating the lexbuf, and the parser cannot see the lexbuf details. My routine might be expensive -- but it only gets called once when there is a parse error, not every token. Whilst I don't think this is a perfect solution, it does seem to partially decouple the parser from the lexbuf by at least abstracting it using a function. Would this interfere with the Ocaml bootstrap? *** a better solution might be to pass this function directly the the parser, thereby decoupling it entirely from the lexer. However that changes the type of parser functions. That can easily be fixed though -- just make a compatibility wrapper which calls the full parser function, passing a default function value. If there was any interest, I could probably provide a design which provided proper decoupling, whilst retaining compatibility using wrappers and defaults. [But I imagine it could also be done easily by someone on the Ocaml team -- and throw in a user state object at the same time please, as has been done for the lexer :] The parser does need to get tokens, and it may need location information, but it should not depend on an object whose principle purpose is to support lexing. In theory this is also true for generated lexers: they shouldn't depend on any lexbufs. However for performance reasons, abstracting a character source probably isn't tolerable. -- John Skaller, mailto:skaller@users.sf.net voice: 061-2-9660-0850, snail: PO BOX 401 Glebe NSW 2037 Australia Checkout the Felix programming language http://felix.sf.net ------------------- To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/ Beginner's list: http://groups.yahoo.com/group/ocaml_beginners