[Caml-list] Problem when parsing

[Caml-list] Problem when parsing

[Laurent Reveillere]

I wrote a small parser for iso C which does nothing (for now).
The lexer and parser files are well compiled by ocamllex and ocamlyacc.
But when I run the generated program, I have the following error :

Fatal error: uncaught exception Failure(lexing: empty token)


If I replace the following rule in the parser :
	
translation_unit:  
     external_declaration   { () }
  |  translation_unit external_declaration   { () }
 
;                                                                             	


by

translation_unit:  
     external_declaration   { () }
 
;                                                                             	

the test file is parsed with no problem.
The test file is just "int x;"


Any idea ...


I know that this error is raised by lex_engine in byterun/lexing.c but I
do not understand why !!!

-- 
Laurent
-------------------
To unsubscribe, mail caml-list-request@inria.fr.  Archives: http://caml.inria.fr

Re: [Caml-list] Problem when parsing

[Jean-Christophe Filliatre]

Laurent Reveillere writes:
 > I wrote a small parser for iso C which does nothing (for now).
 > The lexer and parser files are well compiled by ocamllex and ocamlyacc.
 > But when I run the generated program, I have the following error :
 > 
 > Fatal error: uncaught exception Failure(lexing: empty token)

This exception is raised by the lexer when input does not match any of
the lexer rules.  So it has probably to do with  your lexer rules, not
with your parser rules.

Hope this helps,
-- 
Jean-Christophe FILLIATRE
  mailto:Jean-Christophe.Filliatre@lri.fr
  http://www.lri.fr/~filliatr
-------------------
To unsubscribe, mail caml-list-request@inria.fr.  Archives: http://caml.inria.fr

Re: [Caml-list] Problem when parsing

[Laurent Reveillere]

Jean-Christophe Filliatre wrote:
> 
> Laurent Reveillere writes:
>  > I wrote a small parser for iso C which does nothing (for now).
>  > The lexer and parser files are well compiled by ocamllex and ocamlyacc.
>  > But when I run the generated program, I have the following error :
>  >
>  > Fatal error: uncaught exception Failure(lexing: empty token)
> 
> This exception is raised by the lexer when input does not match any of
> the lexer rules.  So it has probably to do with  your lexer rules, not
> with your parser rules.
> 
> Hope this helps,
> --
> Jean-Christophe FILLIATRE
>   mailto:Jean-Christophe.Filliatre@lri.fr
>   http://www.lri.fr/~filliatr


Yes, I fixed the problem. The error came from the fact that I forgot a
rule for eof in my lexer. Still, I would like to know why the eof
special char is not match by a rule like
	 | _ -> raise (Failure "unknown token") 


-- 
Lauren
-------------------
To unsubscribe, mail caml-list-request@inria.fr.  Archives: http://caml.inria.fr

Re: [Caml-list] Problem when parsing

[Markus Mottl]

On Fri, 02 Mar 2001, Laurent Reveillere wrote:
> I wrote a small parser for iso C which does nothing (for now).
> The lexer and parser files are well compiled by ocamllex and ocamlyacc.
> But when I run the generated program, I have the following error :
> 
> Fatal error: uncaught exception Failure(lexing: empty token)
[snip]

The problem is that your parser rules are recursive, and you do not
handle EOF (end-of-file) in the lexer.

This means that the parser will demand a new token, because the recursive
parser rule says that there could be more of them. The lexer, however,
doesn't return further ones, because it doesn't know how to handle EOF.

There is a special pattern called "eof" (without the double-quotes) that
you can use in the lexer to match EOF. You can introduce a new token
"EOF" in the parser and return this one from the scanner when required.
You'll need parser rules that handle the EOF-token, of course.  Then your
problem should go away.

Regards,
Markus Mottl

-- 
Markus Mottl, mottl@miss.wu-wien.ac.at, http://miss.wu-wien.ac.at/~mottl
-------------------
To unsubscribe, mail caml-list-request@inria.fr.  Archives: http://caml.inria.fr