From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <robert.muller2@gmail.com>
X-Spam-Checker-Version: SpamAssassin 3.1.3 (2006-06-01) on yquem.inria.fr
X-Spam-Level: **
X-Spam-Status: No, score=2.8 required=5.0 tests=DNS_FROM_RFC_POST,SPF_NEUTRAL 
	autolearn=disabled version=3.1.3
X-Original-To: caml-list@yquem.inria.fr
Delivered-To: caml-list@yquem.inria.fr
Received: from mail4-relais-sop.national.inria.fr (mail4-relais-sop.national.inria.fr [192.134.164.105])
	by yquem.inria.fr (Postfix) with ESMTP id CFCD9BBAF
	for <caml-list@yquem.inria.fr>; Tue, 10 Mar 2009 23:44:47 +0100 (CET)
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: AqECAMKKtklKfVwZkGdsb2JhbACUbz8BAQEBCQkMBxEDq3eBB49cAQMBA4QKBoMX
X-IronPort-AV: E=Sophos;i="4.38,338,1233529200"; 
   d="scan'208";a="36370454"
Received: from qw-out-2122.google.com ([74.125.92.25])
  by mail4-smtp-sop.national.inria.fr with ESMTP; 10 Mar 2009 23:44:29 +0100
Received: by qw-out-2122.google.com with SMTP id 3so1150009qwe.15
        for <caml-list@yquem.inria.fr>; Tue, 10 Mar 2009 15:44:28 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=gamma;
        h=domainkey-signature:received:received:message-id:from:to
         :content-type:content-transfer-encoding:mime-version:subject:date
         :x-mailer;
        bh=J1S1vRI6DgRTLwsrk16YgCd0P8XJfDzZAWlVlmKyAs0=;
        b=mWsly+7zESQsIz4VsmjpNbXNQ+rzTlf2yS/g9dNDvI3qRU/Ddi/BY7lJYTkC53hj1p
         LXnBK6URq9N/M1pG9JdEedY0eCg2mCaMQrhbz3WqZMr92hoJBA1OgFI1teczW91a1wiW
         OM7a2XAv18whVfuT/NjB7XARBO01telHUOsAs=
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=message-id:from:to:content-type:content-transfer-encoding
         :mime-version:subject:date:x-mailer;
        b=rvWnehg98wIFltLPGvQUW6d8mbJBIR/WpiXI2MrXv2Et6mA9g6YJ8vS76El7+KbOs0
         HA4VsJVHvAfpf8p+lvzVCKDZ7rgl9uVSECt++DVIGuC4XPL/6/RwjNhTggNm7BlE+wCz
         zSkFTOmTJRRAFbJ/hbl7dt/qdJb5/9zHsDSGM=
Received: by 10.224.46.10 with SMTP id h10mr10191875qaf.176.1236725068613;
        Tue, 10 Mar 2009 15:44:28 -0700 (PDT)
Received: from ?192.168.1.106? (c-24-91-48-211.hsd1.ma.comcast.net [24.91.48.211])
        by mx.google.com with ESMTPS id 7sm5179452qwf.40.2009.03.10.15.44.27
        (version=TLSv1/SSLv3 cipher=RC4-MD5);
        Tue, 10 Mar 2009 15:44:28 -0700 (PDT)
Message-Id: <C312D72A-770B-4BC9-8458-91FD5AA1CB6D@gmail.com>
From: Robert Muller <robert.muller2@gmail.com>
To: O'Caml Mailing List <caml-list@yquem.inria.fr>
Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes
Content-Transfer-Encoding: 7bit
Mime-Version: 1.0 (Apple Message framework v930.3)
Subject: ocamllex question
Date: Tue, 10 Mar 2009 18:44:26 -0400
X-Mailer: Apple Mail (2.930.3)
X-Spam: no; 0.00; ocamllex:01 ocamllex:01 ocamlyacc:01 subset:01 lexer:01 tokens:01 pointer:01 lexer:01 tokens:01 def:01 pointer:01 advancing:98 muller:01 muller:01 token:01 

I am attempting to use ocamllex together with ocamlyacc to parse a  
subset of python. Python uses indentation to denote
statement blocks so a lexer is sometimes required to return a sequence  
of tokens without advancing the input pointer. In
particular, a lexer for python should return a sequence of so-called  
DEDENT tokens when indented fragments
end. E.g.,

def f(x):
	statement1;
	statement2;
		statement3;
		statement4;
A

the lexer should return two consecutive DEDENT tokens between the '\n'  
at the end of statement4 and the token for A.

Looking at the documentation and examples, it isn't clear how to  
convince the generated lexer to not advance the input pointer
so that two consecutive DEDENT tokens can be returned before the token  
for A is returned.

Any ocamllex perts out there?

Thanks,
Bob Muller