[Caml-list] ocamldoc lexer: why tracking line_number and char_number explicitly?

[Caml-list] ocamldoc lexer: why tracking line_number and char_number explicitly?

[Junsong Li]

[-- Attachment #1: Type: text/plain, Size: 238 bytes --]

Dear list,

I am wondering why ocamldoc lexers (odoc_lexer.mll and odoc_text_lexer.mll)
do not use Lexing.position but keep tracking line number and char number
explicitly. Does this have anything to do with the Unicode?

Thanks,
Junsong

[-- Attachment #2: Type: text/html, Size: 318 bytes --]

Re: [Caml-list] ocamldoc lexer: why tracking line_number and char_number explicitly?

[Gabriel Scherer]

[-- Attachment #1: Type: text/plain, Size: 855 bytes --]

Hi Junsong,

I suspect that this simply comes from the fact that ocamldoc started out as
a separate project and thus had to develop its own location handling. If
there is a clean patch to move to Lexing.position, feel free to propose it.
If this requires invasive changes with a larger chance of regressions, it's
maybe not worth it.

(Note that there seem to be some uses of Lexing.position already in the
file, but only in some cases. I don't know the detail and you may have some
"git log" mining to understand them.)

On Wed, Dec 23, 2015 at 6:52 AM, Junsong Li <ljs.darkfish@gmail.com> wrote:

> Dear list,
>
> I am wondering why ocamldoc lexers (odoc_lexer.mll and
> odoc_text_lexer.mll) do not use Lexing.position but keep tracking line
> number and char number explicitly. Does this have anything to do with the
> Unicode?
>
> Thanks,
> Junsong
>

[-- Attachment #2: Type: text/html, Size: 1270 bytes --]

Re: [Caml-list] ocamldoc lexer: why tracking line_number and char_number explicitly?

[Junsong Li]

[-- Attachment #1: Type: text/plain, Size: 1219 bytes --]

Hi Gabriel,

Thanks! The line number and char number in Lexing.position needs to be
initialized and updated (by "Lexing.new_line lexbuf"). This feature only
exists in 3.11 and above. So It could be historic reason. I'll dig more
into it.

On Wed, Dec 23, 2015 at 1:34 AM, Gabriel Scherer <gabriel.scherer@gmail.com>
wrote:

> Hi Junsong,
>
> I suspect that this simply comes from the fact that ocamldoc started out
> as a separate project and thus had to develop its own location handling. If
> there is a clean patch to move to Lexing.position, feel free to propose it.
> If this requires invasive changes with a larger chance of regressions, it's
> maybe not worth it.
>
> (Note that there seem to be some uses of Lexing.position already in the
> file, but only in some cases. I don't know the detail and you may have some
> "git log" mining to understand them.)
>
> On Wed, Dec 23, 2015 at 6:52 AM, Junsong Li <ljs.darkfish@gmail.com>
> wrote:
>
>> Dear list,
>>
>> I am wondering why ocamldoc lexers (odoc_lexer.mll and
>> odoc_text_lexer.mll) do not use Lexing.position but keep tracking line
>> number and char number explicitly. Does this have anything to do with the
>> Unicode?
>>
>> Thanks,
>> Junsong
>>
>
>

[-- Attachment #2: Type: text/html, Size: 1980 bytes --]