Re: mdocml: It's annoying that we don't have preconv, so throw together a

Re: mdocml: It's annoying that we don't have preconv, so throw together a

[Kristaps Dzonsons]

>> It's annoying that we don't have preconv,
>
> All the more thanks for starting that!
>
> However, i see main() in there.
> That will end up with pipes and knobs.
>
> In the long run, i think this needs to be intregrated into mandoc,
> with as few knobs as possible.  OK, maybe we will need a knob to
> specify the input encoding, but maybe even that can sometimes be
> guessed from the file, for example when there is a BOM.
>
> In an UTF-8 terminal,
>
>    mandoc foo.jp.1 | less
>
> ought to be enough for everyone.
>
> Of course, there is nothing wrong with developing new functionality
> stand-alone, it was quite successful with tbl.

Ingo, I hope you don't mind that I cross-post this to discuss@...

For those of you not reading source-changes, there's now a preconv 
utility in mdocml for recoding multibyte manuals as mandoc input.

And it's more or less finished, not started! ;)  I'm able to download 
the Japanese manuals and read through them just fine.  Er... "look at 
them" just fine.  After running them through iconv to UTF-8, of course.

I do agree with what you say.  When you (and other downstream) think the 
time has come, it's ready to be migrated.  Til then, there's plenty of 
catching up to do as it is. ;)

I'll put out a release in the next few days to get eyeballs on all this 
locale stuff that's been checked in since BSDCan.  Once it's been in the 
wild for a little bit, I'll be more prepared to muck around with putting 
it directly into mandoc.

In terms of bloat, if we keep to Latin-1, US-ASCII, and UTF-8, there's 
not very much overhead (as you can see in preconv.c, most of which is 
read_whole_file, which will soon go away to compat.c where it belongs).

Thanks,

Kristaps
--
 To unsubscribe send an email to discuss+unsubscribe@mdocml.bsd.lv