CJK support in ConTeXt

CJK support in ConTeXt

[Jin-Hwan Cho]

It's time to discuss the topic "CJK support in ConTeXt" in a more public
place.

The current version of ConTeXt supports Chinese only. But the same mechasism
enables Korean too. The remaining language is Japanese. In some sense, it is
possible to support Japanese in the current subfont mechanism. But several
issues (e.g. Japanese kerning system as explained by Okumura) may occur.

So, alternative way is to use ASCII pTeX for Japanese. Basically looks no
problem
but I worry eTeX extension features. Is it possible to run ConTeXt without
eTeX
extension features, Hans?

My suggestion for supporting CJK characters in ConTeXt is to write a new
module
using e-Omega. Even though e-Omega does not produce PDF format directly,
there are not much problem with DVIPDFMx.

Best, ChoF.
--
~~~~~~~~~~~~~~~~~~~~~~~~~     ***
| Cho, Jin-Hwan == ChoF |     ^ ^
~~~~~~~~~~~~~~~~~~~~~~~~~      o
| Research Fellow       |     ~~~
| School of Mathematics ~~~~~~~~~~~~~~
| Korea Institute for Advanced Study |
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| chofchof@ktug.or.kr                |
| http://free.kaist.ac.kr/ChoF/      |
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Re: CJK support in ConTeXt

[Hans Hagen]

Hi Chof,

So, alternative way is to use ASCII pTeX for Japanese. Basically looks no
>problem
>but I worry eTeX extension features. Is it possible to run ConTeXt without
>eTeX
>extension features, Hans?

in principle yes, but you loose xml handling and a few other things.

>My suggestion for supporting CJK characters in ConTeXt is to write a new
>module
>using e-Omega. Even though e-Omega does not produce PDF format directly,
>there are not much problem with DVIPDFMx.

sounds ok to me

Hans



-------------------------------------------------------------------------
                                   Hans Hagen | PRAGMA ADE | pragma@wxs.nl
                       Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
  tel: +31 (0)38 477 53 69 | fax: +31 (0)38 477 53 74 | www.pragma-ade.com
-------------------------------------------------------------------------
                        information: http://www.pragma-ade.com/roadmap.pdf
                     documentation: http://www.pragma-ade.com/showcase.pdf
-------------------------------------------------------------------------

Re: CJK support in ConTeXt

[Hong Feng]

> It's time to discuss the topic "CJK support in ConTeXt" in a more public
> place.

I can support this idea!  I can build a mailing list with
GNU Mailman on my upcoming server.

Rgds,
Hong

_________________________________________________________
Do You Yahoo!? 
启用电邮帐号，领会雅虎通[身临其境聊电影]的动感魅力，还有网络摄像头+雅虎通收音机等你来拿
http://cn.rd.yahoo.com/mail_cn/tag/?http://cn.messenger.yahoo.com

Re: Re: CJK support in ConTeXt

[Idris S Hamid]

Hans Hagen wrote:

> Hi Chof,
>
> So, alternative way is to use ASCII pTeX for Japanese. Basically looks no
> >problem
> >but I worry eTeX extension features. Is it possible to run ConTeXt without
> >eTeX
> >extension features, Hans?
>
> in principle yes, but you loose xml handling and a few other things.

And you never know where those "few other things" are going to pop up, as I
painfully learned before eOmega came along. eTeX's use in ConTeXt is by no
means restricted to esoteric areas...

Best
Idris

Re: Re: CJK support in ConTeXt

[Giuseppe Bilotta]

Wednesday, August 27, 2003 Hong Feng wrote:

>> It's time to discuss the topic "CJK support in ConTeXt" in a more public
>> place.

HF> I can support this idea!  I can build a mailing list with
HF> GNU Mailman on my upcoming server.

Isn't this a good enough place?

-- 
Giuseppe "Oblomov" Bilotta

Re: Re: CJK support in ConTeXt

[Idris S Hamid]

Giuseppe Bilotta wrote:

> Wednesday, August 27, 2003 Hong Feng wrote:
>
> >> It's time to discuss the topic "CJK support in ConTeXt" in a more public
> >> place.
>
> HF> I can support this idea!  I can build a mailing list with
> HF> GNU Mailman on my upcoming server.
>
> Isn't this a good enough place?

Since

1. the aleph list is going to be low-traffic;
2. eomega (soon-to-be-Aleph) is geared towards multidirectional/ TeX (CJK,
Arabic, Hebrew, etc.);

The aleph list may be able to serve as an appropriate place for many questions
related to multidirectional or multilingual typesetting under ConTeXt, if it
seems that the regular ConTeXt list is getting clogged. Put another way, these
two lists will probably suffice. For example, some of u mentioned a possible
CJK module for e-Omega; details of that might seem appropriate for the aleph
list. As Arabic, Hebrew, and CJK modules are built for ConTeXt the aleph list
may come in handy to share and/or consolidate ideas, with a view to helping the
future multilingual development of ConTeXt remain consistent.

Best
Idris

Re[2]: Re: CJK support in ConTeXt

[Giuseppe Bilotta]

Wednesday, August 27, 2003 Idris S Hamid wrote:

> Since

> 1. the aleph list is going to be low-traffic;
> 2. eomega (soon-to-be-Aleph) is geared towards multidirectional/ TeX (CJK,
> Arabic, Hebrew, etc.);

> The aleph list may be able to serve as an appropriate place for many questions
> related to multidirectional or multilingual typesetting under ConTeXt, if it
> seems that the regular ConTeXt list is getting clogged. Put another way, these
> two lists will probably suffice. For example, some of u mentioned a possible
> CJK module for e-Omega; details of that might seem appropriate for the aleph
> list. As Arabic, Hebrew, and CJK modules are built for ConTeXt the aleph list
> may come in handy to share and/or consolidate ideas, with a view to helping the
> future multilingual development of ConTeXt remain consistent.

I like the idea, especially since Hans is *very* likely to
follow the Aleph list (I mean, is there a TeX-related mailing
list he doesn't follow? <g>)

I'll wait a little to see if Willi Egger sees the request, then
I'll send him a private message to ask for the creation of the
mailing list.

-- 
Giuseppe "Oblomov" Bilotta

RE: Re: CJK support in ConTeXt

[Tim 't Hart]

Idris S Hamid wrote:

> The aleph list may be able to serve as an appropriate place for many
> questions
> related to multidirectional or multilingual typesetting under ConTeXt, if
> it
> seems that the regular ConTeXt list is getting clogged. Put another way,
> these
> two lists will probably suffice. For example, some of u mentioned a
> possible
> CJK module for e-Omega; details of that might seem appropriate for the
> aleph
> list. As Arabic, Hebrew, and CJK modules are built for ConTeXt the aleph
> list
> may come in handy to share and/or consolidate ideas, with a view to
> helping the
> future multilingual development of ConTeXt remain consistent.

I agree that there should be another list for e-Omega (Aleph) where we can
discuss implementing multilingual code for ConTeXt, because I think it's a
bit off-topic for the regular ConTeXt list.

But on the other hand, I also support Hong Feng when he wants to create a
mailing list just for the developers of the CJK package. Or maybe just a
temporary list, just for organizational reasons.

Also a reason is that it's not already sure if e-Omega is the best option
for this (although after using Giuseppe's latest e-Omega version, I think it
is the best option) and that the creation of the CJK module will need
discussions that have nothing to do with e-Omega, like implementing CJK
chapter headings, East Asian line breaking and numbering systems. Although
some of these are ConTeXt related problems, it's a bit off-topic for the
general ConTeXt list, but also off-topic for the e-Omega list. (Of course,
when the developers of the CJK module decide to use e-Omega to implement
their outcome of the discussions, they can use the e-Omega list)

Anyway, these are my thoughts about it, and it's getting past my bedtime.
I'm sure that someone can persuade me to think otherwise! :-)

Sleep well,
Tim 

Re[2]: Re: CJK support in ConTeXt

[Giuseppe Bilotta]

Wednesday, August 27, 2003 Tim 't Hart wrote:

> I agree that there should be another list for e-Omega (Aleph) where we can
> discuss implementing multilingual code for ConTeXt, because I think it's a
> bit off-topic for the regular ConTeXt list.

> But on the other hand, I also support Hong Feng when he wants to create a
> mailing list just for the developers of the CJK package. Or maybe just a
> temporary list, just for organizational reasons.

> Also a reason is that it's not already sure if e-Omega is the best option
> for this (although after using Giuseppe's latest e-Omega version, I think it
> is the best option) and that the creation of the CJK module will need
> discussions that have nothing to do with e-Omega, like implementing CJK
> chapter headings, East Asian line breaking and numbering systems. Although
> some of these are ConTeXt related problems, it's a bit off-topic for the
> general ConTeXt list, but also off-topic for the e-Omega list. (Of course,
> when the developers of the CJK module decide to use e-Omega to implement
> their outcome of the discussions, they can use the e-Omega list)

> Anyway, these are my thoughts about it, and it's getting past my bedtime.
> I'm sure that someone can persuade me to think otherwise! :-)

While I agree that the CJK module is specialized, I'm not sure
the discussion should necessarily take place somewhere else.
After all, it's most definitely a ConTeXt-specific thing.

Maybe you should go the same route as the other "external"
modules? I don't know, where do the other ConTeXt module get
discussed? Does DocbookInConTeXt have its own mailing list?

Anyway, I'm glad to hear that the latest e-Omega is working for
you too :) these news are the best ones me and the others of
the e-Omega task force can receive :)

But, if you find a way to support CJK *without* e-Omega, you
should also try that route; there already is some support for
Chinese, AFAIK, in ConTeXt, so maybe something else can be
done without making use of Omega-specific features (you're
welcome to make use of e-TeX features, which are now almost
universally available).

In any case, good luck with your project,

-- 
Giuseppe "Oblomov" Bilotta

RE: CJK support in ConTeXt

[Tim 't Hart]

Hong Feng wrote:

> The currently ConTeXt release support UTF-8, CJKV locales can all be
> represented in UTF-8, so the mechanism is already done.
> 
> IMHO, the real one work to do is to make the sets of typesetting
> conventions (typesetting rules) for each locale of CJKV (like
> the Chinese package did), that would varies quite much from
> locale to locale, all all of them are based on the same
> underlayed mechanism.

My very first experiment with trying to use Japanese with ConTeXt made use
of ConTeXt's built-in support for UTF-8. I soon found out that there is no
line breaking present when UTF-8 support is used. So that has to be
implemented. I'm not sure, but I remember that Hans once told me that that
the typesetting mechanism and line breaking algorithm as used in the Chinese
module cannot be directly used for UTF-8 support. Therefore I'm not sure if
we can simply say that the 'mechanism is already done'. Maybe Hans can tell
how difficult it is to add a line breaking mechanism to the UTF-8 support?

It would be really handy if we could use ConTeXt's UTF-8 support so that
some work for a CJK module is already done. But on the other hand, by using
e-Omega, a lot of work is also already done. We have to make sure that
adapting the UTF-8 mechanism doesn't take more time and effort than creating
a module based on e-Omega. A CJK module based on e-Omega is maybe easier to
write and more flexible. For example, not everyone can write documents in
UTF-8. e-Omega will allow almost any kind of file encoding as long as there
is an OTP available to convert it to Unicode. 

So I think the questions we have to ask ourselves are: Do we make line
breaking and typesetting algorithms for ConTeXt's UTF-8 support or for
e-Omega? What is the time and effort needed in creating a CJK module for
each solution? And what solution gives the most powerful typesetting
options?

Personally, I would go for the e-Omega option, but I wouldn't mind seeing a
module based on ConTeXt's UTF-8 support. As long as there is support for
CJK, I'll be happy! :)

My best,
Tim

Re: CJK support in ConTeXt

[Alan Hoenig]

`Aleph' gets my vote.  I think it's a great idea.

---Alan

Re: CJK support in ConTeXt

[Hans Hagen]

At 09:15 27/08/2003 +0200, you wrote:

>quite useable. If anyone else is interested in what I've produced so
>far, please let me know.

can you put together a zip with (1) fonts, (2) macros (3) examples - also 
in pdf, so that i can have a look at it?

Hans
-------------------------------------------------------------------------
                                   Hans Hagen | PRAGMA ADE | pragma@wxs.nl
                       Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
  tel: +31 (0)38 477 53 69 | fax: +31 (0)38 477 53 74 | www.pragma-ade.com
-------------------------------------------------------------------------
                        information: http://www.pragma-ade.com/roadmap.pdf
                     documentation: http://www.pragma-ade.com/showcase.pdf
-------------------------------------------------------------------------

Re: CJK support in ConTeXt

[Hong Feng]

> > It's time to discuss the topic "CJK support in ConTeXt" in a more public
> > place.
> 
> Agree! 

The currently ConTeXt release support UTF-8, CJKV locales can all be 
represented in UTF-8, so the mechanism is already done.

IMHO, the real one work to do is to make the sets of typesetting 
conventions (typesetting rules) for each locale of CJKV (like 
the Chinese package did), that would varies quite much from 
locale to locale, all all of them are based on the same 
underlayed mechanism.

Best,
Hong


_________________________________________________________
Do You Yahoo!? 
启用电邮帐号，领会雅虎通[身临其境聊电影]的动感魅力，还有网络摄像头+雅虎通收音机等你来拿
http://cn.rd.yahoo.com/mail_cn/tag/?http://cn.messenger.yahoo.com

Re: CJK support in ConTeXt

[Idris S Hamid]

Giuseppe Bilotta wrote:

> Wednesday, August 27, 2003 Hans Hagen wrote:
>
> HH> At 12:37 27/08/2003 +0200, you wrote:
> >>(I hate replying to myself, but ...)
> >>
> >>Ok, since it seems that e-Omega is starting to get some "mass"
> >>interest, I'm thinking about setting up a mailing list for it.
> >>Maybe two, one for users and one for developers. Any
> >>suggestions on the listserver to use?
>
> HH> i can ask the ntg, since they host more lists, ok? you can mnage the list
> HH> with mailmanager
>
> AFAIK, the two "biggest" TeX mailing lists holders are ntg and
> tug; either is fine for me. Should I create two lists, or do
> you think that one list is enough?

FWIW, I think that one list is certainly enough. Looking at the Omega list I'm
not sure there was any real benefit in having two lists; we certainly don't need
to divide energies at this stage with two low-traffic lists.

Now when eOmega becomes a hit, we can adjust accordingly.


> Given the ideas on the name change (which will be factual with
> the first officially "stable" release, that is the one
> following RC2, probably) it's probably better to use aleph in
> the mailing list name. So:

BTW, since u and Hans seem to like my idea (and I'm cc'ing the others to see if
they object), is it official? Are we going with Aleph?

Best wishes
Idris

Re: CJK support in ConTeXt

[Tim 't Hart]

Jin-Hwan Cho wrote:

>It's time to discuss the topic "CJK support in ConTeXt" in a more
>public place.

I couldn't agree more! :-)

>The current version of ConTeXt supports Chinese only. But the same
>mechasism enables Korean too. The remaining language is Japanese.
>In some sense, it is possible to support Japanese in the current
>subfont mechanism. But several issues (e.g. Japanese kerning system
>as explained by Okumura) may occur.

Could you explain what these issues are? For the past couple of
months I've been busy writing Japanese support for ConTeXt, based
on the Chinese module. I'm not an expert in this, so I'm sure the
method I used for writing Japanese support is wrong and can be
improved. But for my own personal use it has already proven to be
quite useable. If anyone else is interested in what I've produced so
far, please let me know.

I would like to know what the issues with Japanese kerning are. I do
not know a lot about Japanese typography and would like to know what
the limitations of code based on the Chinese module are.

>My suggestion for supporting CJK characters in ConTeXt is to write a
>new module using e-Omega. Even though e-Omega does not produce PDF
>format directly, there are not much problem with DVIPDFMx.

When I first tried to use Japanese with ConTeXt, I first looked at
e-Omega. The reason I gave up and tried to adapt the Chinese module
was that e-Omega was unstable on my machine. Version 1.23 crashed
immediately (got stuck in a loop producing [] characters). I tried
1.15 because I heard that was more stable. I got some Japanese
output with this version, but when I tried creating more than a
paragraph, it locked as well. But this was all a few months ago.
Does anybody know if things with e-Omega have improved recently?

Anyway, I would like to see a common CJK module for ConTeXt, where
Chinese, Korean and Japanese are based on the same base code. It
doesn't matter to me if it's implemented using subfonts or on using
e-Omega, as long as it's stable and useable. Therefore, I would
like to offer my help in creating CJK support.

My best,
Tim