setuptex changes in minimals (was: bugs in beta)

setuptex changes in minimals (was: bugs in beta)

[Mojca Miklavec]

On Fri, May 8, 2009 at 11:46, Yanrui Li wrote:
>
> Recently I have found some environment variables in setuptex script
> are commented out sometimes, such as TEXMFLOCAL, TEXMFCONTEXT and so
> on. Is this be done deliberately?

Yes (we were bored during some Polish talks at BachoTeX).

The reason for setuptex setting all those variables in the old days
was to override tetex's environmental variables, but in principle this
is not needed at all unless tetex is present on computer. The reason
for removing them again was to make sure that Hans' scripts work
properly even when no environmental variable is set (which will be the
case on TeX Live for example).

The only variable change that's really needed is fixing PATH, but it's
so convenient to be able to run
    ". /path/to/setuptex"
instead of typing the long
    export PATH="/path/to/minimals/bin:$PATH"

We nevertheless left two or three variables since ConTeXt didn't yet
work properly without them, but we'll remove those as well.

If anyone uses minimals on tetex, we can create an additional file
again that would enable complete isolation.

We didn't clean up the windows script yet as it would need some more
testing (and we had enough problem by minimals not running at all on
unix), but that's still on TODO list ...

We also silently added TEXMFHOME (while Hans was not watching), but
maybe we need to change the exact path.

If you have any particular question or request about that, let us know.

Mojca
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: setuptex changes in minimals (was: bugs in beta)

[Yanrui Li]

On Fri, May 8, 2009 at 7:19 PM, Mojca Miklavec
<mojca.miklavec.lists@gmail.com> wrote:
> On Fri, May 8, 2009 at 11:46, Yanrui Li wrote:
>>
>> Recently I have found some environment variables in setuptex script
>> are commented out sometimes, such as TEXMFLOCAL, TEXMFCONTEXT and so
>> on. Is this be done deliberately?
>
> Yes (we were bored during some Polish talks at BachoTeX).
>
> The reason for setuptex setting all those variables in the old days
> was to override tetex's environmental variables, but in principle this
> is not needed at all unless tetex is present on computer. The reason
> for removing them again was to make sure that Hans' scripts work
> properly even when no environmental variable is set (which will be the
> case on TeX Live for example).
>
> The only variable change that's really needed is fixing PATH, but it's
> so convenient to be able to run
>    ". /path/to/setuptex"
> instead of typing the long
>    export PATH="/path/to/minimals/bin:$PATH"
>
> We nevertheless left two or three variables since ConTeXt didn't yet
> work properly without them, but we'll remove those as well.
>
> If anyone uses minimals on tetex, we can create an additional file
> again that would enable complete isolation.
>
> We didn't clean up the windows script yet as it would need some more
> testing (and we had enough problem by minimals not running at all on
> unix), but that's still on TODO list ...
>
> We also silently added TEXMFHOME (while Hans was not watching), but
> maybe we need to change the exact path.
>
> If you have any particular question or request about that, let us know.
>

Thanks for your explanation. Now I see.

I have no questions for these, only I have to modify instruction of these
variables in the tutorial which I wrote for Chinese users :)

-- 
Best wishes,
Li Yanrui
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: setuptex changes in minimals (was: bugs in beta)

[Mojca Miklavec]

On Fri, May 8, 2009 at 14:16, Yanrui Li wrote:
>
> I have no questions for these, only I have to modify instruction of these
> variables in the tutorial which I wrote for Chinese users :)

What exactly do you describe there? Do you write instructions for
users that want to modify their installation?

Mojca
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: setuptex changes in minimals (was: bugs in beta)

[Yanrui Li]

On Sat, May 9, 2009 at 1:43 AM, Mojca Miklavec
<mojca.miklavec.lists@gmail.com> wrote:
> On Fri, May 8, 2009 at 14:16, Yanrui Li wrote:
>>
>> I have no questions for these, only I have to modify instruction of these
>> variables in the tutorial which I wrote for Chinese users :)
>
> What exactly do you describe there? Do you write instructions for
> users that want to modify their installation?
>

MkIV have not support Chinese typesetting now, so I modified
scrp-ini.lua and scrp-cjk.lua for compression of Chinese punctuations.
In addition, I wrote a lua program which can solve the
problem of margin alignment of Chinese puncuations appeared on the
right side. To simplify the usage of these modified script files, I
used TEXMFLOCAL instead of the specific path. But this is not a very
serious matter, because I am going to rewrite the related content
easily and only a few Chinese users are interested in MkIV
.

-- 
Best wishes,
Li Yanrui
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: setuptex changes in minimals (was: bugs in beta)

[Yanrui Li]

On Sat, May 9, 2009 at 2:17 AM, Yanrui Li <liyanrui.m2@gmail.com> wrote:
> On Sat, May 9, 2009 at 1:43 AM, Mojca Miklavec
> <mojca.miklavec.lists@gmail.com> wrote:
>> On Fri, May 8, 2009 at 14:16, Yanrui Li wrote:
>>>
>>> I have no questions for these, only I have to modify instruction of these
>>> variables in the tutorial which I wrote for Chinese users :)
>>
>> What exactly do you describe there? Do you write instructions for
>> users that want to modify their installation?
>>
>
> MkIV have not support Chinese typesetting now, so I modified
> scrp-ini.lua and scrp-cjk.lua for compression of Chinese punctuations.
> In addition, I wrote a lua program which can solve the
> problem of margin alignment of Chinese puncuations appeared on the
> right side. To simplify the usage of these modified script files, I
> used TEXMFLOCAL instead of the specific path. But this is not a very
> serious matter, because I am going to rewrite the related content
> easily and only a few Chinese users are interested in MkIV
> .
>


Sorry!

s/MkIV have not support Chinese typesetting now / MkIV have not
support Chinese typesetting well now/g


-- 
Best wishes,
Li Yanrui
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: setuptex changes in minimals (was: bugs in beta)

[Yue Wang]

On Sat, May 9, 2009 at 2:20 AM, Yanrui Li <liyanrui.m2@gmail.com> wrote:
> Sorry!
>
> s/MkIV have not support Chinese typesetting now / MkIV have not
> support Chinese typesetting well now/g


We can start a fork of scrp-* on code.google.com/p/ctex-kit,
making a usable version, and later ask Hans to merge back into ConTeXt.

Yue Wang

>
>
> --
> Best wishes,
> Li Yanrui
> ___________________________________________________________________________________
> If your question is of interest to others as well, please add an entry to the Wiki!
>
> maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
> webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
> archive  : https://foundry.supelec.fr/projects/contextrev/
> wiki     : http://contextgarden.net
> ___________________________________________________________________________________
>
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: setuptex changes in minimals

[Hans Hagen]

Yue Wang wrote:
> On Sat, May 9, 2009 at 2:20 AM, Yanrui Li <liyanrui.m2@gmail.com> wrote:
>> Sorry!
>>
>> s/MkIV have not support Chinese typesetting now / MkIV have not
>> support Chinese typesetting well now/g
> 
> 
> We can start a fork of scrp-* on code.google.com/p/ctex-kit,
> making a usable version, and later ask Hans to merge back into ConTeXt.

it all depends how scrp evolves as it has to deal with more issues too; 
i will not merge code that i don't fully understand but experimenting 
does not hurt; one problem with script stuff is that (in the future) it 
might be closely cooperate with other mechanisms of context (it used to 
be part of the font analyser, now it's plugged in elsewhere but even 
that might change, for instance when in upcoming versions of luatex we 
have a bit more control)

also, some issues will be dealt with in other code in the future (i 
simply cannot touch all code at the same time) so don't expect mkiv to 
be finished before 2012 (when luatex 1 is ready); it's a stepwise 
process and it's only driven by context (i.e. generic code only shows up 
when context code works okay);

anyhow, as long as nobody tells me what the expected behaviour is, with 
test code etc i can;t fix anything; one complication (e.g. with chinese) 
is that over the last few year is got conflicting requests; now, if 
there are multiple ways to deal with it, then i can implement variants 
but only if i know the exact conventions (with proper names for them)

for instance when i found out that cjk-korean is not the same as 
cjk-chinese (one of them) i split the machinery so we're not talking cjk 
here, but very specific needs; after the split i never tested chinese 
(no samples) and no one else propbably did that either

keep in mind that context is not meant for one language but for many and 
that when possible mechanisms need to be able to used mixed; there will 
never be exclusive solutions for one languages unless it's a very 
specialized module

anyhow, we can best follow the same route as with other language / 
script support

- we need consistent specs, or maybe multiple specs as variants
- as well as test files for each such case
- and then we can look into what can be improved or added

Hans

-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
      tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com
                                              | www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: setuptex changes in minimals

[Yanrui Li]

On Sat, May 9, 2009 at 4:02 PM, Hans Hagen <pragma@wxs.nl> wrote:
> Yue Wang wrote:
>>
>> On Sat, May 9, 2009 at 2:20 AM, Yanrui Li <liyanrui.m2@gmail.com> wrote:
>>>
>>> Sorry!
>>>
>>> s/MkIV have not support Chinese typesetting now / MkIV have not
>>> support Chinese typesetting well now/g
>>
>>
>> We can start a fork of scrp-* on code.google.com/p/ctex-kit,
>> making a usable version, and later ask Hans to merge back into ConTeXt.
>
> it all depends how scrp evolves as it has to deal with more issues too; i
> will not merge code that i don't fully understand but experimenting does not
> hurt; one problem with script stuff is that (in the future) it might be
> closely cooperate with other mechanisms of context (it used to be part of
> the font analyser, now it's plugged in elsewhere but even that might change,
> for instance when in upcoming versions of luatex we have a bit more control)
>

What I have done is only experiment. I will try my best to follow the
evolves of the
 scrp-* files until that day, when MkIV support Chinese typesetting well, comes.
I have to introduce MkIV to Chinese users and make them paying close attention
to it in this way.

> also, some issues will be dealt with in other code in the future (i simply
> cannot touch all code at the same time) so don't expect mkiv to be finished
> before 2012 (when luatex 1 is ready); it's a stepwise process and it's only
> driven by context (i.e. generic code only shows up when context code works
> okay);
>
> anyhow, as long as nobody tells me what the expected behaviour is, with test
> code etc i can;t fix anything; one complication (e.g. with chinese) is that
> over the last few year is got conflicting requests; now, if there are
> multiple ways to deal with it, then i can implement variants but only if i
> know the exact conventions (with proper names for them)
>

Yes. It is a key reson that we are still not familiar with MkIV well
now. But with
in-depth understanding of MkIV, maybe many suggestions will be post to you.
Of course these suggestions should be collected according actual demands by
Chinese TeX users group.

> for instance when i found out that cjk-korean is not the same as cjk-chinese
> (one of them) i split the machinery so we're not talking cjk here, but very
> specific needs; after the split i never tested chinese (no samples) and no
> one else propbably did that either
>
> keep in mind that context is not meant for one language but for many and
> that when possible mechanisms need to be able to used mixed; there will
> never be exclusive solutions for one languages unless it's a very
> specialized module
>
> anyhow, we can best follow the same route as with other language / script
> support
>
> - we need consistent specs, or maybe multiple specs as variants
> - as well as test files for each such case
> - and then we can look into what can be improved or added
>

I think that maybe this needs a very long time until CJK users know what each
other needs. But three years should be enough :)

-- 
Best wishes,
Li Yanrui
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: setuptex changes in minimals

[Yue Wang]

Hi, list:

Well, TeX is evolving rapidly these years (especially in the LuaTeX world).
So there are many things that Chinese typographers never imagine about
in the past can now be solved thanks to LuaTeX's development.

For example:
- Now we can use Lua to detect the character code. so it is possible
to assign different fonts for latin, CJK, CJK-ext A, CJK-extB. font
switching is easy and elegent. This is not possible in tex82.
- It is ugly to produce a paragraph whose last line only consists one
Chinese character. In order to avoid that, tex82 programmers should
write very complicated macros. So many developers here tend to ignore
this problem. However, in LuaTeX, this is quite easy. It is simple to
scan the position before the punctuation and \par in lua, and insert a
line break penalty there.
- Punctuation compression. In the past this is not easy to do (a lot
of macros should be defined), and because each Chinese fonts are so
distinctly designed, their punctuations' positions in the character
box are different. So different parameters should be applied when
different fonts are involved in typesetting. However, LuaTeX make it
possible to calculate character bounding box information, so
punctuation compression is as easy as re-setting the punctuation
width.

There are many more examples, and these are just three. But we can see
that with the development of TeX engine (especially the open-up of TeX
engine), users might consider better ways for old solutions, and even
request for possibilities they never thought about before. So it is
possible for different users to provide different specifications.

But we (me, Li Yanrui, Chen Zhichu) as the board members of the
Chinese TeX Society, will try our best to organized the typesetting
discussion on local forums, taking different opinions into
consideration, and later provide a complete specification as long as
we can.

Chinese support in ConTeXt is not easy, we should thank all the
developers who contribute to our language (esp, Hans, Taco and
Wolfgang), their innovation and hardworking make Chinese typesetting
in ConTeXt possible, and drastically improve the performance of font
loading. Though the current solution is not perfect (eg. there are
many more things we should fix and implement, and the current font
loading time is still a bit too long), I think with the current fast
pace, we will have a very prospective solution near luatex 1.0:)

Yue Wang


On 5/9/09, Yanrui Li <liyanrui.m2@gmail.com> wrote:
> On Sat, May 9, 2009 at 4:02 PM, Hans Hagen <pragma@wxs.nl> wrote:
> > Yue Wang wrote:
> >>
> >> On Sat, May 9, 2009 at 2:20 AM, Yanrui Li <liyanrui.m2@gmail.com> wrote:
> >>>
> >>> Sorry!
> >>>
> >>> s/MkIV have not support Chinese typesetting now / MkIV have not
> >>> support Chinese typesetting well now/g
> >>
> >>
> >> We can start a fork of scrp-* on code.google.com/p/ctex-kit,
> >> making a usable version, and later ask Hans to merge back into ConTeXt.
> >
> > it all depends how scrp evolves as it has to deal with more issues too; i
> > will not merge code that i don't fully understand but experimenting does not
> > hurt; one problem with script stuff is that (in the future) it might be
> > closely cooperate with other mechanisms of context (it used to be part of
> > the font analyser, now it's plugged in elsewhere but even that might change,
> > for instance when in upcoming versions of luatex we have a bit more control)
> >
>
> What I have done is only experiment. I will try my best to follow the
> evolves of the
>  scrp-* files until that day, when MkIV support Chinese typesetting well, comes.
> I have to introduce MkIV to Chinese users and make them paying close attention
> to it in this way.
>
> > also, some issues will be dealt with in other code in the future (i simply
> > cannot touch all code at the same time) so don't expect mkiv to be finished
> > before 2012 (when luatex 1 is ready); it's a stepwise process and it's only
> > driven by context (i.e. generic code only shows up when context code works
> > okay);
> >
> > anyhow, as long as nobody tells me what the expected behaviour is, with test
> > code etc i can;t fix anything; one complication (e.g. with chinese) is that
> > over the last few year is got conflicting requests; now, if there are
> > multiple ways to deal with it, then i can implement variants but only if i
> > know the exact conventions (with proper names for them)
> >
>
> Yes. It is a key reson that we are still not familiar with MkIV well
> now. But with
> in-depth understanding of MkIV, maybe many suggestions will be post to you.
> Of course these suggestions should be collected according actual demands by
> Chinese TeX users group.
>
> > for instance when i found out that cjk-korean is not the same as cjk-chinese
> > (one of them) i split the machinery so we're not talking cjk here, but very
> > specific needs; after the split i never tested chinese (no samples) and no
> > one else propbably did that either
> >
> > keep in mind that context is not meant for one language but for many and
> > that when possible mechanisms need to be able to used mixed; there will
> > never be exclusive solutions for one languages unless it's a very
> > specialized module
> >
> > anyhow, we can best follow the same route as with other language / script
> > support
> >
> > - we need consistent specs, or maybe multiple specs as variants
> > - as well as test files for each such case
> > - and then we can look into what can be improved or added
> >
>
> I think that maybe this needs a very long time until CJK users know what each
> other needs. But three years should be enough :)
>
> --
> Best wishes,
> Li Yanrui
> ___________________________________________________________________________________
> If your question is of interest to others as well, please add an entry to the Wiki!
>
> maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
> webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
> archive  : https://foundry.supelec.fr/projects/contextrev/
> wiki     : http://contextgarden.net
> ___________________________________________________________________________________
>
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: setuptex changes in minimals

[Hans Hagen]

Yue Wang wrote:

> - It is ugly to produce a paragraph whose last line only consists one
> Chinese character. In order to avoid that, tex82 programmers should
> write very complicated macros. So many developers here tend to ignore
> this problem. However, in LuaTeX, this is quite easy. It is simple to
> scan the position before the punctuation and \par in lua, and insert a
> line break penalty there.

ok, but it might even be more clever, like minimum number of glyphs on 
last line or so (and words in case of other scripts)

> - Punctuation compression. In the past this is not easy to do (a lot
> of macros should be defined), and because each Chinese fonts are so
> distinctly designed, their punctuations' positions in the character
> box are different. So different parameters should be applied when
> different fonts are involved in typesetting. However, LuaTeX make it
> possible to calculate character bounding box information, so
> punctuation compression is as easy as re-setting the punctuation
> width.

i had code for that but removes it as fonts are pretty inconsistent; i 
will look into that again when we have (1) a couple of reference fonts 
without bugs in the tex distribution and (2) wiki pages with info about 
the quality of fonts (can later become a database); otherwise we end up 
in messy support situations

> But we (me, Li Yanrui, Chen Zhichu) as the board members of the
> Chinese TeX Society, will try our best to organized the typesetting
> discussion on local forums, taking different opinions into
> consideration, and later provide a complete specification as long as
> we can.

that would be very helpful

> pace, we will have a very prospective solution near luatex 1.0:)

sure


-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
      tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com
                                              | www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________