xml, context and hyphenation

xml, context and hyphenation

[Wolfgang Zillig]

Hello,

I have problems with hyphenation when german öäü are in these words. I'm 
processing xml files which are in unicode. The font encoding for the pdf 
is EC (as I was told there are problems with OT1 encoded fonts).

Any suggestions?

Wolfgang

Re: xml, context and hyphenation

[Hans Hagen]

Wolfgang Zillig wrote:

> Hello,
>
> I have problems with hyphenation when german ��� are in these words. 
> I'm processing xml files which are in unicode. The font encoding for 
> the pdf is EC (as I was told there are problems with OT1 encoded fonts).


normally ec enoding should work ok; the context log mentions what 
patterns are loaded under which encodings.

Hans

Re: xml, context and hyphenation

[Wolfgang Zillig]

Hello,

sory for the late reply but I checked it again and I created an example
document please look at:
http://www.student.kuleuven.ac.be/~s0167070/env.pdf : now one word
"Vollständigkeit" ist
not hyphenated but "übernehmen" is hyphenated. As I process from a XML
file I can not do something like \hyphenation{Voll-stän-dig-keit}. What
can I do?

Thanks

Wolfgang

Hans Hagen schrieb:

> Wolfgang Zillig wrote:
>
>> Hello,
>>
>> I have problems with hyphenation when german ��� are in these words. 
>> I'm processing xml files which are in unicode. The font encoding for 
>> the pdf is EC (as I was told there are problems with OT1 encoded fonts).
>
>
>
> normally ec enoding should work ok; the context log mentions what 
> patterns are loaded under which encodings.
>
> Hans
> _______________________________________________
> ntg-context mailing list
> ntg-context@ntg.nl
> http://www.ntg.nl/mailman/listinfo/ntg-context
>

Re: xml, context and hyphenation

[Taco Hoekwater]

Wolfgang Zillig wrote:
> Hello,
> 
> sory for the late reply but I checked it again and I created an example
> document please look at:
> http://www.student.kuleuven.ac.be/~s0167070/env.pdf : now one word
> "Vollständigkeit" ist
> not hyphenated but "übernehmen" is hyphenated. As I process from a XML
> file I can not do something like \hyphenation{Voll-stän-dig-keit}. What
> can I do?

You can put the \hyphenation command in your cont-sys.tex
or cont-loc.tex. Or you can embed a

  <?context-directive job environment myenv?>

and put the \hyphenation command in myenv.tex.
You have to switch to utf-8 encoding as well, I think.

FYI,  ConTeXt simply seems to miss the first hyphen,
I get the result of \hyphenation{Vollstän-dig-keit} from
\showhyphens (you may want to increase the tolerance when
working with xml input files).

Cheers, Taco

Re: xml, context and hyphenation

[Wolfgang Zillig]

Hello,

when I add \hyphenation{Voll-stön-dig-keit} into my tex file I get the 
following error:

(C:\texmf\tex\context\base\xtag-utf.tex
loading         : Context XML Macros (UTF-8)
)
! Not a letter.
\@default@adiaeresis ->\char 228
                                
\utfunihashglyph ...r \utfdiv {#1}}{#1}\endcsname
                                                  \fi
<inserted text> Voll-stä
                         n-dig-keit
\hyphenation ...malhyphenation {\the \scratchtoks
                                                  }\endgroup
l.118 \hyphenation{Voll-stän-dig-keit}
                                      
?


Any ideas?

Wolfgang


Taco Hoekwater schrieb:

> Wolfgang Zillig wrote:
>
>> Hello,
>>
>> sory for the late reply but I checked it again and I created an example
>> document please look at:
>> http://www.student.kuleuven.ac.be/~s0167070/env.pdf : now one word
>> "Vollständigkeit" ist
>> not hyphenated but "übernehmen" is hyphenated. As I process from a XML
>> file I can not do something like \hyphenation{Voll-stän-dig-keit}. What
>> can I do?
>
>
> You can put the \hyphenation command in your cont-sys.tex
> or cont-loc.tex. Or you can embed a
>
>  <?context-directive job environment myenv?>
>
> and put the \hyphenation command in myenv.tex.
> You have to switch to utf-8 encoding as well, I think.
>
> FYI,  ConTeXt simply seems to miss the first hyphen,
> I get the result of \hyphenation{Vollstän-dig-keit} from
> \showhyphens (you may want to increase the tolerance when
> working with xml input files).
>
> Cheers, Taco
>
>
>
> _______________________________________________
> ntg-context mailing list
> ntg-context@ntg.nl
> http://www.ntg.nl/mailman/listinfo/ntg-context
>

Re: xml, context and hyphenation

[Taco Hoekwater]

[-- Attachment #1: Type: text/plain, Size: 1032 bytes --]

Wolfgang Zillig wrote:
> Hello,
> 
> when I add \hyphenation{Voll-stön-dig-keit} into my tex file I get the 
> following error:
> 
> (C:\texmf\tex\context\base\xtag-utf.tex
> loading         : Context XML Macros (UTF-8)
> )
> ! Not a letter.
> \@default@adiaeresis ->\char 228
>                                \utfunihashglyph ...r \utfdiv 
> {#1}}{#1}\endcsname
>                                                  \fi
> <inserted text> Voll-stä
>                         n-dig-keit
> \hyphenation ...malhyphenation {\the \scratchtoks
>                                                  }\endgroup
> l.118 \hyphenation{Voll-stän-dig-keit}
>                                      ?
> 
> 
> Any ideas?

Perhaps your input is latin1, only pretending to be utf-8.
In a latin-1 text editor, it should look like this:

   \hyphenation{Voll-stän-dig-keit}

But this sort of stuff is tricky to get right in e-mail,
and I am just guessing. What I tried is attached, and that
worked for me.

Cheers, Taco

[-- Attachment #2: vols-utf.tex --]
[-- Type: application/x-tex, Size: 124 bytes --]

[-- Attachment #3: Type: text/plain, Size: 139 bytes --]

_______________________________________________
ntg-context mailing list
ntg-context@ntg.nl
http://www.ntg.nl/mailman/listinfo/ntg-context

Re: xml, context and hyphenation

[Wolfgang Zillig]

Hello Taco,

when I try to compile your example file:

\language[de]
\useencoding[utf]
\hyphenation{Voll-stän-dig-keit}
\starttext
\showhyphens{Vollständigkeit}
\stoptext


Then I get no output! The logfile is at the end of the message! It
complains about that the coding utf is unknown! 

You are right, selecting the right encoding is not always easy. I'm
quite sure that my xml input is utf but I'm not always sure what scite
is doing whatever I select. Perhaps I should change the text editor.

Here the log file:


This is pdfeTeX, Version 3.141592-1.21a-2.2 (MiKTeX 2.4) (preloaded
format=cont-en 2005.10.23)  7 NOV 2005 18:44
entering extended mode
**vols-utf-1.tex
(vols-utf-1.tex

ConTeXt  ver: 2005.08.31  fmt: 2005.10.23  int: english  mes: english

language        : language en is active
<protectionstate 0>
system          : cont-new loaded
(C:\texmf\tex\context\base\cont-new.tex
systems         : beware: some patches loaded from cont-new.tex
color           : palette rollover is available
)
system          : cont-old loaded
(C:\texmf\tex\context\base\cont-old.tex
loading         : Context Old Macros
)
system          : cont-fil loaded
(C:\texmf\tex\context\base\cont-fil.tex
loading         : Context File Synonyms
)
system          : cont-sys.rme loaded
(C:\texmf\tex\context\user\cont-sys.rme
(C:\texmf\tex\context\base\./type-exa.tex)
(C:\texmf\tex\context\base\./type-syn.tex)
(C:\texmf\tex\context\base\./type-enc.tex)
(C:\texmf\tex\context\base\./type-dis.tex)
(C:\texmf\tex\context\base\./type-siz.tex)
(C:\texmf\tex\context\base\./type-map.tex)
(C:\texmf\tex\context\base\./type-spe.tex)
(C:\texmf\tex\context\base\./type-akb.tex))
bodyfont        : 12pt rm is loaded
language        : patterns en->ec:ec->1->2:2 uk->ec:ec->2->2:2
de->texnansi:tex
nansi->3->2:2 de->ec:ec->4->2:2 fr->texnansi:texnansi->5->2:2
fr->ec:ec->6->2:2
 es->ec:ec->7->2:2 pt->texnansi:texnansi->8->2:2 pt->ec:ec->9->2:2
it->texnansi
:texnansi->10->2:2 it->ec:ec->11->2:2 nl->texnansi:texnansi->12->2:2
nl->ec:ec-
>13->2:2 cz->il2:il2->14->2:2 cz->ec:ec->15->2:2 sk->il2:il2->16->2:2
sk->ec:ec
->17->2:2 pl->pl0:pl0->18->2:2 pl->ec:ec->19->2:2 loaded
specials        : tex,postscript,rokicki loaded
system          : vols-utf-1.top loaded
(./vols-utf-1.top
specials        : loading definition file tpd
(C:\texmf\tex\context\base\spec-tpd.tex
specials        : loading definition file fdf
(C:\texmf\tex\context\base\spec-fdf.tex)
specials        : fdf loaded
)
specials        : fdf,tpd loaded
)
encoding        : unknown coding utf8
(./vols-utf-1.tuo) (./vols-utf-1.tuo) (./vols-utf-1.tuo) (./vols-utf-1.tuo)
(./vols-utf-1.tuo) (./vols-utf-1.tuo) (./vols-utf-1.tuo) (./vols-utf-1.tuo)
(./vols-utf-1.tuo) (./vols-utf-1.tuo) (./vols-utf-1.tuo) (./vols-utf-1.tuo)
(./vols-utf-1.tuo)
systems         : begin file vols-utf-1 at line 4

Underfull \hbox (badness 10000) in paragraph at lines 5--5
[] \*10ptrmtf* Voll-stän-dig-keit

\hbox(8.86499+1.94443)x16383.99998, glue set 9784.75977 []

systems         : end file vols-utf-1 at line 6
 ) 
Here is how much of TeX's memory you used:
 1312 strings out of 61144
 19912 string characters out of 647756
 514394 words of memory out of 1076582
 37078 multiletter control sequences out of 60000
 78048 words of font info for 38 fonts, out of 1000000 for 2000
 183 hyphenation exceptions out of 4999
 31i,2n,28p,405b,479s stack positions out of
5000i,500n,10000p,200000b,32768s
PDF statistics:
 0 PDF objects out of 300000
 0 named destinations out of 300000
 1 words of extra memory for PDF output out of 65536

No pages of output.



Zitat von Taco Hoekwater <taco@elvenkind.com>:

> Wolfgang Zillig wrote:
> > Hello,
> > 
> > when I add \hyphenation{Voll-stön-dig-keit} into my tex file I get
> the 
> > following error:
> > 
> > (C:\texmf\tex\context\base\xtag-utf.tex
> > loading         : Context XML Macros (UTF-8)
> > )
> > ! Not a letter.
> > \@default@adiaeresis ->\char 228
> >                                \utfunihashglyph ...r \utfdiv 
> > {#1}}{#1}\endcsname
> >                                                  \fi
> > <inserted text> Voll-stä
> >                         n-dig-keit
> > \hyphenation ...malhyphenation {\the \scratchtoks
> >                                 

                 }\endgroup
> > l.118 \hyphenation{Voll-stän-dig-keit}
> >                                      ?
> > 
> > 
> > Any ideas?
> 
> Perhaps your input is latin1, only pretending to be utf-8.
> In a latin-1 text editor, it should look like this:
> 
>    \hyphenation{Voll-stän-dig-keit}
> 
> But this sort of stuff is tricky to get right in e-mail,
> and I am just guessing. What I tried is attached, and that
> worked for me.
> 
> Cheers, Taco
> 

Re: xml, context and hyphenation

[Taco Hoekwater]

Wolfgang Zillig wrote:
> Hello Taco,
> 
> when I try to compile your example file:
> 
> Then I get no output! The logfile is at the end of the message! 

That was intentional. \showhyphens only writes a message
to the terminal, no typesetting was doen.

> It complains about that the coding utf is unknown! 

That was a mistake. I used \useencoding, where I should have
used \enableregime. Revised version:

   \language[de]
   \enableregime[utf]
   \hyphenation{Voll-stän-dig-keit}
   \starttext
   \showhyphens{Vollständigkeit}
   \stoptext

Either way, your output from my test file looks fine.

Good luck, Taco

Re: xml, context and hyphenation

[Hans Hagen]

Taco Hoekwater wrote:

> Wolfgang Zillig wrote:
>
>> Hello Taco,
>>
>> when I try to compile your example file:
>>
>> Then I get no output! The logfile is at the end of the message! 
>
>
> That was intentional. \showhyphens only writes a message
> to the terminal, no typesetting was doen.
>
>> It complains about that the coding utf is unknown! 
>
>
> That was a mistake. I used \useencoding, where I should have
> used \enableregime. Revised version:
>
>   \language[de]
>   \enableregime[utf]
>   \hyphenation{Voll-stän-dig-keit}
>   \starttext
>   \showhyphens{Vollständigkeit}
>   \stoptext
>
> Either way, your output from my test file looks fine.

if you want to define hyphenation exceptions in XML code:

\useXMLfilter[hyp]

(see xtag-hyp for more info)

Hans

Re: xml, context and hyphenation

[Wolfgang Zillig]

I have created an other hyph.xml file with the following content:
    <hyphenations language="de" regime="utf">
    <hyphenation>Voll-stän-dig-keit</hyphenation></hyphenations>

this file is included by:
    \processXMLfilegrouped {hyph.xml}

In my file which I use for processing I use now:
    \useXMLfilter [utf,hyp]

The compillation stops with the following message:

xml             : loading module hyp
(C:\texmf\tex\context\base\xtag-hyp.tex
loading         : Context XML Macros (hyphenation)
) (./env.tuo) (./env.tuo) (./env.tuo) (./env.tuo) (./env.tuo) (./env.tuo)
(./env.tuo) (./env.tuo) (./env.tuo) (./env.tuo) (./env.tuo) (./env.tuo)
(./env.tuo)
systems         : begin file env at line 127
(./hyph.xml
! Not a letter.
\@ec@adiaeresis ->\char 228
                           
\utfunihashglyph ...r \utfdiv {#1}}{#1}\endcsname
                                                  \fi
<inserted text> Voll-stä
                         n-dig-keit
\hyphenation ...malhyphenation {\the \scratchtoks
                                                  }\endgroup
l.2 <hyphenation>Voll-stän-dig-keit</hyphenation>
                                                  </hyphenations>
?


What do I do wrong?

Many thanks

Wolfgang


Hans Hagen schrieb:

> Taco Hoekwater wrote:
>
>> Wolfgang Zillig wrote:
>>
>>> Hello Taco,
>>>
>>> when I try to compile your example file:
>>>
>>> Then I get no output! The logfile is at the end of the message! 
>>
>>
>>
>> That was intentional. \showhyphens only writes a message
>> to the terminal, no typesetting was doen.
>>
>>> It complains about that the coding utf is unknown! 
>>
>>
>>
>> That was a mistake. I used \useencoding, where I should have
>> used \enableregime. Revised version:
>>
>>   \language[de]
>>   \enableregime[utf]
>>   \hyphenation{Voll-stän-dig-keit}
>>   \starttext
>>   \showhyphens{Vollständigkeit}
>>   \stoptext
>>
>> Either way, your output from my test file looks fine.
>
>
> if you want to define hyphenation exceptions in XML code:
>
> \useXMLfilter[hyp]
>
> (see xtag-hyp for more info)
>
> Hans
> _______________________________________________
> ntg-context mailing list
> ntg-context@ntg.nl
> http://www.ntg.nl/mailman/listinfo/ntg-context
>

Re: xml, context and hyphenation

[Hans Hagen]

Wolfgang Zillig wrote:

>
> I have created an other hyph.xml file with the following content:
>    <hyphenations language="de" regime="utf">
>    <hyphenation>Voll-stän-dig-keit</hyphenation></hyphenations>
>
> this file is included by:
>    \processXMLfilegrouped {hyph.xml}
>
> In my file which I use for processing I use now:
>    \useXMLfilter [utf,hyp]
>
> The compillation stops with the following message:
>
> xml             : loading module hyp
> (C:\texmf\tex\context\base\xtag-hyp.tex
> loading         : Context XML Macros (hyphenation)
> ) (./env.tuo) (./env.tuo) (./env.tuo) (./env.tuo) (./env.tuo) (./env.tuo)
> (./env.tuo) (./env.tuo) (./env.tuo) (./env.tuo) (./env.tuo) (./env.tuo)
> (./env.tuo)
> systems         : begin file env at line 127
> (./hyph.xml
> ! Not a letter.
> \@ec@adiaeresis ->\char 228

which means that in ec encoding is not suited here

btw, if you use a recent context, it has hyphenation patterns that are 
less encoding dependent

Hans

Re: xml, context and hyphenation

[Taco Hoekwater]

Hans Hagen wrote:
> 
> btw, if you use a recent context, it has hyphenation patterns that are 
> less encoding dependent

Is that why it worked for me?

Taco