Hyphenating compound words in MKIV & microtypography

Hyphenating compound words in MKIV & microtypography

[Otso Helenius]

Hi,

MKIV seems to have a problem in hyphenating long finnish compound words in MKIV (2011.05.18 22:26).
Here is a minimalist example (columns added to exaggerate the problem):

\mainlanguage[fi]
\language[fi]
\starttext
\startcolumns[n=3]
kolmivaihekilowattituntimittari ympäristöliiketoiminta epäjärjestelmällisyydellistyttymättömyydellänsäkään järjestelmällisentelentelemättömyydellänsäkään 
\stopcolumns
\stoptext

Similarly with en-hyphenation and \input knuth, some of the lines stretch out in an ugly fashion.
What am I doing wrong? I don't have these problems in LaTeX.

P.S. The punctuation characters should be partially hung out in a justified paragraph (similar to
what microtype package does on LaTeX). Wikipedia says ConTeXt has "interfaces for handling
microtypography" but they don't seem as apparent and as easy as using microtype on LaTeX.

The wiki article Protrusion#MkIV presents \definefontfeature but I find it counterintuitive that one has to use

\definefontfeature[default][default][protrusion=quality,expansion=quality]
\setupalign[hz,hanging]

to enable the features. It is unclear if these lines switch the features on globally for all loaded fonts
(for example ttf/otf ones defined with \starttypescript). IMHO these features should be enabled by default
because they produce better typographic quality by default (which should be the preference), and could be
switched off anyway if wanted. The less setup lines one has to add before \starttext to get high quality
results the better.



Otso Helenius
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: Hyphenating compound words in MKIV & microtypography

[Hans Hagen]

On 23-5-2011 2:07, Otso Helenius wrote:
> Hi,
>
> MKIV seems to have a problem in hyphenating long finnish compound words in MKIV (2011.05.18 22:26).
> Here is a minimalist example (columns added to exaggerate the problem):
>
> \mainlanguage[fi]
> \language[fi]
> \starttext
> \startcolumns[n=3]
> kolmivaihekilowattituntimittari ympäristöliiketoiminta epäjärjestelmällisyydellistyttymättömyydellänsäkään järjestelmällisentelentelemättömyydellänsäkään
> \stopcolumns
> \stoptext
>
> Similarly with en-hyphenation and \input knuth, some of the lines stretch out in an ugly fashion.
> What am I doing wrong? I don't have these problems in LaTeX.

it all depends on the defaults for tolerance and emergencystretch ... 
they determine where tex will break

in such an extreme case (narrow columns and long words) I'd just use

\setupalign[flushleft]


> P.S. The punctuation characters should be partially hung out in a justified paragraph (similar to
> what microtype package does on LaTeX). Wikipedia says ConTeXt has "interfaces for handling
> microtypography" but they don't seem as apparent and as easy as using microtype on LaTeX.

fwiw: one has rather detailed control over what hangs and how much it 
hangs or protrudes (using character classes); we predefine a few cases 
in font-ext.lua but more can be defined if needed (not that I've noticed 
that much demand)

in the case of hz/protrusion "easy"can also easily become ugly

> The wiki article Protrusion#MkIV presents \definefontfeature but I find it counterintuitive that one has to use
>
> \definefontfeature[default][default][protrusion=quality,expansion=quality]
> \setupalign[hz,hanging]

sure, (1) it's a font related feature that should be used with care and 
(2) you want to be able to turn it on and off

> to enable the features. It is unclear if these lines switch the features on globally for all loaded fonts
> (for example ttf/otf ones defined with \starttypescript). IMHO these features should be enabled by default
> because they produce better typographic quality by default (which should be the preference), and could be
> switched off anyway if wanted. The less setup lines one has to add before \starttext to get high quality
> results the better.

it produces larger files, take more runtime and in most cases does not 
look better at all; for instance one can argue that protrusion gives 
more width but at the same time it introduces the boundary condition 
that some characters have to hang; or take hz (expansion) ... when 
applied so much that it gets visible, it will also lead to similar 
shapes in successive lines to look different

so, just enabling these features assuming that the output looks better 
is debatable (sometimes adding or removing a word is more effective)

Hans

-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
     tel: 038 477 53 69 | voip: 087 875 68 74 | www.pragma-ade.com
                                              | www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: Hyphenating compound words in MKIV & microtypographyy

[Hans Hagen]

On 23-5-2011 5:54, Otso Helenius wrote:
>> it produces larger files, take more runtime and in most cases does
>> not look better at all; for instance one can argue that protrusion
>> gives more width but at the same time it introduces the boundary
>> condition that some characters have to hang; or take hz (expansion)
>> ... when applied so much that it gets visible, it will also lead to
>> similar shapes in successive lines to look different
>>
>> so, just enabling these features assuming that the output looks
>> better is debatable (sometimes adding or removing a word is more
>> effective)
>
> I agree, in many cases automatic layout of content can look horrific because the algorithms
> aren't able to solve each possible case in a near optimal way (while we as humans are).
> And also, there is no question about it, *every* setting should be possible to turn on and off
> depending on the situation.
>
> Just as with WYSIWYG tools such as InDesign, manual adjustment of layout is needed to achieve
> optimal results. But my actual complaint is this: the example I pasted was a very trivial
> and simple typesetting task. Even the current versions MS Office can automatically achieve
> acceptable paragraph look with the given input. In such case there should not be any need
> to manually adjust spacing and expansion of words nor forcing the column into a ragged one.
>
> The output of this MKIV source:
>
> \mainlanguage[fi]
> \language[fi]
> \definefontfeature[default][default][protrusion=quality, expansion=quality]
> \setupalign[hz, hanging, spacing]
> \starttext
> \startcolumns[n=2]
> \dorecurse{10}{kolmivaihekilowattituntimittari ympäristöliiketoiminta epäjärjestelmällisyydellistyttymättömyydellänsäkään järjestelmällisentelentelemättömyydellänsäkään hääyöaie kolmivaihekilowattituntimittari ympäristöliiketoiminta epäjärjestelmällisyydellistyttymättömyydellänsäkään järjestelmällisentelentelemättömyydellänsäkään hääyöaie}
> \stopcolumns
> \stoptext
>
> can be downloaded here: https://pi-xi.net/share/context_hyphen.pdf
>
> I also wrote a "close as possible" equivalent of the code above in LaTeX and
> compiled it with pdfTeX (TL 2011):
>
> \documentclass[12pt, twocolumn, a4paper, finnish]{article}
> \usepackage[utf8]{inputenc}
> \usepackage[finnish]{babel}
> \usepackage{ae}
> \usepackage[auto=true, spacing=true, kerning=true, tracking=true, protrusion=true, expansion=true]{microtype}
> \usepackage{pgffor}
> \setlength{\parindent}{0mm}
> \newcommand{\Repeat}[2]{\foreach \n in {1,...,#1}{#2}}
> \begin{document}
> \foreach \n in {1,...,10}{kolmivaihekilowattituntimittari ympäristöliiketoiminta epäjärjestelmällisyydellistyttymättömyydellänsäkään järjestelmällisentelentelemättömyydellänsäkään hääyöaie kolmivaihekilowattituntimittari ympäristöliiketoiminta epäjärjestelmällisyydellistyttymättömyydellänsäkään järjestelmällisentelentelemättömyydellänsäkään hääyöaie}
> \end{document}
>
> The LaTeX example can be downloaded here: https://pi-xi.net/share/latex_hyphen.pdf
>
> As can be seen, the neither versions have any specific adjustments done by hand: both
> only have the microtypographic features turned on. There is no comparison, the pdfTeX
> version looks superior: there are no rivers, all lines are justified to equal length,
> and the expansion and spacing are both acceptable.

(1) The 'spacing' options does not work and even gives an error in mkiv) 
so remove that one. What version of context are you running? I've 
already fotgotten about it but I think it was an experimental pdftesx 
feature with side effects. In pdftex hz and protrusion is also somewhat 
tricky as it can bleed from one font instance to another (due to 
traditional tex's internal font sharing), although context takes some 
precautions.

(2) But, more important: add \setuplayout[width=middle] to give context 
a more similar opportunity to produce something acceptable. You're now 
giving context a disadvantage of narrower columns.

(3) You can remove the \language as you already set \mainlanguage.

(4) In context protrusion and expansion is a font property and applying 
it globally to all fonts (with some flag) will not happen. Actually, in 
some situations you need to turn it on explicitly (imaging hanging 
happening when you're doing a table).

I don't know what latex's default layout setup is but if you want to do 
an experiment, you need to make sure that you use the same widths of 
columns.

Hans

-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
     tel: 038 477 53 69 | voip: 087 875 68 74 | www.pragma-ade.com
                                              | www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________