wrong line break in recent LMTX

wrong line break in recent LMTX

[Pablo Rodriguez]

Dear list,

I have the following sample:

  \starttext
  \startTEXpage[offset=1ex]
  \hsize\zeropoint
  b—.\footnote{}
  \stopTEXpage
  \stoptext

For some strange reason, LMTX from 2021.04.18 18:12 introduce a line
break between the emdash and the point.

LMTX from 2021.04.14 22:58 doesn’t apply horizontal offset to the sample
above.

I’m afraid this might be a bug. Or I may be missing something.

Many thanks for your help,

Pablo
--
http://www.ousia.tk
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: wrong line break in recent LMTX

[Hans Hagen]

On 4/26/2021 6:53 PM, Pablo Rodriguez wrote:
> Dear list,
> 
> I have the following sample:
> 
>    \starttext
>    \startTEXpage[offset=1ex]
>    \hsize\zeropoint
>    b—.\footnote{}
>    \stopTEXpage
>    \stoptext
> 
> For some strange reason, LMTX from 2021.04.18 18:12 introduce a line
> break between the emdash and the point.
> 
> LMTX from 2021.04.14 22:58 doesn’t apply horizontal offset to the sample
> above.
> 
> I’m afraid this might be a bug. Or I may be missing something.
new features ... some mail from last week .. you get unbreakable ones with

\hccode"2013 0
\hccode"2014 0

once i know what' sbest i''l make some option someplace

Hans


-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
        tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: wrong line break in recent LMTX

[Pablo Rodriguez]

On 4/26/21 8:10 PM, Hans Hagen wrote:
> On 4/26/2021 6:53 PM, Pablo Rodriguez wrote:
>> [...]
>> For some strange reason, LMTX from 2021.04.18 18:12 introduce a line
>> break between the emdash and the point.
>>
>> LMTX from 2021.04.14 22:58 doesn’t apply horizontal offset to the sample
>> above.
>>
>> I’m afraid this might be a bug. Or I may be missing something.
> new features ... some mail from last week .. you get unbreakable ones with
>
> \hccode"2013 0
> \hccode"2014 0

I understood
https://mailman.ntg.nl/pipermail/ntg-context/2021/101871.html as a
ligature replacement.

> once i know what' sbest i''l make some option someplace

I think it would be better not to change defaults that have been there
for years (to avoid unpleasant surprises).

I was always able to hyphenate words before and after both — and – and
they didn’t include linebreaks before punctuation chars (unlike --- and --).

BTW, the fourth line in code adds a second page and the third line in
includes an unwanted underline:

  \starttext
  \startTEXpage[offset=1em]
  a—.--\footnote{}
  b—.\footnote{}
  \stopTEXpage
  \stoptext

Many thanks for your help again,

Pablo
--
http://www.ousia.tk
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: wrong line break in recent LMTX

[Hans Hagen]

On 4/27/2021 8:48 AM, Pablo Rodriguez wrote:
> On 4/26/21 8:10 PM, Hans Hagen wrote:
>> On 4/26/2021 6:53 PM, Pablo Rodriguez wrote:
>>> [...]
>>> For some strange reason, LMTX from 2021.04.18 18:12 introduce a line
>>> break between the emdash and the point.
>>>
>>> LMTX from 2021.04.14 22:58 doesn’t apply horizontal offset to the sample
>>> above.
>>>
>>> I’m afraid this might be a bug. Or I may be missing something.
>> new features ... some mail from last week .. you get unbreakable ones with
>>
>> \hccode"2013 0
>> \hccode"2014 0
> 
> I understood
> https://mailman.ntg.nl/pipermail/ntg-context/2021/101871.html as a
> ligature replacement.
> 
>> once i know what' sbest i''l make some option someplace
> 
> I think it would be better not to change defaults that have been there
> for years (to avoid unpleasant surprises).
> 
> I was always able to hyphenate words before and after both — and – and
> they didn’t include linebreaks before punctuation chars (unlike --- and --).

nornmally tex doesn't hyphenate before ... only after (which is why we 
have the || mechanism)

> BTW, the fourth line in code adds a second page and the third line in
> includes an unwanted underline:
> 
>    \starttext
>    \startTEXpage[offset=1em]
>    a—.--\footnote{}
>    b—.\footnote{}
>    \stopTEXpage
>    \stoptext
> 
> Many thanks for your help again,
what is -- supposed to mean here ... in traditional tex setups it's an 
endash just like --- is an emdash

as mentioned in an earlier mail, after decades of utf in tex we should 
use the normal symbols instead ... you can disable collapsing with

\nohyphencollapsing

but then you need to enable the tlig font feature. In days when often 
texts is imported from elsewhere and editors can show these dashes we 
need to adapt

Hans

-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
        tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: wrong line break in recent LMTX

[Hans Hagen]

btw, i'll think of an

--oldschool

switch.

Hans

-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
        tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: wrong line break in recent LMTX

[Pablo Rodriguez]

On 4/27/21 8:57 AM, Hans Hagen wrote:
> [...]
> as mentioned in an earlier mail, after decades of utf in tex we should
> use the normal symbols instead ... you can disable collapsing with
>
> \nohyphencollapsing
>
> but then you need to enable the tlig font feature. In days when often
> texts is imported from elsewhere and editors can show these dashes we
> need to adapt

Not sure I understand your explanation.

I have been using UTF-8 as charset for my documents since 2002
(otherwise polytonic Greek was unreadable for me).

It was also more readable to use real character for em- and en-dashes
than three or two hyphens.

It took me a while since I accidentally discovered a document with a
wrong line break between a real em-dash and a point followed by a
footnote number.

So my question is what \hccode stands for?

From luametatex.pdf in the distribution, I see that this is a LuaTeX
primitive, but luatex.pdf doesn’t mention it.

Many thanks for your help again,

Pablo
--
http://www.ousia.tk
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: wrong line break in recent LMTX

[Hans Hagen]

On 4/27/2021 9:00 PM, Pablo Rodriguez wrote:
> On 4/27/21 8:57 AM, Hans Hagen wrote:
>> [...]
>> as mentioned in an earlier mail, after decades of utf in tex we should
>> use the normal symbols instead ... you can disable collapsing with
>>
>> \nohyphencollapsing
>>
>> but then you need to enable the tlig font feature. In days when often
>> texts is imported from elsewhere and editors can show these dashes we
>> need to adapt
> 
> Not sure I understand your explanation.
> 
> I have been using UTF-8 as charset for my documents since 2002
> (otherwise polytonic Greek was unreadable for me).
> 
> It was also more readable to use real character for em- and en-dashes
> than three or two hyphens.
> 
> It took me a while since I accidentally discovered a document with a
> wrong line break between a real em-dash and a point followed by a
> footnote number.
> 
> So my question is what \hccode stands for?

hyphencharcode

good old tex only has \exhyphenchar, in luametatex we can tag any char 
as such

in traditional tex th ehyphenation, ligature building and kerning wre 
integrated, so handling - -- --- ---- ---------- is kind of complex

in luatex these stages are split but there's still messy logic wrt 
successive hyphens

in luametatex the hyphenation machinery has all kind of extra controls 
and the hccode mechanism was introduced as side effect of a feedback 
mechanism that also can handle compound words better (althoiugh one can 
of course dispute the 'better' part)

>  From luametatex.pdf in the distribution, I see that this is a LuaTeX
> primitive, but luatex.pdf doesn’t mention it.
sure, it's a luametatex primitive, but that is true for some more, like 
the detailed control of how characters are treated in various stages

Hans

-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
        tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki     : http://contextgarden.net
___________________________________________________________________________________