copy/paste text broken with certain ligatures

copy/paste text broken with certain ligatures

[Brian R. Landy]

Hi, sometime between version 2015.08.05 and 2016.04.10 the handling of 
certain ligatures changed (sorry, I don't have intermediate versions to 
test with).  The issue is the ligatures no longer copy/paste as the 
original two characters, which also breaks text searched of the pdf. 
Instead they copy as a single glyph.

I'm seeing the problem with "ti" and "fj", while others like "ffi", 
"fi", and "ff" are fine in both versions.

I see this using Calibri (other fonts I tried don't support those two 
ligatures):

   \starttypescript[cleartype]
     \definetypeface[\typescriptone][ss][sans][calibri][default]
   \stoptypescript
   \usetypescript[cleartype]
   \setupbodyfont[cleartype,sans,10pt]
   \starttext
   ffi\par
   fi\par
   ff\par
   fi\par
   ti\par
   fj\par
   \stoptext

Thank you.

Best regards,
Brian
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: copy/paste text broken with certain ligatures

[Hans Hagen]

On 4/14/2016 3:47 PM, Brian R. Landy wrote:
> Hi, sometime between version 2015.08.05 and 2016.04.10 the handling of
> certain ligatures changed (sorry, I don't have intermediate versions to
> test with).  The issue is the ligatures no longer copy/paste as the
> original two characters, which also breaks text searched of the pdf.
> Instead they copy as a single glyph.
>
> I'm seeing the problem with "ti" and "fj", while others like "ffi",
> "fi", and "ff" are fine in both versions.
>
> I see this using Calibri (other fonts I tried don't support those two
> ligatures):
>
>    \starttypescript[cleartype]
>      \definetypeface[\typescriptone][ss][sans][calibri][default]
>    \stoptypescript
>    \usetypescript[cleartype]
>    \setupbodyfont[cleartype,sans,10pt]
>    \starttext
>    ffi\par
>    fi\par
>    ff\par
>    fi\par
>    ti\par
>    fj\par
>    \stoptext
>
> Thank you.

not all pdf viewers can do that well, unless it's broken, context 
outputs the tounicode vectors needed for it

Hans


-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
       tel: 038 477 53 69 | www.pragma-ade.com | www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: copy/paste text broken with certain ligatures

[Brian R. Landy]

> On Apr 14, 2016, at 6:24 PM, Hans Hagen <pragma@wxs.nl> wrote:
> 
>> On 4/14/2016 3:47 PM, Brian R. Landy wrote:
>> Hi, sometime between version 2015.08.05 and 2016.04.10 the handling of
>> certain ligatures changed (sorry, I don't have intermediate versions to
>> test with).  The issue is the ligatures no longer copy/paste as the
>> original two characters, which also breaks text searched of the pdf.
>> Instead they copy as a single glyph.
>> 
>> I'm seeing the problem with "ti" and "fj", while others like "ffi",
>> "fi", and "ff" are fine in both versions.
>> 
>> I see this using Calibri (other fonts I tried don't support those two
>> ligatures):
>> 
>>   \starttypescript[cleartype]
>>     \definetypeface[\typescriptone][ss][sans][calibri][default]
>>   \stoptypescript
>>   \usetypescript[cleartype]
>>   \setupbodyfont[cleartype,sans,10pt]
>>   \starttext
>>   ffi\par
>>   fi\par
>>   ff\par
>>   fi\par
>>   ti\par
>>   fj\par
>>   \stoptext
>> 
>> Thank you.
> 
> not all pdf viewers can do that well, unless it's broken, context outputs the tounicode vectors needed for it
> 
> Hans

But Acrobat works fine with the pdf generated by the older version of ConTeXt, which also uses the ligatures. So something must be different about the files generated by the two versions?

I can post examples tomorrow if you'd like. 

Thanks,
Brian

> 
> -----------------------------------------------------------------
>                                          Hans Hagen | PRAGMA ADE
>              Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
>      tel: 038 477 53 69 | www.pragma-ade.com | www.pragma-pod.nl
> -----------------------------------------------------------------
> ___________________________________________________________________________________
> If your question is of interest to others as well, please add an entry to the Wiki!
> 
> maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
> webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
> archive  : http://foundry.supelec.fr/projects/contextrev/
> wiki     : http://contextgarden.net
> ___________________________________________________________________________________
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: copy/paste text broken with certain ligatures

[Ulrike Fischer]

Am Fri, 15 Apr 2016 00:24:26 +0200 schrieb Hans Hagen:

> On 4/14/2016 3:47 PM, Brian R. Landy wrote:
>> Hi, sometime between version 2015.08.05 and 2016.04.10 the handling of
>> certain ligatures changed (sorry, I don't have intermediate versions to
>> test with).  The issue is the ligatures no longer copy/paste as the
>> original two characters, which also breaks text searched of the pdf.
>> Instead they copy as a single glyph.
>>
>> I'm seeing the problem with "ti" and "fj", while others like "ffi",
>> "fi", and "ff" are fine in both versions.
>>

 
> not all pdf viewers can do that well, unless it's broken, context 
> outputs the tounicode vectors needed for it

I can't copy the two ligatures either (in Adobe Reader DC), but with
the equivalent latex example (TL2016 pretest) it works fine. So imho
it is not a viewer problem. 

 

-- 
Ulrike Fischer 
http://www.troubleshooting-tex.de/

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: copy/paste text broken with certain ligatures

[Hans Hagen]

On 4/15/2016 1:31 PM, Ulrike Fischer wrote:
> Am Fri, 15 Apr 2016 00:24:26 +0200 schrieb Hans Hagen:
>
>> On 4/14/2016 3:47 PM, Brian R. Landy wrote:
>>> Hi, sometime between version 2015.08.05 and 2016.04.10 the handling of
>>> certain ligatures changed (sorry, I don't have intermediate versions to
>>> test with).  The issue is the ligatures no longer copy/paste as the
>>> original two characters, which also breaks text searched of the pdf.
>>> Instead they copy as a single glyph.
>>>
>>> I'm seeing the problem with "ti" and "fj", while others like "ffi",
>>> "fi", and "ff" are fine in both versions.
>>>
>
>
>> not all pdf viewers can do that well, unless it's broken, context
>> outputs the tounicode vectors needed for it
>
> I can't copy the two ligatures either (in Adobe Reader DC), but with
> the equivalent latex example (TL2016 pretest) it works fine. So imho
> it is not a viewer problem.

i looked into it and it's two things combined ...

(1) when we explicitly pass tounicodes luatex mistakenly falls back to 
the index (a next release repairs this)

(2) this happens here because calibri lacks the information (no names to 
deduce ligature from) .. so, i now have some extra heuristics in the 
font loader that get them from the gsub features

Hans

-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
       tel: 038 477 53 69 | www.pragma-ade.com | www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: copy/paste text broken with certain ligatures

[Brian R. Landy]

On Sat, 16 Apr 2016, Hans Hagen wrote:

> On 4/15/2016 1:31 PM, Ulrike Fischer wrote:
>> Am Fri, 15 Apr 2016 00:24:26 +0200 schrieb Hans Hagen:
>>
>>> On 4/14/2016 3:47 PM, Brian R. Landy wrote:
>>>> Hi, sometime between version 2015.08.05 and 2016.04.10 the handling 
>>>> of
>>>> certain ligatures changed (sorry, I don't have intermediate 
>>>> versions to
>>>> test with).  The issue is the ligatures no longer copy/paste as the
>>>> original two characters, which also breaks text searched of the 
>>>> pdf.
>>>> Instead they copy as a single glyph.
>>>>
>>>> I'm seeing the problem with "ti" and "fj", while others like "ffi",
>>>> "fi", and "ff" are fine in both versions.
>>>>
>>
>>
>>> not all pdf viewers can do that well, unless it's broken, context
>>> outputs the tounicode vectors needed for it
>>
>> I can't copy the two ligatures either (in Adobe Reader DC), but with
>> the equivalent latex example (TL2016 pretest) it works fine. So imho
>> it is not a viewer problem.
>
> i looked into it and it's two things combined ...
>
> (1) when we explicitly pass tounicodes luatex mistakenly falls back to 
> the index (a next release repairs this)
>
> (2) this happens here because calibri lacks the information (no names 
> to deduce ligature from) .. so, i now have some extra heuristics in 
> the font loader that get them from the gsub features
>

Thank you Hans, this worked perfectly.  I appreciate your help with 
this!  (and thanks, too, for fixing the dash issue I raised recently).

Best regards,
Brian
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________