tounicode mapping missing for locl-variants

tounicode mapping missing for locl-variants

[Ulrike Fischer]

When I compile the following document I see in the output localized
variants for the various glyphs. But copy & paste works only for the
last line. The first two gives notdef chars. 

In the pdf e.g. the first glyph has the numbers 2D0E, 2D0F, 2D0D in
the three lines but only for the last there is a unicode mapping:
<2D0D> <5203>.

What can one do to get a mapping also for the other glyphs? 

The font can be downloaded here
https://github.com/adobe-fonts/source-han-sans/blob/release/OTF/SourceHanSansJ.zip
A related bug report is
https://github.com/adobe-fonts/source-han-sans/issues/200


\pdfvariable compresslevel 0
\pdfvariable objcompresslevel 0
\setupheader[state=stop]
\starttext

\font\zhs=
{file:SourceHanSans-Regular.otf:mode=node;script=hani;language=ZHS;+locl}
\zhs 刃令毒骨縣誤

\font\zht=
{file:SourceHanSans-Regular.otf:mode=node;script=hani;language=ZHT;+locl}
\zht 刃令毒骨縣誤

\font\test=
{file:SourceHanSans-Regular.otf:mode=node;script=hani;language=DFLT;}
\test 刃令毒骨縣誤

\stoptext


-- 
Ulrike Fischer 
https://www.troubleshooting-tex.de/

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: tounicode mapping missing for locl-variants

[luigi scarso]

[-- Attachment #1.1: Type: text/plain, Size: 1731 bytes --]

On Wed, Sep 26, 2018 at 11:14 PM Ulrike Fischer <news3@nililand.de> wrote:

>
> When I compile the following document I see in the output localized
> variants for the various glyphs. But copy & paste works only for the
> last line. The first two gives notdef chars.
>
> In the pdf e.g. the first glyph has the numbers 2D0E, 2D0F, 2D0D in
> the three lines but only for the last there is a unicode mapping:
> <2D0D> <5203>.
>
> What can one do to get a mapping also for the other glyphs?
>
> The font can be downloaded here
>
> https://github.com/adobe-fonts/source-han-sans/blob/release/OTF/SourceHanSansJ.zip
> A related bug report is
> https://github.com/adobe-fonts/source-han-sans/issues/200
>
>
> \pdfvariable compresslevel 0
> \pdfvariable objcompresslevel 0
> \setupheader[state=stop]
>



> \starttext
>
> \font\zhs=
> {file:SourceHanSans-Regular.otf:mode=node;script=hani;language=ZHS;+locl}
> \zhs 刃令毒骨縣誤
>
> \font\zht=
> {file:SourceHanSans-Regular.otf:mode=node;script=hani;language=ZHT;+locl}
> \zht 刃令毒骨縣誤
>
> \font\test=
> {file:SourceHanSans-Regular.otf:mode=node;script=hani;language=DFLT;}
> \test 刃令毒骨縣誤
>
> \stoptext
>
>
>
what does the trackers say  at your end ?


\enabletrackers[*]
\nopdfcompression
\setupheader[state=stop]
\starttext

\font\zhs=
{file:SourceHanSans-Regular.otf:mode=node;script=hani;language=ZHS;+locl}
\zhs zhs  刃令毒骨縣誤

\font\zht=
{file:SourceHanSans-Regular.otf:mode=node;script=hani;language=ZHT;+locl}
\zht zht 刃令毒骨縣誤

\font\test=
{file:SourceHanSans-Regular.otf:mode=node;script=hani;language=DFLT;}
\test test 刃令毒骨縣誤

\stoptext


-- 
luigi

[-- Attachment #1.2: Type: text/html, Size: 2835 bytes --]

[-- Attachment #2: Type: text/plain, Size: 492 bytes --]

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: tounicode mapping missing for locl-variants

[Ulrike Fischer]

Am Thu, 27 Sep 2018 09:14:01 +0200 schrieb luigi scarso:

> On Wed, Sep 26, 2018 at 11:14 PM Ulrike Fischer <news3@nililand.de> wrote:
> 
>>
>> When I compile the following document I see in the output localized
>> variants for the various glyphs. But copy & paste works only for the
>> last line. The first two gives notdef chars.
>>
>> In the pdf e.g. the first glyph has the numbers 2D0E, 2D0F, 2D0D in
>> the three lines but only for the last there is a unicode mapping:
>> <2D0D> <5203>.
>>
>> What can one do to get a mapping also for the other glyphs?
>>
>> The font can be downloaded here
>>
>> https://github.com/adobe-fonts/source-han-sans/blob/release/OTF/SourceHanSansJ.zip
>> A related bug report is
>> https://github.com/adobe-fonts/source-han-sans/issues/200
>>
>>
>> \pdfvariable compresslevel 0
>> \pdfvariable objcompresslevel 0
>> \setupheader[state=stop]
>>
> 
> 
> 
>> \starttext
>>
>> \font\zhs=
>> {file:SourceHanSans-Regular.otf:mode=node;script=hani;language=ZHS;+locl}
>> \zhs 刃令毒骨縣誤
>>
>> \font\zht=
>> {file:SourceHanSans-Regular.otf:mode=node;script=hani;language=ZHT;+locl}
>> \zht 刃令毒骨縣誤
>>
>> \font\test=
>> {file:SourceHanSans-Regular.otf:mode=node;script=hani;language=DFLT;}
>> \test 刃令毒骨縣誤
>>
>> \stoptext
>>
>>
>>
> what does the trackers say  at your end ?

> \enabletrackers[*]
> \nopdfcompression
> [...]

a lot ;-) 

https://www.dropbox.com/s/jgwvytl7hi2doi5/test-utf8.log?dl=0

One can see how the glyphs are replaced under locl:

fonts           > otf direct > feature 'locl', type 'gsub_single',
lookup 's_s_42': replacing U+5203 by single U+F0608

but I can find nothing about tounicode/cmap or something like that. 


-- 
Ulrike Fischer 
http://www.troubleshooting-tex.de/

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: tounicode mapping missing for locl-variants

[Hans Hagen]

On 9/27/2018 9:42 AM, Ulrike Fischer wrote:

>>> \pdfvariable compresslevel 0
>>> \pdfvariable objcompresslevel 0

\nopdfcompression

Hans

-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
        tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: tounicode mapping missing for locl-variants

[luigi scarso]

[-- Attachment #1.1: Type: text/plain, Size: 2125 bytes --]

On Thu, Sep 27, 2018 at 9:42 AM Ulrike Fischer <news3@nililand.de> wrote:

> Am Thu, 27 Sep 2018 09:14:01 +0200 schrieb luigi scarso:
>
> > On Wed, Sep 26, 2018 at 11:14 PM Ulrike Fischer <news3@nililand.de>
> wrote:
> >
> >>
> >> When I compile the following document I see in the output localized
> >> variants for the various glyphs. But copy & paste works only for the
> >> last line. The first two gives notdef chars.
> >>
> >> In the pdf e.g. the first glyph has the numbers 2D0E, 2D0F, 2D0D in
> >> the three lines but only for the last there is a unicode mapping:
> >> <2D0D> <5203>.
> >>
> >> What can one do to get a mapping also for the other glyphs?
> >>
> >> The font can be downloaded here
> >>
> >>
> https://github.com/adobe-fonts/source-han-sans/blob/release/OTF/SourceHanSansJ.zip
> >> A related bug report is
> >> https://github.com/adobe-fonts/source-han-sans/issues/200
> >>
> >>
> >> \pdfvariable compresslevel 0
> >> \pdfvariable objcompresslevel 0
> >> \setupheader[state=stop]
> >>
> >
> >
> >
> >> \starttext
> >>
> >> \font\zhs=
> >>
> {file:SourceHanSans-Regular.otf:mode=node;script=hani;language=ZHS;+locl}
> >> \zhs 刃令毒骨縣誤
> >>
> >> \font\zht=
> >>
> {file:SourceHanSans-Regular.otf:mode=node;script=hani;language=ZHT;+locl}
> >> \zht 刃令毒骨縣誤
> >>
> >> \font\test=
> >> {file:SourceHanSans-Regular.otf:mode=node;script=hani;language=DFLT;}
> >> \test 刃令毒骨縣誤
> >>
> >> \stoptext
> >>
> >>
> >>
> > what does the trackers say  at your end ?
>
> > \enabletrackers[*]
> > \nopdfcompression
> > [...]
>
> a lot ;-)
>
> https://www.dropbox.com/s/jgwvytl7hi2doi5/test-utf8.log?dl=0
>
> One can see how the glyphs are replaced under locl:
>
> fonts           > otf direct > feature 'locl', type 'gsub_single',
> lookup 's_s_42': replacing U+5203 by single U+F0608
>
> but I can find nothing about tounicode/cmap or something like that.
>

ok, then you can see the font as seen by context
mtxrun --script fonts --convert SourceHanSans-Regular.otf
(keep track also of its output )

-- 
luigi

[-- Attachment #1.2: Type: text/html, Size: 3361 bytes --]

[-- Attachment #2: Type: text/plain, Size: 492 bytes --]

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: tounicode mapping missing for locl-variants

[Ulrike Fischer]

Am Thu, 27 Sep 2018 09:55:02 +0200 schrieb luigi scarso:

> ok, then you can see the font as seen by context
> mtxrun --script fonts --convert SourceHanSans-Regular.otf
> (keep track also of its output )

That is the output of the mtxrun call:

https://www.dropbox.com/s/xszihzv21snxdiv/contextmtx.log?dl=0


-- 
Ulrike Fischer 
http://www.troubleshooting-tex.de/

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: tounicode mapping missing for locl-variants

[Hans Hagen]

On 9/26/2018 11:13 PM, Ulrike Fischer wrote:
> 
> When I compile the following document I see in the output localized
> variants for the various glyphs. But copy & paste works only for the
> last line. The first two gives notdef chars.
> 
> In the pdf e.g. the first glyph has the numbers 2D0E, 2D0F, 2D0D in
> the three lines but only for the last there is a unicode mapping:
> <2D0D> <5203>.
> 
> What can one do to get a mapping also for the other glyphs?

wait till i update ...

these glyphs have no names so i need to add them to the to-be-checked 
list (which does consult substitutions etc already)
  Hans

-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
        tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: tounicode mapping missing for locl-variants

[luigi scarso]

[-- Attachment #1.1: Type: text/plain, Size: 593 bytes --]

On Thu, Sep 27, 2018 at 10:16 AM Ulrike Fischer <news3@nililand.de> wrote:

> Am Thu, 27 Sep 2018 09:55:02 +0200 schrieb luigi scarso:
>
> > ok, then you can see the font as seen by context
> > mtxrun --script fonts --convert SourceHanSans-Regular.otf
> > (keep track also of its output )
>
> That is the output of the mtxrun call:
>
> https://www.dropbox.com/s/xszihzv21snxdiv/contextmtx.log?dl=0
>
> fine
now, you can use ttx to have an unrelated view of the same font
and then you can find the answer .

(ok, this time hans was faster than you, so warm-up for the next time...  )
-- 
luigi

[-- Attachment #1.2: Type: text/html, Size: 1125 bytes --]

[-- Attachment #2: Type: text/plain, Size: 492 bytes --]

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: tounicode mapping missing for locl-variants

[Ulrike Fischer]

Am Thu, 27 Sep 2018 10:25:51 +0200 schrieb Hans Hagen:

>> What can one do to get a mapping also for the other glyphs?
> 
> wait till i update ...

Thanks.  Will it be also in the generic fontloader?

-- 
Ulrike Fischer 
http://www.troubleshooting-tex.de/

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: tounicode mapping missing for locl-variants

[Ulrike Fischer]

Am Thu, 27 Sep 2018 10:25:51 +0200 schrieb Hans Hagen:

> these glyphs have no names

Side remark: if you think that the font is wrong here I could add a
comment on the github site. 


-- 
Ulrike Fischer 
http://www.troubleshooting-tex.de/

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: tounicode mapping missing for locl-variants

[Hans Hagen]

On 9/27/2018 10:44 AM, Ulrike Fischer wrote:
> Am Thu, 27 Sep 2018 10:25:51 +0200 schrieb Hans Hagen:
> 
>> these glyphs have no names
> 
> Side remark: if you think that the font is wrong here I could add a
> comment on the github site.
i think that there's more wrong with the font and it doesn't look like 
it gets updated so don't bother ... i have no time (or motivation) to 
look into what realy goes on in the font (too large for convenient 
peeking into, these fonts) and the extra check seems to work .. it might 
even be no bug but valid for these huge fonts in which case it's a 'lack 
of feature' on our end

there are more fonts that need this treatment (normally they have 
sensible names for glyphs and I only checked those with names)

Hans


-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
        tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: tounicode mapping missing for locl-variants

[Hans Hagen]

On 9/27/2018 10:32 AM, Ulrike Fischer wrote:
> Am Thu, 27 Sep 2018 10:25:51 +0200 schrieb Hans Hagen:
> 
>>> What can one do to get a mapping also for the other glyphs?
>>
>> wait till i update ...
> 
> Thanks.  Will it be also in the generic fontloader?
sure, as i don't keep that bit of code in split files (in fact most ends 
up in generic anyway)

Hans

-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
        tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki     : http://contextgarden.net
___________________________________________________________________________________