ntg-context - mailing list for ConTeXt users
 help / color / mirror / Atom feed
* [luatex-fonts] non-ascii filenames in font cache
@ 2013-04-28 10:04 Philipp Gesang
  2013-04-28 10:56 ` Hans Hagen
  0 siblings, 1 reply; 6+ messages in thread
From: Philipp Gesang @ 2013-04-28 10:04 UTC (permalink / raw)
  To: ConTeXt ML


[-- Attachment #1.1: Type: text/plain, Size: 970 bytes --]

Hi Hans,

the font cache currently drops non-ascii bytes when creating file
names by means of containers.cleanname(). Dohyun Kim sent a fix
for data-con.lua (see below). My own test with the unicode
library leads to some odd results.

Also I noticed that as a pattern, [^%w%d] is a bit redundant
since %d is a subset of %w in both string and unicode.utf8.

Regards
Philipp



#!/usr/bin/env texlua
local non_ascii_names = {
  [[华文仿宋.ttf]], [[华文细黑.ttf]], [[华文黑体.ttf]],
}

--- [a]: current data-con
--- [b]: include non-ascii (proposed by Dohyun Kim)
--- [c]: with selene unicode

for i = 1, #non_ascii_names do
  local name = non_ascii_names[i]
  print""
  print("[a]", name, string.gsub(string.lower(name),             "[^%w%d]+","-"))
  print("[b]", name, string.gsub(string.lower(name),             "[^%w%d\128-\255]+","-"))
  print("[c]", name, unicode.utf8.gsub(unicode.utf8.lower(name), "[^%w%d]+","-"))
end


[-- Attachment #1.2: Type: application/pgp-signature, Size: 198 bytes --]

[-- Attachment #2: Type: text/plain, Size: 485 bytes --]

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [luatex-fonts] non-ascii filenames in font cache
  2013-04-28 10:04 [luatex-fonts] non-ascii filenames in font cache Philipp Gesang
@ 2013-04-28 10:56 ` Hans Hagen
  2013-04-28 11:17   ` Philipp Gesang
  2013-04-28 12:08   ` Khaled Hosny
  0 siblings, 2 replies; 6+ messages in thread
From: Hans Hagen @ 2013-04-28 10:56 UTC (permalink / raw)
  To: ConTeXt ML

On 4/28/2013 12:04 PM, Philipp Gesang wrote:

> the font cache currently drops non-ascii bytes when creating file
> names by means of containers.cleanname(). Dohyun Kim sent a fix
> for data-con.lua (see below). My own test with the unicode
> library leads to some odd results.

strange that it wasn't noticed before as it's rather old code

function containers.cleanname(name)
     return (gsub(lower(name),"[^%w\128-\255]+","-"))
end

is good enough i guess

Hans

-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
     tel: 038 477 53 69 | voip: 087 875 68 74 | www.pragma-ade.com
                                              | www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________


^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [luatex-fonts] non-ascii filenames in font cache
  2013-04-28 10:56 ` Hans Hagen
@ 2013-04-28 11:17   ` Philipp Gesang
  2013-04-28 12:08   ` Khaled Hosny
  1 sibling, 0 replies; 6+ messages in thread
From: Philipp Gesang @ 2013-04-28 11:17 UTC (permalink / raw)
  To: mailing list for ConTeXt users


[-- Attachment #1.1: Type: text/plain, Size: 651 bytes --]

···<date: 2013-04-28, Sunday>···<from: Hans Hagen>···

> On 4/28/2013 12:04 PM, Philipp Gesang wrote:
> 
> >the font cache currently drops non-ascii bytes when creating file
> >names by means of containers.cleanname(). Dohyun Kim sent a fix
> >for data-con.lua (see below). My own test with the unicode
> >library leads to some odd results.
> 
> strange that it wasn't noticed before as it's rather old code

Personally I would rename the files instead of reporting it.

> function containers.cleanname(name)
>     return (gsub(lower(name),"[^%w\128-\255]+","-"))
> end
> 
> is good enough i guess

Of course, thanks!

Philipp


[-- Attachment #1.2: Type: application/pgp-signature, Size: 198 bytes --]

[-- Attachment #2: Type: text/plain, Size: 485 bytes --]

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [luatex-fonts] non-ascii filenames in font cache
  2013-04-28 10:56 ` Hans Hagen
  2013-04-28 11:17   ` Philipp Gesang
@ 2013-04-28 12:08   ` Khaled Hosny
  2013-04-28 12:15     ` Wolfgang Schuster
  1 sibling, 1 reply; 6+ messages in thread
From: Khaled Hosny @ 2013-04-28 12:08 UTC (permalink / raw)
  To: mailing list for ConTeXt users

On Sun, Apr 28, 2013 at 12:56:25PM +0200, Hans Hagen wrote:
> On 4/28/2013 12:04 PM, Philipp Gesang wrote:
> 
> >the font cache currently drops non-ascii bytes when creating file
> >names by means of containers.cleanname(). Dohyun Kim sent a fix
> >for data-con.lua (see below). My own test with the unicode
> >library leads to some odd results.
> 
> strange that it wasn't noticed before as it's rather old code

I noticed it long ago (by reading the code), but since I didn't have any
fonts with non-ASCII filenames, I didn't bother.

Regards,
Khaled
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________


^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [luatex-fonts] non-ascii filenames in font cache
  2013-04-28 12:08   ` Khaled Hosny
@ 2013-04-28 12:15     ` Wolfgang Schuster
  2013-04-28 13:08       ` Hans Hagen
  0 siblings, 1 reply; 6+ messages in thread
From: Wolfgang Schuster @ 2013-04-28 12:15 UTC (permalink / raw)
  To: mailing list for ConTeXt users


Am 28.04.2013 um 14:08 schrieb Khaled Hosny <khaledhosny@eglug.org>:

> On Sun, Apr 28, 2013 at 12:56:25PM +0200, Hans Hagen wrote:
>> On 4/28/2013 12:04 PM, Philipp Gesang wrote:
>> 
>>> the font cache currently drops non-ascii bytes when creating file
>>> names by means of containers.cleanname(). Dohyun Kim sent a fix
>>> for data-con.lua (see below). My own test with the unicode
>>> library leads to some odd results.
>> 
>> strange that it wasn't noticed before as it's rather old code
> 
> I noticed it long ago (by reading the code), but since I didn't have any
> fonts with non-ASCII filenames, I didn't bother.

IIRC this was on purpose because there had been problems when fonts
used non-ascii characters.

Wolfgang
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________


^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [luatex-fonts] non-ascii filenames in font cache
  2013-04-28 12:15     ` Wolfgang Schuster
@ 2013-04-28 13:08       ` Hans Hagen
  0 siblings, 0 replies; 6+ messages in thread
From: Hans Hagen @ 2013-04-28 13:08 UTC (permalink / raw)
  To: mailing list for ConTeXt users

On 4/28/2013 2:15 PM, Wolfgang Schuster wrote:
>
> Am 28.04.2013 um 14:08 schrieb Khaled Hosny <khaledhosny@eglug.org>:
>
>> On Sun, Apr 28, 2013 at 12:56:25PM +0200, Hans Hagen wrote:
>>> On 4/28/2013 12:04 PM, Philipp Gesang wrote:
>>>
>>>> the font cache currently drops non-ascii bytes when creating file
>>>> names by means of containers.cleanname(). Dohyun Kim sent a fix
>>>> for data-con.lua (see below). My own test with the unicode
>>>> library leads to some odd results.
>>>
>>> strange that it wasn't noticed before as it's rather old code
>>
>> I noticed it long ago (by reading the code), but since I didn't have any
>> fonts with non-ASCII filenames, I didn't bother.
>
> IIRC this was on purpose because there had been problems when fonts
> used non-ascii characters.

indeed, and as this patch only involves caching it means that the 
problem moves elsewhere

Hans

-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
     tel: 038 477 53 69 | voip: 087 875 68 74 | www.pragma-ade.com
                                              | www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________


^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2013-04-28 13:08 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2013-04-28 10:04 [luatex-fonts] non-ascii filenames in font cache Philipp Gesang
2013-04-28 10:56 ` Hans Hagen
2013-04-28 11:17   ` Philipp Gesang
2013-04-28 12:08   ` Khaled Hosny
2013-04-28 12:15     ` Wolfgang Schuster
2013-04-28 13:08       ` Hans Hagen

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).