* [luatex-fonts] non-ascii filenames in font cache
@ 2013-04-28 10:04 Philipp Gesang
2013-04-28 10:56 ` Hans Hagen
0 siblings, 1 reply; 6+ messages in thread
From: Philipp Gesang @ 2013-04-28 10:04 UTC (permalink / raw)
To: ConTeXt ML
[-- Attachment #1.1: Type: text/plain, Size: 970 bytes --]
Hi Hans,
the font cache currently drops non-ascii bytes when creating file
names by means of containers.cleanname(). Dohyun Kim sent a fix
for data-con.lua (see below). My own test with the unicode
library leads to some odd results.
Also I noticed that as a pattern, [^%w%d] is a bit redundant
since %d is a subset of %w in both string and unicode.utf8.
Regards
Philipp
#!/usr/bin/env texlua
local non_ascii_names = {
[[华文仿宋.ttf]], [[华文细黑.ttf]], [[华文黑体.ttf]],
}
--- [a]: current data-con
--- [b]: include non-ascii (proposed by Dohyun Kim)
--- [c]: with selene unicode
for i = 1, #non_ascii_names do
local name = non_ascii_names[i]
print""
print("[a]", name, string.gsub(string.lower(name), "[^%w%d]+","-"))
print("[b]", name, string.gsub(string.lower(name), "[^%w%d\128-\255]+","-"))
print("[c]", name, unicode.utf8.gsub(unicode.utf8.lower(name), "[^%w%d]+","-"))
end
[-- Attachment #1.2: Type: application/pgp-signature, Size: 198 bytes --]
[-- Attachment #2: Type: text/plain, Size: 485 bytes --]
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!
maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage : http://www.pragma-ade.nl / http://tex.aanhet.net
archive : http://foundry.supelec.fr/projects/contextrev/
wiki : http://contextgarden.net
___________________________________________________________________________________
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: [luatex-fonts] non-ascii filenames in font cache
2013-04-28 10:04 [luatex-fonts] non-ascii filenames in font cache Philipp Gesang
@ 2013-04-28 10:56 ` Hans Hagen
2013-04-28 11:17 ` Philipp Gesang
2013-04-28 12:08 ` Khaled Hosny
0 siblings, 2 replies; 6+ messages in thread
From: Hans Hagen @ 2013-04-28 10:56 UTC (permalink / raw)
To: ConTeXt ML
On 4/28/2013 12:04 PM, Philipp Gesang wrote:
> the font cache currently drops non-ascii bytes when creating file
> names by means of containers.cleanname(). Dohyun Kim sent a fix
> for data-con.lua (see below). My own test with the unicode
> library leads to some odd results.
strange that it wasn't noticed before as it's rather old code
function containers.cleanname(name)
return (gsub(lower(name),"[^%w\128-\255]+","-"))
end
is good enough i guess
Hans
-----------------------------------------------------------------
Hans Hagen | PRAGMA ADE
Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
tel: 038 477 53 69 | voip: 087 875 68 74 | www.pragma-ade.com
| www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!
maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage : http://www.pragma-ade.nl / http://tex.aanhet.net
archive : http://foundry.supelec.fr/projects/contextrev/
wiki : http://contextgarden.net
___________________________________________________________________________________
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: [luatex-fonts] non-ascii filenames in font cache
2013-04-28 10:56 ` Hans Hagen
@ 2013-04-28 11:17 ` Philipp Gesang
2013-04-28 12:08 ` Khaled Hosny
1 sibling, 0 replies; 6+ messages in thread
From: Philipp Gesang @ 2013-04-28 11:17 UTC (permalink / raw)
To: mailing list for ConTeXt users
[-- Attachment #1.1: Type: text/plain, Size: 651 bytes --]
···<date: 2013-04-28, Sunday>···<from: Hans Hagen>···
> On 4/28/2013 12:04 PM, Philipp Gesang wrote:
>
> >the font cache currently drops non-ascii bytes when creating file
> >names by means of containers.cleanname(). Dohyun Kim sent a fix
> >for data-con.lua (see below). My own test with the unicode
> >library leads to some odd results.
>
> strange that it wasn't noticed before as it's rather old code
Personally I would rename the files instead of reporting it.
> function containers.cleanname(name)
> return (gsub(lower(name),"[^%w\128-\255]+","-"))
> end
>
> is good enough i guess
Of course, thanks!
Philipp
[-- Attachment #1.2: Type: application/pgp-signature, Size: 198 bytes --]
[-- Attachment #2: Type: text/plain, Size: 485 bytes --]
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!
maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage : http://www.pragma-ade.nl / http://tex.aanhet.net
archive : http://foundry.supelec.fr/projects/contextrev/
wiki : http://contextgarden.net
___________________________________________________________________________________
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [luatex-fonts] non-ascii filenames in font cache
2013-04-28 10:56 ` Hans Hagen
2013-04-28 11:17 ` Philipp Gesang
@ 2013-04-28 12:08 ` Khaled Hosny
2013-04-28 12:15 ` Wolfgang Schuster
1 sibling, 1 reply; 6+ messages in thread
From: Khaled Hosny @ 2013-04-28 12:08 UTC (permalink / raw)
To: mailing list for ConTeXt users
On Sun, Apr 28, 2013 at 12:56:25PM +0200, Hans Hagen wrote:
> On 4/28/2013 12:04 PM, Philipp Gesang wrote:
>
> >the font cache currently drops non-ascii bytes when creating file
> >names by means of containers.cleanname(). Dohyun Kim sent a fix
> >for data-con.lua (see below). My own test with the unicode
> >library leads to some odd results.
>
> strange that it wasn't noticed before as it's rather old code
I noticed it long ago (by reading the code), but since I didn't have any
fonts with non-ASCII filenames, I didn't bother.
Regards,
Khaled
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!
maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage : http://www.pragma-ade.nl / http://tex.aanhet.net
archive : http://foundry.supelec.fr/projects/contextrev/
wiki : http://contextgarden.net
___________________________________________________________________________________
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [luatex-fonts] non-ascii filenames in font cache
2013-04-28 12:08 ` Khaled Hosny
@ 2013-04-28 12:15 ` Wolfgang Schuster
2013-04-28 13:08 ` Hans Hagen
0 siblings, 1 reply; 6+ messages in thread
From: Wolfgang Schuster @ 2013-04-28 12:15 UTC (permalink / raw)
To: mailing list for ConTeXt users
Am 28.04.2013 um 14:08 schrieb Khaled Hosny <khaledhosny@eglug.org>:
> On Sun, Apr 28, 2013 at 12:56:25PM +0200, Hans Hagen wrote:
>> On 4/28/2013 12:04 PM, Philipp Gesang wrote:
>>
>>> the font cache currently drops non-ascii bytes when creating file
>>> names by means of containers.cleanname(). Dohyun Kim sent a fix
>>> for data-con.lua (see below). My own test with the unicode
>>> library leads to some odd results.
>>
>> strange that it wasn't noticed before as it's rather old code
>
> I noticed it long ago (by reading the code), but since I didn't have any
> fonts with non-ASCII filenames, I didn't bother.
IIRC this was on purpose because there had been problems when fonts
used non-ascii characters.
Wolfgang
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!
maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage : http://www.pragma-ade.nl / http://tex.aanhet.net
archive : http://foundry.supelec.fr/projects/contextrev/
wiki : http://contextgarden.net
___________________________________________________________________________________
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [luatex-fonts] non-ascii filenames in font cache
2013-04-28 12:15 ` Wolfgang Schuster
@ 2013-04-28 13:08 ` Hans Hagen
0 siblings, 0 replies; 6+ messages in thread
From: Hans Hagen @ 2013-04-28 13:08 UTC (permalink / raw)
To: mailing list for ConTeXt users
On 4/28/2013 2:15 PM, Wolfgang Schuster wrote:
>
> Am 28.04.2013 um 14:08 schrieb Khaled Hosny <khaledhosny@eglug.org>:
>
>> On Sun, Apr 28, 2013 at 12:56:25PM +0200, Hans Hagen wrote:
>>> On 4/28/2013 12:04 PM, Philipp Gesang wrote:
>>>
>>>> the font cache currently drops non-ascii bytes when creating file
>>>> names by means of containers.cleanname(). Dohyun Kim sent a fix
>>>> for data-con.lua (see below). My own test with the unicode
>>>> library leads to some odd results.
>>>
>>> strange that it wasn't noticed before as it's rather old code
>>
>> I noticed it long ago (by reading the code), but since I didn't have any
>> fonts with non-ASCII filenames, I didn't bother.
>
> IIRC this was on purpose because there had been problems when fonts
> used non-ascii characters.
indeed, and as this patch only involves caching it means that the
problem moves elsewhere
Hans
-----------------------------------------------------------------
Hans Hagen | PRAGMA ADE
Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
tel: 038 477 53 69 | voip: 087 875 68 74 | www.pragma-ade.com
| www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!
maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage : http://www.pragma-ade.nl / http://tex.aanhet.net
archive : http://foundry.supelec.fr/projects/contextrev/
wiki : http://contextgarden.net
___________________________________________________________________________________
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2013-04-28 13:08 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2013-04-28 10:04 [luatex-fonts] non-ascii filenames in font cache Philipp Gesang
2013-04-28 10:56 ` Hans Hagen
2013-04-28 11:17 ` Philipp Gesang
2013-04-28 12:08 ` Khaled Hosny
2013-04-28 12:15 ` Wolfgang Schuster
2013-04-28 13:08 ` Hans Hagen
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).