[9fans] Font behaviour

[9fans] Font behaviour

[Graham Gallagher]

I've been creating a range of fonts via a truetype to font(6) bitmap
generator I wrote.

One thing I noticed was that when trying to render characters possessing
a zero width escapement value, they were transmogrified into PJW. This
is really annoying because some glyphs, Thai vowels and tones for
instance, are intentionally designed to have zero width escapement
vectors for ease of placement.

Investigation of loadchar() in libdraw/font.c (image/font.c under
Inferno) reveals that indeed a test is made to check for zero width
characters. Is there a reason why this test can't be removed?

Re: [9fans] Font behaviour

[rob pike]

[-- Attachment #1: Type: text/plain, Size: 1449 bytes --]

The library forbids zero-width characters because of two issues:
1) they introduce ambiguities into the selection and placement rules,
in particular confusing the algorithm for character-at-position.
2) someone could print an unending stream of zero-width characters
and fill up memory.

One could argue that neither of these is a serious issue, and that
both are raised somewhat by overstriking, which we do accept (badly, at
least in rio), but I was troubled by them anyway.  Plan 9's font code
is quite primitive compared to what some systems do (and some
languages require), but it's served well enough for our purposes.  The
solution I concocted for zero-width characters struck me as rather
elegant, although I knew it would cause trouble for anyone who
legitimately had a use for them.

1) characters absent from the font are by definition zero-width.
2) all zero width characters are rendered as PJWs.

In a stroke, this made sparse Unicode fonts bearable.

You can take out the code that does this, but I think you'll raise some
other issues because of other code that depends on those properties.
A better solution might be to write a special operator that always draws
the character - it's a trivial call to draw().

The real solution is a complete redesign of the character handling model
to deal with all languages, right-to-left, Arabic letter-form-mutations,
etc. etc.  It's beyond my knowledge.

-rob


[-- Attachment #2: Type: message/rfc822, Size: 2261 bytes --]

From: Graham Gallagher <gg@acm.org>
To: 9fans@cse.psu.edu
Subject: [9fans] Font behaviour
Date: Wed, 06 Feb 2002 03:39:14 +1100
Message-ID: <3C600AB2.128AC713@acm.org>

I've been creating a range of fonts via a truetype to font(6) bitmap
generator I wrote.

One thing I noticed was that when trying to render characters possessing
a zero width escapement value, they were transmogrified into PJW. This
is really annoying because some glyphs, Thai vowels and tones for
instance, are intentionally designed to have zero width escapement
vectors for ease of placement.

Investigation of loadchar() in libdraw/font.c (image/font.c under
Inferno) reveals that indeed a test is made to check for zero width
characters. Is there a reason why this test can't be removed?

Re: [9fans] Font behaviour

[Douglas A. Gwyn]

rob pike wrote:
> 1) characters absent from the font are by definition zero-width.

There's another usurpation of a meaningful in-band value
to signal an exception.  We gotta cut that out, it causes
trouble eventually in almost every case I've ever sen.

Re: [9fans] Font behaviour

[Boyd Roberts]

"Douglas A. Gwyn" wrote:
> There's another usurpation of a meaningful in-band value
> to signal an exception.  We gotta cut that out, it causes
> trouble eventually in almost every case I've ever sen.

Absolutely.

Re: [9fans] Font behaviour

[rob pike]

I don't exactly disagree, but there is no in-band data here.
The zero-width character is not returned, it is processed.

-rob

Re: [9fans] Font behaviour

[rob pike]

> There's another usurpation of a meaningful in-band value
> to signal an exception.  We gotta cut that out, it causes
> trouble eventually in almost every case I've ever sen.

Platitudes are fine in their place - I've even written a few
- but you have me at a disadvantage.  My interface is out there
to be criticised, yours is not.  What do you propose to do to
fix this problem?

I have one suggestion.  There are two kinds of zero-width runes,
genuine and synthetic.  (Actually there are three; the third is
an exercise for the reader of the code.) The synthetic ones are not
at issue, I believe - you need to do something when a character
is missing from the font and no one, not even PJW, has argued that
PJW is not an appropriate solution there.  But perhaps we can break
the symmetry in the code to distinguish the genuine zero-width runes
and process them normally, with the understanding that something
may break.  You have the code; read it, try it, play with it. I'm
about to get on a plane or I'd do some experimentation myself.

-rob

Re: [9fans] Font behaviour

[Boyd Roberts]

rob pike wrote:
> Platitudes are fine in their place ...

I think the pjw solution is actually quite elegant.

Re: [9fans] Font behaviour

[Douglas A. Gwyn]

Boyd Roberts wrote:
> rob pike wrote:
> > Platitudes are fine in their place ...

I was just trying to raise the level of awareness about the
meta-issue: any in-band value that might have a valid normal
meaning should not be usurped for any special purpose.  This
is not just about one font package, but applies to a large
number of legacy interfaces.  With hindsight gained from
experience, we should strive not to do this in new designs.
Then the future will be somewhat more pleasant.

> I think the pjw solution is actually quite elegant.

It addresses one issue, namely missing code points, but not
the other, namely genuine 0-width characters (which for some
strange reason all look alike).

Re: [9fans] Font behaviour

[Boyd Roberts]

"Douglas A. Gwyn" wrote:
> I was just trying to raise the level of awareness about the
> meta-issue: ...

I know.

Re: [9fans] Font behaviour

[okamoto]

>the other, namely genuine 0-width characters 

By the way, what is 0-width charcters?
Is that some kind of control code like ^c?

Kenji

Re: [9fans] Font behaviour

[Douglas A. Gwyn]

okamoto@granite.cias.osakafu-u.ac.jp wrote:
> By the way, what is 0-width charcters?
> Is that some kind of control code like ^c?

Not necessarily, although that could be a useful font.