Re: Non-printable Unicode control characters

["Idris Samawi Hamid ادريس سماوي حامد" <ishamid@colostate.edu>]

On Sat, 16 Aug 2008 01:53:06 -0600, Hans Hagen <pragma@wxs.nl> wrote:

> Khaled Hosny wrote:
>> Unicode has many "control characters" that only control text behaviour
>> and shouldn't be rendered visually in the text, such as Bidi_Control and
>> Join_Control chars (see
>> http://www.unicode.org/Public/5.1.0/ucd/PropList.txt and
>> http://unicode.org/Public/UNIDATA/UCD.html)
>> Currently, ConTeXt handles ZWJ and ZWNJ, but other characters get
>> rendered if the font has glyphs for them or make no effect at all if the
>> font has no glyphs for them. I think that the optimum behaviour is to
>> make those characters affect text formatting while not visually rendered
>> whether the font has glyphs for them or not.
>> It might be also useful if we can enable rendering those characters
>> manually, for drafts and such.
>
> actually we need:
>
> - ignore them (like in verbatim)

Eventually we want to be able to show them in verbatim also (provided the  
font has them).

Indeed, I suggest that -- given an appropriate teletype font -- the  
default for _verbatim text_ should be to _show_ the control chars.

> - act upon them
>
> and
>
> - show them (might somehow interfere with other things)

Showing the control chars in typeset text -- non-verbatim -- should be  
rare; more appropriate for verbatim

> - hide them

I suggest that the default for _typeset text_ should definitely be to  
_hide_ the control chars.

> if i'm right, when bidi is turned on, those chars get processed and then
> discarded from the node list, so some more than zwj and zwnj is handled,

It appears to me that zwj and zwnj etc. should be invisible in  
typeset-text output -- as explained above, but should still be encoded in  
the output pdf. Think pdf-text extraction, converting between Arabic and  
Farsi typesetting conventions, etc.

> and of course others need to be handled as well

Even lsep's and psep's should be present in the output pdf (eg \par =>  
psep). Will make text extraction much more useful, etc.

Best wishes
Idris

-- 
Professor Idris Samawi Hamid, Editor-in-Chief
International Journal of Shi`i Studies
Department of Philosophy
Colorado State University
Fort Collins, CO 80523
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________