\framed (and others) don't honor directional switches

\framed (and others) don't honor directional switches

[Khaled Hosny]

[-- Attachment #1.1: Type: text/plain, Size: 490 bytes --]

framed, framedtext, TEXpage (and may be others), no longer honor
directional switches. I'm sure TEXpage at least was OK a month ago.

\pagedir TRT\bodydir TRT\pardir TRT\textdir TRT
\starttext
\startTEXpage
\input tufte
\stopTEXpage
\framed{Hello world}
\startframedtext
\input tufte
\stopframedtext
\startbackground
\input tufte
\stopbackground
\stoptext

Regards,
 Khaled


-- 
 Khaled Hosny
 Arabic localiser and member of Arabeyes.org team
 Free font developer

[-- Attachment #1.2: Digital signature --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

[-- Attachment #2: Type: text/plain, Size: 487 bytes --]

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: \framed (and others) don't honor directional switches

[Hans Hagen]

Khaled Hosny wrote:
> framed, framedtext, TEXpage (and may be others), no longer honor
> directional switches. I'm sure TEXpage at least was OK a month ago.
> 
> \pagedir TRT\bodydir TRT\pardir TRT\textdir TRT
> \starttext
> \startTEXpage
> \input tufte
> \stopTEXpage
> \framed{Hello world}
> \startframedtext
> \input tufte
> \stopframedtext
> \startbackground
> \input tufte
> \stopbackground
> \stoptext

if so then it's a side effect of changesin the engine (changes in rl 
handling are dangerous because macro packages are internally lr in many 
aspects (like positioning data relative to other data) and patching each 
macro for rl is non trivial (due to possible side effects) so first we 
need to be sure that the luatex engine is okay

Hans


-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
      tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com
                                              | www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: \framed (and others) don't honor directional switches

[Khaled Hosny]

[-- Attachment #1.1: Type: text/plain, Size: 1430 bytes --]

On Fri, Apr 03, 2009 at 02:55:56PM +0200, Hans Hagen wrote:
> Khaled Hosny wrote:
>> framed, framedtext, TEXpage (and may be others), no longer honor
>> directional switches. I'm sure TEXpage at least was OK a month ago.
>>
>> \pagedir TRT\bodydir TRT\pardir TRT\textdir TRT
>> \starttext
>> \startTEXpage
>> \input tufte
>> \stopTEXpage
>> \framed{Hello world}
>> \startframedtext
>> \input tufte
>> \stopframedtext
>> \startbackground
>> \input tufte
>> \stopbackground
>> \stoptext
>
> if so then it's a side effect of changesin the engine (changes in rl  
> handling are dangerous because macro packages are internally lr in many  
> aspects (like positioning data relative to other data) and patching each  
> macro for rl is non trivial (due to possible side effects) so first we  
> need to be sure that the luatex engine is okay

I tried to debug thus further, compiling the following document with
both plain and context formats, in plain all boxes are TRT (the part of
the log after "Completed box being shipped out [1]"), while in context
all boxes are TLT except that hbox that contains the text. Though I
don't know how much relevant is this or what it does actually mean.

\pagedir TRT
\bodydir TRT
\pardir TRT
\textdir TRT
\tracingall
Hello \hbox{World}
\bye

Regards,
 Khaled


-- 
 Khaled Hosny
 Arabic localiser and member of Arabeyes.org team
 Free font developer

[-- Attachment #1.2: Digital signature --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

[-- Attachment #2: Type: text/plain, Size: 487 bytes --]

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: \framed (and others) don't honor directional switches

[Hans Hagen]

Khaled Hosny wrote:
> On Fri, Apr 03, 2009 at 02:55:56PM +0200, Hans Hagen wrote:
>> Khaled Hosny wrote:
>>> framed, framedtext, TEXpage (and may be others), no longer honor
>>> directional switches. I'm sure TEXpage at least was OK a month ago.
>>>
>>> \pagedir TRT\bodydir TRT\pardir TRT\textdir TRT
>>> \starttext
>>> \startTEXpage
>>> \input tufte
>>> \stopTEXpage
>>> \framed{Hello world}
>>> \startframedtext
>>> \input tufte
>>> \stopframedtext
>>> \startbackground
>>> \input tufte
>>> \stopbackground
>>> \stoptext
>> if so then it's a side effect of changesin the engine (changes in rl  
>> handling are dangerous because macro packages are internally lr in many  
>> aspects (like positioning data relative to other data) and patching each  
>> macro for rl is non trivial (due to possible side effects) so first we  
>> need to be sure that the luatex engine is okay
> 
> I tried to debug thus further, compiling the following document with
> both plain and context formats, in plain all boxes are TRT (the part of
> the log after "Completed box being shipped out [1]"), while in context
> all boxes are TLT except that hbox that contains the text. Though I
> don't know how much relevant is this or what it does actually mean.
> 
> \pagedir TRT
> \bodydir TRT
> \pardir TRT
> \textdir TRT
> \tracingall
> Hello \hbox{World}
> \bye

you can comment this in cont-new.mkiv

% \def\pagedir{\expandafter\gobblethreearguments}
% \def\bodydir{\expandafter\gobblethreearguments}

i'm trying to figure out what dirs make sense (even when all are enabled 
the alignment is no 100% ok; for instance, why should bodydir influence 
textdir and pardir (i.e. when i would expect the content to be rl 
independent of page/bodydir

(we need some higher level interface and not these low level *dir commands)

best is if you and idris sort this out (is part of idris onthology)

-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
      tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com
                                              | www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: \framed (and others) don't honor directional switches

[Idris Samawi Hamid ادريس سماوي حامد]

On Fri, 03 Apr 2009 15:08:48 -0600, Hans Hagen <pragma@wxs.nl> wrote:

> you can comment this in cont-new.mkiv
>  % \def\pagedir{\expandafter\gobblethreearguments}
> % \def\bodydir{\expandafter\gobblethreearguments}
>  i'm trying to figure out what dirs make sense (even when all are  
> enabled the alignment is no 100% ok; for instance, why should bodydir  
> influence textdir and pardir (i.e. when i would expect the content to be  
> rl independent of page/bodydir
>  (we need some higher level interface and not these low level *dir  
> commands)

Indeed. We may also need Taco's help; he's about the only one I know who  
precisely understands the distinction between \pardir and \bodydir. I did  
some extensive testing on this nearly two years ago and could hardly make  
sense of it. The user should never have to touch \pardir or \bodydir and  
we need a higher level command. Not sure if the same will work at the  
macro level.

I'll search for Taco's old detailed explanations of these two and forward  
them to you and Khalid.

Best wishes
Idris

-- 
Professor Idris Samawi Hamid, Editor-in-Chief
International Journal of Shi`i Studies
Department of Philosophy
Colorado State University
Fort Collins, CO 80523
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: \framed (and others) don't honor directional switches

[Hans Hagen]

Idris Samawi Hamid ادريس سماوي حامد wrote:
> On Fri, 03 Apr 2009 15:08:48 -0600, Hans Hagen <pragma@wxs.nl> wrote:
> 
>> you can comment this in cont-new.mkiv
>>  % \def\pagedir{\expandafter\gobblethreearguments}
>> % \def\bodydir{\expandafter\gobblethreearguments}
>>  i'm trying to figure out what dirs make sense (even when all are 
>> enabled the alignment is no 100% ok; for instance, why should bodydir 
>> influence textdir and pardir (i.e. when i would expect the content to 
>> be rl independent of page/bodydir
>>  (we need some higher level interface and not these low level *dir 
>> commands)
> 
> Indeed. We may also need Taco's help; he's about the only one I know who 
> precisely understands the distinction between \pardir and \bodydir. I 
> did some extensive testing on this nearly two years ago and could hardly 
> make sense of it. The user should never have to touch \pardir or 
> \bodydir and we need a higher level command. Not sure if the same will 
> work at the macro level.
> 
> I'll search for Taco's old detailed explanations of these two and 
> forward them to you and Khalid.

there has been changes (consistency cleasups) to the engine so you might 
want to run some tests again; first we need to be absolutely sure of the 
engine backend does things right (i.e. as we expect)

Hans



-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
      tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com
                                              | www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: \framed (and others) don't honor directional switches

[Hans Hagen]

Khaled Hosny wrote:
> On Fri, Apr 03, 2009 at 02:55:56PM +0200, Hans Hagen wrote:
>> Khaled Hosny wrote:
>>> framed, framedtext, TEXpage (and may be others), no longer honor
>>> directional switches. I'm sure TEXpage at least was OK a month ago.
>>>
>>> \pagedir TRT\bodydir TRT\pardir TRT\textdir TRT
>>> \starttext
>>> \startTEXpage
>>> \input tufte
>>> \stopTEXpage
>>> \framed{Hello world}
>>> \startframedtext
>>> \input tufte
>>> \stopframedtext
>>> \startbackground
>>> \input tufte
>>> \stopbackground
>>> \stoptext
>> if so then it's a side effect of changesin the engine (changes in rl  
>> handling are dangerous because macro packages are internally lr in many  
>> aspects (like positioning data relative to other data) and patching each  
>> macro for rl is non trivial (due to possible side effects) so first we  
>> need to be sure that the luatex engine is okay
> 
> I tried to debug thus further, compiling the following document with
> both plain and context formats, in plain all boxes are TRT (the part of
> the log after "Completed box being shipped out [1]"), while in context
> all boxes are TLT except that hbox that contains the text. Though I
> don't know how much relevant is this or what it does actually mean.

a better example of the mess we're dealing with:

\starttext

\pagedir TRT \bodydir TRT \pardir TRT \textdir TRT \startTEXpage \input 
tufte \stopTEXpage
\pagedir TLT \bodydir TRT \pardir TRT \textdir TRT \startTEXpage \input 
tufte \stopTEXpage
\pagedir TRT \bodydir TLT \pardir TRT \textdir TRT \startTEXpage \input 
tufte \stopTEXpage
\pagedir TLT \bodydir TLT \pardir TRT \textdir TRT \startTEXpage \input 
tufte \stopTEXpage

\stoptext



-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
      tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com
                                              | www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: \framed (and others) don't honor directional switches

[Khaled Hosny]

[-- Attachment #1.1: Type: text/plain, Size: 961 bytes --]

On Fri, Apr 03, 2009 at 11:17:25PM +0200, Hans Hagen wrote:
> a better example of the mess we're dealing with:
>
> \starttext
>
> \pagedir TRT \bodydir TRT \pardir TRT \textdir TRT \startTEXpage \input  
> tufte \stopTEXpage
> \pagedir TLT \bodydir TRT \pardir TRT \textdir TRT \startTEXpage \input  
> tufte \stopTEXpage
> \pagedir TRT \bodydir TLT \pardir TRT \textdir TRT \startTEXpage \input  
> tufte \stopTEXpage
> \pagedir TLT \bodydir TLT \pardir TRT \textdir TRT \startTEXpage \input  
> tufte \stopTEXpage
>
> \stoptext

Thanks for the example, I think I got your point. From my understanding,
I think text and paragraph direction should be only controlled by
\textdir and \pardir respectively, whatever \pagedir or \bodydir are.
Indeed, I don't see why \bodydir would influence the text direction
inside boxes.

Regards,
 Khaled


-- 
 Khaled Hosny
 Arabic localiser and member of Arabeyes.org team
 Free font developer

[-- Attachment #1.2: Digital signature --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

[-- Attachment #2: Type: text/plain, Size: 487 bytes --]

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: \framed (and others) don't honor directional switches

[Hans Hagen]

Khaled Hosny wrote:
> On Fri, Apr 03, 2009 at 11:17:25PM +0200, Hans Hagen wrote:
>> a better example of the mess we're dealing with:
>>
>> \starttext
>>
>> \pagedir TRT \bodydir TRT \pardir TRT \textdir TRT \startTEXpage \input  
>> tufte \stopTEXpage
>> \pagedir TLT \bodydir TRT \pardir TRT \textdir TRT \startTEXpage \input  
>> tufte \stopTEXpage
>> \pagedir TRT \bodydir TLT \pardir TRT \textdir TRT \startTEXpage \input  
>> tufte \stopTEXpage
>> \pagedir TLT \bodydir TLT \pardir TRT \textdir TRT \startTEXpage \input  
>> tufte \stopTEXpage
>>
>> \stoptext
> 
> Thanks for the example, I think I got your point. From my understanding,
> I think text and paragraph direction should be only controlled by
> \textdir and \pardir respectively, whatever \pagedir or \bodydir are.
> Indeed, I don't see why \bodydir would influence the text direction
> inside boxes.

indeed, so we're looking into it now (an dlet's forget about the 
page/bodydir for the moment)

since textdir and pardir go hand-in-hand i suggest the following

\setuplayout[direction=r2l]    % also influences general layout issues
\setupalign [r2l]              % textdir and pardir
\setuparranging[direction=r2l] % will flush pages backward

so l2r and r2l values; as they are part of the align mechanism then, 
they are automatically part of all commands that have a align key

Hans


-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
      tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com
                                              | www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: \framed (and others) don't honor directional switches

[Khaled Hosny]

[-- Attachment #1.1: Type: text/plain, Size: 1710 bytes --]

On Sat, Apr 04, 2009 at 12:35:30PM +0200, Hans Hagen wrote:
> Khaled Hosny wrote:
> indeed, so we're looking into it now (an dlet's forget about the  
> page/bodydir for the moment)
>
> since textdir and pardir go hand-in-hand i suggest the following
>
> \setuplayout[direction=r2l]    % also influences general layout issues

I assume that will mirror all layout elements like double sided page
numbering, right and left pages etc. right?

> \setupalign [r2l]              % textdir and pardir
> \setuparranging[direction=r2l] % will flush pages backward
>
> so l2r and r2l values; as they are part of the align mechanism then,  
> they are automatically part of all commands that have a align key

This sounds very good for me.

While we are in it, I'd suggest that the current implicit bidi to be
revised, currently it is broken. I suggest to be implement Unicode BiDi
algorithm[1], and the "higher-level protocols[2]" part is worthy
investigating. Also, I think it should be decoupled from character
mirroring, so that one can enable mirroring while explicitly setting
text direction. Also, some characters that had assigned mirrored
property in Unicode 5.0.0 has been reversed in later revisions[3]
because it "affects deployed data" (the source of all evil!), this
includes quotation marks, so we need to handle mirroring of those
characters as part of the suggested higher-level protocols as well.


[1]http://www.unicode.org/reports/tr9/
[2]http://www.unicode.org/reports/tr9/#Higher-Level_Protocols
[3]http://unicode.org/versions/corrigendum6.html

Regards,
 Khaled


-- 
 Khaled Hosny
 Arabic localiser and member of Arabeyes.org team
 Free font developer

[-- Attachment #1.2: Digital signature --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

[-- Attachment #2: Type: text/plain, Size: 487 bytes --]

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: \framed (and others) don't honor directional switches

[Hans Hagen]

Khaled Hosny wrote:
> On Sat, Apr 04, 2009 at 12:35:30PM +0200, Hans Hagen wrote:
>> Khaled Hosny wrote:
>> indeed, so we're looking into it now (an dlet's forget about the  
>> page/bodydir for the moment)
>>
>> since textdir and pardir go hand-in-hand i suggest the following
>>
>> \setuplayout[direction=r2l]    % also influences general layout issues
> 
> I assume that will mirror all layout elements like double sided page
> numbering, right and left pages etc. right?

yes, the thing that idris identified in his onthology (which might have 
white spots); we start on this once some other arabic font subsystem, 
has been completed

>> \setupalign [r2l]              % textdir and pardir
>> \setuparranging[direction=r2l] % will flush pages backward
>>
>> so l2r and r2l values; as they are part of the align mechanism then,  
>> they are automatically part of all commands that have a align key
> 
> This sounds very good for me.

ok, the next beta will have the \setupalign and align= available

> While we are in it, I'd suggest that the current implicit bidi to be
> revised, currently it is broken. I suggest to be implement Unicode BiDi
> algorithm[1], and the "higher-level protocols[2]" part is worthy
> investigating. Also, I think it should be decoupled from character
> mirroring, so that one can enable mirroring while explicitly setting
> text direction. Also, some characters that had assigned mirrored
> property in Unicode 5.0.0 has been reversed in later revisions[3]
> because it "affects deployed data" (the source of all evil!), this
> includes quotation marks, so we need to handle mirroring of those
> characters as part of the suggested higher-level protocols as well.

hm, this character mirroring is kind of evil anyway (as it can also 
intefere with font features)

i'll look into it later (first need to finish a few things)

the problem with bidi is that it it has to work ok with the already 
present (of triggered) TRT / TLT nodes which is not that trivial

Hans


-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
      tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com
                                              | www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________