* Paragraph breaking bug with BiDi text
@ 2016-01-09 3:40 Mohammad Hossein Bateni
2016-01-09 3:47 ` Mohammad Hossein Bateni
0 siblings, 1 reply; 6+ messages in thread
From: Mohammad Hossein Bateni @ 2016-01-09 3:40 UTC (permalink / raw)
To: mailing list for ConTeXt users
[-- Attachment #1.1: Type: text/plain, Size: 1344 bytes --]
Hello,
When a line ends with a sequence whose direction differs from that of the
paragraph, we risk pushing some text into the margin (when not necessary).
Here is an example with corresponding output:
\usemodule[simplefonts]
\setmainfont[ALM Fixed][features=arabic,range=arabic]
\setupalign[r2l]
\setupwhitespace[big]
\showframe
\starttext
% 10 copies of Persian word "hello" stay on one line.
\dorecurse{10}{سلام }
% 20 copies makes a 2-line paragraph.
\dorecurse{20}{سلام }
% one copy of the word goes into the margin although the Latin letters
perfectly fit the line.
\dorecurse{10}{سلام }
{\textdir TLT\dorecurse{20}{a}}
\dorecurse{10}{سلام }
% although the Latin string extends into the margin, TeX still puts one
copy of "hello" there as well.
\dorecurse{10}{سلام }
{\textdir TLT\dorecurse{30}{a}}
\dorecurse{10}{سلام }
% something similar happens here with the opposite par/text dir
\pardir TLT
\dorecurse{10}{bidi }
{\textdir TRT\dorecurse{20}{آ}}
\dorecurse{10}{bidi }
\stoptext
The problem seems to be that after typesetting the LTR text within the RTL
paragraph, TeX thinks the current text ends at the left end of the LTR
portion; hence, it tries to add something to the line; it's only after that
that it discovers we ran into the margin!
—MHB
[-- Attachment #1.2: Type: text/html, Size: 1556 bytes --]
[-- Attachment #2: bidi-par-bug.pdf --]
[-- Type: application/pdf, Size: 6055 bytes --]
[-- Attachment #3: Type: text/plain, Size: 485 bytes --]
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!
maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage : http://www.pragma-ade.nl / http://tex.aanhet.net
archive : http://foundry.supelec.fr/projects/contextrev/
wiki : http://contextgarden.net
___________________________________________________________________________________
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: Paragraph breaking bug with BiDi text
2016-01-09 3:40 Paragraph breaking bug with BiDi text Mohammad Hossein Bateni
@ 2016-01-09 3:47 ` Mohammad Hossein Bateni
2016-01-11 13:43 ` Hans Hagen
0 siblings, 1 reply; 6+ messages in thread
From: Mohammad Hossein Bateni @ 2016-01-09 3:47 UTC (permalink / raw)
To: mailing list for ConTeXt users
[-- Attachment #1.1: Type: text/plain, Size: 1714 bytes --]
Minor point: changing "\pardir TLT" to "\pardir TLT\textdir TLT" in the
last paragraph produces better visual, however, the previous paragraphs
already demonstrate the problem sufficiently.
On Fri, Jan 8, 2016 at 10:40 PM, Mohammad Hossein Bateni <bateni@gmail.com>
wrote:
> Hello,
>
> When a line ends with a sequence whose direction differs from that of the
> paragraph, we risk pushing some text into the margin (when not necessary).
> Here is an example with corresponding output:
>
>
> \usemodule[simplefonts]
> \setmainfont[ALM Fixed][features=arabic,range=arabic]
> \setupalign[r2l]
> \setupwhitespace[big]
> \showframe
>
> \starttext
>
> % 10 copies of Persian word "hello" stay on one line.
> \dorecurse{10}{سلام }
> % 20 copies makes a 2-line paragraph.
> \dorecurse{20}{سلام }
>
> % one copy of the word goes into the margin although the Latin letters
> perfectly fit the line.
> \dorecurse{10}{سلام }
> {\textdir TLT\dorecurse{20}{a}}
> \dorecurse{10}{سلام }
>
> % although the Latin string extends into the margin, TeX still puts one
> copy of "hello" there as well.
> \dorecurse{10}{سلام }
> {\textdir TLT\dorecurse{30}{a}}
> \dorecurse{10}{سلام }
>
> % something similar happens here with the opposite par/text dir
> \pardir TLT
> \dorecurse{10}{bidi }
> {\textdir TRT\dorecurse{20}{آ}}
> \dorecurse{10}{bidi }
>
> \stoptext
>
>
> The problem seems to be that after typesetting the LTR text within the RTL
> paragraph, TeX thinks the current text ends at the left end of the LTR
> portion; hence, it tries to add something to the line; it's only after that
> that it discovers we ran into the margin!
>
> —MHB
>
>
[-- Attachment #1.2: Type: text/html, Size: 2199 bytes --]
[-- Attachment #2: Type: text/plain, Size: 485 bytes --]
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!
maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage : http://www.pragma-ade.nl / http://tex.aanhet.net
archive : http://foundry.supelec.fr/projects/contextrev/
wiki : http://contextgarden.net
___________________________________________________________________________________
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: Paragraph breaking bug with BiDi text
2016-01-09 3:47 ` Mohammad Hossein Bateni
@ 2016-01-11 13:43 ` Hans Hagen
2016-01-11 16:58 ` Mohammad Hossein Bateni
0 siblings, 1 reply; 6+ messages in thread
From: Hans Hagen @ 2016-01-11 13:43 UTC (permalink / raw)
To: mailing list for ConTeXt users
On 1/9/2016 4:47 AM, Mohammad Hossein Bateni wrote:
> Minor point: changing "\pardir TLT" to "\pardir TLT\textdir TLT" in the
> last paragraph produces better visual, however, the previous paragraphs
> already demonstrate the problem sufficiently.
It is a side effect of what the the par builder considers to be valid
breakpoints. The current approach is playing very safe but after looking
at it Taco and I decided that it can be a more tolerant with respect to
end dirs so the next luatex version will have that.
Anyway: You need to code carefully: the space after "TRT" in "\textdir
TRT x" is meaningful so in your example you introduce spaces.
Also, in context don't use \textdir etc directly, just use \lefttoright
and \righttoleft in combination with \setupalign as I will not spend
much time on side effects of interfering with these low level dir
changers directly.
> On Fri, Jan 8, 2016 at 10:40 PM, Mohammad Hossein Bateni
> <bateni@gmail.com <mailto:bateni@gmail.com>> wrote:
>
> Hello,
>
> When a line ends with a sequence whose direction differs from that
> of the paragraph, we risk pushing some text into the margin (when
> not necessary). Here is an example with corresponding output:
>
>
> \usemodule[simplefonts]
> \setmainfont[ALM Fixed][features=arabic,range=arabic]
> \setupalign[r2l]
> \setupwhitespace[big]
> \showframe
>
> \starttext
>
> % 10 copies of Persian word "hello" stay on one line.
> \dorecurse{10}{سلام }
> % 20 copies makes a 2-line paragraph.
> \dorecurse{20}{سلام }
>
> % one copy of the word goes into the margin although the Latin
> letters perfectly fit the line.
> \dorecurse{10}{سلام }
> {\textdir TLT\dorecurse{20}{a}}
> \dorecurse{10}{سلام }
>
> % although the Latin string extends into the margin, TeX still puts
> one copy of "hello" there as well.
> \dorecurse{10}{سلام }
> {\textdir TLT\dorecurse{30}{a}}
> \dorecurse{10}{سلام }
>
> % something similar happens here with the opposite par/text dir
> \pardir TLT
> \dorecurse{10}{bidi }
> {\textdir TRT\dorecurse{20}{آ}}
> \dorecurse{10}{bidi }
>
> \stoptext
>
>
> The problem seems to be that after typesetting the LTR text within
> the RTL paragraph, TeX thinks the current text ends at the left end
> of the LTR portion; hence, it tries to add something to the line;
> it's only after that that it discovers we ran into the margin!
>
> —MHB
>
>
>
>
> ___________________________________________________________________________________
> If your question is of interest to others as well, please add an entry to the Wiki!
>
> maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
> webpage : http://www.pragma-ade.nl / http://tex.aanhet.net
> archive : http://foundry.supelec.fr/projects/contextrev/
> wiki : http://contextgarden.net
> ___________________________________________________________________________________
>
--
-----------------------------------------------------------------
Hans Hagen | PRAGMA ADE
Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
tel: 038 477 53 69 | www.pragma-ade.com | www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!
maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage : http://www.pragma-ade.nl / http://tex.aanhet.net
archive : http://foundry.supelec.fr/projects/contextrev/
wiki : http://contextgarden.net
___________________________________________________________________________________
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: Paragraph breaking bug with BiDi text
2016-01-11 13:43 ` Hans Hagen
@ 2016-01-11 16:58 ` Mohammad Hossein Bateni
2016-01-11 19:24 ` Wolfgang Schuster
2016-01-11 21:52 ` Hans Hagen
0 siblings, 2 replies; 6+ messages in thread
From: Mohammad Hossein Bateni @ 2016-01-11 16:58 UTC (permalink / raw)
To: mailing list for ConTeXt users
[-- Attachment #1.1: Type: text/plain, Size: 1552 bytes --]
>
> It is a side effect of what the the par builder considers to be valid
> breakpoints. The current approach is playing very safe but after looking at
> it Taco and I decided that it can be a more tolerant with respect to end
> dirs so the next luatex version will have that.
>
> Anyway: You need to code carefully: the space after "TRT" in "\textdir TRT
> x" is meaningful so in your example you introduce spaces.
>
Thanks for the explanation. So the space after \textdir TLT is meaningful
but the one after \lefttoright isn't. (I guess, I should have used
\textdir TLT\relax. I'm going to use the high-level \lefttoright command
from now on.)
Also, in context don't use \textdir etc directly, just use \lefttoright and
> \righttoleft in combination with \setupalign as I will not spend much time
> on side effects of interfering with these low level dir changers directly.
>
Great! You partly answered another question I meant to ask: what is the
proper way to write \textdir and \pardir in CONTEXT? So \lefttoright and
\righttoleft are replacements for \textdir TLT and \textdir TRT. How
should I code in \pardir TRT in CONTEXT? I couldn't find anything for that
in spac-ali.mkiv.
Those low-level LuaTeX directives were part of an attempt to come to the
core of a problem I'd run into with numbers at the end of a right-to-left
line. The BiDi algorithm correctly gave left-to-right direction to that
digit sequence and the result was that the word following the number had
gone into the margin.
—MHB
[-- Attachment #1.2: Type: text/html, Size: 2109 bytes --]
[-- Attachment #2: Type: text/plain, Size: 485 bytes --]
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!
maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage : http://www.pragma-ade.nl / http://tex.aanhet.net
archive : http://foundry.supelec.fr/projects/contextrev/
wiki : http://contextgarden.net
___________________________________________________________________________________
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2016-01-11 21:52 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-01-09 3:40 Paragraph breaking bug with BiDi text Mohammad Hossein Bateni
2016-01-09 3:47 ` Mohammad Hossein Bateni
2016-01-11 13:43 ` Hans Hagen
2016-01-11 16:58 ` Mohammad Hossein Bateni
2016-01-11 19:24 ` Wolfgang Schuster
2016-01-11 21:52 ` Hans Hagen
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).