Re: French typography is back

Re: French typography is back

[Robert-André Mauchin]

Le 21/05/2009 18:29, Hans Hagen a écrit :
 > \setcharacterspacing[frenchpunctuation]
 >
 > \hsize .1mm a? aa? aaa? aaaa?
 >
 > seems ok here so what version are you using?
 >

Ok, I've made test, I stand corrected, it does add a nnbsp, but does not 
remove the space (if there is) before the punctuation mark. That's why I 
end up with a "?" at the beginning of a new line.

Thanks for the tip.

Bob.

ps: sorry for the bad "reply", but my subscription request has not been 
approved yet.
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: French typography is back

[Arthur Reutenauer]

> Not knowing the internals, that's what I tried to say with adding a space 
> after instead of 1em, i.e. "calculated by the engine". If w an em is added 
> after, it is not stretchable and shrinkable, right?

  Right.  But no particular space is added after '?' or '!' in French
punctuation mode, TeX sets the standard inter-word glue, as it should.
The only intervention by ConTeXt is the addition of the unbreakable
space before those marks (and, obviously, after '«').

> Thank you so much for the detailed technical explanation! So, AFAIK, I 
> believe that the space before should be equivalent to thinspace.

  You'd need to take this up with Olivier Guéry, who suggested the 1/4
em value in the first place (refer to his interventions from last Fall,
e.g., http://www.ntg.nl/pipermail/ntg-context/2008/034845.html).  He
wrote up a small proposal on the wiki page for French Punctuation
(http://wiki.contextgarden.net/French_Punctuation).

> Thanks, I'll see that. Maybe I could write some detailled specs in the wiki.

  Yes, please do so by extending the aforementioned page if needed.

	Arthur
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: French typography is back

[Hans Hagen]

luigi scarso wrote:
> On Fri, May 22, 2009 at 5:51 PM, Arthur Reutenauer <
> arthur.reutenauer@normalesup.org> wrote:
> 
>>>                                but instead of arbitrary adding a 0.25em
>>> before and 1em after the punctuation mark you should use the real nnbsp
>>> (U+202F) before and real normal space (U+0020) after.
>>   I don't think so.  Space characters don't mix very well with TeX glue
>> and should best be avoided, generally speaking.  In particular, all
>> inter-word spaces that are input in the TeX source as one or more of
>> U+0020 are simply ignored, and replaced by normal inter-word glue, with
>> its appropriate stretchability and shrinkability.  This has always been
>> the case in TeX and is not going to change.  All other types of Unicode
>> spaces should really, in my opinion, be processed in the same way, while
>> respecting their additional properties in the case of non-breakable
>> spaces, for instance.
>>
> 
> Maybe one can use
> something like this
> \defineremapper[filterItem]
> \remapcharacter[filterItem][`•]{\item}
> 
> for "spaces"  too

as such a remapper is applied on all input eventually you end up in a 
mess and we don't want a mess ... the current mechanism acts upon the to 
be typeset text which is way more safe



-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
      tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com
                                              | www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: French typography is back

[luigi scarso]

[-- Attachment #1.1: Type: text/plain, Size: 1080 bytes --]

On Fri, May 22, 2009 at 5:51 PM, Arthur Reutenauer <
arthur.reutenauer@normalesup.org> wrote:

> >                                but instead of arbitrary adding a 0.25em
> > before and 1em after the punctuation mark you should use the real nnbsp
> > (U+202F) before and real normal space (U+0020) after.
>
>   I don't think so.  Space characters don't mix very well with TeX glue
> and should best be avoided, generally speaking.  In particular, all
> inter-word spaces that are input in the TeX source as one or more of
> U+0020 are simply ignored, and replaced by normal inter-word glue, with
> its appropriate stretchability and shrinkability.  This has always been
> the case in TeX and is not going to change.  All other types of Unicode
> spaces should really, in my opinion, be processed in the same way, while
> respecting their additional properties in the case of non-breakable
> spaces, for instance.
>

Maybe one can use
something like this
\defineremapper[filterItem]
\remapcharacter[filterItem][`•]{\item}

for "spaces"  too

-- 
luigi

[-- Attachment #1.2: Type: text/html, Size: 1493 bytes --]

[-- Attachment #2: Type: text/plain, Size: 487 bytes --]

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: French typography is back

[Robert-André Mauchin]

Le 22/05/2009 17:51, Arthur Reutenauer a écrit :
>>                                 but instead of arbitrary adding a 0.25em
>> before and 1em after the punctuation mark you should use the real nnbsp
>> (U+202F) before and real normal space (U+0020) after.
>
>    I don't think so.  Space characters don't mix very well with TeX glue
> and should best be avoided, generally speaking.  In particular, all
> inter-word spaces that are input in the TeX source as one or more of
> U+0020 are simply ignored, and replaced by normal inter-word glue, with
> its appropriate stretchability and shrinkability.  This has always been
> the case in TeX and is not going to change.  All other types of Unicode
> spaces should really, in my opinion, be processed in the same way, while
> respecting their additional properties in the case of non-breakable
> spaces, for instance.
>

Not knowing the internals, that's what I tried to say with adding a 
space after instead of 1em, i.e. "calculated by the engine". If w an em 
is added after, it is not stretchable and shrinkable, right?

>    In addition, characters like U+202F are very badly supported across
> fonts, and if you take in account the fact that the most appropriate
> width will probably change depending on the language, you're likely to
> observe much more arbitrary results if you use the glyph for that
> character in font.  I seriously doubt you want to rely on the font for
> that.
>
>> Why? Let me take your example again:
>>
>> {\setcharacterspacing[frenchpunctuation]a? aa? aaa? abba?}
>>
>> a\,? aa\,? aaa\,? abba\,?
>>
>> Surprise: the first line is longer than the second. It's because sizes of
>> the U+0020 and U+202F depend on the font design, their size are not exactly
>> 1em and 0.25em.
>
>    That's not the reason.  The reason is simply that \, is defined as a
> \kern by one sixth of an em (see core-spa.mkiv: it's equivalent to
> \thinspace, which is \kern .16667em).  In the first line, the value of
> .25em is defined in core-spa.mkiv; you can redefine it if you want.
> In any case, every space is completely controlled by ConTeXt, we don't
> let the font mess around.
>
>    For that matter, Latin Modern doesn't have a glyph for U+202F, so if
> we'd use it, we'd just see nothing: there would be no space at all, see
> attached file.
>

Thank you so much for the detailed technical explanation! So, AFAIK, I 
believe that the space before should be equivalent to thinspace.


>    All this really calls for more coordination in order to produce decent
> specifications, in my opinion.  If you think ConTeXt's default should be
> different, it's fine and I encourage you to contact Sébastien to discuss
> about it.  Report then to Hans and Peter for the implementation.
>

Thanks, I'll see that. Maybe I could write some detailled specs in the wiki.


Regards,

Bob.
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: French typography is back

[Arthur Reutenauer]

[-- Attachment #1: Type: text/plain, Size: 3301 bytes --]

>                                but instead of arbitrary adding a 0.25em 
> before and 1em after the punctuation mark you should use the real nnbsp 
> (U+202F) before and real normal space (U+0020) after.

  I don't think so.  Space characters don't mix very well with TeX glue
and should best be avoided, generally speaking.  In particular, all
inter-word spaces that are input in the TeX source as one or more of
U+0020 are simply ignored, and replaced by normal inter-word glue, with
its appropriate stretchability and shrinkability.  This has always been
the case in TeX and is not going to change.  All other types of Unicode
spaces should really, in my opinion, be processed in the same way, while
respecting their additional properties in the case of non-breakable
spaces, for instance.

  In addition, characters like U+202F are very badly supported across
fonts, and if you take in account the fact that the most appropriate
width will probably change depending on the language, you're likely to
observe much more arbitrary results if you use the glyph for that
character in font.  I seriously doubt you want to rely on the font for
that.

> Why? Let me take your example again:
>
> {\setcharacterspacing[frenchpunctuation]a? aa? aaa? abba?}
>
> a\,? aa\,? aaa\,? abba\,?
>
> Surprise: the first line is longer than the second. It's because sizes of 
> the U+0020 and U+202F depend on the font design, their size are not exactly 
> 1em and 0.25em.

  That's not the reason.  The reason is simply that \, is defined as a
\kern by one sixth of an em (see core-spa.mkiv: it's equivalent to
\thinspace, which is \kern .16667em).  In the first line, the value of
.25em is defined in core-spa.mkiv; you can redefine it if you want.
In any case, every space is completely controlled by ConTeXt, we don't
let the font mess around.

  For that matter, Latin Modern doesn't have a glyph for U+202F, so if
we'd use it, we'd just see nothing: there would be no space at all, see
attached file.

> Moreover, this is true that in french typography we should use a thin space 
> before some punctuation mark, and a thin space in France *was* one fourth 
> of an em. BUT due to modern digital typography, a thin space now correspond 
> to something like one fifth of an em.
> In digital typography, the normal space, i.e. inter-word separation, i.e. 
> "espace justifiante" in French, has generally a size around one fourth of 
> an em, calculated by the engine (you know that better than me). The thin 
> space is a bit more than a half of the inter-word separation, i.e. between 
> 1/8 and 1/6 of an em.

  All this really calls for more coordination in order to produce decent
specifications, in my opinion.  If you think ConTeXt's default should be
different, it's fine and I encourage you to contact Sébastien to discuss
about it.  Report then to Hans and Peter for the implementation.

> So I *really* believe that you should not define space before and after 
> punctuation with "arbitrary" em spacing. There must be a way to do a 
> "search and replace" with the correct glyphs (U+202F and U+0020) before and 
> after punctuation marks.

  As my example file demonstrates, we clearly shouldn't use the glyphs
from the font.

	Arthur

[-- Attachment #2: frenchpunctuation.pdf --]
[-- Type: application/pdf, Size: 15357 bytes --]

[-- Attachment #3: frenchpunctuation.tex --]
[-- Type: application/x-tex, Size: 852 bytes --]

[-- Attachment #4: Type: text/plain, Size: 487 bytes --]

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: French typography is back

[Robert-André Mauchin]

 > Hans Hagen a écrit :
 > > Robert-André Mauchin wrote:
 > >
 > > The best would be to get rid off
 > > \setcharacterspacing[frenchpunctuation] and adapt the frenchb Babel
 > > module from Daniel Flipo (source and explanation here
 > > http://daniel.flipo.free.fr/frenchb/ ), as a module (t-french
 > > modification), or when loading the language, but I really
 > > don't know how hard it would be since it handles a lot more than
 > > just punctuation marks.
 >
 > no, active chars for such things are no solution and will not be
 > supported in mkiv; setcharacterspacing is the way to go (and if it
 > breaks that should be fixed instead); language support shoul dbe
 > in the kernel anyway
 >
 > Hans


I don't specifically talk about active chars (I don't really not what 
this is, I'm an user, not a coder), but instead of arbitrary adding a 
0.25em before and 1em after the punctuation mark you should use the real 
nnbsp (U+202F) before and real normal space (U+0020) after.

Why? Let me take your example again:

{\setcharacterspacing[frenchpunctuation]a? aa? aaa? abba?}

a\,? aa\,? aaa\,? abba\,?

Surprise: the first line is longer than the second. It's because sizes 
of the U+0020 and U+202F depend on the font design, their size are not 
exactly 1em and 0.25em.

Moreover, this is true that in french typography we should use a thin 
space before some punctuation mark, and a thin space in France *was* one 
fourth of an em. BUT due to modern digital typography, a thin space now 
correspond to something like one fifth of an em.
In digital typography, the normal space, i.e. inter-word separation, 
i.e. "espace justifiante" in French, has generally a size around one 
fourth of an em, calculated by the engine (you know that better than 
me). The thin space is a bit more than a half of the inter-word 
separation, i.e. between 1/8 and 1/6 of an em.

So I *really* believe that you should not define space before and after 
punctuation with "arbitrary" em spacing. There must be a way to do a 
"search and replace" with the correct glyphs (U+202F and U+0020) before 
and after punctuation marks.

 > language support should be in the kernel anyway

I agree of course, but although I believe this would encourage french 
"texers" to use Context, I don't think typography for a specific 
language is the priority for you&Taco right now.

I wish you luck for the next beta release!

Bob.
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: French typography is back

[Wolfgang Schuster]

Am 21.05.2009 um 18:29 schrieb Hans Hagen:

> Robert-André Mauchin wrote:
>> Hi again,
>> I have been reading all threads about the French typography in this  
>> mailing list, however the current solution is alas far for perfect.  
>> The command \setcharacterspacing[frenchpunctuation] was a good  
>> idea, but it only deals with spacing. French typography rules not  
>> only tells us that there should be a narrow, or a normal space  
>> before some punctuation marks like :, ;, , !, ?, « or », but also  
>> that theses spaces must be non breakable otherwise line breaking  
>> could occur just before a punctuation mark (and this is very bad).
>> Take a look at the following example:
>> \section{Production locale indigène ou production romaine importée?}
>
> \setcharacterspacing[frenchpunctuation]
>
> \hsize .1mm a? aa? aaa? aaaa?
>
> seems ok here so what version are you using?

Not tested but shouldn't be the following two lines (type-spa.lua ->  
line 131/132) be written in the reverse order (penalty before skip).

  	insert_node_after(head,start,make_glue_node(tex.scale(quad,right)))
	insert_node_after(head,start,make_penalty_node(10000))

Wolfgang

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: French typography is back

[Hans Hagen]

Robert-André Mauchin wrote:
> Hi again,
> 
> I have been reading all threads about the French typography in this 
> mailing list, however the current solution is alas far for perfect. The 
> command \setcharacterspacing[frenchpunctuation] was a good idea, but it 
> only deals with spacing. French typography rules not only tells us that 
> there should be a narrow, or a normal space before some punctuation 
> marks like :, ;, , !, ?, « or », but also that theses spaces must be non 
> breakable otherwise line breaking could occur just before a punctuation 
> mark (and this is very bad).
> 
> Take a look at the following example:
> \section{Production locale indigène ou production romaine importée?}

\setcharacterspacing[frenchpunctuation]

\hsize .1mm a? aa? aaa? aaaa?

seems ok here so what version are you using?



-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
      tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com
                                              | www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: French typography is back

[Hans Hagen]

Robert-André Mauchin wrote:

> The best would be to get rid off \setcharacterspacing[frenchpunctuation] 
> and adapt the frenchb Babel module from Daniel Flipo (source and 
> explanation here http://daniel.flipo.free.fr/frenchb/ ), as a module 
> (t-french modification), or when loading the language, but I really 
> don't know how hard it would be since it handles a lot more than just 
> punctuation marks.

no, active chars for such things are no solution and will not be 
supported in mkiv; setcharacterspacing is the way to go (and if it 
breaks that should be fixed instead); language support shoul dbe in the 
kernel anyway

Hans

-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
      tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com
                                              | www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: French typography is back

[Arthur Reutenauer]

> and adapt the frenchb Babel module from Daniel Flipo (source and 
> explanation here http://daniel.flipo.free.fr/frenchb/ ), as a module 
> (t-french modification)

  Contact Sébastien Mengin (sebastien <at> edilibre.net) about that; he
started working on it.

	Arthur
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: French typography is back

[Peter Münster]

On Thu, 21 May 2009, Robert-André Mauchin wrote:

> With the current frenchpunctuation command, we got:
> Production locale indigène ou production romaine importée
> ?

Seems to be a bug. Some months ago, this space was unbreakable as it
should.


> The best would be to get rid off \setcharacterspacing[frenchpunctuation] and
> adapt the frenchb Babel module from Daniel Flipo (source and explanation here
> http://daniel.flipo.free.fr/frenchb/ ), as a module (t-french modification),
> or when loading the language, but I really don't know how hard it would be
> since it handles a lot more than just punctuation marks.

A good idea. This is work in progress:
http://modules.contextgarden.net/t-french
You can contribute.

But I think, \setcharacterspacing[frenchpunctuation] should stay. The
module t-french.tex should just integrate this command, that's all.

Cheers, Peter

-- 
Contact information: http://pmrb.free.fr/contact/

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

French typography is back

[Robert-André Mauchin]

Hi again,

I have been reading all threads about the French typography in this 
mailing list, however the current solution is alas far for perfect. The 
command \setcharacterspacing[frenchpunctuation] was a good idea, but it 
only deals with spacing. French typography rules not only tells us that 
there should be a narrow, or a normal space before some punctuation 
marks like :, ;, , !, ?, « or », but also that theses spaces must be non 
breakable otherwise line breaking could occur just before a punctuation 
mark (and this is very bad).

Take a look at the following example:
\section{Production locale indigène ou production romaine importée?}

With the current frenchpunctuation command, we got:
Production locale indigène ou production romaine importée
?

We should obtain:
Production locale indigène ou production romaine
importée ?

Of course we could achieve this manually by adding the correct NNBSP 
like this:
\section{Production locale indigène ou production romaine importée\,?}
But I believe the whole point of TeX is to let the engine make all the 
typographic stuff.

The best would be to get rid off \setcharacterspacing[frenchpunctuation] 
and adapt the frenchb Babel module from Daniel Flipo (source and 
explanation here http://daniel.flipo.free.fr/frenchb/ ), as a module 
(t-french modification), or when loading the language, but I really 
don't know how hard it would be since it handles a lot more than just 
punctuation marks.

Regards,

Bob.
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________