LaTeX: Individual words in Chinese script

LaTeX: Individual words in Chinese script

['Frederik Elwert' via pandoc-discuss]

[-- Attachment #1.1: Type: text/plain, Size: 2127 bytes --]

Hi!


We are typesetting a lot of academic texts which are mainly in English, but 
contain words in other languages/scripts. At the moment, I need to produce 
an article that contains individual words in traditional Chinese.

With pandoc, I currently face two issues:

First, chinese is not defined in Writers/LaTeX/Lang.hs. This means that, in 
contrast to other scripts, I cannot use `[言]{lang="zh-Hant"}`. For other 
languages that are defined, the LaTeX writer produces 
`\foreignlanguage{}{}` commands, but in this case it produces only `{言}`. 
Thus, the only possibility is to resort to `CJKmainfont`. The issue is that 
this switches on xeCJK for the whole document, which has undesirable side 
effects (mainly concerning spacing around punctuation characters). Thus, I 
have to introduce manual `\makexeCJKactive`/`\makexeCJKinactive` commands 
in my markdown source. For short passages of Chinese, it is actually 
desirable not to use xeCJK, but simply babel, as with other scripts.

Second, I do not see a way to define fonts for other scripts. The default 
LaTeX template calls `\babelprovide` for used fonts, but does not allow to 
define corresponding `\babelfont`s. With the old polyglossia system, IIRC, 
it was possible to use the `fontfamilies` metadata entry to define 
`\chinesefont` etc. for additional scripts. But this does no longer work 
with babel (if I’m not missing something).

So basically my question is twofold: First, how do I define additional 
fonts for foreign scripts with the new babel system, and second, would it 
be possible to support chinese script in the LaTeX writer in the same way 
it supports other scripts?
Thanks,
Frederik

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/cadd3e14-3167-447f-9f64-8158a6613cb6n%40googlegroups.com.

[-- Attachment #1.2: Type: text/html, Size: 2538 bytes --]

Re: LaTeX: Individual words in Chinese script

[John MacFarlane]

Obviously, it is important to support this kind of thing.  Can someone else who uses pandoc with multiple scripts comment on how you handle this?

One solution I see at
https://tex.stackexchange.com/questions/165197/russian-and-chinese-in-the-same-document
is 

\usepackage[encapsulated]{CJK}

and then

\begin{CJK}{UTF8}{gbsn}
你好
\end{CJK}

Perhaps you could use a Lua filter to convert [你好]{lang=zh} to this form?





> On Mar 10, 2023, at 2:35 AM, 'Frederik Elwert' via pandoc-discuss <pandoc-discuss-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org> wrote:
> 
> Hi!
> 
> 
> We are typesetting a lot of academic texts which are mainly in English, but contain words in other languages/scripts. At the moment, I need to produce an article that contains individual words in traditional Chinese.
> 
> With pandoc, I currently face two issues:
> 
> First, chinese is not defined in Writers/LaTeX/Lang.hs. This means that, in contrast to other scripts, I cannot use `[言]{lang="zh-Hant"}`. For other languages that are defined, the LaTeX writer produces `\foreignlanguage{}{}` commands, but in this case it produces only `{言}`. Thus, the only possibility is to resort to `CJKmainfont`. The issue is that this switches on xeCJK for the whole document, which has undesirable side effects (mainly concerning spacing around punctuation characters). Thus, I have to introduce manual `\makexeCJKactive`/`\makexeCJKinactive` commands in my markdown source. For short passages of Chinese, it is actually desirable not to use xeCJK, but simply babel, as with other scripts.
> 
> Second, I do not see a way to define fonts for other scripts. The default LaTeX template calls `\babelprovide` for used fonts, but does not allow to define corresponding `\babelfont`s. With the old polyglossia system, IIRC, it was possible to use the `fontfamilies` metadata entry to define `\chinesefont` etc. for additional scripts. But this does no longer work with babel (if I’m not missing something).
> 
> So basically my question is twofold: First, how do I define additional fonts for foreign scripts with the new babel system, and second, would it be possible to support chinese script in the LaTeX writer in the same way it supports other scripts?
> Thanks,
> Frederik
> 
> -- 
> You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
> To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/cadd3e14-3167-447f-9f64-8158a6613cb6n%40googlegroups.com.

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/667077D3-D8A5-4B6A-9253-F3F569439ADA%40gmail.com.

Re: LaTeX: Individual words in Chinese script

['Frederik Elwert' via pandoc-discuss]

[-- Attachment #1.1: Type: text/plain, Size: 4355 bytes --]

Thanks for the kind reply!

If I see it correctly, modern babel works fine with Chinese the same way as 
other languages. I wrote a small lua filter that inserts raw 
`\foreignlanguage`/`\begin{otherlanguage}` commands for chinese in the same 
way as pandoc by default does for other languages, and it works just fine. 
So maybe the best solution would be to simply add chinese to Lang.hs?

Also, I modified the template a little bit to support a `babelfonts` 
metadata structure. That allows to specify custom fonts for other scripts. 
Would something like this be acceptable for the default template?

You can see my solution in this gist: 
https://gist.github.com/frederik-elwert/fb1ab57bf88fa2c6dbd7253958b64014

An example patch for the template is attached.

If someone sees any issues with this approach, I’m glad if they could point 
them out. I don’t write Chinese myself, so I don’t know if there are subtle 
issues that I’m not aware of.





John MacFarlane schrieb am Freitag, 10. März 2023 um 19:17:56 UTC+1:

> Obviously, it is important to support this kind of thing. Can someone else 
> who uses pandoc with multiple scripts comment on how you handle this?
>
> One solution I see at
>
> https://tex.stackexchange.com/questions/165197/russian-and-chinese-in-the-same-document
> is 
>
> \usepackage[encapsulated]{CJK}
>
> and then
>
> \begin{CJK}{UTF8}{gbsn}
> 你好
> \end{CJK}
>
> Perhaps you could use a Lua filter to convert [你好]{lang=zh} to this form?
>
>
>
>
>
> > On Mar 10, 2023, at 2:35 AM, 'Frederik Elwert' via pandoc-discuss <
> pandoc-...-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org> wrote:
> > 
> > Hi!
> > 
> > 
> > We are typesetting a lot of academic texts which are mainly in English, 
> but contain words in other languages/scripts. At the moment, I need to 
> produce an article that contains individual words in traditional Chinese.
> > 
> > With pandoc, I currently face two issues:
> > 
> > First, chinese is not defined in Writers/LaTeX/Lang.hs. This means that, 
> in contrast to other scripts, I cannot use `[言]{lang="zh-Hant"}`. For other 
> languages that are defined, the LaTeX writer produces 
> `\foreignlanguage{}{}` commands, but in this case it produces only `{言}`. 
> Thus, the only possibility is to resort to `CJKmainfont`. The issue is that 
> this switches on xeCJK for the whole document, which has undesirable side 
> effects (mainly concerning spacing around punctuation characters). Thus, I 
> have to introduce manual `\makexeCJKactive`/`\makexeCJKinactive` commands 
> in my markdown source. For short passages of Chinese, it is actually 
> desirable not to use xeCJK, but simply babel, as with other scripts.
> > 
> > Second, I do not see a way to define fonts for other scripts. The 
> default LaTeX template calls `\babelprovide` for used fonts, but does not 
> allow to define corresponding `\babelfont`s. With the old polyglossia 
> system, IIRC, it was possible to use the `fontfamilies` metadata entry to 
> define `\chinesefont` etc. for additional scripts. But this does no longer 
> work with babel (if I’m not missing something).
> > 
> > So basically my question is twofold: First, how do I define additional 
> fonts for foreign scripts with the new babel system, and second, would it 
> be possible to support chinese script in the LaTeX writer in the same way 
> it supports other scripts?
> > Thanks,
> > Frederik
> > 
> > -- 
> > You received this message because you are subscribed to the Google 
> Groups "pandoc-discuss" group.
> > To unsubscribe from this group and stop receiving emails from it, send 
> an email to pandoc-discus...-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
> > To view this discussion on the web visit 
> https://groups.google.com/d/msgid/pandoc-discuss/cadd3e14-3167-447f-9f64-8158a6613cb6n%40googlegroups.com
> .
>
>

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/a66a708e-6599-4cab-88a5-a89f592e8b8en%40googlegroups.com.

[-- Attachment #1.2: Type: text/html, Size: 5921 bytes --]

[-- Attachment #2: babelfont.patch --]
[-- Type: text/x-patch, Size: 428 bytes --]

--- default.tex	2023-03-10 21:20:49.332547333 +0100
+++ langtest_template.tex	2023-03-10 20:55:38.286983982 +0100
@@ -362,6 +362,9 @@
 $for(babel-otherlangs)$
 \babelprovide[import]{$babel-otherlangs$}
 $endfor$
+$for(babelfonts)$
+\babelfont[$babelfonts.language$]{rm}{$babelfonts.font$}
+$endfor$
 % get rid of language-specific shorthands (see #6817):
 \let\LanguageShortHands\languageshorthands
 \def\languageshorthands#1{}

Re: LaTeX: Individual words in Chinese script

[John MacFarlane]

[-- Attachment #1: Type: text/plain, Size: 5665 bytes --]

I’ve updated pandoc with both the changes you suggested.  chinese is now used with babel, and the template now includes babelfonts.  With the new code you should be able to make this work without a filter.

> On Mar 10, 2023, at 12:32 PM, 'Frederik Elwert' via pandoc-discuss <pandoc-discuss-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org> wrote:
> 
> Thanks for the kind reply!
> 
> If I see it correctly, modern babel works fine with Chinese the same way as other languages. I wrote a small lua filter that inserts raw `\foreignlanguage`/`\begin{otherlanguage}` commands for chinese in the same way as pandoc by default does for other languages, and it works just fine. So maybe the best solution would be to simply add chinese to Lang.hs?
> 
> Also, I modified the template a little bit to support a `babelfonts` metadata structure. That allows to specify custom fonts for other scripts. Would something like this be acceptable for the default template?
> 
> You can see my solution in this gist: https://gist.github.com/frederik-elwert/fb1ab57bf88fa2c6dbd7253958b64014 <https://gist.github.com/frederik-elwert/fb1ab57bf88fa2c6dbd7253958b64014>
> 
> An example patch for the template is attached.
> 
> If someone sees any issues with this approach, I’m glad if they could point them out. I don’t write Chinese myself, so I don’t know if there are subtle issues that I’m not aware of.
> 
> 
> 
> 
> 
> John MacFarlane schrieb am Freitag, 10. März 2023 um 19:17:56 UTC+1:
> Obviously, it is important to support this kind of thing. Can someone else who uses pandoc with multiple scripts comment on how you handle this? 
> 
> One solution I see at 
> https://tex.stackexchange.com/questions/165197/russian-and-chinese-in-the-same-document <https://tex.stackexchange.com/questions/165197/russian-and-chinese-in-the-same-document> 
> is 
> 
> \usepackage[encapsulated]{CJK} 
> 
> and then 
> 
> \begin{CJK}{UTF8}{gbsn} 
> 你好 
> \end{CJK} 
> 
> Perhaps you could use a Lua filter to convert [你好]{lang=zh} to this form? 
> 
> 
> 
> 
> 
> > On Mar 10, 2023, at 2:35 AM, 'Frederik Elwert' via pandoc-discuss <pandoc-...-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org <applewebdata://F64696CA-DF55-4DAE-8142-03F401B85C46>> wrote: 
> > 
> > Hi! 
> > 
> > 
> > We are typesetting a lot of academic texts which are mainly in English, but contain words in other languages/scripts. At the moment, I need to produce an article that contains individual words in traditional Chinese. 
> > 
> > With pandoc, I currently face two issues: 
> > 
> > First, chinese is not defined in Writers/LaTeX/Lang.hs. This means that, in contrast to other scripts, I cannot use `[言]{lang="zh-Hant"}`. For other languages that are defined, the LaTeX writer produces `\foreignlanguage{}{}` commands, but in this case it produces only `{言}`. Thus, the only possibility is to resort to `CJKmainfont`. The issue is that this switches on xeCJK for the whole document, which has undesirable side effects (mainly concerning spacing around punctuation characters). Thus, I have to introduce manual `\makexeCJKactive`/`\makexeCJKinactive` commands in my markdown source. For short passages of Chinese, it is actually desirable not to use xeCJK, but simply babel, as with other scripts. 
> > 
> > Second, I do not see a way to define fonts for other scripts. The default LaTeX template calls `\babelprovide` for used fonts, but does not allow to define corresponding `\babelfont`s. With the old polyglossia system, IIRC, it was possible to use the `fontfamilies` metadata entry to define `\chinesefont` etc. for additional scripts. But this does no longer work with babel (if I’m not missing something). 
> > 
> > So basically my question is twofold: First, how do I define additional fonts for foreign scripts with the new babel system, and second, would it be possible to support chinese script in the LaTeX writer in the same way it supports other scripts? 
> > Thanks, 
> > Frederik 
> > 
> > -- 
> > You received this message because you are subscribed to the Google Groups "pandoc-discuss" group. 
> > To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discus...-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org <applewebdata://F64696CA-DF55-4DAE-8142-03F401B85C46>. 
> > To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/cadd3e14-3167-447f-9f64-8158a6613cb6n%40googlegroups.com <https://groups.google.com/d/msgid/pandoc-discuss/cadd3e14-3167-447f-9f64-8158a6613cb6n%40googlegroups.com>. 
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org <mailto:pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org>.
> To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/a66a708e-6599-4cab-88a5-a89f592e8b8en%40googlegroups.com <https://groups.google.com/d/msgid/pandoc-discuss/a66a708e-6599-4cab-88a5-a89f592e8b8en%40googlegroups.com?utm_medium=email&utm_source=footer>.
> <babelfont.patch>

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/2163FD26-E90E-472A-A94E-D20247FC7A9C%40gmail.com.

[-- Attachment #2: Type: text/html, Size: 20158 bytes --]

Re: LaTeX: Individual words in Chinese script

['Frederik Elwert' via pandoc-discuss]

[-- Attachment #1.1: Type: text/plain, Size: 5575 bytes --]


Wow, thank you so much!
John MacFarlane schrieb am Freitag, 10. März 2023 um 23:20:35 UTC+1:

> I’ve updated pandoc with both the changes you suggested.  chinese is now 
> used with babel, and the template now includes babelfonts.  With the new 
> code you should be able to make this work without a filter.
>
> On Mar 10, 2023, at 12:32 PM, 'Frederik Elwert' via pandoc-discuss <
> pandoc-...-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org> wrote:
>
> Thanks for the kind reply!
>
> If I see it correctly, modern babel works fine with Chinese the same way 
> as other languages. I wrote a small lua filter that inserts raw 
> `\foreignlanguage`/`\begin{otherlanguage}` commands for chinese in the same 
> way as pandoc by default does for other languages, and it works just fine. 
> So maybe the best solution would be to simply add chinese to Lang.hs?
>
> Also, I modified the template a little bit to support a `babelfonts` 
> metadata structure. That allows to specify custom fonts for other scripts. 
> Would something like this be acceptable for the default template?
>
> You can see my solution in this gist: 
> https://gist.github.com/frederik-elwert/fb1ab57bf88fa2c6dbd7253958b64014
>
> An example patch for the template is attached.
>
> If someone sees any issues with this approach, I’m glad if they could 
> point them out. I don’t write Chinese myself, so I don’t know if there are 
> subtle issues that I’m not aware of.
>
>
>
>
>
> John MacFarlane schrieb am Freitag, 10. März 2023 um 19:17:56 UTC+1:
>
>> Obviously, it is important to support this kind of thing. Can someone 
>> else who uses pandoc with multiple scripts comment on how you handle this?
>>  
>>
>> One solution I see at 
>>
>> https://tex.stackexchange.com/questions/165197/russian-and-chinese-in-the-same-document
>>  
>> is 
>>
>> \usepackage[encapsulated]{CJK} 
>>
>> and then 
>>
>> \begin{CJK}{UTF8}{gbsn} 
>> 你好 
>> \end{CJK} 
>>
>> Perhaps you could use a Lua filter to convert [你好]{lang=zh} to this form?
>>  
>>
>>
>>
>>
>>
>> > On Mar 10, 2023, at 2:35 AM, 'Frederik Elwert' via pandoc-discuss <
>> pandoc-...-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org> wrote: 
>> > 
>> > Hi! 
>> > 
>> > 
>> > We are typesetting a lot of academic texts which are mainly in English, 
>> but contain words in other languages/scripts. At the moment, I need to 
>> produce an article that contains individual words in traditional Chinese.
>>  
>> > 
>> > With pandoc, I currently face two issues: 
>> > 
>> > First, chinese is not defined in Writers/LaTeX/Lang.hs. This means 
>> that, in contrast to other scripts, I cannot use `[言]{lang="zh-Hant"}`. For 
>> other languages that are defined, the LaTeX writer produces 
>> `\foreignlanguage{}{}` commands, but in this case it produces only `{言}`. 
>> Thus, the only possibility is to resort to `CJKmainfont`. The issue is that 
>> this switches on xeCJK for the whole document, which has undesirable side 
>> effects (mainly concerning spacing around punctuation characters). Thus, I 
>> have to introduce manual `\makexeCJKactive`/`\makexeCJKinactive` commands 
>> in my markdown source. For short passages of Chinese, it is actually 
>> desirable not to use xeCJK, but simply babel, as with other scripts. 
>> > 
>> > Second, I do not see a way to define fonts for other scripts. The 
>> default LaTeX template calls `\babelprovide` for used fonts, but does not 
>> allow to define corresponding `\babelfont`s. With the old polyglossia 
>> system, IIRC, it was possible to use the `fontfamilies` metadata entry to 
>> define `\chinesefont` etc. for additional scripts. But this does no longer 
>> work with babel (if I’m not missing something). 
>> > 
>> > So basically my question is twofold: First, how do I define additional 
>> fonts for foreign scripts with the new babel system, and second, would it 
>> be possible to support chinese script in the LaTeX writer in the same way 
>> it supports other scripts? 
>> > Thanks, 
>> > Frederik 
>> > 
>> > -- 
>> > You received this message because you are subscribed to the Google 
>> Groups "pandoc-discuss" group. 
>> > To unsubscribe from this group and stop receiving emails from it, send 
>> an email to pandoc-discus...-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org 
>> > To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/pandoc-discuss/cadd3e14-3167-447f-9f64-8158a6613cb6n%40googlegroups.com
>> . 
>>
>>
> -- 
> You received this message because you are subscribed to the Google Groups 
> "pandoc-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to pandoc-discus...-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
>
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/pandoc-discuss/a66a708e-6599-4cab-88a5-a89f592e8b8en%40googlegroups.com 
> <https://groups.google.com/d/msgid/pandoc-discuss/a66a708e-6599-4cab-88a5-a89f592e8b8en%40googlegroups.com?utm_medium=email&utm_source=footer>
> .
> <babelfont.patch>
>
>
>

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/4b263b7a-edbb-4939-86a7-21ca9fb1b8d1n%40googlegroups.com.

[-- Attachment #1.2: Type: text/html, Size: 16380 bytes --]