ligatures in html

ligatures in html

[Chris Wright]

[-- Attachment #1.1: Type: text/plain, Size: 1215 bytes --]

 

I want to publish a document with an \ae ligature to html and to pdf. The 
latex form "\ae robic" converts to the appropriate form and displays 
properly in pdf, but the html just drops the ligature.


Simple test case:


chriswri$ cat > test.txt

\ae robic

chriswri$ more test.txt

\ae robic

chriswri$ pandoc -t native test.txt

[Para [RawInline (Format "tex") "\\ae ",Str "robic"]]

chriswri$ pandoc -t html test.txt

<p>robic</p>


What's the best way around this - write a filter? finding some docs that 
will help? (I've found that ... is automatically converted to an ellipsis 
 - so \dots isn't necessary).


with thanks


Chris



-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/7d633ff1-c25d-436c-a66f-9a8456699db6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[-- Attachment #1.2: Type: text/html, Size: 2387 bytes --]

Re: ligatures in html

['Jason Seeley' via pandoc-discuss]

[-- Attachment #1.1: Type: text/plain, Size: 2008 bytes --]

Hello,

Ligatures like \ae are specific to the LaTeX (and thus PDF) writer, so they 
don't work in any other formats. Pandoc just passes it through unchanged. 
For HTML output, you can use an entity: `Æ` or `æ`, for upper 
case or lower case. Another option is to use the unicode character directly 
(how you do this depends on your system and text editor; in Windows hold 
Alt and type 0230 on the number pad; in vim type CTRL-K a e; use a 
character-map app, etc.) This should work for most output formats. It'll 
work with LaTeX if you use XeLaTeX or LuaLaTeX, as those allow unicode 
input.

Jason

On Monday, September 21, 2015 at 5:57:37 AM UTC-5, Chris Wright wrote:
>
> I want to publish a document with an \ae ligature to html and to pdf. The 
> latex form "\ae robic" converts to the appropriate form and displays 
> properly in pdf, but the html just drops the ligature.
>
>
> Simple test case:
>
>
> chriswri$ cat > test.txt
>
> \ae robic
>
> chriswri$ more test.txt
>
> \ae robic
>
> chriswri$ pandoc -t native test.txt
>
> [Para [RawInline (Format "tex") "\\ae ",Str "robic"]]
>
> chriswri$ pandoc -t html test.txt
>
> <p>robic</p>
>
>
> What's the best way around this - write a filter? finding some docs that 
> will help? (I've found that ... is automatically converted to an ellipsis 
>  - so \dots isn't necessary).
>
>
> with thanks
>
>
> Chris
>
>
>
>

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/bbaae9b2-c139-415f-9063-86a887358b4c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[-- Attachment #1.2: Type: text/html, Size: 2921 bytes --]

Re: ligatures in html

[John MACFARLANE]

Ideally pandoc's latex reader would recognize \ae and
convert it to the proper character, so feel free to put
an issue on the bug tracker about this.

+++ 'Jason Seeley' via pandoc-discuss [Sep 21 15 07:50 ]:
>   Hello,
>   Ligatures like \ae are specific to the LaTeX (and thus PDF) writer, so
>   they don't work in any other formats. Pandoc just passes it through
>   unchanged. For HTML output, you can use an entity: `Æ` or
>   `æ`, for upper case or lower case. Another option is to use the
>   unicode character directly (how you do this depends on your system and
>   text editor; in Windows hold Alt and type 0230 on the number pad; in
>   vim type CTRL-K a e; use a character-map app, etc.) This should work
>   for most output formats. It'll work with LaTeX if you use XeLaTeX or
>   LuaLaTeX, as those allow unicode input.
>   Jason
>   On Monday, September 21, 2015 at 5:57:37 AM UTC-5, Chris Wright wrote:
>
>   I want to publish a document with an \ae ligature to html and to pdf.
>   The latex form "\ae robic" converts to the appropriate form and
>   displays properly in pdf, but the html just drops the ligature.
>
>   Simple test case:
>
>   chriswri$ cat > test.txt
>
>   \ae robic
>
>   chriswri$ more test.txt
>
>   \ae robic
>
>   chriswri$ pandoc -t native test.txt
>
>   [Para [RawInline (Format "tex") "\\ae ",Str "robic"]]
>
>   chriswri$ pandoc -t html test.txt
>
>   <p>robic</p>
>
>   What's the best way around this - write a filter? finding some docs
>   that will help? (I've found that ... is automatically converted to an
>   ellipsis  - so \dots isn't necessary).
>
>   with thanks
>
>   Chris
>
>   --
>   You received this message because you are subscribed to the Google
>   Groups "pandoc-discuss" group.
>   To unsubscribe from this group and stop receiving emails from it, send
>   an email to [1]pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
>   To post to this group, send email to
>   [2]pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
>   To view this discussion on the web visit
>   [3]https://groups.google.com/d/msgid/pandoc-discuss/bbaae9b2-c139-415f-
>   9063-86a887358b4c%40googlegroups.com.
>   For more options, visit [4]https://groups.google.com/d/optout.
>
>References
>
>   1. mailto:pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
>   2. mailto:pandoc-discuss-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
>   3. https://groups.google.com/d/msgid/pandoc-discuss/bbaae9b2-c139-415f-9063-86a887358b4c-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org?utm_medium=email&utm_source=footer
>   4. https://groups.google.com/d/optout

Re: ligatures in html

[Chris Wright]

[-- Attachment #1.1: Type: text/plain, Size: 4903 bytes --]

Thanks for the help folks. 
I don't understand why this would be an issue with the latex reader- that's 
a comment about me, not pandoc! - so I apologise for the laborious pace of 
this question...

If I change the test document to:

$ cat test.md
... \ae robe

that's three periods then the ligature

and look at the native format:

$ pandoc -S -f markdown -t native test.md > test.native
[Para [Str "\8230",Space,RawInline (Format "tex") "\\ae ",Str "robe"]]

so the three periods are converted to the correct ellipsis character, 
and the ligature is parsed to a RawInline

then outputting latex from this native representation:

$ pandoc -S -f native -t latex test.native 
\ldots{} \ae robe

so Str "\8230" is converted to \dots, and the ligature is done from 
RawInline

converting the same native to html:

$ pandoc -S -f native -t html test.native 
<p>… robe </p>

the Str"\8230" is printed as the single ellipsis character, and the 
RawInline is dropped.

It seems as if the md reader can parse three periods to an ellipsis 
character, but doesn't have a representation of the ligature that would 
work in both HTML and LaTex - thought it would work in LuaLaTeX/XeTex if it 
outputted the ligature character (e.g. Str"\8230"). 

Might it work if the ligature was recognised as something like:

Ligature(ae) was generated in the native format, which could then be 
converted by whatever writer produced the output format?

again,  thanks for your patience and help

Chris






On Tuesday, 22 September 2015 06:55:14 UTC+10, John MacFarlane wrote:
Ideally pandoc's latex reader would recognize \ae and 
convert it to the proper character, so feel free to put 
an issue on the bug tracker about this. 

+++ 'Jason Seeley' via pandoc-discuss [Sep 21 15 07:50 ]: 
> Hello, 
> Ligatures like \ae are specific to the LaTeX (and thus PDF) writer, so 
> they don't work in any other formats. Pandoc just passes it through 
> unchanged. For HTML output, you can use an entity: `&AElig;` or 
> `&aelig;`, for upper case or lower case. Another option is to use the 
> unicode character directly (how you do this depends on your system and 
> text editor; in Windows hold Alt and type 0230 on the number pad; in 
> vim type CTRL-K a e; use a character-map app, etc.) This should work 
> for most output formats. It'll work with LaTeX if you use XeLaTeX or 
> LuaLaTeX, as those allow unicode input. 
> Jason 
> On Monday, September 21, 2015 at 5:57:37 AM UTC-5, Chris Wright wrote: 
> 
> I want to publish a document with an \ae ligature to html and to pdf. 
> The latex form "\ae robic" converts to the appropriate form and 
> displays properly in pdf, but the html just drops the ligature. 
> 
> Simple test case: 
> 
> chriswri$ cat > test.txt 
> 
> \ae robic 
> 
> chriswri$ more test.txt 
> 
> \ae robic 
> 
> chriswri$ pandoc -t native test.txt 
> 
> [Para [RawInline (Format "tex") "\\ae ",Str "robic"]] 
> 
> chriswri$ pandoc -t html test.txt 
> 
> <p>robic</p> 
> 
> What's the best way around this - write a filter? finding some docs 
> that will help? (I've found that ... is automatically converted to an 
> ellipsis - so \dots isn't necessary). 
> 
> with thanks 
> 
> Chris 
> 
> -- 
> You received this message because you are subscribed to the Google 
> Groups "pandoc-discuss" group. 
> To unsubscribe from this group and stop receiving emails from it, send 
> an email to [1]pandoc-discus...-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org <javascript:>. 
> To post to this group, send email to 
> [2]pandoc-...-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org <javascript:>. 
> To view this discussion on the web visit 
> [3]https://groups.google.com/d/msgid/pandoc-discuss/bbaae9b2-c139-415f- 
> 9063-86a887358b4c%40googlegroups.com. 
> For more options, visit [4]https://groups.google.com/d/optout. 
> 
>References 
> 
> 1. mailto:pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org <javascript:> 
> 2. mailto:pandoc-...-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org <javascript:> 
> 3. 
https://groups.google.com/d/msgid/pandoc-discuss/bbaae9b2-c139-415f-9063-86a887358b4c-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org?utm_medium=email&utm_source=footer 
> 4. https://groups.google.com/d/optout 

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/ab142d34-ed35-4034-be33-744e955f0329%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[-- Attachment #1.2: Type: text/html, Size: 6681 bytes --]

Re: ligatures in html

[david.pw.smith-Re5JQEeQqe8AvxtiuMwx3w]

[-- Attachment #1.1: Type: text/plain, Size: 2079 bytes --]

I think the more obvious problem is that LaTeX doesn't handle UTF-8 well. 
If UTF-8 is properly supported then you don't need escape characters at 
all, you can just type the æ. These examples would work fine in both html 
and XeTeX/XeLaTeX:

echo "... ærobic" | pandoc -s -f markdown -t html -o test.html

echo "... ærobic" | pandoc -s -f markdown -t latex -o test.tex

You can see for yourself with:

echo "... ærobic" | pandoc -s -f markdown --latex-engine=xelatex -o test.pdf

Indeed, for me, Pandoc's default tex template outputs the unicode 
characters correctly, but this could be because my environment is set up to 
use XeLaTeX.

Hope that clarifies a bit?




On Monday, 21 September 2015 11:57:37 UTC+1, Chris Wright wrote:
>
> I want to publish a document with an \ae ligature to html and to pdf. The 
> latex form "\ae robic" converts to the appropriate form and displays 
> properly in pdf, but the html just drops the ligature.
>
>
> Simple test case:
>
>
> chriswri$ cat > test.txt
>
> \ae robic
>
> chriswri$ more test.txt
>
> \ae robic
>
> chriswri$ pandoc -t native test.txt
>
> [Para [RawInline (Format "tex") "\\ae ",Str "robic"]]
>
> chriswri$ pandoc -t html test.txt
>
> <p>robic</p>
>
>
> What's the best way around this - write a filter? finding some docs that 
> will help? (I've found that ... is automatically converted to an ellipsis 
>  - so \dots isn't necessary).
>
>
> with thanks
>
>
> Chris
>
>
>
>

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/874daeba-ced3-4d7d-b2ad-b0178e5a079b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[-- Attachment #1.2: Type: text/html, Size: 3001 bytes --]

Re: ligatures in html

[Stefan Björk]

[-- Attachment #1: Type: text/plain, Size: 3607 bytes --]

Sorry for cutting in, but I wonder why using TeX macros in markdown in the
first place? As Jason and David points out, the 'Æ' ligature is available
in UTF-8 (and latin-1 as well, I think, since 'æ' is a characther of its
own in some Nordic countries), and since all modern variants of TeX, namely
XeTeX and LuaTeX, more or less requires UTF-8. By far, the easiest solution
should be to simply use the 'æ' character in markdown -- unless there are
some requirements of ascii-only markdown?

And yes, pandoc converts three dots '...' to an ellipsis '…', but '...' is
not a TeX macro.

tis 22 sep. 2015 kl 11:53 skrev <david.pw.smith-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>:

> I think the more obvious problem is that LaTeX doesn't handle UTF-8 well.
> If UTF-8 is properly supported then you don't need escape characters at
> all, you can just type the æ. These examples would work fine in both html
> and XeTeX/XeLaTeX:
>
> echo "... ærobic" | pandoc -s -f markdown -t html -o test.html
>
> echo "... ærobic" | pandoc -s -f markdown -t latex -o test.tex
>
> You can see for yourself with:
>
> echo "... ærobic" | pandoc -s -f markdown --latex-engine=xelatex -o
> test.pdf
>
> Indeed, for me, Pandoc's default tex template outputs the unicode
> characters correctly, but this could be because my environment is set up to
> use XeLaTeX.
>
> Hope that clarifies a bit?
>
>
>
>
>
> On Monday, 21 September 2015 11:57:37 UTC+1, Chris Wright wrote:
>>
>> I want to publish a document with an \ae ligature to html and to pdf. The
>> latex form "\ae robic" converts to the appropriate form and displays
>> properly in pdf, but the html just drops the ligature.
>>
>>
>> Simple test case:
>>
>>
>> chriswri$ cat > test.txt
>>
>> \ae robic
>>
>> chriswri$ more test.txt
>>
>> \ae robic
>>
>> chriswri$ pandoc -t native test.txt
>>
>> [Para [RawInline (Format "tex") "\\ae ",Str "robic"]]
>>
>> chriswri$ pandoc -t html test.txt
>>
>> <p>robic</p>
>>
>>
>> What's the best way around this - write a filter? finding some docs that
>> will help? (I've found that ... is automatically converted to an ellipsis
>>  - so \dots isn't necessary).
>>
>>
>> with thanks
>>
>>
>> Chris
>>
>>
>>
>> --
> You received this message because you are subscribed to the Google Groups
> "pandoc-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
> To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/pandoc-discuss/874daeba-ced3-4d7d-b2ad-b0178e5a079b%40googlegroups.com
> <https://groups.google.com/d/msgid/pandoc-discuss/874daeba-ced3-4d7d-b2ad-b0178e5a079b%40googlegroups.com?utm_medium=email&utm_source=footer>
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/CABsBQU7Qm4W5tnEqXsOz7AW7X%3DBitNP0PWj%2BYo68CiKknnexTA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[-- Attachment #2: Type: text/html, Size: 5067 bytes --]

Re: ligatures in html

[Joost Kremers]

On Tue, Sep 22 2015, david.pw.smith-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org wrote:
> Indeed, for me, Pandoc's default tex template outputs the unicode 
> characters correctly, but this could be because my environment is set up to 
> use XeLaTeX.

Well, that's probably not the reason, unless you've symlinked the
command `pdflatex` to `xelatex`. Pandoc's default latex template uses
the inputenc package with the [utf8] option. This ensures that LaTeX
accepts utf8 encoded input files, whereby non-ASCII-characters are
translated into LaTeX commands that produce them. So `æ` in the input
file will be translated to `\ae`, which will then be replaced with the
glyph `æ` in the pdf output.

One reason to still use XeLaTeX (or LuaLaTeX) is that inputenc does not
support all Unicode characters, though. And It's easier to use system
fonts.

-- 
Joost Kremers
Life has its moments

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/87twqmycla.fsf%40fastmail.fm.
For more options, visit https://groups.google.com/d/optout.

Re: ligatures in html

[Chris Wright]

[-- Attachment #1.1: Type: text/plain, Size: 1743 bytes --]

Thanks for the really good advice, folks

I've upgraded to a more modern workflow - XeTex at the moment

Cheers

Chris

On Tuesday, 22 September 2015 20:26:16 UTC+10, Joost wrote:
>
>
> On Tue, Sep 22 2015, david.p...-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org <javascript:> wrote: 
> > Indeed, for me, Pandoc's default tex template outputs the unicode 
> > characters correctly, but this could be because my environment is set up 
> to 
> > use XeLaTeX. 
>
> Well, that's probably not the reason, unless you've symlinked the 
> command `pdflatex` to `xelatex`. Pandoc's default latex template uses 
> the inputenc package with the [utf8] option. This ensures that LaTeX 
> accepts utf8 encoded input files, whereby non-ASCII-characters are 
> translated into LaTeX commands that produce them. So `æ` in the input 
> file will be translated to `\ae`, which will then be replaced with the 
> glyph `æ` in the pdf output. 
>
> One reason to still use XeLaTeX (or LuaLaTeX) is that inputenc does not 
> support all Unicode characters, though. And It's easier to use system 
> fonts. 
>
> -- 
> Joost Kremers 
> Life has its moments 
>

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/1ce2e029-01fa-405d-94a6-468a12142e47%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[-- Attachment #1.2: Type: text/html, Size: 2596 bytes --]