controlling smart typography with org or markdown

controlling smart typography with org or markdown

[Simon Michael]

G'day all!

As someone documenting command line software, I never want `smart` 
typography. Still, Pandoc lets me control it, right ? In the end, yes.. 
but I must share some notes. Perhaps I'm getting confused between 
readers and writers ? Any comments welcome.


1. A small correction
---------------------

 > https://pandoc.org/MANUAL.html#extensions: "For example, --from 
markdown_strict+footnotes is..."

The underscore is a typo I think.


2. With the org reader, smart can not be disabled
-------------------------------------------------

 > https://pandoc.org/MANUAL.html#extension-smart: "Interpret straight 
quotes as curly quotes, --- as em-dashes, -- as en-dashes, and ... as 
ellipses. "

This suggests smart is disabled by default for org (reader, or so I think):

$ pandoc --version
pandoc 2.14.2
Compiled with pandoc-types 1.22, texmath 0.12.3.1, skylighting 0.11,
citeproc 0.5, ipynb 0.1.0.1
...
$ pandoc --list-extensions=org
-ascii_identifiers
+auto_identifiers
+citations
-east_asian_line_breaks
-gfm_auto_identifiers
-smart

But this shows it enabled by default:

$ echo '--version' | pandoc -f org -t native
[Para [Str "\8211version"]]

And enabling/disabling the extension has no effect:

$ echo '--version' | pandoc -f org-smart -t native
[Para [Str "\8211version"]]
$ echo '--version' | pandoc -f org+smart -t native
[Para [Str "\8211version"]]


3. With the markdown writer, smart is selected oppositely
---------------------------------------------------------

Since I am converting to markdown, maybe I could control it there. I was 
already disabling smart in the markdown writer I thought, but it was not 
working:

$ echo '--version' | pandoc -f org -t markdown-smart
–version

Then I found this:

 > "Note: If you are writing Markdown, then the smart extension has the 
reverse effect: what would have been curly quotes comes out straight."

Which indeed achieves my goal:

$ echo '--version' | pandoc -f org -t markdown+smart
--version

But.. why the reverse effect ? There must be a reason, but I found this 
non-intuitive.


4. More
---------------------------------------------------------

So it seems I have been *enabling* smart in my markdown web docs (with 
-t markdown-smart) for years. But I haven't been seeing smart 
quotes/dashes; apparently they were still being suppressed by other 
means. I investigated:

With the markdown reader, smart is enabled by default, and disabled with 
-smart as one would expect (unlike the markdown writer):

$ echo  '--version' | pandoc -f markdown -t native
[Para [Str "\8211version"]]
$ echo  '--version' | pandoc -f markdown-smart -t native
[Para [Str "--version"]]

Here are the combinations of the markdown reader and markdown writer, as 
I understand them.

A. from smart markdown to non-smart markdown:

$ echo  '--version' | pandoc -f markdown -t markdown
--version

B. from smart markdown to smart markdown:

$ echo  '--version' | pandoc -f markdown -t markdown-smart
–version

C. from non-smart markdown to non-smart markdown (why the backslash ?):

$ echo  '--version' | pandoc -f markdown-smart -t markdown
\--version

D. from non-smart markdown to smart markdown (why no en-dash here ?)

$ echo  '--version' | pandoc -f markdown-smart -t markdown-smart
--version


And now I must go for a lie down.

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/siqibs%242h1%241%40ciao.gmane.io.

Re: controlling smart typography with org or markdown

[Albert Krewinkel]

I remember our [previous discussion] where I promised to think about a
solution. Unfortunately, I still don't have any, ideas welcome.

[previous discussion]: https://github.com/jgm/pandoc/issues/4788

There is a section on `smart` and the org extension on the website.
Here, too, I'd welcome suggestions for improvements.
https://pandoc.org/org.html#smart-extension

Simon Michael <simon-jbxitMBJ2LLQT0dZR+AlfA@public.gmane.org> writes:

> G'day all!
>
> As someone documenting command line software, I never want `smart` typography.
> Still, Pandoc lets me control it, right ? In the end, yes.. but I must share
> some notes. Perhaps I'm getting confused between readers and writers ? Any
> comments welcome.
>
>
> 1. A small correction
> ---------------------
>
>> https://pandoc.org/MANUAL.html#extensions: "For example, --from
> markdown_strict+footnotes is..."
>
> The underscore is a typo I think.
>
>
> 2. With the org reader, smart can not be disabled
> -------------------------------------------------
>
>> https://pandoc.org/MANUAL.html#extension-smart: "Interpret straight
> quotes as curly quotes, --- as em-dashes, -- as en-dashes, and ... as ellipses.
> "
>
> This suggests smart is disabled by default for org (reader, or so I think):
>
> $ pandoc --version
> pandoc 2.14.2
> Compiled with pandoc-types 1.22, texmath 0.12.3.1, skylighting 0.11,
> citeproc 0.5, ipynb 0.1.0.1
> ...
> $ pandoc --list-extensions=org
> -ascii_identifiers
> +auto_identifiers
> +citations
> -east_asian_line_breaks
> -gfm_auto_identifiers
> -smart
>
> But this shows it enabled by default:
>
> $ echo '--version' | pandoc -f org -t native
> [Para [Str "\8211version"]]
>
> And enabling/disabling the extension has no effect:
>
> $ echo '--version' | pandoc -f org-smart -t native
> [Para [Str "\8211version"]]
> $ echo '--version' | pandoc -f org+smart -t native
> [Para [Str "\8211version"]]
>
>
> 3. With the markdown writer, smart is selected oppositely
> ---------------------------------------------------------
>
> Since I am converting to markdown, maybe I could control it there. I was already
> disabling smart in the markdown writer I thought, but it was not working:
>
> $ echo '--version' | pandoc -f org -t markdown-smart
> –version
>
> Then I found this:
>
>> "Note: If you are writing Markdown, then the smart extension has the
> reverse effect: what would have been curly quotes comes out straight."
>
> Which indeed achieves my goal:
>
> $ echo '--version' | pandoc -f org -t markdown+smart
> --version
>
> But.. why the reverse effect ? There must be a reason, but I found this
> non-intuitive.
>
>
> 4. More
> ---------------------------------------------------------
>
> So it seems I have been *enabling* smart in my markdown web docs (with -t
> markdown-smart) for years. But I haven't been seeing smart quotes/dashes;
> apparently they were still being suppressed by other means. I investigated:
>
> With the markdown reader, smart is enabled by default, and disabled with -smart
> as one would expect (unlike the markdown writer):
>
> $ echo  '--version' | pandoc -f markdown -t native
> [Para [Str "\8211version"]]
> $ echo  '--version' | pandoc -f markdown-smart -t native
> [Para [Str "--version"]]
>
> Here are the combinations of the markdown reader and markdown writer, as I
> understand them.
>
> A. from smart markdown to non-smart markdown:
>
> $ echo  '--version' | pandoc -f markdown -t markdown
> --version
>
> B. from smart markdown to smart markdown:
>
> $ echo  '--version' | pandoc -f markdown -t markdown-smart
> –version
>
> C. from non-smart markdown to non-smart markdown (why the backslash ?):
>
> $ echo  '--version' | pandoc -f markdown-smart -t markdown
> \--version
>
> D. from non-smart markdown to smart markdown (why no en-dash here ?)
>
> $ echo  '--version' | pandoc -f markdown-smart -t markdown-smart
> --version
>
>
> And now I must go for a lie down.


--
Albert Krewinkel
GPG: 8eed e3e2 e8c5 6f18 81fe  e836 388d c0b2 1f63 1124

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/87ee9b6pme.fsf%40zeitkraut.de.

Re: controlling smart typography with org or markdown

[John MacFarlane]

[-- Attachment #1.1: Type: text/plain, Size: 5488 bytes --]

I'm not sure what's going on with org (2), but I can illuminate the other 
mysteries.

1)  It's not a typo.  markdown_strict is the name of the format (really 
shorthand for markdown with certain extension settings).

3) Why the reverse effect with `-t markdown-smart`?

Think of it this way.  You're writing markdown for consumption by a 
markdown processor which does not have "smart" support.  How, then, should 
we render Str "\8211"?  Not as "--", because the target processor doesn't 
support smart quotes; it will interpret "--" as two dashes.  Instead, as a 
unicode en-dash.

Another way to think of it is this:  'pandoc -f markdown+smart -t 
markdown+smart' should be ideally be an identity, at least up to the 
semantics.  (There might be changes in indentation, bullet markers, and 
things like that, but we'd hope that the source and target would correspond 
to the same AST.)  So, if '-f markdown+smart' changes "--" to "\8211", then 
"-t markdown+smart" had better change "\8211" back to "--".

4) Hopefully it makes more sense now in light of the above.  I don't really 
understand how you're using "smart markdown" here.  The way we use it, 
"smart markdown" is a markdown dialect in which "--" gets parsed as Str 
"\8211", and Str "\8211" gets rendered as "--".  That is, a dialect in 
which the markdown string "--" *means* Str "\8211".



On Sunday, September 26, 2021 at 12:41:00 PM UTC-7 Simon Michael wrote:

> G'day all!
>
> As someone documenting command line software, I never want `smart` 
> typography. Still, Pandoc lets me control it, right ? In the end, yes.. 
> but I must share some notes. Perhaps I'm getting confused between 
> readers and writers ? Any comments welcome.
>
>
> 1. A small correction
> ---------------------
>
> > https://pandoc.org/MANUAL.html#extensions: "For example, --from 
> markdown_strict+footnotes is..."
>
> The underscore is a typo I think.
>
>
> 2. With the org reader, smart can not be disabled
> -------------------------------------------------
>
> > https://pandoc.org/MANUAL.html#extension-smart: "Interpret straight 
> quotes as curly quotes, --- as em-dashes, -- as en-dashes, and ... as 
> ellipses. "
>
> This suggests smart is disabled by default for org (reader, or so I think):
>
> $ pandoc --version
> pandoc 2.14.2
> Compiled with pandoc-types 1.22, texmath 0.12.3.1, skylighting 0.11,
> citeproc 0.5, ipynb 0.1.0.1
> ...
> $ pandoc --list-extensions=org
> -ascii_identifiers
> +auto_identifiers
> +citations
> -east_asian_line_breaks
> -gfm_auto_identifiers
> -smart
>
> But this shows it enabled by default:
>
> $ echo '--version' | pandoc -f org -t native
> [Para [Str "\8211version"]]
>
> And enabling/disabling the extension has no effect:
>
> $ echo '--version' | pandoc -f org-smart -t native
> [Para [Str "\8211version"]]
> $ echo '--version' | pandoc -f org+smart -t native
> [Para [Str "\8211version"]]
>
>
> 3. With the markdown writer, smart is selected oppositely
> ---------------------------------------------------------
>
> Since I am converting to markdown, maybe I could control it there. I was 
> already disabling smart in the markdown writer I thought, but it was not 
> working:
>
> $ echo '--version' | pandoc -f org -t markdown-smart
> –version
>
> Then I found this:
>
> > "Note: If you are writing Markdown, then the smart extension has the 
> reverse effect: what would have been curly quotes comes out straight."
>
> Which indeed achieves my goal:
>
> $ echo '--version' | pandoc -f org -t markdown+smart
> --version
>
> But.. why the reverse effect ? There must be a reason, but I found this 
> non-intuitive.
>
>
> 4. More
> ---------------------------------------------------------
>
> So it seems I have been *enabling* smart in my markdown web docs (with 
> -t markdown-smart) for years. But I haven't been seeing smart 
> quotes/dashes; apparently they were still being suppressed by other 
> means. I investigated:
>
> With the markdown reader, smart is enabled by default, and disabled with 
> -smart as one would expect (unlike the markdown writer):
>
> $ echo '--version' | pandoc -f markdown -t native
> [Para [Str "\8211version"]]
> $ echo '--version' | pandoc -f markdown-smart -t native
> [Para [Str "--version"]]
>
> Here are the combinations of the markdown reader and markdown writer, as 
> I understand them.
>
> A. from smart markdown to non-smart markdown:
>
> $ echo '--version' | pandoc -f markdown -t markdown
> --version
>
> B. from smart markdown to smart markdown:
>
> $ echo '--version' | pandoc -f markdown -t markdown-smart
> –version
>
> C. from non-smart markdown to non-smart markdown (why the backslash ?):
>
> $ echo '--version' | pandoc -f markdown-smart -t markdown
> \--version
>
> D. from non-smart markdown to smart markdown (why no en-dash here ?)
>
> $ echo '--version' | pandoc -f markdown-smart -t markdown-smart
> --version
>
>
> And now I must go for a lie down.
>
>

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/4331fc4f-e662-4ea0-89c0-c62b0c0228e1n%40googlegroups.com.

[-- Attachment #1.2: Type: text/html, Size: 7203 bytes --]

Re: controlling smart typography with org or markdown

[Ophir Lifshitz]

[-- Attachment #1.1: Type: text/plain, Size: 6731 bytes --]

I must admit, I remember often getting confused too about how to use 
`smart`, and about how `+smart` and `-smart` would often do the opposite of 
what I'd expect.* (That expectation was probably influenced by other 
software that uses something like a `--smart` flag/switch to make 
typography fancier when converting input to rendered output in a single 
direction, instead of an extension model like modern Pandoc.) But the way 
you just explained it has truly helped me sort it out, so here's a thank 
you from me. Perhaps that explanation is something the docs could use!

Ophir

* I think I learned a "reverse psychology" mnemonic/habit for it eventually 
– to do precisely the opposite of what I expect. Naturally that's a great 
way to confuse yourself even more, as once you get used to "no, this is the 
right way" (becoming "yes, this is the right way") you can then easily 
second-guess yourself.
On Monday, September 27, 2021 at 7:35:08 PM UTC-4 John MacFarlane wrote:

> I'm not sure what's going on with org (2), but I can illuminate the other 
> mysteries.
>
> 1)  It's not a typo.  markdown_strict is the name of the format (really 
> shorthand for markdown with certain extension settings).
>
> 3) Why the reverse effect with `-t markdown-smart`?
>
> Think of it this way.  You're writing markdown for consumption by a 
> markdown processor which does not have "smart" support.  How, then, should 
> we render Str "\8211"?  Not as "--", because the target processor doesn't 
> support smart quotes; it will interpret "--" as two dashes.  Instead, as a 
> unicode en-dash.
>
> Another way to think of it is this:  'pandoc -f markdown+smart -t 
> markdown+smart' should be ideally be an identity, at least up to the 
> semantics.  (There might be changes in indentation, bullet markers, and 
> things like that, but we'd hope that the source and target would correspond 
> to the same AST.)  So, if '-f markdown+smart' changes "--" to "\8211", then 
> "-t markdown+smart" had better change "\8211" back to "--".
>
> 4) Hopefully it makes more sense now in light of the above.  I don't 
> really understand how you're using "smart markdown" here.  The way we use 
> it, "smart markdown" is a markdown dialect in which "--" gets parsed as Str 
> "\8211", and Str "\8211" gets rendered as "--".  That is, a dialect in 
> which the markdown string "--" *means* Str "\8211".
>
>
>
> On Sunday, September 26, 2021 at 12:41:00 PM UTC-7 Simon Michael wrote:
>
>> G'day all! 
>>
>> As someone documenting command line software, I never want `smart` 
>> typography. Still, Pandoc lets me control it, right ? In the end, yes.. 
>> but I must share some notes. Perhaps I'm getting confused between 
>> readers and writers ? Any comments welcome. 
>>
>>
>> 1. A small correction 
>> --------------------- 
>>
>> > https://pandoc.org/MANUAL.html#extensions: "For example, --from 
>> markdown_strict+footnotes is..." 
>>
>> The underscore is a typo I think. 
>>
>>
>> 2. With the org reader, smart can not be disabled 
>> ------------------------------------------------- 
>>
>> > https://pandoc.org/MANUAL.html#extension-smart: "Interpret straight 
>> quotes as curly quotes, --- as em-dashes, -- as en-dashes, and ... as 
>> ellipses. " 
>>
>> This suggests smart is disabled by default for org (reader, or so I 
>> think): 
>>
>> $ pandoc --version 
>> pandoc 2.14.2 
>> Compiled with pandoc-types 1.22, texmath 0.12.3.1, skylighting 0.11, 
>> citeproc 0.5, ipynb 0.1.0.1 
>> ... 
>> $ pandoc --list-extensions=org 
>> -ascii_identifiers 
>> +auto_identifiers 
>> +citations 
>> -east_asian_line_breaks 
>> -gfm_auto_identifiers 
>> -smart 
>>
>> But this shows it enabled by default: 
>>
>> $ echo '--version' | pandoc -f org -t native 
>> [Para [Str "\8211version"]] 
>>
>> And enabling/disabling the extension has no effect: 
>>
>> $ echo '--version' | pandoc -f org-smart -t native 
>> [Para [Str "\8211version"]] 
>> $ echo '--version' | pandoc -f org+smart -t native 
>> [Para [Str "\8211version"]] 
>>
>>
>> 3. With the markdown writer, smart is selected oppositely 
>> --------------------------------------------------------- 
>>
>> Since I am converting to markdown, maybe I could control it there. I was 
>> already disabling smart in the markdown writer I thought, but it was not 
>> working: 
>>
>> $ echo '--version' | pandoc -f org -t markdown-smart 
>> –version 
>>
>> Then I found this: 
>>
>> > "Note: If you are writing Markdown, then the smart extension has the 
>> reverse effect: what would have been curly quotes comes out straight." 
>>
>> Which indeed achieves my goal: 
>>
>> $ echo '--version' | pandoc -f org -t markdown+smart 
>> --version 
>>
>> But.. why the reverse effect ? There must be a reason, but I found this 
>> non-intuitive. 
>>
>>
>> 4. More 
>> --------------------------------------------------------- 
>>
>> So it seems I have been *enabling* smart in my markdown web docs (with 
>> -t markdown-smart) for years. But I haven't been seeing smart 
>> quotes/dashes; apparently they were still being suppressed by other 
>> means. I investigated: 
>>
>> With the markdown reader, smart is enabled by default, and disabled with 
>> -smart as one would expect (unlike the markdown writer): 
>>
>> $ echo '--version' | pandoc -f markdown -t native 
>> [Para [Str "\8211version"]] 
>> $ echo '--version' | pandoc -f markdown-smart -t native 
>> [Para [Str "--version"]] 
>>
>> Here are the combinations of the markdown reader and markdown writer, as 
>> I understand them. 
>>
>> A. from smart markdown to non-smart markdown: 
>>
>> $ echo '--version' | pandoc -f markdown -t markdown 
>> --version 
>>
>> B. from smart markdown to smart markdown: 
>>
>> $ echo '--version' | pandoc -f markdown -t markdown-smart 
>> –version 
>>
>> C. from non-smart markdown to non-smart markdown (why the backslash ?): 
>>
>> $ echo '--version' | pandoc -f markdown-smart -t markdown 
>> \--version 
>>
>> D. from non-smart markdown to smart markdown (why no en-dash here ?) 
>>
>> $ echo '--version' | pandoc -f markdown-smart -t markdown-smart 
>> --version 
>>
>>
>> And now I must go for a lie down. 
>>
>>

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/95eec37d-58d5-4b05-be9f-a387ac9471f0n%40googlegroups.com.

[-- Attachment #1.2: Type: text/html, Size: 8593 bytes --]