From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.io/gmane.text.pandoc/29307 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Ophir Lifshitz Newsgroups: gmane.text.pandoc Subject: Re: controlling smart typography with org or markdown Date: Mon, 27 Sep 2021 17:04:01 -0700 (PDT) Message-ID: <95eec37d-58d5-4b05-be9f-a387ac9471f0n@googlegroups.com> References: <4331fc4f-e662-4ea0-89c0-c62b0c0228e1n@googlegroups.com> Reply-To: pandoc-discuss-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_Part_158_1640022082.1632787441713" Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="28053"; mail-complaints-to="usenet@ciao.gmane.io" To: pandoc-discuss Original-X-From: pandoc-discuss+bncBDM4PRVZZYLBB4VXZGFAMGQEQAIA4ZA-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org Tue Sep 28 02:04:05 2021 Return-path: Envelope-to: gtp-pandoc-discuss@m.gmane-mx.org Original-Received: from mail-oo1-f64.google.com ([209.85.161.64]) by ciao.gmane.io with esmtps (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1mV0bN-00072k-3W for gtp-pandoc-discuss@m.gmane-mx.org; Tue, 28 Sep 2021 02:04:05 +0200 Original-Received: by mail-oo1-f64.google.com with SMTP id v11-20020a4a314b000000b002a9d9728566sf21109226oog.1 for ; Mon, 27 Sep 2021 17:04:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20210112; h=sender:date:from:to:message-id:in-reply-to:references:subject :mime-version:x-original-sender:reply-to:precedence:mailing-list :list-id:list-post:list-help:list-archive:list-subscribe :list-unsubscribe; bh=xw5FuqdW0YVmkjIAudDiHz3piiwO4JOPAlE6GUw77PI=; b=SKWFebgwUEGPuEQV5kd6dkme+r9lw1MGmCyuPOumWvZSDfECAXILvmy3Mil44wt9Ak 5hwciOdTz6vWOn5lbUQ88PbfzbJ5KrLVrH5FXz3WRXsxP+awnvhzGtnNM2IN4M1h9Qy2 OS82DxoJAGVm53epkg2141fCpXWARlJ/xrdLPtNdnXt5kURuKhGaFWV68tMuYoRcCypw 5zzhra2NCR/mA1BP+Kc6o5BUmnvU4UztxNqszO0AUzjJsEx6+XL1t+gKgolmD0/eXyoz 4BoOPiqkcjCjc4o6ACBvjKCLVqvhAZriHGRgtkGtscpda91I+gtYaQhmlxsKbe8mibmi aZtA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=date:from:to:message-id:in-reply-to:references:subject:mime-version :x-original-sender:reply-to:precedence:mailing-list:list-id :list-post:list-help:list-archive:list-subscribe:list-unsubscribe; bh=xw5FuqdW0YVmkjIAudDiHz3piiwO4JOPAlE6GUw77PI=; b=ObVkPEGyFBgCvwSlp4CipkWAl4o75MLQSAvyVm/eZ7N8B9JrS4SDE6VVTaVMQ4qavN P65mE7Q9PJSGeAn4DLmligE29xrNK/7G4okSL38bP9Jjt7PXkkoqe0Sn5uaLZ+pSrHuA 61OAblDFmrTuREG17Yixit2DojyfWoamkdlZS+w7SJwzUBib9nWMYd6w/X4/sYZySABG fuQUfPdqtcmZXBdGarcVVfX5dTkKGuNyqm6fQau9GK6RsgpHY5YnnbF0daMyRloPGomu /XhXdxFCEXNXsype3UvGEmcyLA9tvL9MzgABpUlBHCOo41WyEnIveB3ySJ3e4wSy4Lnq 3Diw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=sender:x-gm-message-state:date:from:to:message-id:in-reply-to :references:subject:mime-version:x-original-sender:reply-to :precedence:mailing-list:list-id:x-spam-checked-in-group:list-post :list-help:list-archive:list-subscribe:list-unsubscribe; bh=xw5FuqdW0YVmkjIAudDiHz3piiwO4JOPAlE6GUw77PI=; b=JVciM6Os8SwxdJwbboAjIPl9xsu6dcJTXaH7wNsBw9q3dlhGSTnzv53/qu1n1I/lpE gBWn6JVbW/aI83nCWjrrq4rj1ij3VfnzcSNUDBMbEcnf4naWbrXi17RyPuwNUaKeCFKJ zWV5ry+A8rgcp/8eH6Kbb9maJ+rFwbvVDWA1s4/YA8yAsKLlBKF2WixnGADN+FODhKTW i+7zgjdWp+VN7TnBVH6QaKtpDc5lpNoc0dGrxTsel4YNWQ7iYlqFfiIFSeobaxJokFd5 e6IMHTqhezXs4G9Sd/C2NvltjG1AbrsenWnW5tFpwqAJEa1PWmlHpc8zjWYnoZeWxETS nVpg== Original-Sender: pandoc-discuss-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org X-Gm-Message-State: AOAM53387if2qGWBtuEJ7fXM/C1B/WWlu5HhFFnmW9KZFvvC+YuUbEPS UE91OJe2NHQnRR4O/4Z4bhs= X-Google-Smtp-Source: ABdhPJyM0ZVVox+tkle0WRDBhSl6RmM/nPK3naM/HqxLeRejQbfMmtuHsWtWPLndRGx0PvrSRT2BuA== X-Received: by 2002:a4a:2104:: with SMTP id u4mr2305581oou.60.1632787443980; Mon, 27 Sep 2021 17:04:03 -0700 (PDT) X-BeenThere: pandoc-discuss-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org Original-Received: by 2002:a05:6830:2a0c:: with SMTP id y12ls4846218otu.6.gmail; Mon, 27 Sep 2021 17:04:02 -0700 (PDT) X-Received: by 2002:a9d:6483:: with SMTP id g3mr2453653otl.105.1632787442361; Mon, 27 Sep 2021 17:04:02 -0700 (PDT) In-Reply-To: <4331fc4f-e662-4ea0-89c0-c62b0c0228e1n-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org> X-Original-Sender: hangfromthefloor-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org Precedence: list Mailing-list: list pandoc-discuss-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org; contact pandoc-discuss+owners-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org List-ID: X-Google-Group-Id: 1007024079513 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , Xref: news.gmane.io gmane.text.pandoc:29307 Archived-At: ------=_Part_158_1640022082.1632787441713 Content-Type: multipart/alternative; boundary="----=_Part_159_2031154843.1632787441713" ------=_Part_159_2031154843.1632787441713 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable I must admit, I remember often getting confused too about how to use=20 `smart`, and about how `+smart` and `-smart` would often do the opposite of= =20 what I'd expect.* (That expectation was probably influenced by other=20 software that uses something like a `--smart` flag/switch to make=20 typography fancier when converting input to rendered output in a single=20 direction, instead of an extension model like modern Pandoc.) But the way= =20 you just explained it has truly helped me sort it out, so here's a thank=20 you from me. Perhaps that explanation is something the docs could use! Ophir * I think I learned a "reverse psychology" mnemonic/habit for it eventually= =20 =E2=80=93 to do precisely the opposite of what I expect. Naturally that's a= great=20 way to confuse yourself even more, as once you get used to "no, this is the= =20 right way" (becoming "yes, this is the right way") you can then easily=20 second-guess yourself. On Monday, September 27, 2021 at 7:35:08 PM UTC-4 John MacFarlane wrote: > I'm not sure what's going on with org (2), but I can illuminate the other= =20 > mysteries. > > 1) It's not a typo. markdown_strict is the name of the format (really= =20 > shorthand for markdown with certain extension settings). > > 3) Why the reverse effect with `-t markdown-smart`? > > Think of it this way. You're writing markdown for consumption by a=20 > markdown processor which does not have "smart" support. How, then, shoul= d=20 > we render Str "\8211"? Not as "--", because the target processor doesn't= =20 > support smart quotes; it will interpret "--" as two dashes. Instead, as = a=20 > unicode en-dash. > > Another way to think of it is this: 'pandoc -f markdown+smart -t=20 > markdown+smart' should be ideally be an identity, at least up to the=20 > semantics. (There might be changes in indentation, bullet markers, and= =20 > things like that, but we'd hope that the source and target would correspo= nd=20 > to the same AST.) So, if '-f markdown+smart' changes "--" to "\8211", th= en=20 > "-t markdown+smart" had better change "\8211" back to "--". > > 4) Hopefully it makes more sense now in light of the above. I don't=20 > really understand how you're using "smart markdown" here. The way we use= =20 > it, "smart markdown" is a markdown dialect in which "--" gets parsed as S= tr=20 > "\8211", and Str "\8211" gets rendered as "--". That is, a dialect in=20 > which the markdown string "--" *means* Str "\8211". > > > > On Sunday, September 26, 2021 at 12:41:00 PM UTC-7 Simon Michael wrote: > >> G'day all!=20 >> >> As someone documenting command line software, I never want `smart`=20 >> typography. Still, Pandoc lets me control it, right ? In the end, yes..= =20 >> but I must share some notes. Perhaps I'm getting confused between=20 >> readers and writers ? Any comments welcome.=20 >> >> >> 1. A small correction=20 >> ---------------------=20 >> >> > https://pandoc.org/MANUAL.html#extensions: "For example, --from=20 >> markdown_strict+footnotes is..."=20 >> >> The underscore is a typo I think.=20 >> >> >> 2. With the org reader, smart can not be disabled=20 >> -------------------------------------------------=20 >> >> > https://pandoc.org/MANUAL.html#extension-smart: "Interpret straight=20 >> quotes as curly quotes, --- as em-dashes, -- as en-dashes, and ... as=20 >> ellipses. "=20 >> >> This suggests smart is disabled by default for org (reader, or so I=20 >> think):=20 >> >> $ pandoc --version=20 >> pandoc 2.14.2=20 >> Compiled with pandoc-types 1.22, texmath 0.12.3.1, skylighting 0.11,=20 >> citeproc 0.5, ipynb 0.1.0.1=20 >> ...=20 >> $ pandoc --list-extensions=3Dorg=20 >> -ascii_identifiers=20 >> +auto_identifiers=20 >> +citations=20 >> -east_asian_line_breaks=20 >> -gfm_auto_identifiers=20 >> -smart=20 >> >> But this shows it enabled by default:=20 >> >> $ echo '--version' | pandoc -f org -t native=20 >> [Para [Str "\8211version"]]=20 >> >> And enabling/disabling the extension has no effect:=20 >> >> $ echo '--version' | pandoc -f org-smart -t native=20 >> [Para [Str "\8211version"]]=20 >> $ echo '--version' | pandoc -f org+smart -t native=20 >> [Para [Str "\8211version"]]=20 >> >> >> 3. With the markdown writer, smart is selected oppositely=20 >> ---------------------------------------------------------=20 >> >> Since I am converting to markdown, maybe I could control it there. I was= =20 >> already disabling smart in the markdown writer I thought, but it was not= =20 >> working:=20 >> >> $ echo '--version' | pandoc -f org -t markdown-smart=20 >> =E2=80=93version=20 >> >> Then I found this:=20 >> >> > "Note: If you are writing Markdown, then the smart extension has the= =20 >> reverse effect: what would have been curly quotes comes out straight."= =20 >> >> Which indeed achieves my goal:=20 >> >> $ echo '--version' | pandoc -f org -t markdown+smart=20 >> --version=20 >> >> But.. why the reverse effect ? There must be a reason, but I found this= =20 >> non-intuitive.=20 >> >> >> 4. More=20 >> ---------------------------------------------------------=20 >> >> So it seems I have been *enabling* smart in my markdown web docs (with= =20 >> -t markdown-smart) for years. But I haven't been seeing smart=20 >> quotes/dashes; apparently they were still being suppressed by other=20 >> means. I investigated:=20 >> >> With the markdown reader, smart is enabled by default, and disabled with= =20 >> -smart as one would expect (unlike the markdown writer):=20 >> >> $ echo '--version' | pandoc -f markdown -t native=20 >> [Para [Str "\8211version"]]=20 >> $ echo '--version' | pandoc -f markdown-smart -t native=20 >> [Para [Str "--version"]]=20 >> >> Here are the combinations of the markdown reader and markdown writer, as= =20 >> I understand them.=20 >> >> A. from smart markdown to non-smart markdown:=20 >> >> $ echo '--version' | pandoc -f markdown -t markdown=20 >> --version=20 >> >> B. from smart markdown to smart markdown:=20 >> >> $ echo '--version' | pandoc -f markdown -t markdown-smart=20 >> =E2=80=93version=20 >> >> C. from non-smart markdown to non-smart markdown (why the backslash ?):= =20 >> >> $ echo '--version' | pandoc -f markdown-smart -t markdown=20 >> \--version=20 >> >> D. from non-smart markdown to smart markdown (why no en-dash here ?)=20 >> >> $ echo '--version' | pandoc -f markdown-smart -t markdown-smart=20 >> --version=20 >> >> >> And now I must go for a lie down.=20 >> >> --=20 You received this message because you are subscribed to the Google Groups "= pandoc-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an e= mail to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To view this discussion on the web visit https://groups.google.com/d/msgid/= pandoc-discuss/95eec37d-58d5-4b05-be9f-a387ac9471f0n%40googlegroups.com. ------=_Part_159_2031154843.1632787441713 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable I must admit, I remember often getting confused too about how to use `smart= `, and about how `+smart` and `-smart` would often do the opposite of what = I'd expect.* (That expectation was probably influenced by other software th= at uses something like a `--smart` flag/switch to make typography fancier w= hen converting input to rendered output in a single direction, instead of a= n extension model like modern Pandoc.) But the way you just explained it ha= s truly helped me sort it out, so here's a thank you from me. Perhaps that = explanation is something the docs could use!

Ophir
* I think I learned a "reverse psychology" mnemonic/habit for = it eventually =E2=80=93 to do precisely the opposite of what I expect. Natu= rally that's a great way to confuse yourself even more, as once you get use= d to "no, this is the right way" (becoming "yes, this is the right way") yo= u can then easily second-guess yourself.
On Monday, September 27, 2021 at 7= :35:08 PM UTC-4 John MacFarlane wrote:
I'm not sure what's going on with or= g (2), but I can illuminate the other mysteries.

1= )=C2=A0 It's not a typo.=C2=A0 markdown_strict is the name of the forma= t (really shorthand for markdown with certain extension settings).

3) Why the reverse effect with `-t markdown-smart`?
<= div>
Think of it this way.=C2=A0 You're writing markdown = for consumption by a markdown processor which does not have "smart&quo= t; support.=C2=A0 How, then, should we render Str "\8211"?=C2=A0 = Not as "--", because the target processor doesn't support sma= rt quotes; it will interpret "--" as two dashes.=C2=A0 Instead, a= s a unicode en-dash.

Another way to think of it is= this:=C2=A0 'pandoc -f markdown+smart -t markdown+smart' should be= ideally be an identity, at least up to the semantics.=C2=A0 (There might b= e changes in indentation, bullet markers, and things like that, but we'= d hope that the source and target would correspond to the same AST.)=C2=A0 = So, if '-f markdown+smart' changes "--" to "\8211&qu= ot;, then "-t markdown+smart" had better change "\8211"= back to "--".

4) Hopefully it makes mor= e sense now in light of the above.=C2=A0 I don't really understand how = you're using "smart markdown" here.=C2=A0 The way we use it, = "smart markdown" is a markdown dialect in which "--" ge= ts parsed as Str "\8211", and Str "\8211" gets rendered= as "--".=C2=A0 That is, a dialect in which the markdown string &= quot;--" *means* Str "\8211".


<= /div>

On Sunday, September 26, 2021 at 12:41:00 PM UTC-7 Simon Michael= wrote:
G'day all!

As someone documenting command line software, I never want `smart`=20
typography. Still, Pandoc lets me control it, right ? In the end, yes..= =20
but I must share some notes. Perhaps I'm getting confused between= =20
readers and writers ? Any comments welcome.


1. A small correction
---------------------

> https://pandoc.org/MANUAL.html#extensions: "For example, --from= =20
markdown_strict+footnotes is..."

The underscore is a typo I think.


2. With the org reader, smart can not be disabled
-------------------------------------------------

> https://pandoc.org/MANUAL.html#extension-smart: "Inter= pret straight=20
quotes as curly quotes, --- as em-dashes, -- as en-dashes, and ... as= =20
ellipses. "

This suggests smart is disabled by default for org (reader, or so I thi= nk):

$ pandoc --version
pandoc 2.14.2
Compiled with pandoc-types 1.22, texmath 0.12.3.1, skylighting 0.11,
citeproc 0.5, ipynb 0.1.0.1
...
$ pandoc --list-extensions=3Dorg
-ascii_identifiers
+auto_identifiers
+citations
-east_asian_line_breaks
-gfm_auto_identifiers
-smart

But this shows it enabled by default:

$ echo '--version' | pandoc -f org -t native
[Para [Str "\8211version"]]

And enabling/disabling the extension has no effect:

$ echo '--version' | pandoc -f org-smart -t native
[Para [Str "\8211version"]]
$ echo '--version' | pandoc -f org+smart -t native
[Para [Str "\8211version"]]


3. With the markdown writer, smart is selected oppositely
---------------------------------------------------------

Since I am converting to markdown, maybe I could control it there. I wa= s=20
already disabling smart in the markdown writer I thought, but it was no= t=20
working:

$ echo '--version' | pandoc -f org -t markdown-smart
=E2=80=93version

Then I found this:

> "Note: If you are writing Markdown, then the smart extension= has the=20
reverse effect: what would have been curly quotes comes out straight.&q= uot;

Which indeed achieves my goal:

$ echo '--version' | pandoc -f org -t markdown+smart
--version

But.. why the reverse effect ? There must be a reason, but I found this= =20
non-intuitive.


4. More
---------------------------------------------------------

So it seems I have been *enabling* smart in my markdown web docs (with= =20
-t markdown-smart) for years. But I haven't been seeing smart=20
quotes/dashes; apparently they were still being suppressed by other=20
means. I investigated:

With the markdown reader, smart is enabled by default, and disabled wit= h=20
-smart as one would expect (unlike the markdown writer):

$ echo '--version' | pandoc -f markdown -t native
[Para [Str "\8211version"]]
$ echo '--version' | pandoc -f markdown-smart -t native
[Para [Str "--version"]]

Here are the combinations of the markdown reader and markdown writer, a= s=20
I understand them.

A. from smart markdown to non-smart markdown:

$ echo '--version' | pandoc -f markdown -t markdown
--version

B. from smart markdown to smart markdown:

$ echo '--version' | pandoc -f markdown -t markdown-smart
=E2=80=93version

C. from non-smart markdown to non-smart markdown (why the backslash ?):

$ echo '--version' | pandoc -f markdown-smart -t markdown
\--version

D. from non-smart markdown to smart markdown (why no en-dash here ?)

$ echo '--version' | pandoc -f markdown-smart -t markdown-smar= t
--version


And now I must go for a lie down.

--
You received this message because you are subscribed to the Google Groups &= quot;pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an e= mail to pand= oc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org.
To view this discussion on the web visit https://groups.google.com/d= /msgid/pandoc-discuss/95eec37d-58d5-4b05-be9f-a387ac9471f0n%40googlegroups.= com.
------=_Part_159_2031154843.1632787441713-- ------=_Part_158_1640022082.1632787441713--