Unsupported extension in Commonmark_x resulting in alternative output

Unsupported extension in Commonmark_x resulting in alternative output

[Thomas Byskov Dalgaard]

Hi!

Think I have found something that is unexpected behaviour when using the commonmark_x format with extensions.

Here is the case.

I want to convert from Markdown to Commonmark_x. I have read the Commonmark spec, and from here hard line breaks can be inserted either by two spaces or a backslash at the end of a line inside a paragraph.

This is fine when I write my Markdown document, but when I convert all backslashes are removed and replaced with two spaces.

I have tried to enable the extension escaped_line_breaks but Pandoc says that this extension is not supported.

Why is it that Pandoc allows backslashes to be used but not allowed when converting?

Best regards Thomas 

Re: Unsupported extension in Commonmark_x resulting in alternative output

[John MacFarlane]

This is a bit tricky.  escaped_line_breaks isn't really an
extension for commonmark, because it is part of the core spec.
The writer uses two spaces instead; both are supported according
to the spec, and the writer has to make a choice.

Thomas Byskov Dalgaard <tbdlist-AHfMSizzl90sTMJHDOTzqX9LOBIZ5rWg@public.gmane.org> writes:

> Hi!
>
> Think I have found something that is unexpected behaviour when using the commonmark_x format with extensions.
>
> Here is the case.
>
> I want to convert from Markdown to Commonmark_x. I have read the Commonmark spec, and from here hard line breaks can be inserted either by two spaces or a backslash at the end of a line inside a paragraph.
>
> This is fine when I write my Markdown document, but when I convert all backslashes are removed and replaced with two spaces.
>
> I have tried to enable the extension escaped_line_breaks but Pandoc says that this extension is not supported.
>
> Why is it that Pandoc allows backslashes to be used but not allowed when converting?
>
> Best regards Thomas 
>
> -- 
> You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
> To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/7F040099-8F18-41C2-BB05-0A4658311704%40coolfortheblind.dk.

Line breaks (Was: Re: Unsupported extension in Commonmark_x resulting in alternative output)

[Daniel Staal]

On 6/23/22 2:21 AM, John MacFarlane wrote:
> This is a bit tricky.  escaped_line_breaks isn't really an
> extension for commonmark, because it is part of the core spec.
> The writer uses two spaces instead; both are supported according
> to the spec, and the writer has to make a choice.

As an aside, that particular choice always is a bit touchy to me - while 
it *looks* nice, many text editors (including BBEdit, my preferred) will 
automatically strip whitespace at the end of a line.  Many users won't 
realize it's there in the first place after all, and extra whitespace 
can make handling text documents awkward using automated tools.  (And, 
in case it ever matters, it can bloat filesizes.)

So relying on whitespace at the end of a line for required behavior is 
uncertain at best, in my experience - and this is one thing where I 
often have to go back in and manually edit Markdown files after I've 
formatted them with Pandoc.

Daniel T. Staal

-- 
---------------------------------------------------------------
This email copyright the author.  Unless otherwise noted, you
are expressly allowed to retransmit, quote, or otherwise use
the contents for non-commercial purposes.  This copyright will
expire 5 years after the author's death, or in 30 years,
whichever is longer, unless such a period is in excess of
local copyright law.
---------------------------------------------------------------

Re: Line breaks (Was: Re: Unsupported extension in Commonmark_x resulting in alternative output)

[BPJ]

[-- Attachment #1: Type: text/plain, Size: 2392 bytes --]

+1

Den tors 23 juni 2022 15:20Daniel Staal <DStaal-Jdbf3xiKgS8@public.gmane.org> skrev:

> On 6/23/22 2:21 AM, John MacFarlane wrote:
> > This is a bit tricky.  escaped_line_breaks isn't really an
> > extension for commonmark, because it is part of the core spec.
> > The writer uses two spaces instead; both are supported according
> > to the spec, and the writer has to make a choice.
>
> As an aside, that particular choice always is a bit touchy to me - while
> it *looks* nice, many text editors (including BBEdit, my preferred) will
> automatically strip whitespace at the end of a line.  Many users won't
> realize it's there in the first place after all, and extra whitespace
> can make handling text documents awkward using automated tools.  (And,
> in case it ever matters, it can bloat filesizes.)
>
> So relying on whitespace at the end of a line for required behavior is
> uncertain at best, in my experience - and this is one thing where I
> often have to go back in and manually edit Markdown files after I've
> formatted them with Pandoc.
>
> Daniel T. Staal
>
> --
> ---------------------------------------------------------------
> This email copyright the author.  Unless otherwise noted, you
> are expressly allowed to retransmit, quote, or otherwise use
> the contents for non-commercial purposes.  This copyright will
> expire 5 years after the author's death, or in 30 years,
> whichever is longer, unless such a period is in excess of
> local copyright law.
> ---------------------------------------------------------------
>
> --
> You received this message because you are subscribed to the Google Groups
> "pandoc-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/pandoc-discuss/377aa06e-714c-ba4a-5445-dc9d518b95ab%40usa.net
> .
>

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/CADAJKhBAz2bzrfvAyLRBT7KBKJOmsdqw-0EnV4G3fkxzBoC2Fw%40mail.gmail.com.

[-- Attachment #2: Type: text/html, Size: 3363 bytes --]

Re: Line breaks (Was: Re: Unsupported extension in Commonmark_x resulting in alternative output)

[John MacFarlane]

I'll change the writer so it uses \ + newline for
commonmark + commonmark variants.

Daniel Staal <DStaal-Jdbf3xiKgS8@public.gmane.org> writes:

> On 6/23/22 2:21 AM, John MacFarlane wrote:
>> This is a bit tricky.  escaped_line_breaks isn't really an
>> extension for commonmark, because it is part of the core spec.
>> The writer uses two spaces instead; both are supported according
>> to the spec, and the writer has to make a choice.
>
> As an aside, that particular choice always is a bit touchy to me - while 
> it *looks* nice, many text editors (including BBEdit, my preferred) will 
> automatically strip whitespace at the end of a line.  Many users won't 
> realize it's there in the first place after all, and extra whitespace 
> can make handling text documents awkward using automated tools.  (And, 
> in case it ever matters, it can bloat filesizes.)
>
> So relying on whitespace at the end of a line for required behavior is 
> uncertain at best, in my experience - and this is one thing where I 
> often have to go back in and manually edit Markdown files after I've 
> formatted them with Pandoc.
>
> Daniel T. Staal
>
> -- 
> ---------------------------------------------------------------
> This email copyright the author.  Unless otherwise noted, you
> are expressly allowed to retransmit, quote, or otherwise use
> the contents for non-commercial purposes.  This copyright will
> expire 5 years after the author's death, or in 30 years,
> whichever is longer, unless such a period is in excess of
> local copyright law.
> ---------------------------------------------------------------
>
> -- 
> You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
> To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/377aa06e-714c-ba4a-5445-dc9d518b95ab%40usa.net.

Re: Line breaks (Was: Re: Unsupported extension in Commonmark_x resulting in alternative output)

[Thomas Byskov Dalgaard]

Hi John!

Thanks so much for this!
Can you also look into enable the extension lists_without_preceding_blankline
since the same issue is present for this extension as for the escaped_hard_line_breaks.


Best regards Thomas 

> Den 23. jun. 2022 kl. 22.17 skrev John MacFarlane <jgm-TVLZxgkOlNX2fBVCVOL8/A@public.gmane.org>:
> 
> 
> I'll change the writer so it uses \ + newline for
> commonmark + commonmark variants.
> 
> Daniel Staal <DStaal-Jdbf3xiKgS8@public.gmane.org> writes:
> 
>> On 6/23/22 2:21 AM, John MacFarlane wrote:
>>> This is a bit tricky.  escaped_line_breaks isn't really an
>>> extension for commonmark, because it is part of the core spec.
>>> The writer uses two spaces instead; both are supported according
>>> to the spec, and the writer has to make a choice.
>> 
>> As an aside, that particular choice always is a bit touchy to me - while 
>> it *looks* nice, many text editors (including BBEdit, my preferred) will 
>> automatically strip whitespace at the end of a line.  Many users won't 
>> realize it's there in the first place after all, and extra whitespace 
>> can make handling text documents awkward using automated tools.  (And, 
>> in case it ever matters, it can bloat filesizes.)
>> 
>> So relying on whitespace at the end of a line for required behavior is 
>> uncertain at best, in my experience - and this is one thing where I 
>> often have to go back in and manually edit Markdown files after I've 
>> formatted them with Pandoc.
>> 
>> Daniel T. Staal
>> 
>> -- 
>> ---------------------------------------------------------------
>> This email copyright the author.  Unless otherwise noted, you
>> are expressly allowed to retransmit, quote, or otherwise use
>> the contents for non-commercial purposes.  This copyright will
>> expire 5 years after the author's death, or in 30 years,
>> whichever is longer, unless such a period is in excess of
>> local copyright law.
>> ---------------------------------------------------------------
>> 
>> -- 
>> You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
>> To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
>> To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/377aa06e-714c-ba4a-5445-dc9d518b95ab%40usa.net.
> 
> -- 
> You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
> To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/m25ykrqgmk.fsf%40johnmacfarlane.net.

Re: Line breaks (Was: Re: Unsupported extension in Commonmark_x resulting in alternative output)

[Daniel Staal]

On 6/23/22 4:17 PM, John MacFarlane wrote:
> I'll change the writer so it uses \ + newline for
> commonmark + commonmark variants.

Thanks, that will be a lot more stable to work with I believe.

Daniel T. Staal

-- 
---------------------------------------------------------------
This email copyright the author.  Unless otherwise noted, you
are expressly allowed to retransmit, quote, or otherwise use
the contents for non-commercial purposes.  This copyright will
expire 5 years after the author's death, or in 30 years,
whichever is longer, unless such a period is in excess of
local copyright law.
---------------------------------------------------------------

Re: Unsupported extension in Commonmark_x resulting in alternative output

[Albert Krewinkel]

Obligatory "a Lua filter can help" post:

    function LineBreak ()
      return pandoc.RawInline('markdown', '\\\n')
    end

Save the above to a file and pass it to pandoc via the --lua-filter
command line parameter.

John MacFarlane <jgm-TVLZxgkOlNX2fBVCVOL8/A@public.gmane.org> writes:

> This is a bit tricky.  escaped_line_breaks isn't really an
> extension for commonmark, because it is part of the core spec.
> The writer uses two spaces instead; both are supported according
> to the spec, and the writer has to make a choice.
>
> Thomas Byskov Dalgaard <tbdlist-AHfMSizzl90sTMJHDOTzqX9LOBIZ5rWg@public.gmane.org> writes:
>
>> Hi!
>>
>> Think I have found something that is unexpected behaviour when using the commonmark_x format with extensions.
>>
>> Here is the case.
>>
>> I want to convert from Markdown to Commonmark_x. I have read the Commonmark
>> spec, and from here hard line breaks can be inserted either by two spaces or a
>> backslash at the end of a line inside a paragraph.
>>
>> This is fine when I write my Markdown document, but when I convert all
>> backslashes are removed and replaced with two spaces.
>>
>> I have tried to enable the extension escaped_line_breaks but Pandoc says that this extension is not supported.
>>
>> Why is it that Pandoc allows backslashes to be used but not allowed when converting?
>>
>> Best regards Thomas 


-- 
Albert Krewinkel
GPG: 8eed e3e2 e8c5 6f18 81fe  e836 388d c0b2 1f63 1124