Prevent backslash escaping and kept YAML header in Pandoc Markdown to PHP Extra Markdown conversion

Prevent backslash escaping and kept YAML header in Pandoc Markdown to PHP Extra Markdown conversion

[Sean McDaniel]

[-- Attachment #1.1: Type: text/plain, Size: 1271 bytes --]


All,

I need to convert documents from CommonMark to Markdown PHP Extra. Some of 
my markdown documents have multiple tables in multi-line format that my 
application (Grav CMS) doesn't support.

I'm having two issues with the conversion process that I haven't been able 
to figure out.

First, I have opening and closing square-bracket tags that I don't want 
converted. E.g. [notice] some text... [/notice] gets converted to 
\[notice\] some text... \[/notice\]. I've tried enclosing these sections in 
a plain fenced block but I don't want the text indented.

Is there a way to either prevent backslash escaping of special characters 
(preferred) or keep blocks of text verbatim?

Second, I have a YAML header that I'd like to keep untouched that's being 
removed. Is there a way to output the same YAML header to my converted 
document?

Thank you,

Sean

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/0c95effe-7e6e-4974-a7b7-5981b6b46ed4n%40googlegroups.com.

[-- Attachment #1.2: Type: text/html, Size: 1600 bytes --]

Re: Prevent backslash escaping and kept YAML header in Pandoc Markdown to PHP Extra Markdown conversion

[John MacFarlane]

Sean McDaniel <sean.m.mcdaniel-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> writes:

> All,
>
> I need to convert documents from CommonMark to Markdown PHP Extra. Some of 
> my markdown documents have multiple tables in multi-line format that my 
> application (Grav CMS) doesn't support.
>
> I'm having two issues with the conversion process that I haven't been able 
> to figure out.
>
> First, I have opening and closing square-bracket tags that I don't want 
> converted. E.g. [notice] some text... [/notice] gets converted to 
> \[notice\] some text... \[/notice\]. I've tried enclosing these sections in 
> a plain fenced block but I don't want the text indented.
>
> Is there a way to either prevent backslash escaping of special characters 
> (preferred)

No.  We try to play it safe, and it's very difficult for the
writer to tell if a bracketed text would inadvertently become a link.

> or keep blocks of text verbatim?

Well yes, you can mark something as raw Markdown using the "raw attribute."

    `[my notice]`{=markdown}

You'll need to tell pandoc to use the raw attribute extension:

    pandoc -f commonmark+raw_attribute -t markdown_phpextra

> Second, I have a YAML header that I'd like to keep untouched that's being 
> removed. Is there a way to output the same YAML header to my converted 
> document?

Try with

     pandoc -f commonmark+raw_attribute+yaml_metadata_block -t markdown_phpextra+yaml_metadata_block -s

Re: Prevent backslash escaping and kept YAML header in Pandoc Markdown to PHP Extra Markdown conversion

[Sean McDaniel]

[-- Attachment #1: Type: text/plain, Size: 3215 bytes --]

Thank you John. Your suggestions got me most of the way there: the markdown
code blocks with the +raw_attribute processing are correctly passing
through the sections I want untouched and the +yaml_metadata_block is
keeping the header content I want.

The remaining issue which is giving me trouble is that +yaml_header_block
is also including all other document metadata and not just the variables
I've defined, i.e. stuff like

autoEqnLabels: false
autoSectionLabels: false
ccsDelim: ", "
ccsLabelSep:  —
ccsTemplate: |
  *i*
  *c**c**s**L**a**b**e**l**S**e**p*
  *t*
chapDelim: .

and so forth.

Is there a way to include just the variables I've defined in the converted
document? E.g.

---
title: My Title
metadata:
    description: 'My description'
    keywords: 'keyword1, keyword2, keyword3'
    author: 'Sean McDaniel'
sitemap:
    changefreq: monthly
page-toc:
    active: true
process:
    twig: true
---

I've looked at the appropriate section in the Pandoc documentation and it
seems like I might need to create a custom template. Is this correct?

Thank you,

Sean

On Sun, Aug 8, 2021 at 2:31 PM John MacFarlane <jgm-TVLZxgkOlNX2fBVCVOL8/A@public.gmane.org> wrote:

> Sean McDaniel <sean.m.mcdaniel-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> writes:
>
> > All,
> >
> > I need to convert documents from CommonMark to Markdown PHP Extra. Some
> of
> > my markdown documents have multiple tables in multi-line format that my
> > application (Grav CMS) doesn't support.
> >
> > I'm having two issues with the conversion process that I haven't been
> able
> > to figure out.
> >
> > First, I have opening and closing square-bracket tags that I don't want
> > converted. E.g. [notice] some text... [/notice] gets converted to
> > \[notice\] some text... \[/notice\]. I've tried enclosing these sections
> in
> > a plain fenced block but I don't want the text indented.
> >
> > Is there a way to either prevent backslash escaping of special
> characters
> > (preferred)
>
> No.  We try to play it safe, and it's very difficult for the
> writer to tell if a bracketed text would inadvertently become a link.
>
> > or keep blocks of text verbatim?
>
> Well yes, you can mark something as raw Markdown using the "raw attribute."
>
>     `[my notice]`{=markdown}
>
> You'll need to tell pandoc to use the raw attribute extension:
>
>     pandoc -f commonmark+raw_attribute -t markdown_phpextra
>
> > Second, I have a YAML header that I'd like to keep untouched that's
> being
> > removed. Is there a way to output the same YAML header to my converted
> > document?
>
> Try with
>
>      pandoc -f commonmark+raw_attribute+yaml_metadata_block -t
> markdown_phpextra+yaml_metadata_block -s
>

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/CAML5wTUJ%2BsvOhMUi2GwtPw0-_sx-GrRtG5teCihq6WmGXjZ8Cg%40mail.gmail.com.

[-- Attachment #2: Type: text/html, Size: 4393 bytes --]

Re: Prevent backslash escaping and kept YAML header in Pandoc Markdown to PHP Extra Markdown conversion

[BPJ]

[-- Attachment #1.1: Type: text/plain, Size: 5078 bytes --]

u can use a Lua filter which reads in a whitelist from the metadata itself
and
removes all metadata fields whose keys are not in the whitelist. Such a
filter is attached with this message.

If you have any metadata keys which should be on the whitelist for the
current project put them in your metadata as a list of strings under a key
'meta_whitelist', with one string (metadata key to whitelist) per line.
The field 'meta_whitelist' itself should probably not be on the list!  You
probably want to put the list in a YAML file and include it with the
`--metadata-file` option, or under the `metadata:` field in a defaults
file.  It might look like this:

    meta_whitelist:
      - address
      - phone
      - email

or in a defaults file:

    metadata:
      meta_whitelist:
        - address
        - phone
        - email

# IMPORTANT

You almost certainly want to run this filter after all other filters, since
they may depend on metadata which this filter removes.


Den mån 9 aug. 2021 22:57Sean McDaniel <sean.m.mcdaniel-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> skrev:

> Thank you John. Your suggestions got me most of the way there: the
> markdown code blocks with the +raw_attribute processing are correctly
> passing through the sections I want untouched and the +yaml_metadata_block
> is keeping the header content I want.
>
> The remaining issue which is giving me trouble is that +yaml_header_block
> is also including all other document metadata and not just the variables
> I've defined, i.e. stuff like
>
> autoEqnLabels: false
> autoSectionLabels: false
> ccsDelim: ", "
> ccsLabelSep:  —
> ccsTemplate: |
>   *i*
>   *c**c**s**L**a**b**e**l**S**e**p*
>   *t*
> chapDelim: .
>
> and so forth.
>
> Is there a way to include just the variables I've defined in the converted
> document? E.g.
>
> ---
> title: My Title
> metadata:
>     description: 'My description'
>     keywords: 'keyword1, keyword2, keyword3'
>     author: 'Sean McDaniel'
> sitemap:
>     changefreq: monthly
> page-toc:
>     active: true
> process:
>     twig: true
> ---
>
> I've looked at the appropriate section in the Pandoc documentation and it
> seems like I might need to create a custom template. Is this correct?
>
> Thank you,
>
> Sean
>
> On Sun, Aug 8, 2021 at 2:31 PM John MacFarlane <jgm-TVLZxgkOlNX2fBVCVOL8/A@public.gmane.org> wrote:
>
>> Sean McDaniel <sean.m.mcdaniel-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> writes:
>>
>> > All,
>> >
>> > I need to convert documents from CommonMark to Markdown PHP Extra. Some
>> of
>> > my markdown documents have multiple tables in multi-line format that my
>> > application (Grav CMS) doesn't support.
>> >
>> > I'm having two issues with the conversion process that I haven't been
>> able
>> > to figure out.
>> >
>> > First, I have opening and closing square-bracket tags that I don't want
>> > converted. E.g. [notice] some text... [/notice] gets converted to
>> > \[notice\] some text... \[/notice\]. I've tried enclosing these
>> sections in
>> > a plain fenced block but I don't want the text indented.
>> >
>> > Is there a way to either prevent backslash escaping of special
>> characters
>> > (preferred)
>>
>> No.  We try to play it safe, and it's very difficult for the
>> writer to tell if a bracketed text would inadvertently become a link.
>>
>> > or keep blocks of text verbatim?
>>
>> Well yes, you can mark something as raw Markdown using the "raw
>> attribute."
>>
>>     `[my notice]`{=markdown}
>>
>> You'll need to tell pandoc to use the raw attribute extension:
>>
>>     pandoc -f commonmark+raw_attribute -t markdown_phpextra
>>
>> > Second, I have a YAML header that I'd like to keep untouched that's
>> being
>> > removed. Is there a way to output the same YAML header to my converted
>> > document?
>>
>> Try with
>>
>>      pandoc -f commonmark+raw_attribute+yaml_metadata_block -t
>> markdown_phpextra+yaml_metadata_block -s
>>
> --
> You received this message because you are subscribed to the Google Groups
> "pandoc-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/pandoc-discuss/CAML5wTUJ%2BsvOhMUi2GwtPw0-_sx-GrRtG5teCihq6WmGXjZ8Cg%40mail.gmail.com
> <https://groups.google.com/d/msgid/pandoc-discuss/CAML5wTUJ%2BsvOhMUi2GwtPw0-_sx-GrRtG5teCihq6WmGXjZ8Cg%40mail.gmail.com?utm_medium=email&utm_source=footer>
> .
>

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/CADAJKhDj%3Dk9Ue0_iL2wWdQkpHXB0Em9RcnqeZXu5sHrAHOjYSA%40mail.gmail.com.

[-- Attachment #1.2: Type: text/html, Size: 7231 bytes --]

[-- Attachment #2: meta-whitelist.lua --]
[-- Type: application/octet-stream, Size: 3663 bytes --]

--[===================================[
Pandoc filter which removes all fields whose key are not in a whitelist in the
metadata itself.

## Description

If you have any metadata keys which should be on the whitelist for the
current project put them in your metadata as a list of strings under a key 'meta_whitelist', with one string
(metadata key to whitelist) per line.
The field 'meta_whitelist' itself should probably not be on the list!
You probably want to put the list in a YAML file and include it with the
`--metadata-file` option, or under the `metadata:` field in a defaults file.
It might look like this:
 
    meta_whitelist:
      - address
      - phone
      - email

or in a defaults file:

    metadata:
      meta_whitelist:
        - address
        - phone
        - email

# IMPORTANT

You almost certainly want to run this filter after all other filters, since
they may depend on metadata which this filter removes.

# Author

Benct Philip Jonsson <bpjonsson@gmail.com>

# Copyright and license

This software is Copyright (c) 2021 by Benct Philip Jonsson.

This is free software, licensed under:

  The MIT (X11) License

The MIT License

Permission is hereby granted, free of charge, to any person
obtaining a copy of this software and associated
documentation files (the "Software"), to deal in the Software
without restriction, including without limitation the rights to
use, copy, modify, merge, publish, distribute, sublicense,
and/or sell copies of the Software, and to permit persons to
whom the Software is furnished to do so, subject to the
following conditions:

The above copyright notice and this permission notice shall
be included in all copies or substantial portions of the
Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT
WARRANTY OF ANY KIND, EXPRESS OR IMPLIED,
INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR
PURPOSE AND NONINFRINGEMENT. IN NO EVENT
SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
CONNECTION WITH THE SOFTWARE OR THE USE OR
OTHER DEALINGS IN THE SOFTWARE.

--]===================================]

-- List the metadata keys which should always be on your whitelist here
local whitelist = {
  'title',
  'author',
  'date',
}

local ok = {}
-- Process the permanent whitelist
for _,k in ipairs(whitelist) do
  ok[k] = true
end

function Meta (meta)
  -- Read in the whitelist from the metadata itself if it exists
  local mwl = meta.whitelist
  if mwl then
    if ('table' == type(mwl) ) and ( 'MetaList' == mwl.tag ) then
      for _,item in ipairs(mwl) do
        local item_type = type(item)
        local key
        if 'table' == item_type then -- always true?
          -- It's a metadata object so extract its string content
          key = pandoc.utils.stringify(item)
        elseif 'string' == item_type then
          key = item
        else
          error "Expected items in meta.meta_whitelist to be strings"
        end
        ok[key] = true
      end
    else
      error "Expected meta.meta_whitelist to be list of strings"
    end
  end
  -- Since the Lua pairs iterator is not stateful and we are going to modidy the
  -- mapping whose keys we are looping over we must first create a list of keys
  -- and loop over that!
  local keys
  for k in pairs(meta) do
    keys[#keys+1] = k
  end
  -- Now we loop over the list of keys
  for _,k in ipairs(keys) do
    if not ok[k] then -- if the key is not in the whitelist
      meta[k] = nil   -- remove the field
    end
  end
  -- Done!
  return meta
end

-- The End

Re: Prevent backslash escaping and kept YAML header in Pandoc Markdown to PHP Extra Markdown conversion

[John MacFarlane]

Sean McDaniel <sean.m.mcdaniel-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> writes:

> Thank you John. Your suggestions got me most of the way there: the markdown
> code blocks with the +raw_attribute processing are correctly passing
> through the sections I want untouched and the +yaml_metadata_block is
> keeping the header content I want.
>
> The remaining issue which is giving me trouble is that +yaml_header_block
> is also including all other document metadata and not just the variables
> I've defined, i.e. stuff like
>
> autoEqnLabels: false
> autoSectionLabels: false
> ccsDelim: ", "
> ccsLabelSep:  —
> ccsTemplate: |
>   *i*
>   *c**c**s**L**a**b**e**l**S**e**p*
>   *t*
> chapDelim: .
>
> and so forth.

I'm not sure why these are going in the metadata -- what is
setting these metadata elements, if you're not doing it?
maybe I don't understand the issue.

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/m21r71895k.fsf%40MacBook-Pro-2.hsd1.ca.comcast.net.

Re: Prevent backslash escaping and kept YAML header in Pandoc Markdown to PHP Extra Markdown conversion

[BPJ]

[-- Attachment #1: Type: text/plain, Size: 2077 bytes --]

They are pandoc-crossref configuration options, passed through the metadata
since that is the most convenient way to let the user "communicate" with
filters.

Den tis 10 aug. 2021 20:09John MacFarlane <jgm-TVLZxgkOlNX2fBVCVOL8/A@public.gmane.org> skrev:

> Sean McDaniel <sean.m.mcdaniel-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> writes:
>
> > Thank you John. Your suggestions got me most of the way there: the
> markdown
> > code blocks with the +raw_attribute processing are correctly passing
> > through the sections I want untouched and the +yaml_metadata_block is
> > keeping the header content I want.
> >
> > The remaining issue which is giving me trouble is that +yaml_header_block
> > is also including all other document metadata and not just the variables
> > I've defined, i.e. stuff like
> >
> > autoEqnLabels: false
> > autoSectionLabels: false
> > ccsDelim: ", "
> > ccsLabelSep:  —
> > ccsTemplate: |
> >   *i*
> >   *c**c**s**L**a**b**e**l**S**e**p*
> >   *t*
> > chapDelim: .
> >
> > and so forth.
>
> I'm not sure why these are going in the metadata -- what is
> setting these metadata elements, if you're not doing it?
> maybe I don't understand the issue.
>
> --
> You received this message because you are subscribed to the Google Groups
> "pandoc-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/pandoc-discuss/m21r71895k.fsf%40MacBook-Pro-2.hsd1.ca.comcast.net
> .
>

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/CADAJKhBhk%3DM15HuN-4b%2BbG8NCGBeS6A7YYa3Y2n3Dax22m8MJA%40mail.gmail.com.

[-- Attachment #2: Type: text/html, Size: 3176 bytes --]

Re: Prevent backslash escaping and kept YAML header in Pandoc Markdown to PHP Extra Markdown conversion

[John MacFarlane]

I see. Well, messing with templates won't help.
You could write a filter that is applied AFTER
pandoc-crossref and removes these metadata fields.

BPJ <melroch-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> writes:

> They are pandoc-crossref configuration options, passed through the metadata
> since that is the most convenient way to let the user "communicate" with
> filters.
>
> Den tis 10 aug. 2021 20:09John MacFarlane <jgm-TVLZxgkOlNX2fBVCVOL8/A@public.gmane.org> skrev:
>
>> Sean McDaniel <sean.m.mcdaniel-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> writes:
>>
>> > Thank you John. Your suggestions got me most of the way there: the
>> markdown
>> > code blocks with the +raw_attribute processing are correctly passing
>> > through the sections I want untouched and the +yaml_metadata_block is
>> > keeping the header content I want.
>> >
>> > The remaining issue which is giving me trouble is that +yaml_header_block
>> > is also including all other document metadata and not just the variables
>> > I've defined, i.e. stuff like
>> >
>> > autoEqnLabels: false
>> > autoSectionLabels: false
>> > ccsDelim: ", "
>> > ccsLabelSep:  —
>> > ccsTemplate: |
>> >   *i*
>> >   *c**c**s**L**a**b**e**l**S**e**p*
>> >   *t*
>> > chapDelim: .
>> >
>> > and so forth.
>>
>> I'm not sure why these are going in the metadata -- what is
>> setting these metadata elements, if you're not doing it?
>> maybe I don't understand the issue.
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "pandoc-discuss" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/pandoc-discuss/m21r71895k.fsf%40MacBook-Pro-2.hsd1.ca.comcast.net
>> .
>>
>
> -- 
> You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
> To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/CADAJKhBhk%3DM15HuN-4b%2BbG8NCGBeS6A7YYa3Y2n3Dax22m8MJA%40mail.gmail.com.

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/yh480ktujxw08h.fsf%40johnmacfarlane.net.

Re: Prevent backslash escaping and kept YAML header in Pandoc Markdown to PHP Extra Markdown conversion

[Sean McDaniel]

[-- Attachment #1: Type: text/plain, Size: 4317 bytes --]

Thank you BP for the Lua filter and the explanation. I have attempted to
run your Lua filter without modifications as a test and I get the following
error:

Error running filter meta-whitelist.lua:
meta-whitelist.lua:114: attempt to get length of a nil value (local 'keys')
stack traceback:
meta-whitelist.lua:114: in function 'Meta'

The issued pandoc command was (note that I put the Lua filter after the
crossref filter):

pandoc -s --markdown-headings=atx \
    --filter=pandoc-crossref --lua-filter=meta-whitelist.lua \
    --from=markdown+raw_attribute+yaml_metadata_block \
    --to=markdown_phpextra+yaml_metadata_block \
    -o myfile_phpextra.md myfile.md

I get the same error with or without the crossref filter.

As you've explained (thanks) the metadata fields are created by the
pandoc-crossref filter. The following without filters works as
expected---both my YAML header and Markdown blocks are preserved:

pandoc -s --markdown-headings=atx \
    --from=markdown+raw_attribute+yaml_metadata_block \
    --to=markdown_phpextra+yaml_metadata_block \
    -o myfile_phpextra.md myfile.md

The above will probably work for most of my needs, but I was wondering if
there's a simple Lua filter fix to get it working for my case.

Regards,

Sean

On Tue, Aug 10, 2021 at 3:47 PM John MacFarlane <jgm-TVLZxgkOlNX2fBVCVOL8/A@public.gmane.org> wrote:

>
> I see. Well, messing with templates won't help.
> You could write a filter that is applied AFTER
> pandoc-crossref and removes these metadata fields.
>
> BPJ <melroch-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> writes:
>
> > They are pandoc-crossref configuration options, passed through the
> metadata
> > since that is the most convenient way to let the user "communicate" with
> > filters.
> >
> > Den tis 10 aug. 2021 20:09John MacFarlane <jgm-TVLZxgkOlNX2fBVCVOL8/A@public.gmane.org> skrev:
> >
> >> Sean McDaniel <sean.m.mcdaniel-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> writes:
> >>
> >> > Thank you John. Your suggestions got me most of the way there: the
> >> markdown
> >> > code blocks with the +raw_attribute processing are correctly passing
> >> > through the sections I want untouched and the +yaml_metadata_block is
> >> > keeping the header content I want.
> >> >
> >> > The remaining issue which is giving me trouble is that
> +yaml_header_block
> >> > is also including all other document metadata and not just the
> variables
> >> > I've defined, i.e. stuff like
> >> >
> >> > autoEqnLabels: false
> >> > autoSectionLabels: false
> >> > ccsDelim: ", "
> >> > ccsLabelSep:  —
> >> > ccsTemplate: |
> >> >   *i*
> >> >   *c**c**s**L**a**b**e**l**S**e**p*
> >> >   *t*
> >> > chapDelim: .
> >> >
> >> > and so forth.
> >>
> >> I'm not sure why these are going in the metadata -- what is
> >> setting these metadata elements, if you're not doing it?
> >> maybe I don't understand the issue.
> >>
> >> --
> >> You received this message because you are subscribed to the Google
> Groups
> >> "pandoc-discuss" group.
> >> To unsubscribe from this group and stop receiving emails from it, send
> an
> >> email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
> >> To view this discussion on the web visit
> >>
> https://groups.google.com/d/msgid/pandoc-discuss/m21r71895k.fsf%40MacBook-Pro-2.hsd1.ca.comcast.net
> >> .
> >>
> >
> > --
> > You received this message because you are subscribed to the Google
> Groups "pandoc-discuss" group.
> > To unsubscribe from this group and stop receiving emails from it, send
> an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
> > To view this discussion on the web visit
> https://groups.google.com/d/msgid/pandoc-discuss/CADAJKhBhk%3DM15HuN-4b%2BbG8NCGBeS6A7YYa3Y2n3Dax22m8MJA%40mail.gmail.com
> .
>

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/CAML5wTWbn0S2Ps%2BROAOqjpHvv1ygfqWjx1FL0nj_2DiNSjjhdQ%40mail.gmail.com.

[-- Attachment #2: Type: text/html, Size: 6417 bytes --]

Re: Prevent backslash escaping and kept YAML header in Pandoc Markdown to PHP Extra Markdown conversion

[Sean McDaniel]

[-- Attachment #1: Type: text/plain, Size: 4610 bytes --]

*BJP. Sorry about the misspelling.

On Fri, Aug 13, 2021 at 1:01 PM Sean McDaniel <sean.m.mcdaniel-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
wrote:

> Thank you BP for the Lua filter and the explanation. I have attempted to
> run your Lua filter without modifications as a test and I get the following
> error:
>
> Error running filter meta-whitelist.lua:
> meta-whitelist.lua:114: attempt to get length of a nil value (local 'keys')
> stack traceback:
> meta-whitelist.lua:114: in function 'Meta'
>
> The issued pandoc command was (note that I put the Lua filter after the
> crossref filter):
>
> pandoc -s --markdown-headings=atx \
>     --filter=pandoc-crossref --lua-filter=meta-whitelist.lua \
>     --from=markdown+raw_attribute+yaml_metadata_block \
>     --to=markdown_phpextra+yaml_metadata_block \
>     -o myfile_phpextra.md myfile.md
>
> I get the same error with or without the crossref filter.
>
> As you've explained (thanks) the metadata fields are created by the
> pandoc-crossref filter. The following without filters works as
> expected---both my YAML header and Markdown blocks are preserved:
>
> pandoc -s --markdown-headings=atx \
>     --from=markdown+raw_attribute+yaml_metadata_block \
>     --to=markdown_phpextra+yaml_metadata_block \
>     -o myfile_phpextra.md myfile.md
>
> The above will probably work for most of my needs, but I was wondering if
> there's a simple Lua filter fix to get it working for my case.
>
> Regards,
>
> Sean
>
> On Tue, Aug 10, 2021 at 3:47 PM John MacFarlane <jgm-TVLZxgkOlNX2fBVCVOL8/A@public.gmane.org> wrote:
>
>>
>> I see. Well, messing with templates won't help.
>> You could write a filter that is applied AFTER
>> pandoc-crossref and removes these metadata fields.
>>
>> BPJ <melroch-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> writes:
>>
>> > They are pandoc-crossref configuration options, passed through the
>> metadata
>> > since that is the most convenient way to let the user "communicate" with
>> > filters.
>> >
>> > Den tis 10 aug. 2021 20:09John MacFarlane <jgm-TVLZxgkOlNX2fBVCVOL8/A@public.gmane.org> skrev:
>> >
>> >> Sean McDaniel <sean.m.mcdaniel-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> writes:
>> >>
>> >> > Thank you John. Your suggestions got me most of the way there: the
>> >> markdown
>> >> > code blocks with the +raw_attribute processing are correctly passing
>> >> > through the sections I want untouched and the +yaml_metadata_block is
>> >> > keeping the header content I want.
>> >> >
>> >> > The remaining issue which is giving me trouble is that
>> +yaml_header_block
>> >> > is also including all other document metadata and not just the
>> variables
>> >> > I've defined, i.e. stuff like
>> >> >
>> >> > autoEqnLabels: false
>> >> > autoSectionLabels: false
>> >> > ccsDelim: ", "
>> >> > ccsLabelSep:  —
>> >> > ccsTemplate: |
>> >> >   *i*
>> >> >   *c**c**s**L**a**b**e**l**S**e**p*
>> >> >   *t*
>> >> > chapDelim: .
>> >> >
>> >> > and so forth.
>> >>
>> >> I'm not sure why these are going in the metadata -- what is
>> >> setting these metadata elements, if you're not doing it?
>> >> maybe I don't understand the issue.
>> >>
>> >> --
>> >> You received this message because you are subscribed to the Google
>> Groups
>> >> "pandoc-discuss" group.
>> >> To unsubscribe from this group and stop receiving emails from it, send
>> an
>> >> email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
>> >> To view this discussion on the web visit
>> >>
>> https://groups.google.com/d/msgid/pandoc-discuss/m21r71895k.fsf%40MacBook-Pro-2.hsd1.ca.comcast.net
>> >> .
>> >>
>> >
>> > --
>> > You received this message because you are subscribed to the Google
>> Groups "pandoc-discuss" group.
>> > To unsubscribe from this group and stop receiving emails from it, send
>> an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
>> > To view this discussion on the web visit
>> https://groups.google.com/d/msgid/pandoc-discuss/CADAJKhBhk%3DM15HuN-4b%2BbG8NCGBeS6A7YYa3Y2n3Dax22m8MJA%40mail.gmail.com
>> .
>>
>

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/CAML5wTWgCA_xEMP%2BpUQvKJQbbq%2B14BWQUZ%3DfSY%2B5cwP2gj435A%40mail.gmail.com.

[-- Attachment #2: Type: text/html, Size: 6923 bytes --]

Re: Prevent backslash escaping and kept YAML header in Pandoc Markdown to PHP Extra Markdown conversion

[BPJ]

[-- Attachment #1: Type: text/plain, Size: 4672 bytes --]

Sorry a typo on line 112: change `local keys` into `local keys = {}`

I must have hit undo one time too many!

Den fre 13 aug. 2021 19:01Sean McDaniel <sean.m.mcdaniel-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> skrev:

> Thank you BP for the Lua filter and the explanation. I have attempted to
> run your Lua filter without modifications as a test and I get the following
> error:
>
> Error running filter meta-whitelist.lua:
> meta-whitelist.lua:114: attempt to get length of a nil value (local 'keys')
> stack traceback:
> meta-whitelist.lua:114: in function 'Meta'
>
> The issued pandoc command was (note that I put the Lua filter after the
> crossref filter):
>
> pandoc -s --markdown-headings=atx \
>     --filter=pandoc-crossref --lua-filter=meta-whitelist.lua \
>     --from=markdown+raw_attribute+yaml_metadata_block \
>     --to=markdown_phpextra+yaml_metadata_block \
>     -o myfile_phpextra.md myfile.md
>
> I get the same error with or without the crossref filter.
>
> As you've explained (thanks) the metadata fields are created by the
> pandoc-crossref filter. The following without filters works as
> expected---both my YAML header and Markdown blocks are preserved:
>
> pandoc -s --markdown-headings=atx \
>     --from=markdown+raw_attribute+yaml_metadata_block \
>     --to=markdown_phpextra+yaml_metadata_block \
>     -o myfile_phpextra.md myfile.md
>
> The above will probably work for most of my needs, but I was wondering if
> there's a simple Lua filter fix to get it working for my case.
>
> Regards,
>
> Sean
>
> On Tue, Aug 10, 2021 at 3:47 PM John MacFarlane <jgm-TVLZxgkOlNX2fBVCVOL8/A@public.gmane.org> wrote:
>
>>
>> I see. Well, messing with templates won't help.
>> You could write a filter that is applied AFTER
>> pandoc-crossref and removes these metadata fields.
>>
>> BPJ <melroch-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> writes:
>>
>> > They are pandoc-crossref configuration options, passed through the
>> metadata
>> > since that is the most convenient way to let the user "communicate" with
>> > filters.
>> >
>> > Den tis 10 aug. 2021 20:09John MacFarlane <jgm-TVLZxgkOlNX2fBVCVOL8/A@public.gmane.org> skrev:
>> >
>> >> Sean McDaniel <sean.m.mcdaniel-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> writes:
>> >>
>> >> > Thank you John. Your suggestions got me most of the way there: the
>> >> markdown
>> >> > code blocks with the +raw_attribute processing are correctly passing
>> >> > through the sections I want untouched and the +yaml_metadata_block is
>> >> > keeping the header content I want.
>> >> >
>> >> > The remaining issue which is giving me trouble is that
>> +yaml_header_block
>> >> > is also including all other document metadata and not just the
>> variables
>> >> > I've defined, i.e. stuff like
>> >> >
>> >> > autoEqnLabels: false
>> >> > autoSectionLabels: false
>> >> > ccsDelim: ", "
>> >> > ccsLabelSep:  —
>> >> > ccsTemplate: |
>> >> >   *i*
>> >> >   *c**c**s**L**a**b**e**l**S**e**p*
>> >> >   *t*
>> >> > chapDelim: .
>> >> >
>> >> > and so forth.
>> >>
>> >> I'm not sure why these are going in the metadata -- what is
>> >> setting these metadata elements, if you're not doing it?
>> >> maybe I don't understand the issue.
>> >>
>> >> --
>> >> You received this message because you are subscribed to the Google
>> Groups
>> >> "pandoc-discuss" group.
>> >> To unsubscribe from this group and stop receiving emails from it, send
>> an
>> >> email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
>> >> To view this discussion on the web visit
>> >>
>> https://groups.google.com/d/msgid/pandoc-discuss/m21r71895k.fsf%40MacBook-Pro-2.hsd1.ca.comcast.net
>> >> .
>> >>
>> >
>> > --
>> > You received this message because you are subscribed to the Google
>> Groups "pandoc-discuss" group.
>> > To unsubscribe from this group and stop receiving emails from it, send
>> an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
>> > To view this discussion on the web visit
>> https://groups.google.com/d/msgid/pandoc-discuss/CADAJKhBhk%3DM15HuN-4b%2BbG8NCGBeS6A7YYa3Y2n3Dax22m8MJA%40mail.gmail.com
>> .
>>
>

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/CADAJKhDU-03GUEA3_5W7ThQMuMU9_bqPH0_ZsWHUm6ptq1HBsQ%40mail.gmail.com.

[-- Attachment #2: Type: text/html, Size: 7108 bytes --]

Re: Prevent backslash escaping and kept YAML header in Pandoc Markdown to PHP Extra Markdown conversion

[Sean McDaniel]

[-- Attachment #1: Type: text/plain, Size: 4951 bytes --]

BPJ, that did the trick. Thank you for your help!

On Fri, Aug 13, 2021 at 3:41 PM BPJ <melroch-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:

> Sorry a typo on line 112: change `local keys` into `local keys = {}`
>
> I must have hit undo one time too many!
>
> Den fre 13 aug. 2021 19:01Sean McDaniel <sean.m.mcdaniel-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> skrev:
>
>> Thank you BP for the Lua filter and the explanation. I have attempted to
>> run your Lua filter without modifications as a test and I get the following
>> error:
>>
>> Error running filter meta-whitelist.lua:
>> meta-whitelist.lua:114: attempt to get length of a nil value (local
>> 'keys')
>> stack traceback:
>> meta-whitelist.lua:114: in function 'Meta'
>>
>> The issued pandoc command was (note that I put the Lua filter after the
>> crossref filter):
>>
>> pandoc -s --markdown-headings=atx \
>>     --filter=pandoc-crossref --lua-filter=meta-whitelist.lua \
>>     --from=markdown+raw_attribute+yaml_metadata_block \
>>     --to=markdown_phpextra+yaml_metadata_block \
>>     -o myfile_phpextra.md myfile.md
>>
>> I get the same error with or without the crossref filter.
>>
>> As you've explained (thanks) the metadata fields are created by the
>> pandoc-crossref filter. The following without filters works as
>> expected---both my YAML header and Markdown blocks are preserved:
>>
>> pandoc -s --markdown-headings=atx \
>>     --from=markdown+raw_attribute+yaml_metadata_block \
>>     --to=markdown_phpextra+yaml_metadata_block \
>>     -o myfile_phpextra.md myfile.md
>>
>> The above will probably work for most of my needs, but I was wondering if
>> there's a simple Lua filter fix to get it working for my case.
>>
>> Regards,
>>
>> Sean
>>
>> On Tue, Aug 10, 2021 at 3:47 PM John MacFarlane <jgm-TVLZxgkOlNX2fBVCVOL8/A@public.gmane.org> wrote:
>>
>>>
>>> I see. Well, messing with templates won't help.
>>> You could write a filter that is applied AFTER
>>> pandoc-crossref and removes these metadata fields.
>>>
>>> BPJ <melroch-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> writes:
>>>
>>> > They are pandoc-crossref configuration options, passed through the
>>> metadata
>>> > since that is the most convenient way to let the user "communicate"
>>> with
>>> > filters.
>>> >
>>> > Den tis 10 aug. 2021 20:09John MacFarlane <jgm-TVLZxgkOlNX2fBVCVOL8/A@public.gmane.org> skrev:
>>> >
>>> >> Sean McDaniel <sean.m.mcdaniel-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> writes:
>>> >>
>>> >> > Thank you John. Your suggestions got me most of the way there: the
>>> >> markdown
>>> >> > code blocks with the +raw_attribute processing are correctly passing
>>> >> > through the sections I want untouched and the +yaml_metadata_block
>>> is
>>> >> > keeping the header content I want.
>>> >> >
>>> >> > The remaining issue which is giving me trouble is that
>>> +yaml_header_block
>>> >> > is also including all other document metadata and not just the
>>> variables
>>> >> > I've defined, i.e. stuff like
>>> >> >
>>> >> > autoEqnLabels: false
>>> >> > autoSectionLabels: false
>>> >> > ccsDelim: ", "
>>> >> > ccsLabelSep:  —
>>> >> > ccsTemplate: |
>>> >> >   *i*
>>> >> >   *c**c**s**L**a**b**e**l**S**e**p*
>>> >> >   *t*
>>> >> > chapDelim: .
>>> >> >
>>> >> > and so forth.
>>> >>
>>> >> I'm not sure why these are going in the metadata -- what is
>>> >> setting these metadata elements, if you're not doing it?
>>> >> maybe I don't understand the issue.
>>> >>
>>> >> --
>>> >> You received this message because you are subscribed to the Google
>>> Groups
>>> >> "pandoc-discuss" group.
>>> >> To unsubscribe from this group and stop receiving emails from it,
>>> send an
>>> >> email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
>>> >> To view this discussion on the web visit
>>> >>
>>> https://groups.google.com/d/msgid/pandoc-discuss/m21r71895k.fsf%40MacBook-Pro-2.hsd1.ca.comcast.net
>>> >> .
>>> >>
>>> >
>>> > --
>>> > You received this message because you are subscribed to the Google
>>> Groups "pandoc-discuss" group.
>>> > To unsubscribe from this group and stop receiving emails from it, send
>>> an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
>>> > To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/pandoc-discuss/CADAJKhBhk%3DM15HuN-4b%2BbG8NCGBeS6A7YYa3Y2n3Dax22m8MJA%40mail.gmail.com
>>> .
>>>
>>

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/CAML5wTV5_Zxz7_ABo%2Bp-22AF9obc4aH1d8uMCc4bzGnyvcKENg%40mail.gmail.com.

[-- Attachment #2: Type: text/html, Size: 7603 bytes --]