From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.io/gmane.text.pandoc/29023 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Sean McDaniel Newsgroups: gmane.text.pandoc Subject: Re: Prevent backslash escaping and kept YAML header in Pandoc Markdown to PHP Extra Markdown conversion Date: Fri, 13 Aug 2021 13:01:21 -0400 Message-ID: References: <0c95effe-7e6e-4974-a7b7-5981b6b46ed4n@googlegroups.com> Reply-To: pandoc-discuss-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org Mime-Version: 1.0 Content-Type: multipart/alternative; boundary="000000000000b5cafb05c973cdc1" Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="7520"; mail-complaints-to="usenet@ciao.gmane.io" Cc: BPJ , pandoc-discuss To: John MacFarlane Original-X-From: pandoc-discuss+bncBCQZFQXCXYGRB3OK3KEAMGQE2DPV4AI-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org Fri Aug 13 19:01:37 2021 Return-path: Envelope-to: gtp-pandoc-discuss@m.gmane-mx.org Original-Received: from mail-il1-f192.google.com ([209.85.166.192]) by ciao.gmane.io with esmtps (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1mEaYq-0001kp-2X for gtp-pandoc-discuss@m.gmane-mx.org; Fri, 13 Aug 2021 19:01:36 +0200 Original-Received: by mail-il1-f192.google.com with SMTP id z5-20020a92cd050000b029022368ee85c8sf5363658iln.16 for ; Fri, 13 Aug 2021 10:01:35 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1628874095; cv=pass; d=google.com; s=arc-20160816; b=syyQhbSh6f7O4x292gQbnJQYH0Kh6oOhTxe5xjfnEF2qwHJiTB4tTC6Y6Ywi9uDmRQ rtj6VmAiiHvlfO/myDLP5OvJpeHhDWTF3LLYTfKbNKz1pOW8qn+wJs9rXx+tt0LVw43k H3Vn2aG5bYJNYu5VeHbthWtnCMQFmg1V4b6HwyPeBiV41RBDyJINCvgy+dpvBHkLEYH4 a4S3uftfRmwu4hQNalOf1IZAticKG385Fzbd6Iad4p/RV3klcH00y1v8TAUqgEe++UF8 pTYbrIfex8klUxwjYBRtvpnsuRcS31uV3gZYZxy+6mitGP8dcmzgzDVK+O6QkaxkBSgJ 3B3Q== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:reply-to:cc:to:subject:message-id :date:from:in-reply-to:references:mime-version:sender:dkim-signature :dkim-signature; bh=uPnVThPzusD4lN/Og6qb5FmUhk3DvsZxAAyJ6PvW998=; b=b17D3doCEjHupuOgY4acf9Z2Isk6FglF46YpAMv9PdfXHnXQiW6z4EweaWj/QRWwbM nCC/1KqSRUejLnFQ5/a94au7tqixLuxTdNzscoOksUiQh8LhXGyOV+Z+l6LiGF+rCTtb XMskp0Aa/6VurJ7xaILq6ysM8Iw966ExLUBinADL+Glxa9oMXpsoeeoJ7XjbbpG9onyH jkH+KxhbRapjHQR0EigKFyfCX9mdMoYw1/Yv6rso8wQdk4RbFU0iTvqBlG6uYzEyxX4z 9jL10vTQJTvo1Km/orf+M0jeeD8usEw7z8HybMwCp9qQnK1roEZ8Vk8U/hWEQ5n9KdPY aSZw== ARC-Authentication-Results: i=2; gmr-mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=HmlHcaL4; spf=pass (google.com: domain of sean.m.mcdaniel-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org designates 2607:f8b0:4864:20::d2f as permitted sender) smtp.mailfrom=sean.m.mcdaniel-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20161025; h=sender:mime-version:references:in-reply-to:from:date:message-id :subject:to:cc:x-original-sender:x-original-authentication-results :reply-to:precedence:mailing-list:list-id:list-post:list-help :list-archive:list-subscribe:list-unsubscribe; bh=uPnVThPzusD4lN/Og6qb5FmUhk3DvsZxAAyJ6PvW998=; b=PlUV5S56B6UZ8lP4r+QJvY/luX5F6bm2sM3MU6+RwjG7A0mE+3STAjyVP7yFMqoYS4 fA0UsR2KvyaceYLFdbSmLzihDNCLvlKRAHRHJtHtjyyJga86BxsbI7B9aHeW90KwdVNL H/Bd8Op1UQDxhoni1uf+3yHg+VcIFF1a6hJnGb/UUkWRG5UUMkBdZHY/KR+XVIrIKgqS 2Pk2n0PIIH9LsWG3g5c5Lznz31OqW4VvTgdwjs7tqKIxO5c89E32c0CPKZgO1yNYj9nK gjx5kpGoG+rHRqmYqrZfx6SDQ3IIJGeCpvN8j+16S+RBpJH80PkTVqpl2J2Nq9EAcTL4 BXXw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:x-original-sender:x-original-authentication-results:reply-to :precedence:mailing-list:list-id:list-post:list-help:list-archive :list-subscribe:list-unsubscribe; bh=uPnVThPzusD4lN/Og6qb5FmUhk3DvsZxAAyJ6PvW998=; b=P/MLGN+uvQpF+N7maBMyRzptn07kDLk/WCFxpamInl4BjA4xOh4jDYmGV9LO95ievL YVIgrcGd4w8hB9ecmVQ0HEmHevmvf2eCExbUGjyTVcSj2CsQaV1xE0RKZtnHS6B2nL9Z pJI/7nDaTcXHzF/Qi18NiZoiHgVeDjHoUmWRY+nWUKGgMrAW0zhnMMz8ipvmCh71Uxji 3p61uFT2Fbpi5Rn38738azOokkUpEtoSmffDjCydM+ZFBuxWi7TFTOgG/R7jUi1ekzUM UjEVhOu2n313MsOGyomXLNc6LK/gB23fG9dP+mdbD41OWfKHAIqV4J/CmZt2Cjm52+FU HUQw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=sender:x-gm-message-state:mime-version:references:in-reply-to:from :date:message-id:subject:to:cc:x-original-sender :x-original-authentication-results:reply-to:precedence:mailing-list :list-id:x-spam-checked-in-group:list-post:list-help:list-archive :list-subscribe:list-unsubscribe; bh=uPnVThPzusD4lN/Og6qb5FmUhk3DvsZxAAyJ6PvW998=; b=l2v5CZsA0iiBjNQfyqXRtjbcchSPQ9WXRRZUjkkHyJcuS8Tqc2tW/PCTfwWO1y04Lf eH4ysXd2S216RdR+aKD1tNsCzOT+XvqStp1pmwrMpPe8jTvv3z4A2lneSt9wOdbU0Ojs Zs2wmRBFGcpoI3I27lCegDFykU9S3g4CqB4HhEBuwXaU0WF0+6QprwXsrlZwZ9Xcoq8L eyG1scg1zQLGvKxRCC6N9judeWvCNag5ydMrfsNL65i7x2B6scsnVd2CE0fzXzxaZkI4 9N74MD12E9fDDpK+otOvpYy+5wgE00ev7A2Mpk9WyJ538LsdT6fB50blptLoqyEi6Kdt 6g5w== Original-Sender: pandoc-discuss-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org X-Gm-Message-State: AOAM533wGvAGQmKf7ZGZgBSpNaXcwrbK75pK8xD6XHOumiMqaZOsR4GS ShwXWtXB2QWQnhpGTczRkiY= X-Google-Smtp-Source: ABdhPJzlEpnRhjx1meFhwvZyMtCZ7O0s3+V6SQ//fPp5GNDg2VHLQyEFiIGVZ6Ys45DxVcrWOQAGLw== X-Received: by 2002:a6b:8ed8:: with SMTP id q207mr2674383iod.29.1628874094995; Fri, 13 Aug 2021 10:01:34 -0700 (PDT) X-BeenThere: pandoc-discuss-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org Original-Received: by 2002:a05:6638:1345:: with SMTP id u5ls270293jad.9.gmail; Fri, 13 Aug 2021 10:01:33 -0700 (PDT) X-Received: by 2002:a02:caa1:: with SMTP id e1mr3199643jap.107.1628874093554; Fri, 13 Aug 2021 10:01:33 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1628874093; cv=none; d=google.com; s=arc-20160816; b=ixdiuMRjPajCJMPARh3gOJp1tArOLhq6ACRbh0QL68paQnbrS5r8qAuHzfU9tLexV7 nE/z1zC7oTEpp/SsjbvdDLQGhS4zBl3LIQgiOTDGCSLjWEhEXP6us/3FM6CoVBvqGQ7D 3DJR3j2GAbyEbWPBs/m/CmcPCl7+xPDDEDgKE7YBcfjn1oEN+VAzehHj5JHDH6S+fTx3 114SYrRl/mOW7uyjnXocuxp2solSKr2vx7OZuLp7GUbbZ3Ty5ahloQLzOqmbHNNvdWIl PbEPZ8N6LY7v4rjF0d//j1Sl6D85BqWKuv8G1lMidShD1/W621n5/beyfj+EmwPmGbV4 UvZA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:dkim-signature; bh=O8s+HyGI3H7iBoQ4HT87wXqm8igpNkXMPnj6MfhLluw=; b=KZeT7k+bAbGFXUVEY0ftt8rAvV97StYWQUmtoAsG51gxXfrjmy6GPrcmcNhgrioAzJ PYL4WR6QLmQQXwyXRbBonmbvF1xzgArxK7V+o4S8ifGbdVFj64kMinDwbIB7oq8aGdtn XLm7dUl6hW8AjF9aGfKoMo1Ck42P1yKnH+Ew3wyYQjmm8ER5syMTA9cfWa7gkI5j5rq1 mk6wCB3EfUAFKv1fS9XPuEjsehzQ0GQR4dToU4DJKb26MEg7uHQTlBqzsQxmSGjmoWtK j7RDXDZ/gULoqrrzQzGgs/sXdmlZ0hrCE9994vtSIIfwgjkhYdkqNvZSjNRE4Mb4Hbcl cCng== ARC-Authentication-Results: i=1; gmr-mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=HmlHcaL4; spf=pass (google.com: domain of sean.m.mcdaniel-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org designates 2607:f8b0:4864:20::d2f as permitted sender) smtp.mailfrom=sean.m.mcdaniel-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Original-Received: from mail-io1-xd2f.google.com (mail-io1-xd2f.google.com. [2607:f8b0:4864:20::d2f]) by gmr-mx.google.com with ESMTPS id j13si68797iow.1.2021.08.13.10.01.33 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 13 Aug 2021 10:01:33 -0700 (PDT) Received-SPF: pass (google.com: domain of sean.m.mcdaniel-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org designates 2607:f8b0:4864:20::d2f as permitted sender) client-ip=2607:f8b0:4864:20::d2f; Original-Received: by mail-io1-xd2f.google.com with SMTP id y1so14160871iod.10 for ; Fri, 13 Aug 2021 10:01:33 -0700 (PDT) X-Received: by 2002:a6b:8d8a:: with SMTP id p132mr2735321iod.81.1628874093216; Fri, 13 Aug 2021 10:01:33 -0700 (PDT) In-Reply-To: X-Original-Sender: sean.m.mcdaniel-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org X-Original-Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=HmlHcaL4; spf=pass (google.com: domain of sean.m.mcdaniel-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org designates 2607:f8b0:4864:20::d2f as permitted sender) smtp.mailfrom=sean.m.mcdaniel-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Precedence: list Mailing-list: list pandoc-discuss-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org; contact pandoc-discuss+owners-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org List-ID: X-Google-Group-Id: 1007024079513 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , Xref: news.gmane.io gmane.text.pandoc:29023 Archived-At: --000000000000b5cafb05c973cdc1 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Thank you BP for the Lua filter and the explanation. I have attempted to run your Lua filter without modifications as a test and I get the following error: Error running filter meta-whitelist.lua: meta-whitelist.lua:114: attempt to get length of a nil value (local 'keys') stack traceback: meta-whitelist.lua:114: in function 'Meta' The issued pandoc command was (note that I put the Lua filter after the crossref filter): pandoc -s --markdown-headings=3Datx \ --filter=3Dpandoc-crossref --lua-filter=3Dmeta-whitelist.lua \ --from=3Dmarkdown+raw_attribute+yaml_metadata_block \ --to=3Dmarkdown_phpextra+yaml_metadata_block \ -o myfile_phpextra.md myfile.md I get the same error with or without the crossref filter. As you've explained (thanks) the metadata fields are created by the pandoc-crossref filter. The following without filters works as expected---both my YAML header and Markdown blocks are preserved: pandoc -s --markdown-headings=3Datx \ --from=3Dmarkdown+raw_attribute+yaml_metadata_block \ --to=3Dmarkdown_phpextra+yaml_metadata_block \ -o myfile_phpextra.md myfile.md The above will probably work for most of my needs, but I was wondering if there's a simple Lua filter fix to get it working for my case. Regards, Sean On Tue, Aug 10, 2021 at 3:47 PM John MacFarlane wrote: > > I see. Well, messing with templates won't help. > You could write a filter that is applied AFTER > pandoc-crossref and removes these metadata fields. > > BPJ writes: > > > They are pandoc-crossref configuration options, passed through the > metadata > > since that is the most convenient way to let the user "communicate" wit= h > > filters. > > > > Den tis 10 aug. 2021 20:09John MacFarlane skrev: > > > >> Sean McDaniel writes: > >> > >> > Thank you John. Your suggestions got me most of the way there: the > >> markdown > >> > code blocks with the +raw_attribute processing are correctly passing > >> > through the sections I want untouched and the +yaml_metadata_block i= s > >> > keeping the header content I want. > >> > > >> > The remaining issue which is giving me trouble is that > +yaml_header_block > >> > is also including all other document metadata and not just the > variables > >> > I've defined, i.e. stuff like > >> > > >> > autoEqnLabels: false > >> > autoSectionLabels: false > >> > ccsDelim: ", " > >> > ccsLabelSep: =E2=80=94 > >> > ccsTemplate: | > >> > *i* > >> > *c**c**s**L**a**b**e**l**S**e**p* > >> > *t* > >> > chapDelim: . > >> > > >> > and so forth. > >> > >> I'm not sure why these are going in the metadata -- what is > >> setting these metadata elements, if you're not doing it? > >> maybe I don't understand the issue. > >> > >> -- > >> You received this message because you are subscribed to the Google > Groups > >> "pandoc-discuss" group. > >> To unsubscribe from this group and stop receiving emails from it, send > an > >> email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org > >> To view this discussion on the web visit > >> > https://groups.google.com/d/msgid/pandoc-discuss/m21r71895k.fsf%40MacBook= -Pro-2.hsd1.ca.comcast.net > >> . > >> > > > > -- > > You received this message because you are subscribed to the Google > Groups "pandoc-discuss" group. > > To unsubscribe from this group and stop receiving emails from it, send > an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org > > To view this discussion on the web visit > https://groups.google.com/d/msgid/pandoc-discuss/CADAJKhBhk%3DM15HuN-4b%2= BbG8NCGBeS6A7YYa3Y2n3Dax22m8MJA%40mail.gmail.com > . > --=20 You received this message because you are subscribed to the Google Groups "= pandoc-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an e= mail to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To view this discussion on the web visit https://groups.google.com/d/msgid/= pandoc-discuss/CAML5wTWbn0S2Ps%2BROAOqjpHvv1ygfqWjx1FL0nj_2DiNSjjhdQ%40mail= .gmail.com. --000000000000b5cafb05c973cdc1 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Thank you BP for the Lua filter and the explanation. = I have attempted to run your Lua filter without modifications as a test and= I get the following error:

Error running filter meta-whi= telist.lua:
meta-whitelist.lua:114: attempt to get length of a nil val= ue (local 'keys')
stack traceback:
meta-whitelist.lua:1= 14: in function 'Meta'

The issued pandoc c= ommand was (note that I put the Lua filter after the crossref filter):

pandoc -s --markdown-headings=3Datx \
=C2=A0= =C2=A0=C2=A0 --filter=3Dpandoc-crossref --lua-filter=3Dmeta-whitelist.lua \=
=C2=A0=C2=A0=C2=A0 --from=3Dmarkdown+raw_attribute+yaml_metadata= _block \
=C2=A0=C2=A0=C2=A0 --to=3Dmarkdown_phpextra+yaml_met= adata_block \
=C2=A0=C2=A0=C2=A0 -o myfile_phpextra.md myfile.md<= /div>

I get the same error with or without the crossref = filter.

As you've explained (thanks) the metad= ata fields are created by the pandoc-crossref filter. The following without= filters works as expected---both my YAML header and Markdown blocks are pr= eserved:

pandoc -s --markdown-headings=3D= atx \
=C2=A0=C2=A0=C2=A0 --from=3Dmarkdown+raw_attribute+yaml_met= adata_block \
=C2=A0=C2=A0=C2=A0 --to=3Dmarkdown_phpextra+yam= l_metadata_block \
=C2=A0=C2=A0=C2=A0 -o myfile_phpextra.md myfil= e.md

The above will probably work for most o= f my needs, but I was wondering if there's a simple Lua filter fix to g= et it working for my case.

Regards,
=
Sean

On Tue, Aug 10, 2021 at 3:47 PM John MacFarla= ne <jgm@berkeley.e= du> wrote:
melroch@gmai= l.com> writes:

> They are pandoc-crossref configuration options, passed through the met= adata
> since that is the most convenient way to let the user "communicat= e" with
> filters.
>
> Den tis 10 aug. 2021 20:09John MacFarlane <jgm-TVLZxgkOlNX2fBVCVOL8/A@public.gmane.org> skrev:
>
>> Sean McDaniel <sean.m.mcdaniel-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> writes:
>>
>> > Thank you John. Your suggestions got me most of the way there= : the
>> markdown
>> > code blocks with the +raw_attribute processing are correctly = passing
>> > through the sections I want untouched and the +yaml_metadata_= block is
>> > keeping the header content I want.
>> >
>> > The remaining issue which is giving me trouble is that +yaml_= header_block
>> > is also including all other document metadata and not just th= e variables
>> > I've defined, i.e. stuff like
>> >
>> > autoEqnLabels: false
>> > autoSectionLabels: false
>> > ccsDelim: ", "
>> > ccsLabelSep:=C2=A0 =E2=80=94
>> > ccsTemplate: |
>> >=C2=A0 =C2=A0*i*
>> >=C2=A0 =C2=A0*c**c**s**L**a**b**e**l**S**e**p*
>> >=C2=A0 =C2=A0*t*
>> > chapDelim: .
>> >
>> > and so forth.
>>
>> I'm not sure why these are going in the metadata -- what is >> setting these metadata elements, if you're not doing it?
>> maybe I don't understand the issue.
>>
>> --
>> You received this message because you are subscribed to the Google= Groups
>> "pandoc-discuss" group.
>> To unsubscribe from this group and stop receiving emails from it, = send an
>> email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org.<= br> >> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/pandoc-discuss/m21r71895k.fsf%40M= acBook-Pro-2.hsd1.ca.comcast.net
>> .
>>
>
> --
> You received this message because you are subscribed to the Google Gro= ups "pandoc-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send= an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org.
> To view this discussion on the web visit https://= groups.google.com/d/msgid/pandoc-discuss/CADAJKhBhk%3DM15HuN-4b%2BbG8NCGBeS= 6A7YYa3Y2n3Dax22m8MJA%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups &= quot;pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an e= mail to pand= oc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org.
To view this discussion on the web visit https://group= s.google.com/d/msgid/pandoc-discuss/CAML5wTWbn0S2Ps%2BROAOqjpHvv1ygfqWjx1FL= 0nj_2DiNSjjhdQ%40mail.gmail.com.
--000000000000b5cafb05c973cdc1--