From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.io/gmane.text.pandoc/16620 Path: news.gmane.org!.POSTED!not-for-mail From: BPJ Newsgroups: gmane.text.pandoc Subject: Re: Filter with arguments Date: Fri, 06 Jan 2017 19:47:45 +0000 Message-ID: References: <20170106114647.GG791@Administrateurs-iMac-3.local> <8760ls8jby.fsf@fastmail.fm> <20170106125852.GA1662@Administrateurs-iMac-3.local> Reply-To: pandoc-discuss-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: multipart/alternative; boundary=001a1141852c522f7b0545724bd0 X-Trace: blaine.gmane.org 1483732085 3432 195.159.176.226 (6 Jan 2017 19:48:05 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Fri, 6 Jan 2017 19:48:05 +0000 (UTC) To: pandoc-discuss-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org Original-X-From: pandoc-discuss+bncBCWMVYEK54FRB3PIX7BQKGQEQLVPI3Q-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org Fri Jan 06 20:48:00 2017 Return-path: Envelope-to: gtp-pandoc-discuss@m.gmane.org Original-Received: from mail-lf0-f60.google.com ([209.85.215.60]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cPaUT-0007yc-83 for gtp-pandoc-discuss@m.gmane.org; Fri, 06 Jan 2017 20:47:53 +0100 Original-Received: by mail-lf0-f60.google.com with SMTP id o12sf4876779lfg.1 for ; Fri, 06 Jan 2017 11:47:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20161025; h=sender:mime-version:references:in-reply-to:from:date:message-id :subject:to:x-original-sender:x-original-authentication-results :reply-to:precedence:mailing-list:list-id:x-spam-checked-in-group :list-post:list-help:list-archive:list-subscribe:list-unsubscribe; bh=UDLFSF44F7fbpExng9qU+rGMgjY4kwmavp73Hs5UCfA=; b=kVfXu4sJaqTyG9xeQTnyds/9NlCMxmWZMnDVUweuq0TVPS9xC5e8jQavswnp9btDdO JJFuc2unAlwxAiY+sQsAlij2i1Uuh5u01EQyQj6bmeUfk1bECF1UMmms+BltQS7+8PUx SUigC0nztFq3q63vb6IjJzN/27m3CbhYLouYnXMmy7oa/WLp26o2wtqe8Ujl9P/zNoSV lPMSurP8FFu4JFg2hd0l2OxYUVeoscSfleXPO/ln3B43Q8vPYpN5Fhaa77HMFWRCTwqk hPpiRnGRJFiVcz/SzTDQeqwuSibIFb/mTOOX1L8RMQfhKCMaK9yjJ8exjmuHNdMYtyNZ EUfA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=sender:x-gm-message-state:mime-version:references:in-reply-to:from :date:message-id:subject:to:x-original-sender :x-original-authentication-results:reply-to:precedence:mailing-list :list-id:x-spam-checked-in-group:list-post:list-help:list-archive :list-subscribe:list-unsubscribe; bh=UDLFSF44F7fbpExng9qU+rGMgjY4kwmavp73Hs5UCfA=; b=I2qzcyfXlFfSgeE0zfTmgXlkt1H3EP1XKJsqtQMRKLhKYtXe2LgJuHwhinZopgFfjb 8sFweDvAR/Riqwsid+XIb4MGLGapGqy+nR191e+x+94R+Wee2E9MmsxIRvqLh7olJxul A7P5nbrg6gozy/qqim2nfKTaY0ghcEVMjLvNfwOneuzIQD9i3RnyCYTuI5rw80arXPq4 aFkJGkPzleBOHwgGl4Iw740c2+vl7J354aJw4danTMozVbALDHqc73VeGAfubrOuoPKQ hq6otRUw64a6WGOHTWzfm/1FthAaSeCsusnkotvGi20gcmxzywL4TrMF3bitWczQC1Vw luFA== Original-Sender: pandoc-discuss-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org X-Gm-Message-State: AIkVDXIeQdvVeTvYJD4BI0VXG2cqgmo9SeSXsO5rcg7c5GSHa0Zo+E0OtMQno5xEHZJHQg== X-Received: by 10.46.5.193 with SMTP id 184mr695981ljf.18.1483732077853; Fri, 06 Jan 2017 11:47:57 -0800 (PST) X-BeenThere: pandoc-discuss-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org Original-Received: by 10.46.75.10 with SMTP id y10ls3386704lja.26.gmail; Fri, 06 Jan 2017 11:47:56 -0800 (PST) X-Received: by 10.25.145.13 with SMTP id t13mr6550652lfd.5.1483732076825; Fri, 06 Jan 2017 11:47:56 -0800 (PST) Original-Received: from mail-lf0-f67.google.com (mail-lf0-f67.google.com. [209.85.215.67]) by gmr-mx.google.com with ESMTPS id f71si171894wmf.2.2017.01.06.11.47.56 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 06 Jan 2017 11:47:56 -0800 (PST) Received-SPF: pass (google.com: domain of melroch-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org designates 209.85.215.67 as permitted sender) client-ip=209.85.215.67; Original-Received: by mail-lf0-f67.google.com with SMTP id k62so4057904lfg.0 for ; Fri, 06 Jan 2017 11:47:56 -0800 (PST) X-Received: by 10.25.207.196 with SMTP id f187mr28184307lfg.15.1483732075817; Fri, 06 Jan 2017 11:47:55 -0800 (PST) In-Reply-To: X-Original-Sender: melroch-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of melroch-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org designates 209.85.215.67 as permitted sender) smtp.mailfrom=melroch-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org Precedence: list Mailing-list: list pandoc-discuss-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org; contact pandoc-discuss+owners-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org List-ID: X-Google-Group-Id: 1007024079513 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , Xref: news.gmane.org gmane.text.pandoc:16620 Archived-At: --001a1141852c522f7b0545724bd0 Content-Type: text/plain; charset=UTF-8 It would work but it would need to be `-F myfilter --filter-option myfilter='-foo'` so that pandoc sends the right options to the right filter when running multiple filters. I still think '-M myfilter_foo=bar` is not only sufficient but better. What we need is a way to specify non-markdown metadata, e.g. that values of keys beginning with an underscore are non-markdown. That's no big deal though. Filters can convert their metadata options to plain data themselves. /bpj fre 6 jan. 2017 kl. 19:43 skrev Matthew Pickering < matthewtpickering-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>: > Communicating arguments though env vars is very ad-hoc, an option > > "--filter-arguments" (like Joost suggests) which took a string and > > passed it to the filter when invoked would be sufficient. > > > > pandoc --filter myfilter --filter-options="--foo --baz --bar" > > > > results in pandoc calling > > > > myfilter csv --foo --baz --bar > > > > I investigated reimplementing pandoc-lit as a filter but it was not > > possible because the program has a lot of command line arguments you > > need to specify. > > > > The worry about passing the format as the first argument being > > confusing could be overcome by embedding more metadata into the JSON > > input. > > > > Matt > > > > On Fri, Jan 6, 2017 at 6:32 PM, BPJ wrote: > > > The best way to pass options to filters is through metadata. That gets > > > considerably easier if the filter can use a function which converts a > > > structure of metadata objects into a plain data structure so that > `MetsMap`s > > > get converted to plain associative arrays, `MetaList`s to plain arrays > and > > > other `Meta` elements to plain strings or booleans. It is easy to write > such > > > a function using existing walking and steingification functions. > > > > > > Once you get the hang of that you don't feel any need to pass custom > > > arguments to filters because it is so convenient to do filter config in > the > > > document. A useful way to avoid a lot of metadata keys prefixed with > filter > > > names is to have a single key equal to the filter name with a mapping as > > > value which contains the options for that filter. Nothing hinders you > from > > > also checking environment variables if metadata options are undefined and > > > use hardcoded defaults only if the env vars are also undefined. I do so > > > often enough to have written a helper function. > > > > > > /bpj > > > > > > fre 6 jan. 2017 kl. 14:00 skrev John MacFarlane : > > >> > > >> +++ Joost Kremers [Jan 06 17 13:27 ]: > > >> > > >> > > > >> > > >> >On Fri, Jan 06 2017, John MacFarlane wrote: > > >> > > >> >>This is a pretty common request. A couple of complications: > > >> > > >> >> > > >> > > >> >>1. Pandoc calls each filter with one argument, the name of > > >> > > >> >>the output format. (Filters thus have access to this.) > > >> > > >> >>If we allowed `-F "pantable -t csv"`, how would this work? > > >> > > >> >>Would pandoc call `pantable latex -t csv`? Would the filter > > >> > > >> >>be expecting this? > > >> > > >> > > > >> > > >> >For a properly written filter that shouldn't be a problem, should it? > > >> > > >> >You just need to decide on one way of doing it and stick to it. > > >> > > >> >Filters then need to be updated accordingly. Or are you worried about > > >> > > >> >backward compatibility? > > >> > > >> > > >> > > >> Well, I'm anticipating people getting confused. "I said to > > >> > > >> run --filter 'myfilter myarg', and for some reason 'myarg' > > >> > > >> isn't the first argument!" > > >> > > >> > > >> > > >> What's wrong with communicating options to filters through > > >> > > >> environment variables, which you can already do? > > >> > > >> > > >> > > >> PANTABLE_OUTPUT="csv" pandoc --filter pantable > > >> > > >> > > >> > > >> >pandoc --filter "My Cool Filter.py" --filter-args "-t csv" > > >> > > >> > > > >> > > >> >with each --filter-args referring to the immediately preceding > > >> > > >> >--filters option. Depending on how Pandoc calls filters, it may even > > >> > > >> >be possible to just pass it the literal string argument of > > >> > > >> >--filter-args without having to parse it into different options. > > >> > > >> > > >> > > >> I think this is too complex, with --filter-args affecting > > >> > > >> the immediately preceding --filter option. > > >> > > >> > > >> > > >> Again, I'd have to be convinced that environment variables > > >> > > >> don't already provide a perfectly adequate way to do this. > > >> > > >> > > >> > > >> -- > > >> > > >> You received this message because you are subscribed to the Google > Groups > > >> "pandoc-discuss" group. > > >> > > >> To unsubscribe from this group and stop receiving emails from it, send > an > > >> email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org > > >> > > >> To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org > > >> > > >> To view this discussion on the web visit > > >> > https://groups.google.com/d/msgid/pandoc-discuss/20170106125852.GA1662%40Administrateurs-iMac-3.local > . > > >> > > >> For more options, visit https://groups.google.com/d/optout. > > >> > > > -- > > > You received this message because you are subscribed to the Google Groups > > > "pandoc-discuss" group. > > > To unsubscribe from this group and stop receiving emails from it, send an > > > email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org > > > To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org > > > To view this discussion on the web visit > > > > https://groups.google.com/d/msgid/pandoc-discuss/CADAJKhCPn81DsWpzWSQHv59ii%2BSLXmz4Ww5ru1mSmP4eRQozow%40mail.gmail.com > . > > > > > > For more options, visit https://groups.google.com/d/optout. > > > > -- > > You received this message because you are subscribed to the Google Groups > "pandoc-discuss" group. > > To unsubscribe from this group and stop receiving emails from it, send an > email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org > > To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org > > To view this discussion on the web visit > https://groups.google.com/d/msgid/pandoc-discuss/CALuQ0m8Jx2N_nKKOL5i5pNM9houf_iUe2Z54uWeajm2oVEZW5w%40mail.gmail.com > . > > For more options, visit https://groups.google.com/d/optout. > > -- You received this message because you are subscribed to the Google Groups "pandoc-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/CADAJKhCf3X7ETGQ2DJpZ3Y5gWQid2mUGWJCrpdmpJKWXWb6XBw%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout. --001a1141852c522f7b0545724bd0 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
It would work but it would need to be `-F myfilter --filter-option myf= ilter=3D'-foo'` so that pandoc sends the right options to the right= filter when running multiple filters. I still think '-M myfilter_foo= =3Dbar` is not only sufficient but better. What we need is a way to specify= non-markdown metadata, e.g. that values of keys beginning with an undersco= re are non-markdown. That's no big deal though. Filters can convert the= ir metadata options to plain data themselves.

/bpj=

fre 6 jan. 2017 kl. 19:43 sk= rev Matthew Pickering <ma= tthewtpickering-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>:
= Communicating arguments though env vars is very ad-hoc, an option

"--filter-arguments" (like Joost suggests) whi= ch took a string and

passed it to the filter whe= n invoked would be sufficient.



pandoc --filter myfilter --filter-options=3D"--foo --baz --= bar"



results in= pandoc calling



myfi= lter csv --foo --baz --bar



I investigated reimplementing pandoc-lit as a filter but it was not<= br class=3D"gmail_msg">
possible because the program has a lot of comman= d line arguments you

need to specify.



The worry about passing the = format as the first argument being

confusing cou= ld be overcome by embedding more metadata into the JSON

input.



Matt<= br class=3D"gmail_msg">


On Fri, Jan 6, 2017 = at 6:32 PM, BPJ <bpj-J3H7GcXPSITLoDKTGw+V6w@public.gmane.org> wrote:

&= gt; The best way to pass options to filters is through metadata. That gets<= br class=3D"gmail_msg">
> considerably easier if the filter can use a= function which converts a

> structure of met= adata objects into a plain data structure so that `MetsMap`s

> get converted to plain associative arrays, `MetaList`s to= plain arrays and

> other `Meta` elements to = plain strings or booleans. It is easy to write such
=
> a function using existing walking and steingification functions.
>

> Once you get= the hang of that you don't feel any need to pass custom

> arguments to filters because it is so convenient to do fi= lter config in the

> document. A useful way t= o avoid a lot of metadata keys prefixed with filter
=
> names is to have a single key equal to the filter name with a mapp= ing as

> value which contains the options for= that filter. Nothing hinders you from

> also= checking environment variables if metadata options are undefined and

> use hardcoded defaults only if the env vars are = also undefined. I do so

> often enough to hav= e written a helper function.

>

> /bpj

>

> fre 6 jan. 2017 kl. 14:00 skrev John MacFarlane <jgm@berkeley= .edu>:

>>
>> +++ Joost Kremers [Jan 06 17 13:27 ]:
>>

>> >
<= br>>>

>> >On Fri, Jan 06 2017, Jo= hn MacFarlane wrote:

>>

>> >>This is a pretty common request.=C2=A0 A couple o= f complications:

>>

>> >>

>>

>> >>1.=C2=A0 Pandoc calls each filter with one argu= ment, the name of

>>

>> >>the output format.=C2=A0 (Filters thus have access t= o this.)

>>

>= ;> >>If we allowed `-F "pantable -t csv"`, how would thi= s work?

>>

>= > >>Would pandoc call `pantable latex -t csv`?=C2=A0 Would the fil= ter

>>

>>= >>be expecting this?

>>

>> >

>>

>> >For a properly written filter that shouldn&= #39;t be a problem, should it?

>>

>> >You just need to decide on one way of doing= it and stick to it.

>>

>> >Filters then need to be updated accordingly. Or are y= ou worried about

>>

>> >backward compatibility?

>&g= t;

>>

>><= br class=3D"gmail_msg">
>> Well, I'm anticipating people getti= ng confused.=C2=A0 "I said to

>>

>> run --filter 'myfilter myarg', and = for some reason 'myarg'

>>

>> isn't the first argument!"

>>

>>

>>

>> What's wro= ng with communicating options to filters through
>>

>> environment variables, which = you can already do?

>>

>>

>>

>>=C2=A0 =C2=A0 =C2=A0PANTABLE_OUTPUT=3D"csv" pandoc -= -filter pantable

>>

>>

>>
>> >pandoc --filter "My Cool Filter.py" --filter-args = "-t csv"

>>

>> >

>>

>> >with each --filter-args referring to the immediately = preceding

>>

&g= t;> >--filters option. Depending on how Pandoc calls filters, it may = even

>>

>>= ; >be possible to just pass it the literal string argument of

>>

>> >--filte= r-args without having to parse it into different options.

>>

>>

>>

>> I think this is too com= plex, with --filter-args affecting

>>

>> the immediately preceding --filter option.<= br class=3D"gmail_msg">
>>

>>

>>

>> Again,= I'd have to be convinced that environment variables

>>

>> don't already pro= vide a perfectly adequate way to do this.

>&g= t;

>>

>><= br class=3D"gmail_msg">
>> --

>><= br class=3D"gmail_msg">
>> You received this message because you a= re subscribed to the Google Groups

>> &quo= t;pandoc-discuss" group.

>>

>> To unsubscribe from this group and stop receivi= ng emails from it, send an

>> email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org.
>>

>> To p= ost to this group, send email to pandoc-discuss@googlegroups.= com.

>>

>= ;> To view this discussion on the web visit

&= gt;> https://groups.google.com/d/msgid/pandoc-discu= ss/20170106125852.GA1662%40Administrateurs-iMac-3.local.

>>

>> For more options,= visit https://groups.google.com/d/optout.=

>>

> --

> You received this message because you are subs= cribed to the Google Groups

> "pandoc-di= scuss" group.

> To unsubscribe from this= group and stop receiving emails from it, send an
> email to pandoc-discuss+unsubscribe@goog= legroups.com.

> To post to this group, se= nd email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org.

> To view this discussion on the web visit

> https://groups.goo= gle.com/d/msgid/pandoc-discuss/CADAJKhCPn81DsWpzWSQHv59ii%2BSLXmz4Ww5ru1mSm= P4eRQozow%40mail.gmail.com.

>

> For more options, visit h= ttps://groups.google.com/d/optout.



--

You received this message = because you are subscribed to the Google Groups "pandoc-discuss" = group.

To unsubscribe from this group and stop r= eceiving emails from it, send an email to pando= c-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org.

To p= ost to this group, send email to pandoc-discuss@googlegroups.= com.

To view this discussion on the web visi= t https://groups.google.com/d/msgid/pa= ndoc-discuss/CALuQ0m8Jx2N_nKKOL5i5pNM9houf_iUe2Z54uWeajm2oVEZW5w%40mail.gma= il.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups &= quot;pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an e= mail to pand= oc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org.
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org.
To view this discussion on the web visit https://groups.= google.com/d/msgid/pandoc-discuss/CADAJKhCf3X7ETGQ2DJpZ3Y5gWQid2mUGWJCrpdmp= JKWXWb6XBw%40mail.gmail.com.
For more options, visit http= s://groups.google.com/d/optout.
--001a1141852c522f7b0545724bd0--