* [Feature Request] Filter with arguments @ 2017-01-06 4:32 Kolen Cheung [not found] ` <e8cf9c92-7aa7-4841-85d0-7464c771cf6e-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org> 0 siblings, 1 reply; 20+ messages in thread From: Kolen Cheung @ 2017-01-06 4:32 UTC (permalink / raw) To: pandoc-discuss [-- Attachment #1.1: Type: text/plain, Size: 1494 bytes --] Currently, if one need to pass arguments to a filter, one need to pipe it through. It would be convenient in some cases to allow passing argument to filters. e.g. pandoc -F "pantable -t csv" -s -o example.tex example.md# which is equiv. to pandoc -t json -s example.md | pantable -t csv | pandoc -f json -s -o example.tex To elaborate a little bit further: I want to write a filter that at the same time do: 1. convert code block to table 2. convert pandoc tables to code block, potentially with different output code formats (e.g. csv, yaml, json, etc. or even other kinds e.g. HTML, Div with CSS table, etc.) 3. standalone cli. i.e. not a filter Right now (1) and (2) are mutually exclusive and has to be split into 2 filters. (1) and (3) can potentially be written in the same executable, but it is uglier and pointless to do so without including (2). -- You received this message because you are subscribed to the Google Groups "pandoc-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/e8cf9c92-7aa7-4841-85d0-7464c771cf6e%40googlegroups.com. For more options, visit https://groups.google.com/d/optout. [-- Attachment #1.2: Type: text/html, Size: 5866 bytes --] ^ permalink raw reply [flat|nested] 20+ messages in thread
[parent not found: <e8cf9c92-7aa7-4841-85d0-7464c771cf6e-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org>]
* Re: Filter with arguments [not found] ` <e8cf9c92-7aa7-4841-85d0-7464c771cf6e-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org> @ 2017-01-06 11:46 ` John MacFarlane [not found] ` <20170106114647.GG791-BKjuZOBx5Kn2N3qrpRCZGbhGAdq7xJNKhPhL2mjWHbk@public.gmane.org> 2017-01-08 20:51 ` [Feature Request] " Kolen Cheung 2018-06-20 14:58 ` Peter Zagubisalo 2 siblings, 1 reply; 20+ messages in thread From: John MacFarlane @ 2017-01-06 11:46 UTC (permalink / raw) To: pandoc-discuss-/JYPxA39Uh5TLH3MbocFFw This is a pretty common request. A couple of complications: 1. Pandoc calls each filter with one argument, the name of the output format. (Filters thus have access to this.) If we allowed `-F "pantable -t csv"`, how would this work? Would pandoc call `pantable latex -t csv`? Would the filter be expecting this? 2. We'd get a quoted argument like `"pantable -t csv"` as a single string from the environment. So pandoc would have to do its own parsing of this string into a command, options and arguments. This is tricky. Maybe some people are already using filters with spaces in them, like `-F "My Cool Filter.py". This kind of thing would break if pandoc assumed that the command was "My" and "Cool" an argument. +++ Kolen Cheung [Jan 05 17 20:32 ]: > Currently, if one need to pass arguments to a filter, one need to pipe > it through. > > It would be convenient in some cases to allow passing argument to > filters. e.g. >pandoc -F "pantable -t csv" -s -o example.tex example.md ># which is equiv. to >pandoc -t json -s example.md | pantable -t csv | pandoc -f json -s -o example.te >x > > To elaborate a little bit further: I want to write a filter that at the > same time do: > 1. convert code block to table > 2. convert pandoc tables to code block, potentially with different > output code formats (e.g. csv, yaml, json, etc. or even other kinds > e.g. HTML, Div with CSS table, etc.) > 3. standalone cli. i.e. not a filter > > Right now (1) and (2) are mutually exclusive and has to be split into 2 > filters. (1) and (3) can potentially be written in the same executable, > but it is uglier and pointless to do so without including (2). > > > -- > You received this message because you are subscribed to the Google > Groups "pandoc-discuss" group. > To unsubscribe from this group and stop receiving emails from it, send > an email to [1]pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org > To post to this group, send email to > [2]pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org > To view this discussion on the web visit > [3]https://groups.google.com/d/msgid/pandoc-discuss/e8cf9c92-7aa7-4841- > 85d0-7464c771cf6e%40googlegroups.com. > For more options, visit [4]https://groups.google.com/d/optout. > >References > > 1. mailto:pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org > 2. mailto:pandoc-discuss-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org > 3. https://groups.google.com/d/msgid/pandoc-discuss/e8cf9c92-7aa7-4841-85d0-7464c771cf6e-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org?utm_medium=email&utm_source=footer > 4. https://groups.google.com/d/optout -- You received this message because you are subscribed to the Google Groups "pandoc-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/20170106114647.GG791%40Administrateurs-iMac-3.local. For more options, visit https://groups.google.com/d/optout. ^ permalink raw reply [flat|nested] 20+ messages in thread
[parent not found: <20170106114647.GG791-BKjuZOBx5Kn2N3qrpRCZGbhGAdq7xJNKhPhL2mjWHbk@public.gmane.org>]
* Re: Filter with arguments [not found] ` <20170106114647.GG791-BKjuZOBx5Kn2N3qrpRCZGbhGAdq7xJNKhPhL2mjWHbk@public.gmane.org> @ 2017-01-06 12:27 ` Joost Kremers [not found] ` <8760ls8jby.fsf-97jfqw80gc6171pxa8y+qA@public.gmane.org> 2017-01-06 23:10 ` Kolen Cheung 1 sibling, 1 reply; 20+ messages in thread From: Joost Kremers @ 2017-01-06 12:27 UTC (permalink / raw) To: pandoc-discuss-/JYPxA39Uh5TLH3MbocFFw On Fri, Jan 06 2017, John MacFarlane wrote: > This is a pretty common request. A couple of complications: > > 1. Pandoc calls each filter with one argument, the name of > the output format. (Filters thus have access to this.) > If we allowed `-F "pantable -t csv"`, how would this work? > Would pandoc call `pantable latex -t csv`? Would the filter > be expecting this? For a properly written filter that shouldn't be a problem, should it? You just need to decide on one way of doing it and stick to it. Filters then need to be updated accordingly. Or are you worried about backward compatibility? > 2. We'd get a quoted argument like `"pantable -t csv"` as a > single string from the environment. So pandoc would have to > do its own parsing of this string into a command, options and > arguments. This is tricky. Maybe some people are already > using filters with spaces in them, like `-F "My Cool Filter.py". > This kind of thing would break if pandoc assumed that the > command was "My" and "Cool" an argument. Perhaps the way to overcome this problem is to introduce another option, e.g., --filter-args, so you'd get: pandoc --filter "My Cool Filter.py" --filter-args "-t csv" with each --filter-args referring to the immediately preceding --filters option. Depending on how Pandoc calls filters, it may even be possible to just pass it the literal string argument of --filter-args without having to parse it into different options. -- Joost Kremers Life has its moments ^ permalink raw reply [flat|nested] 20+ messages in thread
[parent not found: <8760ls8jby.fsf-97jfqw80gc6171pxa8y+qA@public.gmane.org>]
* Re: Filter with arguments [not found] ` <8760ls8jby.fsf-97jfqw80gc6171pxa8y+qA@public.gmane.org> @ 2017-01-06 12:58 ` John MacFarlane [not found] ` <20170106125852.GA1662-BKjuZOBx5Kn2N3qrpRCZGbhGAdq7xJNKhPhL2mjWHbk@public.gmane.org> 0 siblings, 1 reply; 20+ messages in thread From: John MacFarlane @ 2017-01-06 12:58 UTC (permalink / raw) To: pandoc-discuss-/JYPxA39Uh5TLH3MbocFFw +++ Joost Kremers [Jan 06 17 13:27 ]: > >On Fri, Jan 06 2017, John MacFarlane wrote: >>This is a pretty common request. A couple of complications: >> >>1. Pandoc calls each filter with one argument, the name of >>the output format. (Filters thus have access to this.) >>If we allowed `-F "pantable -t csv"`, how would this work? >>Would pandoc call `pantable latex -t csv`? Would the filter >>be expecting this? > >For a properly written filter that shouldn't be a problem, should it? >You just need to decide on one way of doing it and stick to it. >Filters then need to be updated accordingly. Or are you worried about >backward compatibility? Well, I'm anticipating people getting confused. "I said to run --filter 'myfilter myarg', and for some reason 'myarg' isn't the first argument!" What's wrong with communicating options to filters through environment variables, which you can already do? PANTABLE_OUTPUT="csv" pandoc --filter pantable >pandoc --filter "My Cool Filter.py" --filter-args "-t csv" > >with each --filter-args referring to the immediately preceding >--filters option. Depending on how Pandoc calls filters, it may even >be possible to just pass it the literal string argument of >--filter-args without having to parse it into different options. I think this is too complex, with --filter-args affecting the immediately preceding --filter option. Again, I'd have to be convinced that environment variables don't already provide a perfectly adequate way to do this. ^ permalink raw reply [flat|nested] 20+ messages in thread
[parent not found: <20170106125852.GA1662-BKjuZOBx5Kn2N3qrpRCZGbhGAdq7xJNKhPhL2mjWHbk@public.gmane.org>]
* Re: Filter with arguments [not found] ` <20170106125852.GA1662-BKjuZOBx5Kn2N3qrpRCZGbhGAdq7xJNKhPhL2mjWHbk@public.gmane.org> @ 2017-01-06 18:32 ` BPJ [not found] ` <CADAJKhCPn81DsWpzWSQHv59ii+SLXmz4Ww5ru1mSmP4eRQozow-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org> 2017-01-08 19:40 ` Joost Kremers 1 sibling, 1 reply; 20+ messages in thread From: BPJ @ 2017-01-06 18:32 UTC (permalink / raw) To: pandoc-discuss-/JYPxA39Uh5TLH3MbocFFw [-- Attachment #1: Type: text/plain, Size: 3939 bytes --] The best way to pass options to filters is through metadata. That gets considerably easier if the filter can use a function which converts a structure of metadata objects into a plain data structure so that `MetsMap`s get converted to plain associative arrays, `MetaList`s to plain arrays and other `Meta` elements to plain strings or booleans. It is easy to write such a function using existing walking and steingification functions. Once you get the hang of that you don't feel any need to pass custom arguments to filters because it is so convenient to do filter config in the document. A useful way to avoid a lot of metadata keys prefixed with filter names is to have a single key equal to the filter name with a mapping as value which contains the options for that filter. Nothing hinders you from also checking environment variables if metadata options are undefined and use hardcoded defaults only if the env vars are also undefined. I do so often enough to have written a helper function. /bpj fre 6 jan. 2017 kl. 14:00 skrev John MacFarlane <jgm-TVLZxgkOlNX2fBVCVOL8/A@public.gmane.org>: > +++ Joost Kremers [Jan 06 17 13:27 ]: > > > > > >On Fri, Jan 06 2017, John MacFarlane wrote: > > >>This is a pretty common request. A couple of complications: > > >> > > >>1. Pandoc calls each filter with one argument, the name of > > >>the output format. (Filters thus have access to this.) > > >>If we allowed `-F "pantable -t csv"`, how would this work? > > >>Would pandoc call `pantable latex -t csv`? Would the filter > > >>be expecting this? > > > > > >For a properly written filter that shouldn't be a problem, should it? > > >You just need to decide on one way of doing it and stick to it. > > >Filters then need to be updated accordingly. Or are you worried about > > >backward compatibility? > > > > Well, I'm anticipating people getting confused. "I said to > > run --filter 'myfilter myarg', and for some reason 'myarg' > > isn't the first argument!" > > > > What's wrong with communicating options to filters through > > environment variables, which you can already do? > > > > PANTABLE_OUTPUT="csv" pandoc --filter pantable > > > > >pandoc --filter "My Cool Filter.py" --filter-args "-t csv" > > > > > >with each --filter-args referring to the immediately preceding > > >--filters option. Depending on how Pandoc calls filters, it may even > > >be possible to just pass it the literal string argument of > > >--filter-args without having to parse it into different options. > > > > I think this is too complex, with --filter-args affecting > > the immediately preceding --filter option. > > > > Again, I'd have to be convinced that environment variables > > don't already provide a perfectly adequate way to do this. > > > > -- > > You received this message because you are subscribed to the Google Groups > "pandoc-discuss" group. > > To unsubscribe from this group and stop receiving emails from it, send an > email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org > > To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org > > To view this discussion on the web visit > https://groups.google.com/d/msgid/pandoc-discuss/20170106125852.GA1662%40Administrateurs-iMac-3.local > . > > For more options, visit https://groups.google.com/d/optout. > > -- You received this message because you are subscribed to the Google Groups "pandoc-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/CADAJKhCPn81DsWpzWSQHv59ii%2BSLXmz4Ww5ru1mSmP4eRQozow%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout. [-- Attachment #2: Type: text/html, Size: 6170 bytes --] ^ permalink raw reply [flat|nested] 20+ messages in thread
[parent not found: <CADAJKhCPn81DsWpzWSQHv59ii+SLXmz4Ww5ru1mSmP4eRQozow-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>]
* Re: Filter with arguments [not found] ` <CADAJKhCPn81DsWpzWSQHv59ii+SLXmz4Ww5ru1mSmP4eRQozow-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org> @ 2017-01-06 18:42 ` Matthew Pickering [not found] ` <CALuQ0m8Jx2N_nKKOL5i5pNM9houf_iUe2Z54uWeajm2oVEZW5w-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org> 0 siblings, 1 reply; 20+ messages in thread From: Matthew Pickering @ 2017-01-06 18:42 UTC (permalink / raw) To: pandoc-discuss-/JYPxA39Uh5TLH3MbocFFw Communicating arguments though env vars is very ad-hoc, an option "--filter-arguments" (like Joost suggests) which took a string and passed it to the filter when invoked would be sufficient. pandoc --filter myfilter --filter-options="--foo --baz --bar" results in pandoc calling myfilter csv --foo --baz --bar I investigated reimplementing pandoc-lit as a filter but it was not possible because the program has a lot of command line arguments you need to specify. The worry about passing the format as the first argument being confusing could be overcome by embedding more metadata into the JSON input. Matt On Fri, Jan 6, 2017 at 6:32 PM, BPJ <bpj-J3H7GcXPSITLoDKTGw+V6w@public.gmane.org> wrote: > The best way to pass options to filters is through metadata. That gets > considerably easier if the filter can use a function which converts a > structure of metadata objects into a plain data structure so that `MetsMap`s > get converted to plain associative arrays, `MetaList`s to plain arrays and > other `Meta` elements to plain strings or booleans. It is easy to write such > a function using existing walking and steingification functions. > > Once you get the hang of that you don't feel any need to pass custom > arguments to filters because it is so convenient to do filter config in the > document. A useful way to avoid a lot of metadata keys prefixed with filter > names is to have a single key equal to the filter name with a mapping as > value which contains the options for that filter. Nothing hinders you from > also checking environment variables if metadata options are undefined and > use hardcoded defaults only if the env vars are also undefined. I do so > often enough to have written a helper function. > > /bpj > > fre 6 jan. 2017 kl. 14:00 skrev John MacFarlane <jgm-TVLZxgkOlNX2fBVCVOL8/A@public.gmane.org>: >> >> +++ Joost Kremers [Jan 06 17 13:27 ]: >> >> > >> >> >On Fri, Jan 06 2017, John MacFarlane wrote: >> >> >>This is a pretty common request. A couple of complications: >> >> >> >> >> >>1. Pandoc calls each filter with one argument, the name of >> >> >>the output format. (Filters thus have access to this.) >> >> >>If we allowed `-F "pantable -t csv"`, how would this work? >> >> >>Would pandoc call `pantable latex -t csv`? Would the filter >> >> >>be expecting this? >> >> > >> >> >For a properly written filter that shouldn't be a problem, should it? >> >> >You just need to decide on one way of doing it and stick to it. >> >> >Filters then need to be updated accordingly. Or are you worried about >> >> >backward compatibility? >> >> >> >> Well, I'm anticipating people getting confused. "I said to >> >> run --filter 'myfilter myarg', and for some reason 'myarg' >> >> isn't the first argument!" >> >> >> >> What's wrong with communicating options to filters through >> >> environment variables, which you can already do? >> >> >> >> PANTABLE_OUTPUT="csv" pandoc --filter pantable >> >> >> >> >pandoc --filter "My Cool Filter.py" --filter-args "-t csv" >> >> > >> >> >with each --filter-args referring to the immediately preceding >> >> >--filters option. Depending on how Pandoc calls filters, it may even >> >> >be possible to just pass it the literal string argument of >> >> >--filter-args without having to parse it into different options. >> >> >> >> I think this is too complex, with --filter-args affecting >> >> the immediately preceding --filter option. >> >> >> >> Again, I'd have to be convinced that environment variables >> >> don't already provide a perfectly adequate way to do this. >> >> >> >> -- >> >> You received this message because you are subscribed to the Google Groups >> "pandoc-discuss" group. >> >> To unsubscribe from this group and stop receiving emails from it, send an >> email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org >> >> To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org >> >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/pandoc-discuss/20170106125852.GA1662%40Administrateurs-iMac-3.local. >> >> For more options, visit https://groups.google.com/d/optout. >> > -- > You received this message because you are subscribed to the Google Groups > "pandoc-discuss" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org > To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org > To view this discussion on the web visit > https://groups.google.com/d/msgid/pandoc-discuss/CADAJKhCPn81DsWpzWSQHv59ii%2BSLXmz4Ww5ru1mSmP4eRQozow%40mail.gmail.com. > > For more options, visit https://groups.google.com/d/optout. ^ permalink raw reply [flat|nested] 20+ messages in thread
[parent not found: <CALuQ0m8Jx2N_nKKOL5i5pNM9houf_iUe2Z54uWeajm2oVEZW5w-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>]
* Re: Filter with arguments [not found] ` <CALuQ0m8Jx2N_nKKOL5i5pNM9houf_iUe2Z54uWeajm2oVEZW5w-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org> @ 2017-01-06 19:47 ` BPJ 2017-01-06 21:11 ` John MacFarlane 1 sibling, 0 replies; 20+ messages in thread From: BPJ @ 2017-01-06 19:47 UTC (permalink / raw) To: pandoc-discuss-/JYPxA39Uh5TLH3MbocFFw [-- Attachment #1: Type: text/plain, Size: 7167 bytes --] It would work but it would need to be `-F myfilter --filter-option myfilter='-foo'` so that pandoc sends the right options to the right filter when running multiple filters. I still think '-M myfilter_foo=bar` is not only sufficient but better. What we need is a way to specify non-markdown metadata, e.g. that values of keys beginning with an underscore are non-markdown. That's no big deal though. Filters can convert their metadata options to plain data themselves. /bpj fre 6 jan. 2017 kl. 19:43 skrev Matthew Pickering < matthewtpickering-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>: > Communicating arguments though env vars is very ad-hoc, an option > > "--filter-arguments" (like Joost suggests) which took a string and > > passed it to the filter when invoked would be sufficient. > > > > pandoc --filter myfilter --filter-options="--foo --baz --bar" > > > > results in pandoc calling > > > > myfilter csv --foo --baz --bar > > > > I investigated reimplementing pandoc-lit as a filter but it was not > > possible because the program has a lot of command line arguments you > > need to specify. > > > > The worry about passing the format as the first argument being > > confusing could be overcome by embedding more metadata into the JSON > > input. > > > > Matt > > > > On Fri, Jan 6, 2017 at 6:32 PM, BPJ <bpj-J3H7GcXPSITLoDKTGw+V6w@public.gmane.org> wrote: > > > The best way to pass options to filters is through metadata. That gets > > > considerably easier if the filter can use a function which converts a > > > structure of metadata objects into a plain data structure so that > `MetsMap`s > > > get converted to plain associative arrays, `MetaList`s to plain arrays > and > > > other `Meta` elements to plain strings or booleans. It is easy to write > such > > > a function using existing walking and steingification functions. > > > > > > Once you get the hang of that you don't feel any need to pass custom > > > arguments to filters because it is so convenient to do filter config in > the > > > document. A useful way to avoid a lot of metadata keys prefixed with > filter > > > names is to have a single key equal to the filter name with a mapping as > > > value which contains the options for that filter. Nothing hinders you > from > > > also checking environment variables if metadata options are undefined and > > > use hardcoded defaults only if the env vars are also undefined. I do so > > > often enough to have written a helper function. > > > > > > /bpj > > > > > > fre 6 jan. 2017 kl. 14:00 skrev John MacFarlane <jgm-TVLZxgkOlNX2fBVCVOL8/A@public.gmane.org>: > > >> > > >> +++ Joost Kremers [Jan 06 17 13:27 ]: > > >> > > >> > > > >> > > >> >On Fri, Jan 06 2017, John MacFarlane wrote: > > >> > > >> >>This is a pretty common request. A couple of complications: > > >> > > >> >> > > >> > > >> >>1. Pandoc calls each filter with one argument, the name of > > >> > > >> >>the output format. (Filters thus have access to this.) > > >> > > >> >>If we allowed `-F "pantable -t csv"`, how would this work? > > >> > > >> >>Would pandoc call `pantable latex -t csv`? Would the filter > > >> > > >> >>be expecting this? > > >> > > >> > > > >> > > >> >For a properly written filter that shouldn't be a problem, should it? > > >> > > >> >You just need to decide on one way of doing it and stick to it. > > >> > > >> >Filters then need to be updated accordingly. Or are you worried about > > >> > > >> >backward compatibility? > > >> > > >> > > >> > > >> Well, I'm anticipating people getting confused. "I said to > > >> > > >> run --filter 'myfilter myarg', and for some reason 'myarg' > > >> > > >> isn't the first argument!" > > >> > > >> > > >> > > >> What's wrong with communicating options to filters through > > >> > > >> environment variables, which you can already do? > > >> > > >> > > >> > > >> PANTABLE_OUTPUT="csv" pandoc --filter pantable > > >> > > >> > > >> > > >> >pandoc --filter "My Cool Filter.py" --filter-args "-t csv" > > >> > > >> > > > >> > > >> >with each --filter-args referring to the immediately preceding > > >> > > >> >--filters option. Depending on how Pandoc calls filters, it may even > > >> > > >> >be possible to just pass it the literal string argument of > > >> > > >> >--filter-args without having to parse it into different options. > > >> > > >> > > >> > > >> I think this is too complex, with --filter-args affecting > > >> > > >> the immediately preceding --filter option. > > >> > > >> > > >> > > >> Again, I'd have to be convinced that environment variables > > >> > > >> don't already provide a perfectly adequate way to do this. > > >> > > >> > > >> > > >> -- > > >> > > >> You received this message because you are subscribed to the Google > Groups > > >> "pandoc-discuss" group. > > >> > > >> To unsubscribe from this group and stop receiving emails from it, send > an > > >> email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org > > >> > > >> To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org > > >> > > >> To view this discussion on the web visit > > >> > https://groups.google.com/d/msgid/pandoc-discuss/20170106125852.GA1662%40Administrateurs-iMac-3.local > . > > >> > > >> For more options, visit https://groups.google.com/d/optout. > > >> > > > -- > > > You received this message because you are subscribed to the Google Groups > > > "pandoc-discuss" group. > > > To unsubscribe from this group and stop receiving emails from it, send an > > > email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org > > > To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org > > > To view this discussion on the web visit > > > > https://groups.google.com/d/msgid/pandoc-discuss/CADAJKhCPn81DsWpzWSQHv59ii%2BSLXmz4Ww5ru1mSmP4eRQozow%40mail.gmail.com > . > > > > > > For more options, visit https://groups.google.com/d/optout. > > > > -- > > You received this message because you are subscribed to the Google Groups > "pandoc-discuss" group. > > To unsubscribe from this group and stop receiving emails from it, send an > email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org > > To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org > > To view this discussion on the web visit > https://groups.google.com/d/msgid/pandoc-discuss/CALuQ0m8Jx2N_nKKOL5i5pNM9houf_iUe2Z54uWeajm2oVEZW5w%40mail.gmail.com > . > > For more options, visit https://groups.google.com/d/optout. > > -- You received this message because you are subscribed to the Google Groups "pandoc-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/CADAJKhCf3X7ETGQ2DJpZ3Y5gWQid2mUGWJCrpdmpJKWXWb6XBw%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout. [-- Attachment #2: Type: text/html, Size: 13379 bytes --] ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: Filter with arguments [not found] ` <CALuQ0m8Jx2N_nKKOL5i5pNM9houf_iUe2Z54uWeajm2oVEZW5w-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org> 2017-01-06 19:47 ` BPJ @ 2017-01-06 21:11 ` John MacFarlane [not found] ` <20170106211115.GA30105-l/d5Ua9yGnxXsXJlQylH7w@public.gmane.org> 1 sibling, 1 reply; 20+ messages in thread From: John MacFarlane @ 2017-01-06 21:11 UTC (permalink / raw) To: pandoc-discuss-/JYPxA39Uh5TLH3MbocFFw +++ Matthew Pickering [Jan 06 17 18:42 ]: >Communicating arguments though env vars is very ad-hoc, an option I still haven't understood the case against using environment variables. In what way are they "ad hoc"? They are pretty standardly used to communicate options to 'make', for example. It's just about the same effort to write MYARG=blah pandoc --filter foo as to write pandoc -filter foo --filter-args blah And an added advantage of environment variables is that you can set them once and then not have to repeat. ^ permalink raw reply [flat|nested] 20+ messages in thread
[parent not found: <20170106211115.GA30105-l/d5Ua9yGnxXsXJlQylH7w@public.gmane.org>]
* Re: Filter with arguments [not found] ` <20170106211115.GA30105-l/d5Ua9yGnxXsXJlQylH7w@public.gmane.org> @ 2017-01-06 21:36 ` BPJ 0 siblings, 0 replies; 20+ messages in thread From: BPJ @ 2017-01-06 21:36 UTC (permalink / raw) To: pandoc-discuss-/JYPxA39Uh5TLH3MbocFFw [-- Attachment #1: Type: text/plain, Size: 1973 bytes --] It may seem different to Windows users. Setting environment variables on Windows is a pain. fre 6 jan. 2017 kl. 22:12 skrev John MacFarlane <jgm-TVLZxgkOlNX2fBVCVOL8/A@public.gmane.org>: > +++ Matthew Pickering [Jan 06 17 18:42 ]: > > >Communicating arguments though env vars is very ad-hoc, an option > > > > I still haven't understood the case against using > > environment variables. In what way are they "ad hoc"? > > They are pretty standardly used to communicate options to > > 'make', for example. > > > > It's just about the same effort to write > > > > MYARG=blah pandoc --filter foo > > > > as to write > > > > pandoc -filter foo --filter-args blah > > > > And an added advantage of environment variables is that > > you can set them once and then not have to repeat. > > > > -- > > You received this message because you are subscribed to the Google Groups > "pandoc-discuss" group. > > To unsubscribe from this group and stop receiving emails from it, send an > email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org > > To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org > > To view this discussion on the web visit > https://groups.google.com/d/msgid/pandoc-discuss/20170106211115.GA30105%40Johns-MBP.home > . > > For more options, visit https://groups.google.com/d/optout. > > -- You received this message because you are subscribed to the Google Groups "pandoc-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/CADAJKhB9TuDXh7OEELzMPjHBAabmLzgfq%3D-NzzGN9%3DUkvxSpbg%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout. [-- Attachment #2: Type: text/html, Size: 3644 bytes --] ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: Filter with arguments [not found] ` <20170106125852.GA1662-BKjuZOBx5Kn2N3qrpRCZGbhGAdq7xJNKhPhL2mjWHbk@public.gmane.org> 2017-01-06 18:32 ` BPJ @ 2017-01-08 19:40 ` Joost Kremers 1 sibling, 0 replies; 20+ messages in thread From: Joost Kremers @ 2017-01-08 19:40 UTC (permalink / raw) To: pandoc-discuss-/JYPxA39Uh5TLH3MbocFFw On Fri, Jan 06 2017, John MacFarlane wrote: > I think this is too complex, with --filter-args affecting > the immediately preceding --filter option. It's not unprecedented, though. pdfjam,[1] for example, allows you to specify multiple file names on the command line, with each file followed by the pages from the preceding file that pdfjam is to extract. But, although I brought it up, I'm now hoping you won't go this way, because my pandoc-mode for Emacs doesn't have any infrastructure set up for this kind of thing... Normally, new options in Pandoc can be supported in pandoc-mode with a single line of code, but --filter-args would require quite a bit more work... ;-) Joost [1] http://www2.warwick.ac.uk/fac/sci/statistics/staff/academic-research/firth/software/pdfjam -- Joost Kremers Life has its moments ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: Filter with arguments [not found] ` <20170106114647.GG791-BKjuZOBx5Kn2N3qrpRCZGbhGAdq7xJNKhPhL2mjWHbk@public.gmane.org> 2017-01-06 12:27 ` Joost Kremers @ 2017-01-06 23:10 ` Kolen Cheung [not found] ` <1239e0f1-2538-4781-8a75-90c59f86aabc-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org> 1 sibling, 1 reply; 20+ messages in thread From: Kolen Cheung @ 2017-01-06 23:10 UTC (permalink / raw) To: pandoc-discuss [-- Attachment #1.1: Type: text/plain, Size: 3539 bytes --] Oops! Sorry again to have a duplicated post… a post on the same topic is in pandoc-discuss/support arguments on filters <https://groups.google.com/d/msg/pandoc-discuss/Wb20JhBlQLE/72zsvBbEU4sJ>. To compensate, let me wrote a combined summary of both posts (please expand or correct if incomplete/incorrect): <#>Current method 1. pipe it through - Cons: - the filter would not know the output format without duplication - related question: how to pass output format when using pandoc -t json | filter1 ... | ...? e.g. I often see filters checking the output format from the json rather than the 1st arg. - ugly, “unnecessary” pipe - Questions: how about Windows users? Does it work the same way? 2. Metadata, either through YAML or -M command line option - Cons: - treated as Markdown - metadata key collision (e.g. hierarchy of metadata key is needed, which make it uglier) 3. panzer - Cons: - extra dependency - written in Python (for filter not written in Python it is not desirable) - it has its own rules (which some *might* not like) 4. environmental variables - Cons: suffer the same deficiency of -M option in (2) - Questions: how about Windows users? Does it work the same way? (@bpj has mentioned it is a pain to do so.) Comments: As as filter writer, one would want their filter as easy to use as possible, and as few extra dependencies as possible. These criteria eliminates (1, 4) for easiness and (3) for dependencies. This leaves (2) as the only viable solution, which still has some cons needed to be dealt with. I personally don’t worry too much about things treated as Markdown, but metadata key collision is unavoidable if one want short, easy to remember keys (e.g. pantable -s -t csv, anyone used to pandoc will almost be able to remember this.). In addition, the above discussion hasn’t covered the case that one might want to write a program that can acts as both a filter and a standalone cli. e.g. I am thinking about using -s to indicate it is a standalone cli (and without it, a pandoc filter). Using -M means the options is parsed differently (and duplicated) in these 2 cases. <#>Suggested method 1. -O key=value/--filter-option=KEY:VAL/--filter-args/--filter-arguments - Cons - what if there’s multiple filters? special ordering is needed. 2. -F "filter --split --on --hyphens"/-F filter --split --on --space - Cons: - pandoc is already passing the output format as the 1st arg - potentially break existing ill-named filters Comments: Just for the sake of letting the -F option be backward compatible, (1) might be better. But either way it will be fine with me. -- You received this message because you are subscribed to the Google Groups "pandoc-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/1239e0f1-2538-4781-8a75-90c59f86aabc%40googlegroups.com. For more options, visit https://groups.google.com/d/optout. [-- Attachment #1.2: Type: text/html, Size: 21206 bytes --] ^ permalink raw reply [flat|nested] 20+ messages in thread
[parent not found: <1239e0f1-2538-4781-8a75-90c59f86aabc-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org>]
* Re: Filter with arguments [not found] ` <1239e0f1-2538-4781-8a75-90c59f86aabc-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org> @ 2017-01-07 19:05 ` BP Jonsson [not found] ` <c21c773d-c958-2e55-725d-90863f5f9792-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> 0 siblings, 1 reply; 20+ messages in thread From: BP Jonsson @ 2017-01-07 19:05 UTC (permalink / raw) To: pandoc-discuss-/JYPxA39Uh5TLH3MbocFFw Den 2017-01-07 kl. 00:10, skrev Kolen Cheung: > In addition, the above discussion hasn’t covered the case that one might > want to write a program that can acts as both a filter and a standalone > cli. Is that a realistic scenario? Either the program would have to expect pandoc-style JSON as input when used standalone, or it could check whether the input is (pandoc-style) JSON and do something completely different if it isn't! I have some scripts which do essentially the same things to HTML documents as some of my pandoc filters, but they use entirely different libraries: an HTML parser instead of a JSON parser, and a different library for manipulating the data structure produced by the respective parser. Explicit piping of `-t json` output is also a rather outlandish problem: it is now effectively only used by filter authors debugging their filters, who then really should not expect to not have to specify the output format as an argument to the program on the rhs of the pipe. As for the problem of metadata being interpreted as markdown the code which Pandoc::Elements uses to convert metadata to plain data is not too complicated (the `string` methods are already there for other purposes anyway): sub Pandoc::Document::metavalue { my $meta = $_[0]->meta; return { map { $_ => $meta->{$_}->metavalue } keys %$meta } } sub Pandoc::Document::MetaMap::metavalue { my $map = $_[0]->{c}; return { map { $_ => $map->{$_}->metavalue } keys %$map } } sub Pandoc::Document::MetaList::metavalue { [ map { $_->metavalue } @{$_[0]->{c}} ] } sub Pandoc::Document::MetaInlines::metavalue { join '', map { $_->string } @{$_[0]->{c}} } sub Pandoc::Document::MetaBlocks::metavalue { join "\n", map { $_->string } @{$_[0]->{c}} } sub Pandoc::Document::MetaString::metavalue { $_[0]->{c} } sub Pandoc::Document::MetaBool::metavalue { $_[0]->{c} ? 1 : 0 } As for hierarchical metadata structures being ugly I don't think they are. They create a namespace and keep related data together without relying on sorting order, and often hierarchical data make things easier to do compared to non-hierarchical data. That's the reason data structures in general and hierarchical data structures in particular were invented. At a user experience level they keep each mapping key shorter and less repetitive, since otherwise you would in principle have to make every key a 'path' in order to avoid field name clashes. I have been in the situation of having to emulate hierarcical data structures in languages which don't have them, or even associative arrays, in languages which don't have them. It's no fun. Consider a case where the user would supply different data to be used in processing elements with different classes with different output formats, in an hierachical version myfilter: class: foo: latex: ... html: ... bar: latex: ... html: ... ... ... and a 'non-hierarchical' version myfilter-class-foo-latex: ... myfilter-class-foo-html: ... myfilter-class-bar-latex: ... myfilter-class-bar-html: ... I vastly prefer the first both as a document author and as a filter author! /bpj -- You received this message because you are subscribed to the Google Groups "pandoc-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/c21c773d-c958-2e55-725d-90863f5f9792%40gmail.com. For more options, visit https://groups.google.com/d/optout. ^ permalink raw reply [flat|nested] 20+ messages in thread
[parent not found: <c21c773d-c958-2e55-725d-90863f5f9792-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>]
* Re: Filter with arguments [not found] ` <c21c773d-c958-2e55-725d-90863f5f9792-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> @ 2017-01-07 20:51 ` Kolen Cheung [not found] ` <de2e97b1-17a4-4f70-8c17-b3cd960dceb2-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org> 0 siblings, 1 reply; 20+ messages in thread From: Kolen Cheung @ 2017-01-07 20:51 UTC (permalink / raw) To: pandoc-discuss [-- Attachment #1.1: Type: text/plain, Size: 1852 bytes --] Is that a realistic scenario? At least in my case of pantable it actually is. Because either way it is calling pandoc to do some heavy lifting. So many of the functions I wrote will be the same. The only differences are how the options are given, and then how the input/output format might be different (e.g. from csv as a filter expects the csv in CodeBlock, but from csv on cli expects just a plain csv). As for hierarchical metadata structures being ugly I don’t think they are. They aren’t if in YAML, but they seems to be when specified using -M (or am I missing something?). In the case of any of the suggested method, the filter arg is only available to that particular filter, so there’s no mixing of name spaces. And just to add that sometimes it is not desirable to put this info in the YAML. e.g. when converting all pandoc Table to other formats (currently I only have CSV in CodeBlock output, but I might add some others soon, and not to mention it can works the other way around), one want to be able to input any random sources (it would very well be from a .docx) and have the ability to specifies the output format (say CSV in ClodeBlock, CSS tables using Div, etc.) *without* touching the documents. -- You received this message because you are subscribed to the Google Groups "pandoc-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/de2e97b1-17a4-4f70-8c17-b3cd960dceb2%40googlegroups.com. For more options, visit https://groups.google.com/d/optout. [-- Attachment #1.2: Type: text/html, Size: 6353 bytes --] ^ permalink raw reply [flat|nested] 20+ messages in thread
[parent not found: <de2e97b1-17a4-4f70-8c17-b3cd960dceb2-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org>]
* Re: Filter with arguments [not found] ` <de2e97b1-17a4-4f70-8c17-b3cd960dceb2-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org> @ 2017-01-07 22:53 ` BP Jonsson [not found] ` <CAFC_yuTR=2UQZPN4EFbKTCOAwR9tOoS1+sYy5rLRViNo908s-Q-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org> 0 siblings, 1 reply; 20+ messages in thread From: BP Jonsson @ 2017-01-07 22:53 UTC (permalink / raw) To: pandoc-discuss-/JYPxA39Uh5TLH3MbocFFw [-- Attachment #1: Type: text/plain, Size: 4103 bytes --] Yes of course with `-M` you get all the ugliness (IMO) of a single level of key-value pairs which was my point, but that is no different from traditional command line options, apart from `--option=key=value` where you get an additional level. Sure there are things which you may want to communicate to a filter on the command line and not in the document but those are in my experience in the minority. What do you mean with outputting different formats? If you mean what we usually mean by "output format" in a pandoc context I'd expect the format argument which pandoc gives to filters to suffice in most cases, wouldn't it? It would be a little ironic to start adding command line options to suit different people's filter writing style, since we got filters to avoid adding support for everyone's needed features in the pandoc ececutable. That said if a separate filter option option is implemented it would probably be wiser to implement it analogously to the -M option, something like `--filter-option myfilter:option[=value]` and pass those options to the filter as a second 'line' of JSON, thereby not breaking backwards compatibility. Dual filter/cli scripts can just (a) check whether there are any command line options and (b) whether the first two input records are JSON and do their thing based on that. Den 7 jan 2017 21:51 skrev "Kolen Cheung" <christian.kolen-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>: > Is that a realistic scenario? > > At least in my case of pantable it actually is. Because either way it is > calling pandoc to do some heavy lifting. So many of the functions I wrote > will be the same. The only differences are how the options are given, and > then how the input/output format might be different (e.g. from csv as a > filter expects the csv in CodeBlock, but from csv on cli expects just a > plain csv). > > As for hierarchical metadata structures being ugly I don’t think they are. > > They aren’t if in YAML, but they seems to be when specified using -M (or > am I missing something?). In the case of any of the suggested method, the > filter arg is only available to that particular filter, so there’s no > mixing of name spaces. And just to add that sometimes it is not desirable > to put this info in the YAML. e.g. when converting all pandoc Table to > other formats (currently I only have CSV in CodeBlock output, but I might > add some others soon, and not to mention it can works the other way > around), one want to be able to input any random sources (it would very > well be from a .docx) and have the ability to specifies the output format > (say CSV in ClodeBlock, CSS tables using Div, etc.) *without* touching > the documents. > > > -- > You received this message because you are subscribed to the Google Groups > "pandoc-discuss" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org > To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org > To view this discussion on the web visit https://groups.google.com/d/ > msgid/pandoc-discuss/de2e97b1-17a4-4f70-8c17-b3cd960dceb2% > 40googlegroups.com > <https://groups.google.com/d/msgid/pandoc-discuss/de2e97b1-17a4-4f70-8c17-b3cd960dceb2%40googlegroups.com?utm_medium=email&utm_source=footer> > . > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "pandoc-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/CAFC_yuTR%3D2UQZPN4EFbKTCOAwR9tOoS1%2BsYy5rLRViNo908s-Q%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout. [-- Attachment #2: Type: text/html, Size: 9207 bytes --] ^ permalink raw reply [flat|nested] 20+ messages in thread
[parent not found: <CAFC_yuTR=2UQZPN4EFbKTCOAwR9tOoS1+sYy5rLRViNo908s-Q-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>]
* Re: Filter with arguments [not found] ` <CAFC_yuTR=2UQZPN4EFbKTCOAwR9tOoS1+sYy5rLRViNo908s-Q-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org> @ 2017-01-08 9:09 ` John MacFarlane [not found] ` <CADAJKhAwBq+mxsimD83aefMEUCm025i5QaNvfpGBzrAWrB7atQ@mail.gmail.com> 0 siblings, 1 reply; 20+ messages in thread From: John MacFarlane @ 2017-01-08 9:09 UTC (permalink / raw) To: pandoc-discuss-/JYPxA39Uh5TLH3MbocFFw +++ BP Jonsson [Jan 07 17 23:53 ]: > That said if a separate filter option option is implemented it would > probably be wiser to implement it analogously to the -M option, > something like `--filter-option myfilter:option[=value]` and pass those I like the idea of specifying the filter name explicitly. > options to the filter as a second 'line' of JSON, thereby not breaking > backwards compatibility. Changing the JSON will affect backwards compatibility. Better just to add these as arguments to the filter (after the initial 'format' argument). But I'm still not *entirely* convinced that this is needed. ^ permalink raw reply [flat|nested] 20+ messages in thread
[parent not found: <CADAJKhAwBq+mxsimD83aefMEUCm025i5QaNvfpGBzrAWrB7atQ@mail.gmail.com>]
[parent not found: <CADAJKhACOpd-yQHTBTrL4VbCg8zyGiJX7WwECsifGtihbyyzCA@mail.gmail.com>]
[parent not found: <CADAJKhACOpd-yQHTBTrL4VbCg8zyGiJX7WwECsifGtihbyyzCA-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>]
* Re: Filter with arguments [not found] ` <CADAJKhACOpd-yQHTBTrL4VbCg8zyGiJX7WwECsifGtihbyyzCA-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org> @ 2017-01-08 11:21 ` Melroch 0 siblings, 0 replies; 20+ messages in thread From: Melroch @ 2017-01-08 11:21 UTC (permalink / raw) To: pandoc-discuss-/JYPxA39Uh5TLH3MbocFFw [-- Attachment #1: Type: text/plain, Size: 2658 bytes --] What I propose is not to alter the current AST JSON document but to add a second JSON document after it containing the filter options, each terminated by its own newline. As for specifying the filter which the option(s) belong to explicitly I think that is pretty much necessary to avoid confusion. Alternatively the `-F` option might be altered to take an optional 'value' which would be the options to the filter specified as 'key': `-F FILTERNAME[=FILTEROPTIONS]`. Another question is whether the filter options should be parsed by pandoc or that should be left to the filter. At least perl and python already have libraries which can parse options from a string. Other filters may wish to receive their options e.g. as a JSON object string, since any filter has to be able to decode JSON. Den 8 jan 2017 10:10 skrev "John MacFarlane" <jgm-TVLZxgkOlNX2fBVCVOL8/A@public.gmane.org>: +++ BP Jonsson [Jan 07 17 23:53 ]: That said if a separate filter option option is implemented it would > probably be wiser to implement it analogously to the -M option, > something like `--filter-option myfilter:option[=value]` and pass those > I like the idea of specifying the filter name explicitly. options to the filter as a second 'line' of JSON, thereby not breaking > backwards compatibility. > Changing the JSON will affect backwards compatibility. Better just to add these as arguments to the filter (after the initial 'format' argument). But I'm still not *entirely* convinced that this is needed. -- You received this message because you are subscribed to the Google Groups "pandoc-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To view this discussion on the web visit https://groups.google.com/d/ms gid/pandoc-discuss/20170108090942.GA41105%40Johns-MBP.home. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "pandoc-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/CADAJKhBvu9uTL3SgXPdUEeMJDziRi-eWNf6MuZjS6MacRgwA7Q%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout. [-- Attachment #2: Type: text/html, Size: 4290 bytes --] ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [Feature Request] Filter with arguments [not found] ` <e8cf9c92-7aa7-4841-85d0-7464c771cf6e-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org> 2017-01-06 11:46 ` John MacFarlane @ 2017-01-08 20:51 ` Kolen Cheung [not found] ` <434dc10f-02a1-4bbc-a41c-3a29398ca635-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org> 2018-06-20 14:58 ` Peter Zagubisalo 2 siblings, 1 reply; 20+ messages in thread From: Kolen Cheung @ 2017-01-08 20:51 UTC (permalink / raw) To: pandoc-discuss [-- Attachment #1.1: Type: text/plain, Size: 1466 bytes --] Just to mention another disadvantage of -M option that I haven’t included in the summary above: if the options is specified in this way, it will “burn-into” the document. But one of the application I need is to go freely in a round trip (converting pandoc Table to CSV CodeBlock in one pass, and convert the CSV CodeBlock back to pandoc Table in another pass). But since the -M metadata will accumulate, this is a no-go at least for me. Because of this, I now see why @jgm recommends using environment variables among the currently working methods. But the potential difficulties for Windows users is still a concern. And the new suggestions of -F FILTERNAME[=FILTEROPTIONS] seems cleanest. For ugly filter name: -F "My Cool Filter.py"="--some --cool --arg". Don’t know if this will make the life of pandoc-mode or panzer and alike easier too. -- You received this message because you are subscribed to the Google Groups "pandoc-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/434dc10f-02a1-4bbc-a41c-3a29398ca635%40googlegroups.com. For more options, visit https://groups.google.com/d/optout. [-- Attachment #1.2: Type: text/html, Size: 6160 bytes --] ^ permalink raw reply [flat|nested] 20+ messages in thread
[parent not found: <434dc10f-02a1-4bbc-a41c-3a29398ca635-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org>]
* Re: [Feature Request] Filter with arguments [not found] ` <434dc10f-02a1-4bbc-a41c-3a29398ca635-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org> @ 2017-01-09 11:26 ` BP Jonsson [not found] ` <CAFC_yuRuuDDfbok7Gds-kdj_ySqvpF0psnTUn0-TRvdEhRUs+Q-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org> 0 siblings, 1 reply; 20+ messages in thread From: BP Jonsson @ 2017-01-09 11:26 UTC (permalink / raw) To: pandoc-discuss-/JYPxA39Uh5TLH3MbocFFw [-- Attachment #1: Type: text/plain, Size: 3045 bytes --] There clearly are situations where metadata are the best option, but I realize that there are situations where they may be unsuitable, depending on the nature of the filter, the data and the way the data is used by the filter. For now if you think that the metadata get in the way later the filter which uses them can remove or alter them. As for filter arguments making things easier/harder for this or that wrapper it pretty much depends on how the wrapper interacts with pandoc and/or filters and how (manner and form) filter arguments are passed to filters. Clearly the possible need of wrappers to see some/all filter arguments need to be taken into account. /bpj Den 8 jan 2017 21:51 skrev "Kolen Cheung" <christian.kolen-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>: > Just to mention another disadvantage of -M option that I haven’t included > in the summary above: if the options is specified in this way, it will > “burn-into” the document. But one of the application I need is to go freely > in a round trip (converting pandoc Table to CSV CodeBlock in one pass, and > convert the CSV CodeBlock back to pandoc Table in another pass). But since > the -M metadata will accumulate, this is a no-go at least for me. Because > of this, I now see why @jgm recommends using environment variables among > the currently working methods. But the potential difficulties for Windows > users is still a concern. > > And the new suggestions of -F FILTERNAME[=FILTEROPTIONS] seems cleanest. > For ugly filter name: -F "My Cool Filter.py"="--some --cool --arg". Don’t > know if this will make the life of pandoc-mode or panzer and alike easier > too. > > > -- > You received this message because you are subscribed to the Google Groups > "pandoc-discuss" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org > To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org > To view this discussion on the web visit https://groups.google.com/d/ > msgid/pandoc-discuss/434dc10f-02a1-4bbc-a41c-3a29398ca635% > 40googlegroups.com > <https://groups.google.com/d/msgid/pandoc-discuss/434dc10f-02a1-4bbc-a41c-3a29398ca635%40googlegroups.com?utm_medium=email&utm_source=footer> > . > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "pandoc-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/CAFC_yuRuuDDfbok7Gds-kdj_ySqvpF0psnTUn0-TRvdEhRUs%2BQ%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout. [-- Attachment #2: Type: text/html, Size: 8209 bytes --] ^ permalink raw reply [flat|nested] 20+ messages in thread
[parent not found: <CAFC_yuRuuDDfbok7Gds-kdj_ySqvpF0psnTUn0-TRvdEhRUs+Q-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>]
* Re: [Feature Request] Filter with arguments [not found] ` <CAFC_yuRuuDDfbok7Gds-kdj_ySqvpF0psnTUn0-TRvdEhRUs+Q-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org> @ 2017-08-06 23:33 ` Kolen Cheung 0 siblings, 0 replies; 20+ messages in thread From: Kolen Cheung @ 2017-08-06 23:33 UTC (permalink / raw) To: pandoc-discuss [-- Attachment #1.1: Type: text/plain, Size: 806 bytes --] Hi, @jgm, what are your thoughts on this feature now? Will that be under consideration or will it not be considered at all? If this situation won't change, I'll start implementing what I want to do under these constraints. Thanks. -- You received this message because you are subscribed to the Google Groups "pandoc-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/ecbd2112-8af8-4875-be29-9172828e3208%40googlegroups.com. For more options, visit https://groups.google.com/d/optout. [-- Attachment #1.2: Type: text/html, Size: 1223 bytes --] ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [Feature Request] Filter with arguments [not found] ` <e8cf9c92-7aa7-4841-85d0-7464c771cf6e-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org> 2017-01-06 11:46 ` John MacFarlane 2017-01-08 20:51 ` [Feature Request] " Kolen Cheung @ 2018-06-20 14:58 ` Peter Zagubisalo 2 siblings, 0 replies; 20+ messages in thread From: Peter Zagubisalo @ 2018-06-20 14:58 UTC (permalink / raw) To: pandoc-discuss [-- Attachment #1.1: Type: text/plain, Size: 2389 bytes --] > > In order to have Pandoc filters with arguments I wrote pandoctools - > https://github.com/kiwi0fruit/pandoctools - a profile manager of text > processing pipelines that consist of Pandoc filters or any text CLI > filters. Profiles are special bash scripts so they are quite flexible. > > Introduction about workflow that uses pandoctools: > https://github.com/kiwi0fruit/atom-jupyter-pandoc-markdown > In order to have Pandoc filters with arguments I wrote pandoctools - https://github.com/kiwi0fruit/pandoctools - a profile manager of text processing pipelines that consist of Pandoc filters or any text CLI filters. Profiles are special bash scripts so they are quite flexible. Introduction about workflow that uses pandoctools: https://github.com/kiwi0fruit/atom-jupyter-pandoc-markdown Kolen Cheung написал: Currently, if one need to pass arguments to a filter, one need to pipe it > through. > > It would be convenient in some cases to allow passing argument to filters. > e.g. > > pandoc -F "pantable -t csv" -s -o example.tex example.md# which is equiv. to > pandoc -t json -s example.md | pantable -t csv | pandoc -f json -s -o example.tex > > To elaborate a little bit further: I want to write a filter that at the > same time do: > > 1. convert code block to table > 2. convert pandoc tables to code block, potentially with different > output code formats (e.g. csv, yaml, json, etc. or even other kinds e.g. > HTML, Div with CSS table, etc.) > 3. standalone cli. i.e. not a filter > > Right now (1) and (2) are mutually exclusive and has to be split into 2 > filters. (1) and (3) can potentially be written in the same executable, but > it is uglier and pointless to do so without including (2). > > -- You received this message because you are subscribed to the Google Groups "pandoc-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/7998338d-31dc-4a97-b938-6a035e2cd702%40googlegroups.com. For more options, visit https://groups.google.com/d/optout. [-- Attachment #1.2: Type: text/html, Size: 15124 bytes --] ^ permalink raw reply [flat|nested] 20+ messages in thread
end of thread, other threads:[~2018-06-20 14:58 UTC | newest] Thread overview: 20+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2017-01-06 4:32 [Feature Request] Filter with arguments Kolen Cheung [not found] ` <e8cf9c92-7aa7-4841-85d0-7464c771cf6e-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org> 2017-01-06 11:46 ` John MacFarlane [not found] ` <20170106114647.GG791-BKjuZOBx5Kn2N3qrpRCZGbhGAdq7xJNKhPhL2mjWHbk@public.gmane.org> 2017-01-06 12:27 ` Joost Kremers [not found] ` <8760ls8jby.fsf-97jfqw80gc6171pxa8y+qA@public.gmane.org> 2017-01-06 12:58 ` John MacFarlane [not found] ` <20170106125852.GA1662-BKjuZOBx5Kn2N3qrpRCZGbhGAdq7xJNKhPhL2mjWHbk@public.gmane.org> 2017-01-06 18:32 ` BPJ [not found] ` <CADAJKhCPn81DsWpzWSQHv59ii+SLXmz4Ww5ru1mSmP4eRQozow-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org> 2017-01-06 18:42 ` Matthew Pickering [not found] ` <CALuQ0m8Jx2N_nKKOL5i5pNM9houf_iUe2Z54uWeajm2oVEZW5w-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org> 2017-01-06 19:47 ` BPJ 2017-01-06 21:11 ` John MacFarlane [not found] ` <20170106211115.GA30105-l/d5Ua9yGnxXsXJlQylH7w@public.gmane.org> 2017-01-06 21:36 ` BPJ 2017-01-08 19:40 ` Joost Kremers 2017-01-06 23:10 ` Kolen Cheung [not found] ` <1239e0f1-2538-4781-8a75-90c59f86aabc-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org> 2017-01-07 19:05 ` BP Jonsson [not found] ` <c21c773d-c958-2e55-725d-90863f5f9792-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> 2017-01-07 20:51 ` Kolen Cheung [not found] ` <de2e97b1-17a4-4f70-8c17-b3cd960dceb2-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org> 2017-01-07 22:53 ` BP Jonsson [not found] ` <CAFC_yuTR=2UQZPN4EFbKTCOAwR9tOoS1+sYy5rLRViNo908s-Q-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org> 2017-01-08 9:09 ` John MacFarlane [not found] ` <CADAJKhAwBq+mxsimD83aefMEUCm025i5QaNvfpGBzrAWrB7atQ@mail.gmail.com> [not found] ` <CADAJKhACOpd-yQHTBTrL4VbCg8zyGiJX7WwECsifGtihbyyzCA@mail.gmail.com> [not found] ` <CADAJKhACOpd-yQHTBTrL4VbCg8zyGiJX7WwECsifGtihbyyzCA-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org> 2017-01-08 11:21 ` Melroch 2017-01-08 20:51 ` [Feature Request] " Kolen Cheung [not found] ` <434dc10f-02a1-4bbc-a41c-3a29398ca635-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org> 2017-01-09 11:26 ` BP Jonsson [not found] ` <CAFC_yuRuuDDfbok7Gds-kdj_ySqvpF0psnTUn0-TRvdEhRUs+Q-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org> 2017-08-06 23:33 ` Kolen Cheung 2018-06-20 14:58 ` Peter Zagubisalo
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).