Pandoc sinks and sources

Pandoc sinks and sources

[Francesco Occhipinti]

[-- Attachment #1.1: Type: text/plain, Size: 1030 bytes --]


Hello pandoc users,

 While assisting my users in a project that relies on pandoc, i found 
myself developing some small scripts which made me think of a new 
convention: pandoc sinks and pandoc sources. Here 
https://github.com/italia/docs-italia-comandi-conversione/blob/master/doc/pandoc-sinks-and-sources.md 
the idea is explained in detail, let me know what you think about it. If 
you find this useful, we could add it to the wiki.

Cheers,
Francesco

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/78277484-e7f4-421f-b5d0-ff7e98ad76a9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[-- Attachment #1.2: Type: text/html, Size: 1540 bytes --]

Re: Pandoc sinks and sources

[mb21]

[-- Attachment #1.1: Type: text/plain, Size: 1569 bytes --]

Not sure people talk about these kinds of programs enough to warrant 
specific terminology, but the concepts certainly make sense.

For example my pandoc-placetable <https://github.com/mb21/pandoc-placetable> 
filter acts as a "source" if not called as a filter but as a standalone 
program. So you can use it as an ad-hoc CSV reader:

    pandoc-placetable --file=foo.csv --widths="0.2 0.8" | pandoc -f json -o 
output.html

cheers


On Thursday, June 14, 2018 at 11:04:53 PM UTC+2, Francesco Occhipinti wrote:
>
>
> Hello pandoc users,
>
>  While assisting my users in a project that relies on pandoc, i found 
> myself developing some small scripts which made me think of a new 
> convention: pandoc sinks and pandoc sources. Here 
> https://github.com/italia/docs-italia-comandi-conversione/blob/master/doc/pandoc-sinks-and-sources.md 
> the idea is explained in detail, let me know what you think about it. If 
> you find this useful, we could add it to the wiki.
>
> Cheers,
> Francesco
>
>

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/aa144587-a080-41cb-99a6-95c754336c3f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[-- Attachment #1.2: Type: text/html, Size: 3006 bytes --]

Re: Pandoc sinks and sources

[Francesco Occhipinti]

[-- Attachment #1.1: Type: text/plain, Size: 2066 bytes --]


Thanks, i added your command to the list. Well you know how it works with 
language, it's less likely to talk about something whose name is unknown. 
This could be the start of something. I hope that other users will give us 
their feedback. I realised that a sink is similar to the concept of a 
custom writer, except it can be written in any programming language rather 
than just in Lua

On Tuesday, June 19, 2018 at 2:25:35 PM UTC+2, mb21 wrote:
>
> Not sure people talk about these kinds of programs enough to warrant 
> specific terminology, but the concepts certainly make sense.
>
> For example my pandoc-placetable 
> <https://github.com/mb21/pandoc-placetable> filter acts as a "source" if 
> not called as a filter but as a standalone program. So you can use it as an 
> ad-hoc CSV reader:
>
>     pandoc-placetable --file=foo.csv --widths="0.2 0.8" | pandoc -f json 
> -o output.html
>
> cheers
>
>
> On Thursday, June 14, 2018 at 11:04:53 PM UTC+2, Francesco Occhipinti 
> wrote:
>>
>>
>> Hello pandoc users,
>>
>>  While assisting my users in a project that relies on pandoc, i found 
>> myself developing some small scripts which made me think of a new 
>> convention: pandoc sinks and pandoc sources. Here 
>> https://github.com/italia/docs-italia-comandi-conversione/blob/master/doc/pandoc-sinks-and-sources.md 
>> the idea is explained in detail, let me know what you think about it. If 
>> you find this useful, we could add it to the wiki.
>>
>> Cheers,
>> Francesco
>>
>>

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/6710819f-2df7-4311-bcc8-5275ad6e4272%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[-- Attachment #1.2: Type: text/html, Size: 4058 bytes --]

Re: Pandoc sinks and sources

[BP Jonsson]

[-- Attachment #1: Type: text/plain, Size: 5655 bytes --]

I wrote JSON filters using pipes already before there was any official
filter support or dedicated support libraries using the Perl data walking
module Data::Rmap and a bunch of support functions of my own (basically a
wrapper which took an AST and a dispatch table mapping element type names
to function references, replacing original element hashes with any defined
return values. Very primitive.) At first your suggestion seemed like a step
backwards, but I supposed you meant that these programs should respectively
take/return non-document data.

I have written some scripts which mined data from Pandoc ASTs but they all
took a filename and called out to Pandoc, basically because I didn't want
to type the same command line options over and over.

I have implemented some "custom writers" as filters which modify the AST so
that the output of the plain writer looks like my idea of what email ought
to look like or like valid Perl POD documentation. They could certainly be
rewritten as Lua writers or to output a document instead of modified JSON
but my priority was on getting the output I wanted using something written
in Perl.

I also have several filters which read config and/or partial input from
external files. It seems writing a CSV to table converter is a rite of
passage among Pandoc filter/addon authors.
Mine would take a code block with CSV, TSV or a file name, parse the CSV
with Text::CSV or simply split TSV on tabs and build a Pandoc table element
from the parsed structure. Where it maybe stood out was in that it could
convert in either direction between all three of Pandoc tables or
definition lists of definition lists and CSV/TSV.

tis 19 juni 2018 kl. 16:31 skrev Francesco Occhipinti <
f.occhipinti-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>:

>
> Thanks, i added your command to the list. Well you know how it works with
> language, it's less likely to talk about something whose name is unknown.
> This could be the start of something. I hope that other users will give us
> their feedback. I realised that a sink is similar to the concept of a
> custom writer, except it can be written in any programming language rather
> than just in Lua
>
> On Tuesday, June 19, 2018 at 2:25:35 PM UTC+2, mb21 wrote:
>>
>> Not sure people talk about these kinds of programs enough to warrant
>> specific terminology, but the concepts certainly make sense.
>>
>> For example my pandoc-placetable
>> <https://github.com/mb21/pandoc-placetable> filter acts as a "source" if
>> not called as a filter but as a standalone program. So you can use it as an
>> ad-hoc CSV reader:
>>
>>     pandoc-placetable --file=foo.csv --widths="0.2 0.8" | pandoc -f json
>> -o output.html
>>
>> cheers
>>
>>
>> On Thursday, June 14, 2018 at 11:04:53 PM UTC+2, Francesco Occhipinti
>> wrote:
>>>
>>>
>>> Hello pandoc users,
>>>
>>>  While assisting my users in a project that relies on pandoc, i found
>>> myself developing some small scripts which made me think of a new
>>> convention: pandoc sinks and pandoc sources. Here
>>> https://github.com/italia/docs-italia-comandi-conversione/blob/master/doc/pandoc-sinks-and-sources.md
>>> the idea is explained in detail, let me know what you think about it. If
>>> you find this useful, we could add it to the wiki.
>>>
>>> Cheers,
>>> Francesco
>>>
>>> --
> You received this message because you are subscribed to the Google Groups
> "pandoc-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
> To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/pandoc-discuss/6710819f-2df7-4311-bcc8-5275ad6e4272%40googlegroups.com
> <https://groups.google.com/d/msgid/pandoc-discuss/6710819f-2df7-4311-bcc8-5275ad6e4272%40googlegroups.com?utm_medium=email&utm_source=footer>
> .
> For more options, visit https://groups.google.com/d/optout.
>
-- 

------------------------------
SavedURI :Show URLShow URLSavedURI :
SavedURI :Hide URLHide URLSavedURI :
https://mail.google.com/_/scs/mail-static/_/js/k=gmail.main.sv.G3GZFwvcniQ.O/m=m_i,t,it/am=fUAcTAoZawdGHAZ2YD-g9N_f7LL4CX7WlSgHQKgABHaCv9kToPiBD8qOMw/rt=h/d=1/rs=AItRSTO5CF1YB_frDRXLXTeUsQ1zItcBvwhttps://mail.google.com/_/scs/mail-static/_/js/k=gmail.main.sv.G3GZFwvcniQ.O/m=m_i,t,it/am=fUAcTAoZawdGHAZ2YD-g9N_f7LL4CX7WlSgHQKgABHaCv9kToPiBD8qOMw/rt=h/d=1/rs=AItRSTO5CF1YB_frDRXLXTeUsQ1zItcBvw
<https://mail.google.com/_/scs/mail-static/_/js/k=gmail.main.sv.G3GZFwvcniQ.O/m=m_i,t,it/am=fUAcTAoZawdGHAZ2YD-g9N_f7LL4CX7WlSgHQKgABHaCv9kToPiBD8qOMw/rt=h/d=1/rs=AItRSTO5CF1YB_frDRXLXTeUsQ1zItcBvw>
<https://mail.google.com/_/scs/mail-static/_/js/k=gmail.main.sv.G3GZFwvcniQ.O/m=m_i,t,it/am=fUAcTAoZawdGHAZ2YD-g9N_f7LL4CX7WlSgHQKgABHaCv9kToPiBD8qOMw/rt=h/d=1/rs=AItRSTO5CF1YB_frDRXLXTeUsQ1zItcBvw>
------------------------------

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/CAFC_yuRb4M2LruCd0jq50ZjWgt5Q5rpoByYYD3MnAH1f3kjhqQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[-- Attachment #2: Type: text/html, Size: 7697 bytes --]

Re: Pandoc sinks and sources

[BP Jonsson]

[-- Attachment #1: Type: text/plain, Size: 7266 bytes --]

I forgot to say that while I appreciate the usefulness of terms for these
programs I'm not wild about the terms "source" and "sink". When I think
about "pandoc source" I think about the document source file, and to
non-programmers "sink" may sound like you are discarding something.

Perhaps "feeders" and "consumers" would be better. No rime or alliteration
but perhaps less ambiguous and more descriptive for non-programmers.

tis 19 juni 2018 kl. 17:55 skrev BP Jonsson <bpjonsson-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>:

> I wrote JSON filters using pipes already before there was any official
> filter support or dedicated support libraries using the Perl data walking
> module Data::Rmap and a bunch of support functions of my own (basically a
> wrapper which took an AST and a dispatch table mapping element type names
> to function references, replacing original element hashes with any defined
> return values. Very primitive.) At first your suggestion seemed like a step
> backwards, but I supposed you meant that these programs should respectively
> take/return non-document data.
>
> I have written some scripts which mined data from Pandoc ASTs but they all
> took a filename and called out to Pandoc, basically because I didn't want
> to type the same command line options over and over.
>
> I have implemented some "custom writers" as filters which modify the AST
> so that the output of the plain writer looks like my idea of what email
> ought to look like or like valid Perl POD documentation. They could
> certainly be rewritten as Lua writers or to output a document instead of
> modified JSON but my priority was on getting the output I wanted using
> something written in Perl.
>
> I also have several filters which read config and/or partial input from
> external files. It seems writing a CSV to table converter is a rite of
> passage among Pandoc filter/addon authors.
> Mine would take a code block with CSV, TSV or a file name, parse the CSV
> with Text::CSV or simply split TSV on tabs and build a Pandoc table element
> from the parsed structure. Where it maybe stood out was in that it could
> convert in either direction between all three of Pandoc tables or
> definition lists of definition lists and CSV/TSV.
>
> tis 19 juni 2018 kl. 16:31 skrev Francesco Occhipinti <
> f.occhipinti-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>:
>
>>
>> Thanks, i added your command to the list. Well you know how it works with
>> language, it's less likely to talk about something whose name is unknown.
>> This could be the start of something. I hope that other users will give us
>> their feedback. I realised that a sink is similar to the concept of a
>> custom writer, except it can be written in any programming language rather
>> than just in Lua
>>
>> On Tuesday, June 19, 2018 at 2:25:35 PM UTC+2, mb21 wrote:
>>>
>>> Not sure people talk about these kinds of programs enough to warrant
>>> specific terminology, but the concepts certainly make sense.
>>>
>>> For example my pandoc-placetable
>>> <https://github.com/mb21/pandoc-placetable> filter acts as a "source"
>>> if not called as a filter but as a standalone program. So you can use it as
>>> an ad-hoc CSV reader:
>>>
>>>     pandoc-placetable --file=foo.csv --widths="0.2 0.8" | pandoc -f json
>>> -o output.html
>>>
>>> cheers
>>>
>>>
>>> On Thursday, June 14, 2018 at 11:04:53 PM UTC+2, Francesco Occhipinti
>>> wrote:
>>>>
>>>>
>>>> Hello pandoc users,
>>>>
>>>>  While assisting my users in a project that relies on pandoc, i found
>>>> myself developing some small scripts which made me think of a new
>>>> convention: pandoc sinks and pandoc sources. Here
>>>> https://github.com/italia/docs-italia-comandi-conversione/blob/master/doc/pandoc-sinks-and-sources.md
>>>> the idea is explained in detail, let me know what you think about it. If
>>>> you find this useful, we could add it to the wiki.
>>>>
>>>> Cheers,
>>>> Francesco
>>>>
>>>> --
>> You received this message because you are subscribed to the Google Groups
>> "pandoc-discuss" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
>> To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/pandoc-discuss/6710819f-2df7-4311-bcc8-5275ad6e4272%40googlegroups.com
>> <https://groups.google.com/d/msgid/pandoc-discuss/6710819f-2df7-4311-bcc8-5275ad6e4272%40googlegroups.com?utm_medium=email&utm_source=footer>
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
> --
>
> ------------------------------
> SavedURI :Show URLShow URLSavedURI :
> SavedURI :Hide URLHide URLSavedURI :
>
> https://mail.google.com/_/scs/mail-static/_/js/k=gmail.main.sv.G3GZFwvcniQ.O/m=m_i,t,it/am=fUAcTAoZawdGHAZ2YD-g9N_f7LL4CX7WlSgHQKgABHaCv9kToPiBD8qOMw/rt=h/d=1/rs=AItRSTO5CF1YB_frDRXLXTeUsQ1zItcBvwhttps://mail.google.com/_/scs/mail-static/_/js/k=gmail.main.sv.G3GZFwvcniQ.O/m=m_i,t,it/am=fUAcTAoZawdGHAZ2YD-g9N_f7LL4CX7WlSgHQKgABHaCv9kToPiBD8qOMw/rt=h/d=1/rs=AItRSTO5CF1YB_frDRXLXTeUsQ1zItcBvw
> <https://mail.google.com/_/scs/mail-static/_/js/k=gmail.main.sv.G3GZFwvcniQ.O/m=m_i,t,it/am=fUAcTAoZawdGHAZ2YD-g9N_f7LL4CX7WlSgHQKgABHaCv9kToPiBD8qOMw/rt=h/d=1/rs=AItRSTO5CF1YB_frDRXLXTeUsQ1zItcBvw>
> <https://mail.google.com/_/scs/mail-static/_/js/k=gmail.main.sv.G3GZFwvcniQ.O/m=m_i,t,it/am=fUAcTAoZawdGHAZ2YD-g9N_f7LL4CX7WlSgHQKgABHaCv9kToPiBD8qOMw/rt=h/d=1/rs=AItRSTO5CF1YB_frDRXLXTeUsQ1zItcBvw>
> ------------------------------
>
-- 

------------------------------
SavedURI :Show URLShow URLSavedURI :
SavedURI :Hide URLHide URLSavedURI :
https://mail.google.com/_/scs/mail-static/_/js/k=gmail.main.sv.G3GZFwvcniQ.O/m=m_i,t,it/am=fUAcTAoZawdGHAZ2YD-g9N_f7LL4CX7WlSgHQKgABHaCv9kToPiBD8qOMw/rt=h/d=1/rs=AItRSTO5CF1YB_frDRXLXTeUsQ1zItcBvwhttps://mail.google.com/_/scs/mail-static/_/js/k=gmail.main.sv.G3GZFwvcniQ.O/m=m_i,t,it/am=fUAcTAoZawdGHAZ2YD-g9N_f7LL4CX7WlSgHQKgABHaCv9kToPiBD8qOMw/rt=h/d=1/rs=AItRSTO5CF1YB_frDRXLXTeUsQ1zItcBvw
<https://mail.google.com/_/scs/mail-static/_/js/k=gmail.main.sv.G3GZFwvcniQ.O/m=m_i,t,it/am=fUAcTAoZawdGHAZ2YD-g9N_f7LL4CX7WlSgHQKgABHaCv9kToPiBD8qOMw/rt=h/d=1/rs=AItRSTO5CF1YB_frDRXLXTeUsQ1zItcBvw>
<https://mail.google.com/_/scs/mail-static/_/js/k=gmail.main.sv.G3GZFwvcniQ.O/m=m_i,t,it/am=fUAcTAoZawdGHAZ2YD-g9N_f7LL4CX7WlSgHQKgABHaCv9kToPiBD8qOMw/rt=h/d=1/rs=AItRSTO5CF1YB_frDRXLXTeUsQ1zItcBvw>
------------------------------

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/CAFC_yuQEaYZ_zk_F2eOM005evfoJkbrmYENb1dFaBYVuyEfWjw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[-- Attachment #2: Type: text/html, Size: 9876 bytes --]

Re: Pandoc sinks and sources

[Francesco Occhipinti]

[-- Attachment #1.1: Type: text/plain, Size: 2264 bytes --]

Hi, thanks for your useful comments, i will reply inline to specific 
sections

On Tuesday, June 19, 2018 at 5:56:09 PM UTC+2, BP Jonsson wrote:
>
> At first your suggestion seemed like a step backwards, but I supposed you 
> meant that these programs should respectively take/return non-document data.
>

Exactly. I would be happy if you could suggest me any change to the text 
https://github.com/italia/docs-italia-comandi-conversione/blob/master/doc/pandoc-sinks-and-sources.md 
that makes this clearer
 

>
> I have written some scripts which mined data from Pandoc ASTs but they all 
> took a filename and called out to Pandoc, basically because I didn't want 
> to type the same command line options over and over.
>

This is something i considered in order to make life easier to our users, 
the drawback is maintaining the code that passes command line options from 
the wrapper to pandoc, or imposing restrictions on the users.
 

>
> I also have several filters which read config and/or partial input from 
> external files. It seems writing a CSV to table converter is a rite of 
> passage among Pandoc filter/addon authors.
> Mine would take a code block with CSV, TSV or a file name, parse the CSV 
> with Text::CSV or simply split TSV on tabs and build a Pandoc table element 
> from the parsed structure. Where it maybe stood out was in that it could 
> convert in either direction between all three of Pandoc tables or 
> definition lists of definition lists and CSV/TSV.
>

Well hopefully having a name that describes this sort of tools could help 
their findability. If you would like to send me some links i would add them 
to the list

> ------------------------------
>

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/72a730cb-1f8f-4d7a-8e21-a241da71cc11%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[-- Attachment #1.2: Type: text/html, Size: 3555 bytes --]

Re: Pandoc sinks and sources

[Francesco Occhipinti]

[-- Attachment #1.1: Type: text/plain, Size: 7611 bytes --]


"feeders" and "consumers" sound good to me, thanks for the suggestion

On Tuesday, June 19, 2018 at 6:11:52 PM UTC+2, BP Jonsson wrote:
>
> I forgot to say that while I appreciate the usefulness of terms for these 
> programs I'm not wild about the terms "source" and "sink". When I think 
> about "pandoc source" I think about the document source file, and to 
> non-programmers "sink" may sound like you are discarding something.
>
> Perhaps "feeders" and "consumers" would be better. No rime or alliteration 
> but perhaps less ambiguous and more descriptive for non-programmers.
>
> tis 19 juni 2018 kl. 17:55 skrev BP Jonsson <bpjo...-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org 
> <javascript:>>:
>
>> I wrote JSON filters using pipes already before there was any official 
>> filter support or dedicated support libraries using the Perl data walking 
>> module Data::Rmap and a bunch of support functions of my own (basically a 
>> wrapper which took an AST and a dispatch table mapping element type names 
>> to function references, replacing original element hashes with any defined 
>> return values. Very primitive.) At first your suggestion seemed like a step 
>> backwards, but I supposed you meant that these programs should respectively 
>> take/return non-document data.
>>
>> I have written some scripts which mined data from Pandoc ASTs but they 
>> all took a filename and called out to Pandoc, basically because I didn't 
>> want to type the same command line options over and over.
>>
>> I have implemented some "custom writers" as filters which modify the AST 
>> so that the output of the plain writer looks like my idea of what email 
>> ought to look like or like valid Perl POD documentation. They could 
>> certainly be rewritten as Lua writers or to output a document instead of 
>> modified JSON but my priority was on getting the output I wanted using 
>> something written in Perl.
>>
>> I also have several filters which read config and/or partial input from 
>> external files. It seems writing a CSV to table converter is a rite of 
>> passage among Pandoc filter/addon authors.
>> Mine would take a code block with CSV, TSV or a file name, parse the CSV 
>> with Text::CSV or simply split TSV on tabs and build a Pandoc table element 
>> from the parsed structure. Where it maybe stood out was in that it could 
>> convert in either direction between all three of Pandoc tables or 
>> definition lists of definition lists and CSV/TSV.
>>
>> tis 19 juni 2018 kl. 16:31 skrev Francesco Occhipinti <
>> f.occh...-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org <javascript:>>:
>>
>>>
>>> Thanks, i added your command to the list. Well you know how it works 
>>> with language, it's less likely to talk about something whose name is 
>>> unknown. This could be the start of something. I hope that other users will 
>>> give us their feedback. I realised that a sink is similar to the concept of 
>>> a custom writer, except it can be written in any programming language 
>>> rather than just in Lua
>>>
>>> On Tuesday, June 19, 2018 at 2:25:35 PM UTC+2, mb21 wrote:
>>>>
>>>> Not sure people talk about these kinds of programs enough to warrant 
>>>> specific terminology, but the concepts certainly make sense.
>>>>
>>>> For example my pandoc-placetable 
>>>> <https://github.com/mb21/pandoc-placetable> filter acts as a "source" 
>>>> if not called as a filter but as a standalone program. So you can use it as 
>>>> an ad-hoc CSV reader:
>>>>
>>>>     pandoc-placetable --file=foo.csv --widths="0.2 0.8" | pandoc -f 
>>>> json -o output.html
>>>>
>>>> cheers
>>>>
>>>>
>>>> On Thursday, June 14, 2018 at 11:04:53 PM UTC+2, Francesco Occhipinti 
>>>> wrote:
>>>>>
>>>>>
>>>>> Hello pandoc users,
>>>>>
>>>>>  While assisting my users in a project that relies on pandoc, i found 
>>>>> myself developing some small scripts which made me think of a new 
>>>>> convention: pandoc sinks and pandoc sources. Here 
>>>>> https://github.com/italia/docs-italia-comandi-conversione/blob/master/doc/pandoc-sinks-and-sources.md 
>>>>> the idea is explained in detail, let me know what you think about it. If 
>>>>> you find this useful, we could add it to the wiki.
>>>>>
>>>>> Cheers,
>>>>> Francesco
>>>>>
>>>>> -- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "pandoc-discuss" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to pandoc-discus...-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org <javascript:>.
>>> To post to this group, send email to pandoc-...-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org 
>>> <javascript:>.
>>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/pandoc-discuss/6710819f-2df7-4311-bcc8-5275ad6e4272%40googlegroups.com 
>>> <https://groups.google.com/d/msgid/pandoc-discuss/6710819f-2df7-4311-bcc8-5275ad6e4272%40googlegroups.com?utm_medium=email&utm_source=footer>
>>> .
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>> -- 
>>
>> ------------------------------
>> SavedURI :Show URLShow URLSavedURI :
>> SavedURI :Hide URLHide URLSavedURI :
>>
>> https://mail.google.com/_/scs/mail-static/_/js/k=gmail.main.sv.G3GZFwvcniQ.O/m=m_i,t,it/am=fUAcTAoZawdGHAZ2YD-g9N_f7LL4CX7WlSgHQKgABHaCv9kToPiBD8qOMw/rt=h/d=1/rs=AItRSTO5CF1YB_frDRXLXTeUsQ1zItcBvwhttps://mail.google.com/_/scs/mail-static/_/js/k=gmail.main.sv.G3GZFwvcniQ.O/m=m_i,t,it/am=fUAcTAoZawdGHAZ2YD-g9N_f7LL4CX7WlSgHQKgABHaCv9kToPiBD8qOMw/rt=h/d=1/rs=AItRSTO5CF1YB_frDRXLXTeUsQ1zItcBvw 
>> <https://mail.google.com/_/scs/mail-static/_/js/k=gmail.main.sv.G3GZFwvcniQ.O/m=m_i,t,it/am=fUAcTAoZawdGHAZ2YD-g9N_f7LL4CX7WlSgHQKgABHaCv9kToPiBD8qOMw/rt=h/d=1/rs=AItRSTO5CF1YB_frDRXLXTeUsQ1zItcBvw> 
>> <https://mail.google.com/_/scs/mail-static/_/js/k=gmail.main.sv.G3GZFwvcniQ.O/m=m_i,t,it/am=fUAcTAoZawdGHAZ2YD-g9N_f7LL4CX7WlSgHQKgABHaCv9kToPiBD8qOMw/rt=h/d=1/rs=AItRSTO5CF1YB_frDRXLXTeUsQ1zItcBvw>
>> ------------------------------
>>
> -- 
>
> ------------------------------
> SavedURI :Show URLShow URLSavedURI :
> SavedURI :Hide URLHide URLSavedURI :
>
> https://mail.google.com/_/scs/mail-static/_/js/k=gmail.main.sv.G3GZFwvcniQ.O/m=m_i,t,it/am=fUAcTAoZawdGHAZ2YD-g9N_f7LL4CX7WlSgHQKgABHaCv9kToPiBD8qOMw/rt=h/d=1/rs=AItRSTO5CF1YB_frDRXLXTeUsQ1zItcBvwhttps://mail.google.com/_/scs/mail-static/_/js/k=gmail.main.sv.G3GZFwvcniQ.O/m=m_i,t,it/am=fUAcTAoZawdGHAZ2YD-g9N_f7LL4CX7WlSgHQKgABHaCv9kToPiBD8qOMw/rt=h/d=1/rs=AItRSTO5CF1YB_frDRXLXTeUsQ1zItcBvw 
> <https://mail.google.com/_/scs/mail-static/_/js/k=gmail.main.sv.G3GZFwvcniQ.O/m=m_i,t,it/am=fUAcTAoZawdGHAZ2YD-g9N_f7LL4CX7WlSgHQKgABHaCv9kToPiBD8qOMw/rt=h/d=1/rs=AItRSTO5CF1YB_frDRXLXTeUsQ1zItcBvw> 
> <https://mail.google.com/_/scs/mail-static/_/js/k=gmail.main.sv.G3GZFwvcniQ.O/m=m_i,t,it/am=fUAcTAoZawdGHAZ2YD-g9N_f7LL4CX7WlSgHQKgABHaCv9kToPiBD8qOMw/rt=h/d=1/rs=AItRSTO5CF1YB_frDRXLXTeUsQ1zItcBvw>
> ------------------------------
>

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/958da4cc-24fd-4d71-94df-6d4e226b9ca4%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[-- Attachment #1.2: Type: text/html, Size: 14353 bytes --]

Re: Pandoc sinks and sources

[Francesco Occhipinti]

[-- Attachment #1.1: Type: text/plain, Size: 7631 bytes --]


Thinking about this again, i think i would use "producers" and "consumers" 
for symmetry

On Tuesday, June 19, 2018 at 6:11:52 PM UTC+2, BP Jonsson wrote:
>
> I forgot to say that while I appreciate the usefulness of terms for these 
> programs I'm not wild about the terms "source" and "sink". When I think 
> about "pandoc source" I think about the document source file, and to 
> non-programmers "sink" may sound like you are discarding something.
>
> Perhaps "feeders" and "consumers" would be better. No rime or alliteration 
> but perhaps less ambiguous and more descriptive for non-programmers.
>
> tis 19 juni 2018 kl. 17:55 skrev BP Jonsson <bpjo...-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org 
> <javascript:>>:
>
>> I wrote JSON filters using pipes already before there was any official 
>> filter support or dedicated support libraries using the Perl data walking 
>> module Data::Rmap and a bunch of support functions of my own (basically a 
>> wrapper which took an AST and a dispatch table mapping element type names 
>> to function references, replacing original element hashes with any defined 
>> return values. Very primitive.) At first your suggestion seemed like a step 
>> backwards, but I supposed you meant that these programs should respectively 
>> take/return non-document data.
>>
>> I have written some scripts which mined data from Pandoc ASTs but they 
>> all took a filename and called out to Pandoc, basically because I didn't 
>> want to type the same command line options over and over.
>>
>> I have implemented some "custom writers" as filters which modify the AST 
>> so that the output of the plain writer looks like my idea of what email 
>> ought to look like or like valid Perl POD documentation. They could 
>> certainly be rewritten as Lua writers or to output a document instead of 
>> modified JSON but my priority was on getting the output I wanted using 
>> something written in Perl.
>>
>> I also have several filters which read config and/or partial input from 
>> external files. It seems writing a CSV to table converter is a rite of 
>> passage among Pandoc filter/addon authors.
>> Mine would take a code block with CSV, TSV or a file name, parse the CSV 
>> with Text::CSV or simply split TSV on tabs and build a Pandoc table element 
>> from the parsed structure. Where it maybe stood out was in that it could 
>> convert in either direction between all three of Pandoc tables or 
>> definition lists of definition lists and CSV/TSV.
>>
>> tis 19 juni 2018 kl. 16:31 skrev Francesco Occhipinti <
>> f.occh...-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org <javascript:>>:
>>
>>>
>>> Thanks, i added your command to the list. Well you know how it works 
>>> with language, it's less likely to talk about something whose name is 
>>> unknown. This could be the start of something. I hope that other users will 
>>> give us their feedback. I realised that a sink is similar to the concept of 
>>> a custom writer, except it can be written in any programming language 
>>> rather than just in Lua
>>>
>>> On Tuesday, June 19, 2018 at 2:25:35 PM UTC+2, mb21 wrote:
>>>>
>>>> Not sure people talk about these kinds of programs enough to warrant 
>>>> specific terminology, but the concepts certainly make sense.
>>>>
>>>> For example my pandoc-placetable 
>>>> <https://github.com/mb21/pandoc-placetable> filter acts as a "source" 
>>>> if not called as a filter but as a standalone program. So you can use it as 
>>>> an ad-hoc CSV reader:
>>>>
>>>>     pandoc-placetable --file=foo.csv --widths="0.2 0.8" | pandoc -f 
>>>> json -o output.html
>>>>
>>>> cheers
>>>>
>>>>
>>>> On Thursday, June 14, 2018 at 11:04:53 PM UTC+2, Francesco Occhipinti 
>>>> wrote:
>>>>>
>>>>>
>>>>> Hello pandoc users,
>>>>>
>>>>>  While assisting my users in a project that relies on pandoc, i found 
>>>>> myself developing some small scripts which made me think of a new 
>>>>> convention: pandoc sinks and pandoc sources. Here 
>>>>> https://github.com/italia/docs-italia-comandi-conversione/blob/master/doc/pandoc-sinks-and-sources.md 
>>>>> the idea is explained in detail, let me know what you think about it. If 
>>>>> you find this useful, we could add it to the wiki.
>>>>>
>>>>> Cheers,
>>>>> Francesco
>>>>>
>>>>> -- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "pandoc-discuss" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to pandoc-discus...-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org <javascript:>.
>>> To post to this group, send email to pandoc-...-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org 
>>> <javascript:>.
>>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/pandoc-discuss/6710819f-2df7-4311-bcc8-5275ad6e4272%40googlegroups.com 
>>> <https://groups.google.com/d/msgid/pandoc-discuss/6710819f-2df7-4311-bcc8-5275ad6e4272%40googlegroups.com?utm_medium=email&utm_source=footer>
>>> .
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>> -- 
>>
>> ------------------------------
>> SavedURI :Show URLShow URLSavedURI :
>> SavedURI :Hide URLHide URLSavedURI :
>>
>> https://mail.google.com/_/scs/mail-static/_/js/k=gmail.main.sv.G3GZFwvcniQ.O/m=m_i,t,it/am=fUAcTAoZawdGHAZ2YD-g9N_f7LL4CX7WlSgHQKgABHaCv9kToPiBD8qOMw/rt=h/d=1/rs=AItRSTO5CF1YB_frDRXLXTeUsQ1zItcBvwhttps://mail.google.com/_/scs/mail-static/_/js/k=gmail.main.sv.G3GZFwvcniQ.O/m=m_i,t,it/am=fUAcTAoZawdGHAZ2YD-g9N_f7LL4CX7WlSgHQKgABHaCv9kToPiBD8qOMw/rt=h/d=1/rs=AItRSTO5CF1YB_frDRXLXTeUsQ1zItcBvw 
>> <https://mail.google.com/_/scs/mail-static/_/js/k=gmail.main.sv.G3GZFwvcniQ.O/m=m_i,t,it/am=fUAcTAoZawdGHAZ2YD-g9N_f7LL4CX7WlSgHQKgABHaCv9kToPiBD8qOMw/rt=h/d=1/rs=AItRSTO5CF1YB_frDRXLXTeUsQ1zItcBvw> 
>> <https://mail.google.com/_/scs/mail-static/_/js/k=gmail.main.sv.G3GZFwvcniQ.O/m=m_i,t,it/am=fUAcTAoZawdGHAZ2YD-g9N_f7LL4CX7WlSgHQKgABHaCv9kToPiBD8qOMw/rt=h/d=1/rs=AItRSTO5CF1YB_frDRXLXTeUsQ1zItcBvw>
>> ------------------------------
>>
> -- 
>
> ------------------------------
> SavedURI :Show URLShow URLSavedURI :
> SavedURI :Hide URLHide URLSavedURI :
>
> https://mail.google.com/_/scs/mail-static/_/js/k=gmail.main.sv.G3GZFwvcniQ.O/m=m_i,t,it/am=fUAcTAoZawdGHAZ2YD-g9N_f7LL4CX7WlSgHQKgABHaCv9kToPiBD8qOMw/rt=h/d=1/rs=AItRSTO5CF1YB_frDRXLXTeUsQ1zItcBvwhttps://mail.google.com/_/scs/mail-static/_/js/k=gmail.main.sv.G3GZFwvcniQ.O/m=m_i,t,it/am=fUAcTAoZawdGHAZ2YD-g9N_f7LL4CX7WlSgHQKgABHaCv9kToPiBD8qOMw/rt=h/d=1/rs=AItRSTO5CF1YB_frDRXLXTeUsQ1zItcBvw 
> <https://mail.google.com/_/scs/mail-static/_/js/k=gmail.main.sv.G3GZFwvcniQ.O/m=m_i,t,it/am=fUAcTAoZawdGHAZ2YD-g9N_f7LL4CX7WlSgHQKgABHaCv9kToPiBD8qOMw/rt=h/d=1/rs=AItRSTO5CF1YB_frDRXLXTeUsQ1zItcBvw> 
> <https://mail.google.com/_/scs/mail-static/_/js/k=gmail.main.sv.G3GZFwvcniQ.O/m=m_i,t,it/am=fUAcTAoZawdGHAZ2YD-g9N_f7LL4CX7WlSgHQKgABHaCv9kToPiBD8qOMw/rt=h/d=1/rs=AItRSTO5CF1YB_frDRXLXTeUsQ1zItcBvw>
> ------------------------------
>

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/317048ac-6182-46fb-9c1c-bb8735da0902%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[-- Attachment #1.2: Type: text/html, Size: 14341 bytes --]