Re: Markdown, tables and CSV

[Kolen Cheung <christian.kolen-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>]

[-- Attachment #1: Type: text/plain, Size: 3847 bytes --]

Great insight. This is the disadvantage of the easiness or writing filters, together with the fact that they are decentralized. The result is the design of filters usually only go through a brief period and not a lot of input from the others. While syntaxes and features in pandoc took a long time to discuss and settle, it focuses on the big picture (both in terms of syntax and generality).

I find it difficult to find a solution to the problem, here's my thought:

First, consider only CSV tables, I think it is actually more natural to write a CSV reader rather than a filter, hence it will read a csv file and convert it to AST on cli. I suggested it to pandoc-placetable and I think I should do the same to mine. Whether or not it will make into pandoc is irrelevant (but I very much hope so!), the important point is it becomes a reader that turns CSV into AST. Then the second part would be most useful: define a standard syntax that *everyone* will be happy with, that includes arbitrary raw formats in the markdown source. (There were already efforts on, say, raw TeX because sometimes pandoc will accidentally parsed the TeX. )The point here is to provide a syntax such that it is clear that the following codeblock will be included (HTML, LaTeX, rst, even docx with a file name, etc.) and optionally parsed by the reader pandoc/filter knows (for binary format, parsing will be mandatory).

Second, that's why I suggested here and there (#2 misc idea on panflute) that there should be a centralized filter gallery. The point is not all filters should go to there (you can't force them anyway), but that for people care enough to share filters for reuse together will go through the design *together*. People can have their own repo for testing or even distributing over standard channel like cabal/pip, but when they submitted to the gallery, it will be santinized (for security/quality). It doesn't mean it will be very rigorous either (who has the time to do that?), but as long as it is centralized, the awareness of the other filters in the repo will also remind them to think about the big picture (currently it is very difficult to have a big picture of the current filters, which is why I started to catalog current filters and hopes to analyze them). And then the issue tracker there may also serves as a discussion on creating fitlers (or may be just here).

Lastly, this is most difficult but exciting. Bringing a analogy from LaTeX: something like the memoir class will be awesome. i.e. a certain guy knows the existing filters and use cases enough, and has good taste and the skills, and have enough time and will to do it: a big filter that is general enough to do most of the things and has a syntax clear and general enough to build compatible filters to use with it. (by the way, currently the compatibility between filters is not known, at least not well documented.)

To summaries, the ability to use filters (on internal AST) is what sets apart pandoc. But it has a lot of potential than currently realized. To bring the analogy from TeX and LaTeX again, filter+pandoc can be like La+TeX.

P.S. I realized I digressed, only the 1st point belongs to this thread, the others should probably belongs to the "ecosystem" thread.

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/620ebba4-4753-4909-a0a2-cc3f2218cc26%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.