Re: Markdown, tables and CSV

[Kolen Cheung <christian.kolen-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>]

[-- Attachment #1: Type: text/plain, Size: 5355 bytes --]

The panflute filter I wrote on csv-tables is almost finishing, I wrote extensive testing today, and probably I would submit it to PyPI tomorrow. Some of the unique features is auto width and talb-width, design not to fail on crappy source (following pandoc's practice). I decided to call it pantable as a wordplay on pandoc, and means a subset of pandoc, and probably will support xlsx or html_tables with slight modification.

I understand that "yet another pandoc csv table filter" seems not a good idea. And believe me, I strongly believe in streamlining pandoc experience, and hope to integrate filters to work together in harmony. So may be let me explain why I wrote mine:

I've been considering writing (and re-writing) filters in haskwell or python with panflute. Haskell almost won me over with the performance and tigher interaction with pandoc. But, (besides learning curve on picking up Haskell) the main difficulties I have around haskell filters is distribution, basically only those who have programming and command-line background can handle it. This might not be a big deal for some situation. But for the project I'm working on, it is extremely important. My department hired me this semester to rewrite one of the introductory physics course workbooks. What they ask for is the "traditional" way of writing workbooks. The old sources were partly Word partly TeX. Last update is about a decade ago. And they hire one person for a semester to finish the job and that's it. So the quality of the workbook is, as you can imagine, sub-par. But my idea is to:

1. turn it into GitHub repo, openning it up for all GSIs teaching those courses for download and edit. (open-source is under consideration, but I can guess it is not their priority even if they don't object to the idea)
2. use of pandoc, rather than pure LaTeX, so that multiple output can be simultaneously targetted. It also lower the barrier for any GSIs to contribute (many new graduate students do not know LaTeX)

All these is supposed to accerlerate development, partly by platform and partly by lowering the barrier (technological, mental, etc.). The last thing I want is to make the build process daunting or fragile (cabal dependency hell?)

I have suggested on the first meeting to use pandoc, and they amost immediately rejected it. But since I need to use pandoc to convert the doc (not docx! i.e. indirectly), I demostrated the capability of pandoc and what it is. I prepared it to be an intermediate process only. And in my design of the makefile, I prepare to nuke any existance of pandoc and leave a way to export the project in pure LaTeX only. However, "they" are convinced  to use pandoc markdown as the source after seeing what it is capable of.

Even after we settled for using pandoc, there's still a lot of uncertainties. The "they" I refered to who are convinced to adopt pandoc, are fired deal to budget cut. Who knows who is to take over and what he will think about pandoc (probably will be an old senior staff, used to old tools). And also because of the budget cut, they probably are not hiring me to continue to develop the workbook (I got teaching offer instead). (a hint on the poor (in both sense) university that got so much budget cut since 2009, it is the same as pandoc's author's university.)

That's the primary reason I've been thinking about the pandoc ecosystem and how to lower the barrier to use filters. The last thing I want to see is whoever the future hired person to work on the project nuke the pandoc source (and yes, I will provide the "nuke button"). (I even afraid they will nuke my makefile and the whole build process that I keep perfecting for weeks to handle single source to develop multiple series of workbooks. You know, I can imagine someone picking it up and say, "What's that makefile? Let's double click the tex and build in TeXShop" (or even worse convert it to LyX).)

P.S. However, the situation of pandoc filters might get brighter soon. There's some sort of pandoc package manager in development (so far panflute only). And I have a vague idea on a pre-build big binaries to include some of the useful filters (including those in Haskell). Using TeX terminologies, they are something like tlmgr and texlive distribution. I personally believe that these are very important for the longevity of the tool we all loves about: they are all about lower barrier, easier to use, easier to write filters (I believe addressed by panflute), zero maintenance, easier configuration, even GUI (say, Atom packages). (Right now pandoc seems to be for "hackers" only.) And let's not to mention ARM and appstore compatibility (podoc seems to have limited capability to parse pandoc in Python and has an appstore friendly license).

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/d847d3af-73fd-41d1-96e8-2c3a0dc9d70a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.