From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.io/gmane.text.pandoc/14984 Path: news.gmane.org!not-for-mail From: BP Jonsson Newsgroups: gmane.text.pandoc Subject: Re: Markdown, tables and CSV Date: Fri, 20 May 2016 21:15:09 +0200 Message-ID: References: <047d7b86ebe83c062b05332eab9b@google.com> <20BF19CB-A2B0-4B19-A749-D750CDD89736@martinfenner.org> <1463766905.1918988.613990665.6CD67781@webmail.messagingengine.com> Reply-To: pandoc-discuss-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: multipart/alternative; boundary=001a1149e452cdb9a705334ae806 X-Trace: ger.gmane.org 1463772114 18105 80.91.229.3 (20 May 2016 19:21:54 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Fri, 20 May 2016 19:21:54 +0000 (UTC) To: "pandoc-discuss-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org" Original-X-From: pandoc-discuss+bncBDIY76M674FRBPWE7W4QKGQE3RWKBQI-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org Fri May 20 21:21:54 2016 Return-path: Envelope-to: gtp-pandoc-discuss@m.gmane.org Original-Received: from mail-pf0-f187.google.com ([209.85.192.187]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1b3pt9-0004w8-VK for gtp-pandoc-discuss@m.gmane.org; Fri, 20 May 2016 21:15:12 +0200 Original-Received: by mail-pf0-f187.google.com with SMTP id g132sf41383692pfb.0 for ; Fri, 20 May 2016 12:15:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20120806; h=sender:mime-version:in-reply-to:references:date:message-id:subject :from:to:x-original-sender:x-original-authentication-results :reply-to:precedence:mailing-list:list-id:x-spam-checked-in-group :list-post:list-help:list-archive:list-subscribe:list-unsubscribe; bh=i2gBeecidb2Sh7E6Ad+xr9Wg1dvGfC/yvWDdmACjntQ=; b=vwk8nhBPy9/A/ArGOzLDhym+uUBJ/XPWJovCQaw+ZXd735DAD6lbz+tlYRDKJEbmcR 8Mb0cqoIbC/1VBBNUzNZ77NyQtM1t1nttJtw86eFEuMD4eUYA9UKwP4t2eFL/UVJSZaU 9Qcg3kZw4XezRKSxR4zGzKzhr3aVcx/ghREIq6Ot3X433b742EO9NFQh/GwpZjRqM3SX 8TMkOEVMIyiCQ1jiI5SPQznkxSIasXUHXO0ZHPQOyoBDovG5MVECvbL8owI4O2uosQ0z +4+mBMA0i6fLxttl9YkIyEfgpOWuoBMgAUEti1HBLgpKJ63kM+sHYI0z4dojwix7vQIi 2elQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :x-original-sender:x-original-authentication-results:reply-to :precedence:mailing-list:list-id:x-spam-checked-in-group:list-post :list-help:list-archive:list-subscribe:list-unsubscribe; bh=i2gBeecidb2Sh7E6Ad+xr9Wg1dvGfC/yvWDdmACjntQ=; b=JljH4Wu3TW+zOn6kBb/ANIh4wsFOER4V+KKLufNPz4JqwJFDO+fE6Q5v15mVmHQTi/ PhqONFoJYNlao5b/NlaIEO4mg/XZyo5bv/Z/fOyMDsBnrPg82yVEAYQI8ka0WLq84KNR i8OtEnHSCnX43ra53Wb4d2hHZ+SG9EaORTKmDU2zQEAeheeIP1Q9gR5C5vdEXShxekQf 6rHGlH4bqBgPAlLWaZMPJfQY8+gD1vs4LlJsw3BMbZIJorMcNe9JIcTUxt/jBf1fGC2X 1Jv3Zi1lwCQG/YAdUCnsOASLB3hElkr2mxoZ7YITeUxW9A+tofucvKEF1EIt9d5MqrYF 25Qw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=sender:x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:x-original-sender :x-original-authentication-results:reply-to:precedence:mailing-list :list-id:x-spam-checked-in-group:list-post:list-help:list-archive :list-subscribe:list-unsubscribe; bh=i2gBeecidb2Sh7E6Ad+xr9Wg1dvGfC/yvWDdmACjntQ=; b=NSYkzAS1B42z2p2f/EQnac9I7N1/nrQ+kPfsPDTnGXueVpLRGLCyNBaJBQXKbd4ZEh cGwYB7Mmvp3JYeIyaagVLuP//DlYfwkutA+EDbDr9sQVrSx8MsqAZeJZ6hRg6F6r+y+s oVQWf0aHhMpi1e/WQW/PxiaqfB10zSth4oghNm+crljMAwfxkAflpE+vx9Xff3Lu003I CSlixol/08I6vhlDb3ogex57+0rPhFbzYbHoWNqVhsyCJo+gNDGxPfoz9OChVq3J/K/W LSL9fxfX3oACnUY7jel21FYdoroxT935u7M1ZQHuwUU1FFYmzBct+eIB15SKZd1OtfEo qmCQ== Original-Sender: pandoc-discuss-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org X-Gm-Message-State: AOPr4FXzx0UGK0QKj3sJZZSjhb4KaXTO92U9j1z/2rbRyReROIqlnmx4wyzKeA5r15E+oQ== X-Received: by 10.140.43.161 with SMTP id e30mr160309qga.17.1463771710831; Fri, 20 May 2016 12:15:10 -0700 (PDT) X-BeenThere: pandoc-discuss-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org Original-Received: by 10.140.104.36 with SMTP id z33ls2792851qge.17.gmail; Fri, 20 May 2016 12:15:10 -0700 (PDT) X-Received: by 10.140.29.198 with SMTP id b64mr3103090qgb.29.1463771710321; Fri, 20 May 2016 12:15:10 -0700 (PDT) Original-Received: from mail-oi0-x22d.google.com (mail-oi0-x22d.google.com. [2607:f8b0:4003:c06::22d]) by gmr-mx.google.com with ESMTPS id l184si307594itg.0.2016.05.20.12.15.10 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 20 May 2016 12:15:10 -0700 (PDT) Received-SPF: pass (google.com: domain of bpjonsson-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org designates 2607:f8b0:4003:c06::22d as permitted sender) client-ip=2607:f8b0:4003:c06::22d; Original-Received: by mail-oi0-x22d.google.com with SMTP id x19so192781315oix.2 for ; Fri, 20 May 2016 12:15:10 -0700 (PDT) X-Received: by 10.157.20.101 with SMTP id h92mr3106855oth.114.1463771709942; Fri, 20 May 2016 12:15:09 -0700 (PDT) Original-Received: by 10.157.13.136 with HTTP; Fri, 20 May 2016 12:15:09 -0700 (PDT) In-Reply-To: <1463766905.1918988.613990665.6CD67781-2RFepEojUI2N1INw9kWLP6GC3tUn3ZHUQQ4Iyu8u01E@public.gmane.org> X-Original-Sender: bpjonsson-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org X-Original-Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@gmail.com; spf=pass (google.com: domain of bpjonsson-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org designates 2607:f8b0:4003:c06::22d as permitted sender) smtp.mailfrom=bpjonsson-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org; dmarc=pass (p=NONE dis=NONE) header.from=gmail.com Precedence: list Mailing-list: list pandoc-discuss-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org; contact pandoc-discuss+owners-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org List-ID: X-Spam-Checked-In-Group: pandoc-discuss-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org X-Google-Group-Id: 1007024079513 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , Xref: news.gmane.org gmane.text.pandoc:14984 Archived-At: --001a1149e452cdb9a705334ae806 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable I like the idea of overloading image syntax for includes, but I'm a little concerned how format recognition by extension would work out in practice. Most extensions that would likely come up map only to one fiile format but some file file formats map to multiple extensions, notably markdown, and for backwards compatibility's sake one would have to assume that an 'unknown' extension is an image. However now that image elements take attributes it would be possible to use an "include" pseudo-attribute with the same formats as used for the --from option as values, including extensions if any. I would guess that a filter implementing that would not be too hard to implement in Haskell, which can call pandoc as a library. Shelling out to pandoc from python or perl would be possible. Also the include would probably have to be an image element as only content since the included document would consist of block elements. /bpj fredag 20 maj 2016 skrev John Gabriele : > Hi Martin, > > There's also [issue 553](https://github.com/jgm/pandoc/issues/553). > > Personally, I think I like (from that issue thread) anton-k's original > idea: > > ![an image](foo.jpg) > > ![an include](foo.txt) > > ![a csv to be rendered as a table](foo.csv) > > (that is, based on filename extension) > > Those seem sensible, symmetrical, easy to remember, and I think fit well > with what pandoc already does (`![]()` is already like an include). > > As for a syntax to allow writing your csv data right into your md file, > ... Pandoc already supports a generous number of table formats that are > pretty easy to type. And for larger tables that you might be tempted to > copy/paste in, might be better easier to bang-include them (as in, > `![]()`), rather than muck up your pretty markdown file with a giant bunc= h > of csv data. :) > > -- John > > > > > On Fri, May 20, 2016, at 05:38 AM, Martin Fenner wrote: > > Dear group, > > The topic of CSV support in Pandoc has come up several times on this list= , > includes this thread from 2014: > https://groups.google.com/forum/#!topic/pandoc-discuss/kBdJU_JktzI > > Since last year I work for an organisation that frequently deals with > tabular data (and helped organize CSVconf earlier this month), and I have > done some thinking on how CSV could fit into Pandoc. I see two important > use cases: > > * CSV reader that converts to tables in HTML, docx, latex, etc. > * CSV has a format to describe tables in markdown > > For the first use case I wrote a hack for the Jekyll blogging platform > this week that turns CSV files into markdown grid tables format that is > then processed by Pandoc (https://github.com/datacite/jekyll-csvy). I > would rather use Pandoc with a CSV reader, but my Haskell isn't good enou= gh > to write one. But for now I can generate blog posts directly from CSV > files. Other people have done similar things with Pandoc and CSV. > > For the second use case I see a clear advantage of CSV over the various > attempts to format tables in markdown (simple_tables, multiline_tables, > grid_tables, pipe_tables). Everyone (and many tools) understands the CSV > format, and you can do most of the things with CSV that the other table > formats allow (multi-column formats and column alignment are a bit > trickier). This has been done before using Pandoc filters, but I think a > Pandoc "csv_tables" Pandoc extension would make this easier for the casua= l > user. Using the grid_tables example from the Pandoc documentation, this > could look like this: > > : Sample csv table. > > ,,, > Fruit,Price,Advantages > Bananas,$1.34,- built-in wrapper\n- bright color > Oranges,$2.10, - cures scurvy\n- tasty > ,,, > > I like three commas on a new line to indicate the start and end of a > table, but that is of course open for discussion. The format is much easi= er > to read and edit for humans compared to grid tables, the only tricky bit = is > maybe the \n for multiline columns. I would think we could add metadata t= o > the fenced table blog similar to code blocks, e.g. > > ,,,{ #mytable .numberRows } > > One challenge with CSV is that it is an ill-defined format somewhat > similar to markdown before CommonMark. It may make things easier to only > support a specific CSV variant (e.g. comma as separator, header required, > comment lines not allowed). > > Thoughts? > > Best, > > Martin > > > > > > -- > You received this message because you are subscribed to the Google Groups > "pandoc-discuss" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org > To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org > To view this discussion on the web visit > https://groups.google.com/d/msgid/pandoc-discuss/20BF19CB-A2B0-4B19-A749-= D750CDD89736%40martinfenner.org > > . > For more options, visit https://groups.google.com/d/optout. > > > > -- > You received this message because you are subscribed to the Google Groups > "pandoc-discuss" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org > To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org > To view this discussion on the web visit > https://groups.google.com/d/msgid/pandoc-discuss/1463766905.1918988.61399= 0665.6CD67781%40webmail.messagingengine.com > > . > For more options, visit https://groups.google.com/d/optout. --=20 ------------------------------ SavedURI :Show URLShow URLSavedURI : SavedURI :Hide URLHide URLSavedURI : https://mail.google.com/_/scs/mail-static/_/js/k=3Dgmail.main.sv.G3GZFwvcni= Q.O/m=3Dm_i,t,it/am=3DfUAcTAoZawdGHAZ2YD-g9N_f7LL4CX7WlSgHQKgABHaCv9kToPiBD= 8qOMw/rt=3Dh/d=3D1/rs=3DAItRSTO5CF1YB_frDRXLXTeUsQ1zItcBvwhttps://mail.goog= le.com/_/scs/mail-static/_/js/k=3Dgmail.main.sv.G3GZFwvcniQ.O/m=3Dm_i,t,it/= am=3DfUAcTAoZawdGHAZ2YD-g9N_f7LL4CX7WlSgHQKgABHaCv9kToPiBD8qOMw/rt=3Dh/d=3D= 1/rs=3DAItRSTO5CF1YB_frDRXLXTeUsQ1zItcBvw ------------------------------ --=20 You received this message because you are subscribed to the Google Groups "= pandoc-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an e= mail to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To view this discussion on the web visit https://groups.google.com/d/msgid/= pandoc-discuss/CAFC_yuQe9oFtnkjLYfAYgAqgzYrM6XBZrBPOHui%3DHG14U4X2Zg%40mail= .gmail.com. For more options, visit https://groups.google.com/d/optout. --001a1149e452cdb9a705334ae806 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable I lik= e the idea of overloading image syntax for includes, but I'm a little c= oncerned how format recognition by extension would work out in practice. Mo= st extensions that would likely come up map only to one fiile format=C2=A0b= ut some file file formats map to multiple extensions, notably markdown, and= for backwards compatibility's sake=C2=A0one would have to assume that = an 'unknown' extension is an image. However now that image elements= take attributes it would be possible to use an "include" pseudo-= attribute with the same formats as used for the --from option as values, in= cluding extensions if any. I would=C2=A0guess that a filter implementing th= at would not be too hard to implement in Haskell, which can call pandoc as = a library. Shelling out to pandoc from python or perl would be possible. Al= so the include would probably have to be an image element as only content s= ince the included document would consist of block elements.
/bpj

fredag 20 maj 2016 skrev John Gabriele &l= t;jgabriele-97jfqw80gc6171pxa8y+qA@public.gmane.org>:
Hi Martin,
=C2=A0<= /span>
=C2=A0
Personally= , I think I like (from that issue thread) anton-k's original idea:
<= /span>
=C2=A0
=C2=A0=C2=A0=C2=A0 ![an image]= (foo.jpg)
=C2=A0
=C2=A0=C2=A0=C2= =A0 ![an include](foo.txt)
=C2=A0
=C2=A0=C2=A0=C2=A0 ![a csv to be rendered as a table](foo.csv)
<= /font>
=C2=A0
(that is, based on filename extension= )
=C2=A0
Those seem sensible, sym= metrical, easy to remember, and I think fit well with what pandoc already d= oes (`![]()` is already like an include).
=C2=A0
As for a syntax to allow writing your csv data right into y= our md file, ... Pandoc already supports a generous number of table formats= that are pretty easy to type. And for larger tables that you might be temp= ted to copy/paste in, might be better easier to bang-include them (as in, `= ![]()`), rather than muck up your pretty markdown file with a giant bunch o= f csv data. :)
=C2=A0
-- John
=
=C2=A0
=C2=A0
=C2= =A0
=C2=A0
On Fri, May 20, 2016, at 0= 5:38 AM, Martin Fenner wrote:
Dear group,
=C2=A0
The topic = of CSV support in Pandoc has come up several times on this list, includes t= his thread from 2014:
=C2=A0
Since last year I = work for an organisation that frequently deals with tabular data (and helpe= d organize CSVconf earlier this month), and I have done some thinking on ho= w CSV could fit into Pandoc. I see two important use cases:
=C2=A0
* CSV reader that converts to tables in H= TML, docx, latex, etc.
* CSV has a format to describ= e tables in markdown
=C2=A0
= For t= he first use case I wrote a hack for the Jekyll blogging platform this week= that turns CSV files into markdown grid tables format that is then process= ed by Pandoc (https://github.com/datacite/jekyll-csvy). I would rather use P= andoc with a CSV reader, but my Haskell isn't good enough to write one.= But for now I can generate blog posts directly from CSV files. Other peopl= e have done similar things with Pandoc and CSV.
=C2= =A0
For the second use case I see a clear advantage of C= SV over the various attempts to format tables in markdown (simple_tables, m= ultiline_tables, grid_tables, pipe_tables). Everyone (and many tools) under= stands the CSV format, and you can do most of the things with CSV that the = other table formats allow (multi-column formats and column alignment are a = bit trickier). This has been done before using Pandoc filters, but I think = a Pandoc "csv_tables" Pandoc extension would make this easier for= the casual user. Using the grid_tables example from the Pandoc documentati= on, this could look like this:
=C2=A0<= /div>
: Sample csv table.
= =C2=A0
,,,
Fruit,Price,Advantages
Bananas,$1.34,- built-in wrapper\n- bright color
<= div>O= ranges,$2.10, - cures scurvy\n- tasty
,,,
=
=C2=A0
I like three commas on a new l= ine to indicate the start and end of a table, but that is of course open fo= r discussion. The format is much easier to read and edit for humans compare= d to grid tables, the only tricky bit is maybe the \n for multiline columns= . I would think we could add metadata to the fenced table blog similar to c= ode blocks, e.g.
=C2=A0
,,,{ #m= ytable .numberRows }
=C2=A0
= One c= hallenge with CSV is that it is an ill-defined format somewhat similar to m= arkdown before CommonMark. It may make things easier to only support a spec= ific CSV variant (e.g. comma as separator, header required, comment lines n= ot allowed).
=C2=A0
Thoughts?
=
=C2=A0
Best,
=
= =C2=A0
Martin
=C2=A0
=C2=A0
=C2=A0


=

--
You received this message because you are sub= scribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it= , send an email to=C2=A0pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org.=
To post to this group, send email to=C2=A0pandoc= -discuss-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org.
For more options, visit=C2=A0https://groups.googl= e.com/d/optout.
=C2=A0

--=C2=A0
You received this message because you are subscr= ibed to the Google Groups "pandoc-discuss" group.
To unsubscri= be from this group and stop receiving emails from it, send an email to=C2= =A0pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org.
To post to this g= roup, send email to=C2=A0pandoc-discuss-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org.
To view= this discussion on the web visit=C2=A0https://groups.google.com/d/msgid/pandoc-discuss/1463766905.1918988.613= 990665.6CD67781%40webmail.messagingengine.com.
For more options, vis= it=C2=A0ht= tps://groups.google.com/d/optout.

<= br>--

--
You received this message because you are subscribed to the Google Groups &= quot;pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an e= mail to pand= oc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org.
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org.
To view this discussion on the web visit https://group= s.google.com/d/msgid/pandoc-discuss/CAFC_yuQe9oFtnkjLYfAYgAqgzYrM6XBZrBPOHu= i%3DHG14U4X2Zg%40mail.gmail.com.
For more options, visit http= s://groups.google.com/d/optout.
--001a1149e452cdb9a705334ae806--