Pandoc Projects Ideas for Summer of Haskell 2018?

Pandoc Projects Ideas for Summer of Haskell 2018?

[Jasper Van der Jeugt]

Hey all,

Google Summer of Code will take place again in 2018.  Last year,
Haskell.org was not selected, and we decided to run our own program,
which ended very successfully.

Currently, we are in the process of gathering ideas for the students.  I
think it would be great to have a Pandoc project in there.  Projects
should be concrete and small enough in scope such that they can be
finished by a student in three months.

Adding a specific output or input format to Pandoc sounds like a solid
project that is not too large.  Which are the most requested/needed
backends?  Would any Pandoc developer be willing to mentor such a
project?

For context, Google Summer of Code is a program where Google sponsors
students to work on open-source projects during the summer. Haskell.org
has taken part in this program from 2006 until 2015.  Many important
improvements to the ecosystem have been the direct or indirect result of
Google Summer of Code projects, and it has also connected new people
with the existing community.

Kind regards, on behalf of the Haskell.org committee
Jasper Van der Jeugt

Re: Pandoc Projects Ideas for Summer of Haskell 2018?

[John MacFarlane]

I've mentored a SOC student before on a pandoc-related project,
and I'd certainly consider doing it again.

A couple years ago we put up an issue with some ideas:
https://github.com/jgm/pandoc/issues/1852

The highest priority item would be
https://github.com/jgm/pandoc/issues/1024,
improving the internal representation of tables so that
we can have colspans and other features.  Not sure I want
to wait all the way til summer for this, though.

A pure Haskell commonmark parser would be fantastic.

A direct writer for the Kindle format K8 (#1865) would
also be interesting.

Looking through the issue tracker would be a good first
step.

+++ Jasper Van der Jeugt [Dec 26 17 12:32 ]:
>Hey all,
>
>Google Summer of Code will take place again in 2018.  Last year,
>Haskell.org was not selected, and we decided to run our own program,
>which ended very successfully.
>
>Currently, we are in the process of gathering ideas for the students.  I
>think it would be great to have a Pandoc project in there.  Projects
>should be concrete and small enough in scope such that they can be
>finished by a student in three months.
>
>Adding a specific output or input format to Pandoc sounds like a solid
>project that is not too large.  Which are the most requested/needed
>backends?  Would any Pandoc developer be willing to mentor such a
>project?
>
>For context, Google Summer of Code is a program where Google sponsors
>students to work on open-source projects during the summer. Haskell.org
>has taken part in this program from 2006 until 2015.  Many important
>improvements to the ecosystem have been the direct or indirect result of
>Google Summer of Code projects, and it has also connected new people
>with the existing community.
>
>Kind regards, on behalf of the Haskell.org committee
>Jasper Van der Jeugt
>
>-- 
>You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
>To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
>To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
>To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/20171226113216.GA9551%40colony6.localdomain.
>For more options, visit https://groups.google.com/d/optout.

Re: Pandoc Projects Ideas for Summer of Haskell 2018?

[Albert Krewinkel]

Another project could be the development of a standardized system to run
code snippets, similar to org-mode's "babel" system or RStudio's
rmarkdown. I started "rundoc" some years ago, but I never got far (due
to time constraints and prioritizing on the org reader). Rebuilding the
core of this tool in an extensible way should fit well within the three
month time frame, and I'd be willing to serve as a mentor for that
project.

Re: Pandoc Projects Ideas for Summer of Haskell 2018?

[Jasper Van der Jeugt]

Thanks for the suggestions.

I think the CommonMark suggestion sounds good.  The column span issue
might be too urgent, and also too hard since it touches almost all
readers and writers.  I can write this up and then link to the list
you provided for more suggestions.

Albert, I'm not familiar enough with babel/org-mode to summarize it
into an idea.  Would you be willing to contribute that?

Jasper

Re: Pandoc Projects Ideas for Summer of Haskell 2018?

[John MacFarlane]

>I think the CommonMark suggestion sounds good.

An addendum on this:  we have written some very efficient
CommonMark parsers (cmark and commonmark.js).  However,
the parsing algorithms are very imperative and rely on
mutable data structures.  It's an interesting question
how to write nice functional CommonMark parsers that
have some of the performance properties of these imperative
parsers:  fast, non-backtracking, no space/time blowups
parsing pathological cases.  Ideally, it would be easy
to extend them.

Currently pandoc uses a wrapper around libcmark (the
C parser) for commonmark and gfm.  Having a pure Haskell
parser would improve security and allow us to add more
extensions.  It would also allow compilation to JavaScript
with ghcjs.

Re: Pandoc Projects Ideas for Summer of Haskell 2018?

[lf-araujo]

[-- Attachment #1.1: Type: text/plain, Size: 1129 bytes --]



Another project could be the development of a standardized system to run 
> code snippets, similar to org-mode's "babel" system or RStudio's 
> rmarkdown. I started "rundoc" some years ago, but I never got far (due 
> to time constraints and prioritizing on the org reader). Rebuilding the 
> core of this tool in an extensible way should fit well within the three 
> month time frame, and I'd be willing to serve as a mentor for that 
> project. 
>
It is already possible to achieve this with pp 
<https://github.com/CDSoft/pp>. 
​

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/ebfb4a5f-d2b6-46bc-b5ea-9b725f88c69a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[-- Attachment #1.2: Type: text/html, Size: 3125 bytes --]

Re: Pandoc Projects Ideas for Summer of Haskell 2018?

[Albert Krewinkel]

This is cool, thanks for the link! I have to examine pp a bit more, but
my understanding so far is that Org-mode's Babel is quite a bit more
powerful/flexible, as is RStudio. E.g., there seems to be no
(interpreter) session handling in pp.

Regardless, it seems better to extend pp than to rebuild rundoc, which
is why I retract my proposal.

Re: Pandoc Projects Ideas for Summer of Haskell 2018?

[lf-araujo]

[-- Attachment #1.1: Type: text/plain, Size: 1516 bytes --]

RRstudio + rmarkdown is more powerful than pandoc alone. However, rmarkdown 
design decisions are questionable, I take an issue with the amount of files 
that get thrown into the directory one is working with. PP as it is right 
now, may be able to allow a "rebuild" of something similar to rmarkdown. At 
least potentially. 

I did something with it, a memoir-based tufte-like style that also 
incorporates R using pp. It is a WIP at the moment: 
[here](https://lf-araujo.github.io/2017/11/10/tuftish.html).

On Friday, 29 December 2017 08:36:48 UTC+11, Albert Krewinkel wrote:
>
> This is cool, thanks for the link! I have to examine pp a bit more, but 
> my understanding so far is that Org-mode's Babel is quite a bit more 
> powerful/flexible, as is RStudio. E.g., there seems to be no 
> (interpreter) session handling in pp. 
>
> Regardless, it seems better to extend pp than to rebuild rundoc, which 
> is why I retract my proposal. 
>

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/d01524c7-8705-41a5-a2b4-9499bdadc43c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[-- Attachment #1.2: Type: text/html, Size: 2090 bytes --]

Re: Pandoc Projects Ideas for Summer of Haskell 2018?

[Jasper Van der Jeugt]

[-- Attachment #1.1: Type: text/plain, Size: 732 bytes --]

I have added an idea proposal here:

    https://summer.haskell.org/ideas.html#pandoc-commonmark

Let me know if you want to change anything about the wording.

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/1df69e0b-adf1-4f56-a590-b44473269eb2%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[-- Attachment #1.2: Type: text/html, Size: 1163 bytes --]

Re: Pandoc Projects Ideas for Summer of Haskell 2018?

[John MACFARLANE]

I'd suggest having the project create a standalone
pure Haskell commonmark parsing library that exports
something like

    parseCommonMark :: Options -> Text -> Pandoc

rather than contributing code directly to pandoc.

This library would have to depend on pandoc-types, but not
on pandoc as a whole. That would allow the library to be
used by other things (e.g. a future incarnation of Haddock?)
without drawing in all of pandoc.  Pandoc could use it too.

Note that pandoc-types is dual-licensed, so people
could use the library without making their software
GPL. That would also be an advantage for a lot of people.

The only architectural drawback of this approach is that
the parser would not be in PandocMonad, so it couldn't
take advantage of the logging, media, and file handling
capabilities that gives us.  But, if we needed these
things, we could always copy the code and create a new
module in pandoc later.

+++ Jasper Van der Jeugt [Jan 06 18 05:55 ]:
>   I have added an idea proposal here:
>       https://summer.haskell.org/ideas.html#pandoc-commonmark
>   Let me know if you want to change anything about the wording.
>
>   --
>   You received this message because you are subscribed to the Google
>   Groups "pandoc-discuss" group.
>   To unsubscribe from this group and stop receiving emails from it, send
>   an email to [1]pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
>   To post to this group, send email to
>   [2]pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
>   To view this discussion on the web visit
>   [3]https://groups.google.com/d/msgid/pandoc-discuss/1df69e0b-adf1-4f56-
>   a590-b44473269eb2%40googlegroups.com.
>   For more options, visit [4]https://groups.google.com/d/optout.
>
>References
>
>   1. mailto:pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
>   2. mailto:pandoc-discuss-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
>   3. https://groups.google.com/d/msgid/pandoc-discuss/1df69e0b-adf1-4f56-a590-b44473269eb2-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org?utm_medium=email&utm_source=footer
>   4. https://groups.google.com/d/optout

Re: Pandoc Projects Ideas for Summer of Haskell 2018?

['Nick Bart' via pandoc-discuss]

[-- Attachment #1: Type: text/plain, Size: 909 bytes --]

Another interesting project might be to port citeproc-js’s “CSL-M” multilingual and other extensions to CSL (see [https://citeproc-js.readthedocs.io/en/latest/csl-m/](https://citeproc-js.readthedocs.io/en/latest/csl-m/index.html)) to pandoc-citeproc.

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/3fNY86BIJDPGkDAsZQrqXuOK0LqNc_I5XjLYYcpYKN96rHxpmhAFMrfzi2XS2NQIXujNz5DcPtjdU3nv81noBKPnMm-i4k9rVwjH6ph6Ljo%3D%40protonmail.com.
For more options, visit https://groups.google.com/d/optout.

[-- Attachment #2: Type: text/html, Size: 1386 bytes --]

Re: Pandoc Projects Ideas for Summer of Haskell 2018?

[Philip Lee]

[-- Attachment #1.1: Type: text/plain, Size: 1085 bytes --]

I'd suggest pandoc to add DjVu to PDF with searchable text preserved 
conversion support . 
We often need comment and markup(highlight , underline , side note etc.) on 
the DjVu ebook while reading  , but there is nearly no DjVu reader or 
editor allow us to do all these kinds of comment and markup, while if 
converted to PDF with searchable text preserved, we have plenty of tools to 
do these things.
There are online service to do such conversion currently , but all lost 
searchable text after conversion .

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/917ef4fc-b55c-4046-bd37-56926ed1bb02%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[-- Attachment #1.2: Type: text/html, Size: 1519 bytes --]

Re: Pandoc Projects Ideas for Summer of Haskell 2018?

[John MACFARLANE]

One further comment.  I've learned recently that several
projects are still using my cheapskate library, which I more
or less abandoned in favor of writing commonmark parsers.
(It has some serious parsing bugs, and I no longer really
maintain it.)

These include the server that powers Hackage, Idris, and Elm.
Someone has just contacted me about using it in Haddock.

I recommended using cmark, but many of these users are
reluctant to use a bundled C library.

Clearly there's a big need in the community for an
efficient, liberally licensed, extensible,
_pure Haskell_ CommonMark parser.


+++ John MACFARLANE [Jan 08 18 09:46 ]:
>I'd suggest having the project create a standalone
>pure Haskell commonmark parsing library that exports
>something like
>
>   parseCommonMark :: Options -> Text -> Pandoc
>
>rather than contributing code directly to pandoc.
>
>This library would have to depend on pandoc-types, but not
>on pandoc as a whole. That would allow the library to be
>used by other things (e.g. a future incarnation of Haddock?)
>without drawing in all of pandoc.  Pandoc could use it too.
>
>Note that pandoc-types is dual-licensed, so people
>could use the library without making their software
>GPL. That would also be an advantage for a lot of people.
>
>The only architectural drawback of this approach is that
>the parser would not be in PandocMonad, so it couldn't
>take advantage of the logging, media, and file handling
>capabilities that gives us.  But, if we needed these
>things, we could always copy the code and create a new
>module in pandoc later.
>
>+++ Jasper Van der Jeugt [Jan 06 18 05:55 ]:
>>  I have added an idea proposal here:
>>      https://summer.haskell.org/ideas.html#pandoc-commonmark
>>  Let me know if you want to change anything about the wording.
>>