Building pandoc "ecosystems": package manger, "central gallery", "wikibook", etc.?

Building pandoc "ecosystems": package manger, "central gallery", "wikibook", etc.?

[Kolen Cheung]

[-- Attachment #1.1: Type: text/plain, Size: 4370 bytes --]



Hi, all,

I’m very new in the pandoc community. I kind of searched a bit about what I 
wanted to suggest and doesn’t seem to find any significant thing. Pardon me 
if this idea has already been visited or even been existed already.

So far, the centralized places for pandoc seems to be:

   - pandoc official website <http://pandoc.org> 
   - GitHub Wiki like https://github.com/jgm/pandoc/wiki/Pandoc-Extras 
   - here in pandoc-discuss 

And as one goes down the list, the information is lesser and lesser 
coherent but scattered around. Often some pandoc tricks, either only 
involve native pandoc but in an obscure way, or involves other 
tools/filters/templates that solves some common problems that pandoc 
cannot/not intended to solve, are very difficult to find. Most of them are 
out there, may be in pandoc-discuss, github issue tracker, github wiki, 
stackexchange, etc.

So I’m thinking if there’s a need of a better “ecosystem” around pandoc, 
and if we are interested into building it.
<#>Filters, Templates, etc. 

For example, besides of what pandoc (and pandoc-templates, pandoc-citeproc) 
offers officially, pandoc allows one to customize how it behaves, which is 
great. But it would be better if those tools are shown in a “central 
gallery”, a place for people to browse filters, templates, makefile, 
scripts, etc. Again, GitHub wiki is already doing it, but kind of 
disorganized and difficult to discover tools. I tried to clean it up and 
reorganize it in the past, but the wiki just seem too limited to be a good 
platform for it.

To be clear, while it can be fancy, something as primitive as 
http://macappstore.org (which allow one browse through homebrew packages 
among other things) will be good enough. The point is it will be 
centralize. So people will be consciously going there to showcase there 
tools or look for a tool they need.

And a closely related subject will be package manager. At least for pandoc 
filters, it seems easy to be implemented and will make the barrier of using 
a filter much lower than before. Basically there’s a centralize, and 
automatic way of installing/uninstalling filters. (by the way, issue #3127 
<https://github.com/jgm/pandoc/issues/3127> decided to make the 
data-dir/filters to be the location for filters, which is great for setting 
up a package manager)

One possible tool to use here is brew-cask. It seems not very difficult to 
set it up (writing formulas pointing to the source of the binary, etc.).
<#>Wikibook, Blog, etc. 

Another way to gathering useful information is some sort of wikibook, or a 
blog. An example would be https://en.wikibooks.org/wiki/LaTeX for wikibook, 
https://github.com/DesignOpen/designopen.github.io for a blog that use 
gh-pages to collaborate the blog by pull request.

Again, most of the information is already out there, some are centralized 
in GitHub wiki already. But a better, more deliberate efforts might be 
worth it. In terms of wikibook, it can be a systematic “textbook” to teach 
on things about pandoc (again, like the LaTeX wikibook, or for example some 
programming books in the market (not the manual)). In terms of a 
collaborative blog, different people might be invited or volunteer to write 
a post to explain something. Some people are already doing on their blogs, 
but again, here it is about centralization.

I don’t know how you guys will think. May be the pandoc communities are 
mostly hackers or in the academia that either are too smart or too busy to 
bother about these. But may be there are some hackers out there enjoy 
setting up these ecosystems, or academics pedagogical enough to bother 
writing about it.

What are your thoughts?
​

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/01d303b0-f034-4065-98ca-f27314a9e308%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[-- Attachment #1.2: Type: text/html, Size: 12810 bytes --]

Re: Building pandoc "ecosystems": package manger, "central gallery", "wikibook", etc.?

[Sergio Correia]

[-- Attachment #1.1: Type: text/plain, Size: 5670 bytes --]

Hi Kolen,

I completely agree and would personally benefit a lot from a centralized 
"repo" where I can both store and load extras.

Sadly, I don't have any experience with writing package managers, but as a 
Python user that suffered from the pip/setuptools/easy_install problem, my 
main advice would be to keep it as simple as possible, because complexity 
creeps up very quickly.

I also have no experience with using homebrew, but I would be wary of using 
something that is not cross platform with Linux and Windows. Other 
alternatives are the package managers for text editors such as atom 
<https://atom.io/packages> , subime text <https://packagecontrol.io/>, etc. 
Their source code is open and their design choices might be useful. For 
instance, the sublime text package manager relies on a github repo 
<https://github.com/wbond/package_control_channel/blob/master/repository/a.json>

Best,
S


On Saturday, October 8, 2016 at 10:03:28 PM UTC-4, Kolen Cheung wrote:
>
> Hi, all,
>
> I’m very new in the pandoc community. I kind of searched a bit about what 
> I wanted to suggest and doesn’t seem to find any significant thing. Pardon 
> me if this idea has already been visited or even been existed already.
>
> So far, the centralized places for pandoc seems to be:
>
>    - pandoc official website <http://pandoc.org> 
>    - GitHub Wiki like https://github.com/jgm/pandoc/wiki/Pandoc-Extras 
>    - here in pandoc-discuss 
>
> And as one goes down the list, the information is lesser and lesser 
> coherent but scattered around. Often some pandoc tricks, either only 
> involve native pandoc but in an obscure way, or involves other 
> tools/filters/templates that solves some common problems that pandoc 
> cannot/not intended to solve, are very difficult to find. Most of them are 
> out there, may be in pandoc-discuss, github issue tracker, github wiki, 
> stackexchange, etc.
>
> So I’m thinking if there’s a need of a better “ecosystem” around pandoc, 
> and if we are interested into building it.
> <#01d303b0-f034-4065-98ca-f27314a9e308-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org_>Filters, 
> Templates, etc. 
>
> For example, besides of what pandoc (and pandoc-templates, 
> pandoc-citeproc) offers officially, pandoc allows one to customize how it 
> behaves, which is great. But it would be better if those tools are shown in 
> a “central gallery”, a place for people to browse filters, templates, 
> makefile, scripts, etc. Again, GitHub wiki is already doing it, but kind of 
> disorganized and difficult to discover tools. I tried to clean it up and 
> reorganize it in the past, but the wiki just seem too limited to be a good 
> platform for it.
>
> To be clear, while it can be fancy, something as primitive as 
> http://macappstore.org (which allow one browse through homebrew packages 
> among other things) will be good enough. The point is it will be 
> centralize. So people will be consciously going there to showcase there 
> tools or look for a tool they need.
>
> And a closely related subject will be package manager. At least for pandoc 
> filters, it seems easy to be implemented and will make the barrier of using 
> a filter much lower than before. Basically there’s a centralize, and 
> automatic way of installing/uninstalling filters. (by the way, issue #3127 
> <https://github.com/jgm/pandoc/issues/3127> decided to make the 
> data-dir/filters to be the location for filters, which is great for 
> setting up a package manager)
>
> One possible tool to use here is brew-cask. It seems not very difficult to 
> set it up (writing formulas pointing to the source of the binary, etc.).
> <#01d303b0-f034-4065-98ca-f27314a9e308-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org_>Wikibook, Blog, 
> etc. 
>
> Another way to gathering useful information is some sort of wikibook, or a 
> blog. An example would be https://en.wikibooks.org/wiki/LaTeX for 
> wikibook, https://github.com/DesignOpen/designopen.github.io for a blog 
> that use gh-pages to collaborate the blog by pull request.
>
> Again, most of the information is already out there, some are centralized 
> in GitHub wiki already. But a better, more deliberate efforts might be 
> worth it. In terms of wikibook, it can be a systematic “textbook” to teach 
> on things about pandoc (again, like the LaTeX wikibook, or for example some 
> programming books in the market (not the manual)). In terms of a 
> collaborative blog, different people might be invited or volunteer to write 
> a post to explain something. Some people are already doing on their blogs, 
> but again, here it is about centralization.
>
> I don’t know how you guys will think. May be the pandoc communities are 
> mostly hackers or in the academia that either are too smart or too busy to 
> bother about these. But may be there are some hackers out there enjoy 
> setting up these ecosystems, or academics pedagogical enough to bother 
> writing about it.
>
> What are your thoughts?
> ​
>

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/06e06db0-d418-44a8-841a-6d072f31b200%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[-- Attachment #1.2: Type: text/html, Size: 17166 bytes --]

Re: Building pandoc "ecosystems": package manger, "central gallery", "wikibook", etc.?

['Jakob Voss' via pandoc-discuss]

[-- Attachment #1.1: Type: text/plain, Size: 2757 bytes --]

Hi Kolen,

you wrote:

> So I’m thinking if there’s a need of a better “ecosystem” around pandoc, 
and if we are interested into building it.
>
> Filter, Templates, etc.

filters and templates are just files installed at $DATA_DIR/filters and 
$DATA_DIR/templates plus additional requirements such as programming 
languages and their libraries or LaTeX-packages. The additional 
requirements depend on language and system anyway so it does not make sense 
to create an infrastructure around them. Pandoc filters and templates could 
very well be collected in a registry but in my opinion we don't need a 
complex infrastructure for this. It just needs a set of best practice how 
to best package a filter and/or template.

How about this:

- put you filter/template in one git repository
- specify which files must be in this repository (README.md, about.yaml or 
similar?)
- collect the URLs to this git repositories in the pandoc Wiki

> And a closely related subject will be package manager. At least for 
pandoc filters, it seems easy to be implemented and will make the barrier 
of using a filter much lower than before. Basically there’s a centralize, 
and automatic way of installing/uninstalling filters. (by the way, issue 
#3127 <https://github.com/jgm/pandoc/issues/3127> decided to make the 
data-dir/filters to be the location for filters, which is great for setting 
up a package manager)

Such package manager could be part of pandoc executable given a list of 
public packages (=git repositories) and standard how to structure each 
package. See here for similar package standards:

https://docs.npmjs.com/files/package.json
https://getcomposer.org/doc/04-schema.md

About your idea of a pandoc textbook or similar: I think the best way is to 
clean up and better structure the wiki instead of creating yet another 
place with documentation. The wiki can also be accessed as git repository 
and it is based on markdown so one could use it as source to generate a 
textbook. I did similar with another project: the page 
http://librecat.org/Catmandu/ is fully generated from a GitHub wiki.

Cheers
Jakob

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/61d8e4cf-4945-4991-825a-17da1fbd990c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[-- Attachment #1.2: Type: text/html, Size: 4220 bytes --]

Re: Building pandoc "ecosystems": package manger, "central gallery", "wikibook", etc.?

[Kolen Cheung]

[-- Attachment #1: Type: text/plain, Size: 2775 bytes --]

I'm thinking about pandoc itself being a package manager too. The concept behind it is a mix of LaTeX and homebrew, let's call it the lapandoc and pandoc-cask for the moment.

In lapandoc, an extra YAML front matter key `usefilter` is given, for each of the usefilter value, say, `pandoc-abc`, pandoc will first look for it locally in the PATH and then then `data-dir/filters`, if it doesn't exist, then look for it in pandoc-cask, a GitHub repository.

Then in pandoc-cask, there's a bunch of formulas (similar to homebrew-cask), each formula has the

1. name of the filter
2. description
3. version
4. url to the binary
5. sha-256 checksum
6. prerequisite if any (as a warning message perhaps)

etc. Again, the idea is very much like brew-cask.

The benifits are

- pandoc itself as the package manager eliminates the problem about cross-platform compatibility
- relationship between pandoc and lapandoc is then the same as that between TeX and LaTeX.
- popular features like pandoc include, csv support etc. that has good reason to not be included in core pandoc can live happily in lapandoc. pandoc focuses on core, secure, reliable, long-term tools, and lapandoc can evolve much quicker, allowing commonly needed syntax which might be less secure (I remember somewhere a discussion about pandoc include is pandoc by design doesn't I/O other files. Important for server use, etc.), only support a few input/output format (less universial than pandoc want), etc. Perhaps a command line option will be given to toggle the use of `usefilter` to auto include the filters for security reason.
- the pandoc-cask can be seperately maintained, any people writing new filters can submit it there, like CTAN for TeX.

Cons:

- since it involves the pandoc program itself, it needs to involve the core developers (probably @jgm himself) to agree and implement it. And for one thing I have no idea how to do it (given what I know, the kind of package manager I can create right now is using brew-cask and write formulas in a repo for it to tap into). And you know how busy they are and how many things are already waiting in line (er... say, native pandoc div and span syntaxes)

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/e003b963-6aab-4a6d-977f-ee05c083e91f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Building pandoc "ecosystems": package manger, "central gallery", "wikibook", etc.?

[Kolen Cheung]

[-- Attachment #1: Type: text/plain, Size: 1770 bytes --]

And regarding the wikibook thing, I agree having one more site to visit is less integrated and can cause confusion.

But the problem about GitHub's wiki is it is kind of like out of the spotlight. It is just not intuitive to go there and find "tips and tricks". And while GitHub's html output is better than plain text, the styling is still, arguably, not very presentable (compare, say, wikibook's LaTeX page in https://en.wikibooks.org/wiki/LaTeX and pandoc extras and how they looks.)

One idea I have (perhaps suggested in the past?) is to open source pandoc.org, at least part of it, as a GitHub repository. Others can then make pull request there to submit "wiki pages" there. And actually GitIt can even be served there.

By the way, one thing about how we feel the sites are integrated together is linking back to where I was from. The reason the pandoc extra page feels odd, although linked from pandoc.org directly (front and center, on the left actually), is it can't be linked back. And incidentally I found that the GitHub.com/jgm/pandoc's url (the one on the very top, just above all the files) actually links to http://johnmacfarlane.net/pandoc and it can't be loaded.

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/6d3a0f04-da84-4711-9963-3a252c717587%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Building pandoc "ecosystems": package manger, "central gallery", "wikibook", etc.?

[Kolen Cheung]

[-- Attachment #1.1: Type: text/plain, Size: 2088 bytes --]

Never mind about the http://johnmacfarlane.net/pandoc cannot be loaded. It 
is ok now. Don't know if it's Comcast's problem here or the server there. 
But should it be changed to pandoc.org instead?

On Tuesday, October 11, 2016 at 2:27:28 PM UTC-7, Kolen Cheung wrote:
>
> And regarding the wikibook thing, I agree having one more site to visit is 
> less integrated and can cause confusion.
>
> But the problem about GitHub's wiki is it is kind of like out of the 
> spotlight. It is just not intuitive to go there and find "tips and tricks". 
> And while GitHub's html output is better than plain text, the styling is 
> still, arguably, not very presentable (compare, say, wikibook's LaTeX page 
> in https://en.wikibooks.org/wiki/LaTeX and pandoc extras and how they 
> looks.)
>
> One idea I have (perhaps suggested in the past?) is to open source 
> pandoc.org, at least part of it, as a GitHub repository. Others can then 
> make pull request there to submit "wiki pages" there. And actually GitIt 
> can even be served there.
>
> By the way, one thing about how we feel the sites are integrated together 
> is linking back to where I was from. The reason the pandoc extra page feels 
> odd, although linked from pandoc.org directly (front and center, on the 
> left actually), is it can't be linked back. And incidentally I found that 
> the GitHub.com/jgm/pandoc's url (the one on the very top, just above all 
> the files) actually links to http://johnmacfarlane.net/pandoc and it 
> can't be loaded.
>
>

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/b31b366c-eb47-42d1-a44f-9eedd2e4d259%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[-- Attachment #1.2: Type: text/html, Size: 4383 bytes --]

Re: Building pandoc "ecosystems": package manger, "central gallery", "wikibook", etc.?

[Sergio Correia]

[-- Attachment #1.1: Type: text/plain, Size: 5333 bytes --]

I'm *really* against having this integrated with pandoc itself:

If you look at the project, it's already at the risk of becoming too 
complex, which is why I think John has split the project into pandoc, 
pandoc-templates, pandocfilters, pandoc-types, etc. Further, there are 
other packages (panzer, and even panflute) that would not have happened if 
pandoc was too tightly integrated with these extra tools. Also, I know John 
is a one-man-army, but he is still one man, so I agree that our objective 
should be to avoid giving him work and instead try to take it away from him 
so he can focus on the important core stuff.

I'm also *really* in favor of having something simple:

Package managers are hard, *really hard 
<https://medium.com/@sdboyer/so-you-want-to-write-a-package-manager-4ae9c17d9527#.ph76p1t5y> *and 
trying to do a proper package manager would be insane in my opinion. 
Support for binaries which have their own dependencies, support for 
multiple OSes, support for multiple pandoc versions (so package xyz needs 
at least pandoc 1.2.3 because it uses the new [span] feature), etc.

Finally, I think it would be more practical is *discovery* is centralized 
but not hosting. In that way, everyone can have their own packages in their 
own repos or even folders, and there is a master repo that links to them.

Now, how would that work? Just to give a quick (and probably flawed) sketch:


   - There is a centralized repo with a file like this 
   one: https://github.com/wbond/package_control_channel/blob/master/channel.json
   - It just lists the repos or urls that contain packages (it should 
   probably just be a simple yaml list)
   - Now, suppose one of the links is 
   to https://github.com/sergiocorreia/panflute-filters
   - In that location, there is a packages.yaml file with the list of 
   packages and its details (name, category, tags, list of files, etc.)

Now, how would discovery work?

   - there is a central server (e.g. an AWS instance) that every hour goes 
   through all the repos and creates an index.yaml file.
   - There is a simple python/perl/whatever script that installs and 
   updates packages. EG:

panman install csv-tables --> install it if the name is unique
panman install csv-tables --from=github/sergiocorreia/foobar
panman uninstall csv-tables
panman index
panman update

And what "panman" (or whatever name) does is just update a local yaml file 
and sync the local folder with the remote locations (based on a hash or on 
the version number)

I also thought about doing more complicated stuff, like a script that uses 
git and updates only some files, but feels too hackish and prone to 
problems. In the example above, the only problem would be the need for an 
instance that regularly creates the package index.

Now, I'm not an expert in this, so take my idea with a grain of salt, but 
we should still aim for something as simple as possible, even if that 
implies cutting some corners (e.g. let's just assume we all use the latest 
pandoc version, let's assume packages have no external dependencies and are 
the same file for each OS version, etc.)

Best,
S

On Tuesday, October 11, 2016 at 10:52:06 PM UTC-4, Kolen Cheung wrote:
>
> Never mind about the http://johnmacfarlane.net/pandoc cannot be loaded. 
> It is ok now. Don't know if it's Comcast's problem here or the server 
> there. But should it be changed to pandoc.org instead?
>
> On Tuesday, October 11, 2016 at 2:27:28 PM UTC-7, Kolen Cheung wrote:
>>
>> And regarding the wikibook thing, I agree having one more site to visit 
>> is less integrated and can cause confusion.
>>
>> But the problem about GitHub's wiki is it is kind of like out of the 
>> spotlight. It is just not intuitive to go there and find "tips and tricks". 
>> And while GitHub's html output is better than plain text, the styling is 
>> still, arguably, not very presentable (compare, say, wikibook's LaTeX page 
>> in https://en.wikibooks.org/wiki/LaTeX and pandoc extras and how they 
>> looks.)
>>
>> One idea I have (perhaps suggested in the past?) is to open source 
>> pandoc.org, at least part of it, as a GitHub repository. Others can then 
>> make pull request there to submit "wiki pages" there. And actually GitIt 
>> can even be served there.
>>
>> By the way, one thing about how we feel the sites are integrated together 
>> is linking back to where I was from. The reason the pandoc extra page feels 
>> odd, although linked from pandoc.org directly (front and center, on the 
>> left actually), is it can't be linked back. And incidentally I found that 
>> the GitHub.com/jgm/pandoc's url (the one on the very top, just above all 
>> the files) actually links to http://johnmacfarlane.net/pandoc and it 
>> can't be loaded.
>>
>>

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/45c9b77f-5289-4e16-8600-8a86560279f6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[-- Attachment #1.2: Type: text/html, Size: 9309 bytes --]

Re: Building pandoc "ecosystems": package manger, "central gallery", "wikibook", etc.?

[John MacFarlane]

Two small notes on this thread (and I agree that I don't
want to create any additional maintenance duties for myself).

- the pandoc.org source is already on github, jgm/pandoc-website

- In effect, we already have a package manager for pandoc filters
  written in Haskell.  You can publish them on Hackage
  and install them with cabal or stack.  (There are several
  there already.) For python filters you can similarly
  publish them on PyPi and use pip.

Re: Building pandoc "ecosystems": package manger, "central gallery", "wikibook", etc.?

[Kolen Cheung]

[-- Attachment #1.1: Type: text/plain, Size: 3217 bytes --]



@jgm:

Thanks for the link.

About the package managers like Hackage/Pip:

   1. pandoc filters can be written in a lot of different programming 
   languages, making those programming-language-centric package managers not 
   centralize for pandoc filters 
   2. For quite some pandoc filters, I don’t see the author bother to put 
   it there, say, pip. Then I guess it is rude if one fork it and put it in 
   pip. 

And more about the lapandoc and pandoc-cask:

   1. from the reaction, lapandoc is clearly a no go 
   2. pandoc-cask do not depends on the existence of lapandoc. The concept 
   around pandoc-cask is very simple, like brew-cask. 

And actually I agree with Sergio a lot.

A possible confusion here: homebrew is different from brew-cask. In fact 
they are completely unrelated, except you can call brew-cask by brew. brew 
compiles from source code, manage dependency, update, etc. brew-cask 
install sbinaries, does not manage dependency (except, say, when a 
different binary is needed for a different OS version. *i.e.* manage 
dependency is not the core of it and mostly not needed/done), does not even 
support update natively (install --force can overwrite old version, which 
is sufficient for pandoc filters).

The pandoc cask concept above is as simple as said there. It is separated 
into 2 components:

   1. pandoc itself 
   2. and a GitHub repository storing “forumlas” 

say when pandoc cask install abc:

   1. pandoc load a formula in the said GitHub repository called abc.ext 
   2. pandoc downloads the abc filter from the url in the formula (which 
   points to the official url the author distributes the binary) 
   3. compute sha256 and verify 
   4. move it to data-dir/filters 

Basically, it is the implementation of the simplest feature of brew cask 
(brew cask did a lot because it needs to know how to deal with .pkg, .app, 
etc.).

So to maintain it (once the pandoc code that reads the formula and acts 
accordingly is done), it is all about that GitHub repository.

The magic of it is the experience is centralized, but it is actually 
decentralized (since the binary is at their original url, not copied to the 
repo). But the end user do not need to worry about where the binaries come 
from, just pandoc cask install abc/xyz.

brew cask can already do all this. The need of pandoc cask is for Windows 
support (brew has linuxbrew), and tighter integration. *e.g.* potentially 
if someone did pandoc ... --filter=abc.py ... pandoc can automatically 
suggest pandoc cask install abc when it doesn’t find it.
​

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/d63aa0bf-dbfc-4f72-98a6-46f96f59c593%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[-- Attachment #1.2: Type: text/html, Size: 14869 bytes --]

Re: Building pandoc "ecosystems": package manger, "central gallery", "wikibook", etc.?

[BP Jonsson]

[-- Attachment #1: Type: text/plain, Size: 2881 bytes --]

Since filters can be written in any language -- in addition to Haskell and
Python at least Perl and JavaScript are already in the game -- using the
respective languages' package managers is of doubtful value as an
improvement. Moreover most published filters are already on GitHub, so it
seems to me that the best solution would be for each author to put their
filters in a repository and then add a link to a list on a designated page
on pandoc's GitHub wiki [^1]

The wiki is at present a very underused resource. In my opinion it is the
natural home for examples, how-tos etc. related to pandoc, as well as the
natural place for lists of links to other repositories of pandoc-related
projects. The wiki has the advantage that it doesn't require
administrative by one person. It could certainly use a little more love,
but anyone can give that love. If jgm thinks it's OK I'm willing to invest
a bit of my time in looking after the wiki.

The main problem with GitHub wikis is that they aren't very visible. The
easiest remedy for that would be to add a section about the wiki to the
README.

[^1]: https://github.com/jgm/pandoc/wiki

Den 12 okt 2016 11:05 skrev "John MacFarlane" <jgm-TVLZxgkOlNX2fBVCVOL8/A@public.gmane.org>:

> Two small notes on this thread (and I agree that I don't
> want to create any additional maintenance duties for myself).
>
> - the pandoc.org source is already on github, jgm/pandoc-website
>
> - In effect, we already have a package manager for pandoc filters
>  written in Haskell.  You can publish them on Hackage
>  and install them with cabal or stack.  (There are several
>  there already.) For python filters you can similarly
>  publish them on PyPi and use pip.
>
> --
> You received this message because you are subscribed to the Google Groups
> "pandoc-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
> To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
> To view this discussion on the web visit https://groups.google.com/d/ms
> gid/pandoc-discuss/20161012090425.GB30660%40Administrateurs-iMac-3.local.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/CAFC_yuQtzVAWtDdz29S5usjmEGNtMHx%2Bidz3YDzDs9Udv-AE_g%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[-- Attachment #2: Type: text/html, Size: 4215 bytes --]

Re: Building pandoc "ecosystems": package manger, "central gallery", "wikibook", etc.?

['Jakob Voss' via pandoc-discuss]

[-- Attachment #1.1: Type: text/plain, Size: 2426 bytes --]

Sergio Correia wrote:

trying to do a proper package manager would be insane in my opinion
>

Agreed. As John pointed out, there already are package managers for each 
particular programming language.

Finally, I think it would be more practical is *discovery* is centralized 
> but not hosting.
>

[...]

Now, how would that work? Just to give a quick (and probably flawed) sketch:
>
>    - There is a centralized repo with a file like this one: 
>    https://github.com/wbond/package_control_channel/blob/master/channel.json
>    - It just lists the repos or urls that contain packages (it should 
>    probably just be a simple yaml list)
>    - Now, suppose one of the links is to 
>    https://github.com/sergiocorreia/panflute-filters
>    - In that location, there is a packages.yaml file with the list of 
>    packages and its details (name, category, tags, list of files, etc.)
>
> The centralized repo is already there at 
https://github.com/jgm/pandoc/wiki/Pandoc-Filters, at least for filters. 
The page just needs some cleanup and may better be split into description 
of filters but there is no need to create another centralized repo.
 

> Now, how would discovery work?
>
>    - there is a central server (e.g. an AWS instance) that every hour 
>    goes through all the repos and creates an index.yaml file.
>
> No need for a central server to start with. We need a script that parses 
filter locations from https://github.com/jgm/pandoc/wiki/Pandoc-Filters (or 
another wiki page), looks out for package.yaml and creates an index.yaml. 
Everyone can run this script locally.

>
>    - There is a simple python/perl/whatever script that installs and 
>    updates packages.
>    
> This is the harder part, I don't expect that this will work. The script 
would need to delegate installation to pip, cabal, npm, cpanm etc.

Cheers
Jakob

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/92db4f26-cda6-4dd9-b9b1-6bc3bc68be52%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[-- Attachment #1.2: Type: text/html, Size: 4838 bytes --]

Re: Building pandoc "ecosystems": package manger, "central gallery", "wikibook", etc.?

[Sergio Correia]

[-- Attachment #1.1: Type: text/plain, Size: 2235 bytes --]

On Wednesday, October 12, 2016 at 6:40:47 AM UTC-4, Jakob Voss wrote:
 

> The centralized repo is already there at 
> https://github.com/jgm/pandoc/wiki/Pandoc-Filters, at least for filters. 
> The page just needs some cleanup and may better be split into description 
> of filters but there is no need to create another centralized repo.
>

As long as it's something easy to parse by a script, I'm fine. One 
advantage of just having a list of repos is that whenever I add a new 
package in my repo I don't have to go back and add the entry for the 
package, and that you can auto-populate a table of packages based on the 
yaml files of each repo.

No need for a central server to start with. We need a script that parses 
> filter locations from https://github.com/jgm/pandoc/wiki/Pandoc-Filters 
> (or another wiki page), looks out for package.yaml and creates an 
> index.yaml. Everyone can run this script locally.
>

Agreed. If at some point someone wants to create a nice server that parses 
the repos and presents them nicely (like this <https://packagecontrol.io/>) 
that is fine but no need for that now.
 

>
>>    - There is a simple python/perl/whatever script that installs and 
>>    updates packages.
>>    
>> This is the harder part, I don't expect that this will work. The script 
> would need to delegate installation to pip, cabal, npm, cpanm etc.
>

What would you suggest? the script can either i) just copy a file, or ii) 
run pip/cabal/etc for more complex cases. In the case of python there is no 
absolutely no need for -pip- as filters are standalone files, but maybe for 
haskell or other languages you need to compile the files.
 

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/36d68af7-f16e-4466-bc9b-15dea01e05fa%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[-- Attachment #1.2: Type: text/html, Size: 4447 bytes --]

Re: Building pandoc "ecosystems": package manger, "central gallery", "wikibook", etc.?

[BP Jonsson]

Den 2016-10-12 kl. 20:29, skrev Sergio Correia:
> As long as it's something easy to parse by a script, I'm fine. One
> advantage of just having a list of repos is that whenever I add a new
> package in my repo I don't have to go back and add the entry for the
> package, and that you can auto-populate a table of packages based on the
> yaml files of each repo.

My approach to this is a commit hook which extracts an abstract 
from the documentation of each individual script/filter and builds 
the repo README with a boilerplate introduction and a heading for 
each script name followed by a link to the script file and the 
abstract.

Just my two worthless coins...

/bpj

Re: Building pandoc "ecosystems": package manger, "central gallery", "wikibook", etc.?

[Kolen Cheung]

[-- Attachment #1.1: Type: text/plain, Size: 2793 bytes --]

Going back to the idea about "lapandoc". I'm thinking if it is doable, and 
convenient, to have a big class of filters that combine multiple filters 
together, so that installing one such filter equals to installing a number 
of them. This solve part of the problem of having the filters being 
decentralized. Essentially this is like creating a document class in LaTeX. 
It doesn't seem to work to incorporate different filters from different 
languages. But at least, say, for filters that's already on cabal, they 
could be put in a big class.

And about the package manager: today I got a problem that I cannot install 
pandoc-placetable. Basically if the filter is written in python, it would 
be so portable since it doesn't need to be compile. The said hypothetical 
package manager can just copy the python script in 
`$HOME/.pandoc/filters/`. But for executable like that, it will be 
problematic. *If* they offer binaries like pandoc do in "releases", it will 
be much easier. But the few I tried to install today all didn't distribute 
a binary but through cabal. So it adds an extra layer of complexity. Using 
the `brew` analogy again, `brew cask` (the simple tool) won't work in the 
later case, which forces one to use something like `brew` directly which is 
complicated (and even `brew` don't install cabal's packages. It allows the 
user to install cabal-install and hand over the package managing part to 
cabal). 

On Thursday, October 13, 2016 at 2:23:00 AM UTC-7, BP Jonsson wrote:
>
> Den 2016-10-12 kl. 20:29, skrev Sergio Correia: 
> > As long as it's something easy to parse by a script, I'm fine. One 
> > advantage of just having a list of repos is that whenever I add a new 
> > package in my repo I don't have to go back and add the entry for the 
> > package, and that you can auto-populate a table of packages based on the 
> > yaml files of each repo. 
>
> My approach to this is a commit hook which extracts an abstract 
> from the documentation of each individual script/filter and builds 
> the repo README with a boilerplate introduction and a heading for 
> each script name followed by a link to the script file and the 
> abstract. 
>
> Just my two worthless coins... 
>
> /bpj 
>

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/919b7057-8bce-4e2c-9bf1-0909a2a606c0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[-- Attachment #1.2: Type: text/html, Size: 3470 bytes --]

Re: Building pandoc "ecosystems": package manger, "central gallery", "wikibook", etc.?

[Kolen Cheung]

[-- Attachment #1.1: Type: text/plain, Size: 4516 bytes --]



I just opened a public Google spreadsheet named pandoc filters 
<https://docs.google.com/spreadsheets/d/1eqMwPyxT0rN3z_tXpsISGBys0QR25W0x-tYDRsFBKAE/edit?usp=sharing>. 
The purposes are:

   1. 
   
   have a “bird view” on all pandoc filters available, including syntax 
   involved, short description, programming language, platform, *etc.* (
   *i.e.* metadata about the filters). This will gives us more concrete 
   data on what could/has to be done to, say, build a package manager.
   2. 
   
   Initially the info will be ported from Pandoc Filters · jgm/pandoc Wiki 
   <https://github.com/jgm/pandoc/wiki/Pandoc-Filters>. But there are some 
   filters not mentioned there, including some actually reside on 
   pandoc-discuss, stack exchange etc.
   3. 
   
   to test how many people care about having a centralize place to discover 
   filters. If lots of people is using and editing the spreadsheet, we can 
   then convert the spreadsheet into a more useful format (*e.g.* CSV, 
   YAML, JSON?), and possibly turn it into a “gallery” using ghpages. Or else 
   if no body cares, the experiment will be dead, *but* we still have a 
   useful spreadsheet as our “gallery”.
   
Just in case, had any people done a spreadsheet about it before?

And I encourage those who want to improve the situation of pandoc filter 
discovery to contribute to the spreadsheet. If you have written pandoc 
filters, please add there as well, especially the short description. Thanks!

On Monday, October 24, 2016 at 8:40:33 PM UTC-7, Kolen Cheung wrote:

Going back to the idea about "lapandoc". I'm thinking if it is doable, and 
> convenient, to have a big class of filters that combine multiple filters 
> together, so that installing one such filter equals to installing a number 
> of them. This solve part of the problem of having the filters being 
> decentralized. Essentially this is like creating a document class in LaTeX. 
> It doesn't seem to work to incorporate different filters from different 
> languages. But at least, say, for filters that's already on cabal, they 
> could be put in a big class.
>
> And about the package manager: today I got a problem that I cannot install 
> pandoc-placetable. Basically if the filter is written in python, it would 
> be so portable since it doesn't need to be compile. The said hypothetical 
> package manager can just copy the python script in 
> `$HOME/.pandoc/filters/`. But for executable like that, it will be 
> problematic. *If* they offer binaries like pandoc do in "releases", it will 
> be much easier. But the few I tried to install today all didn't distribute 
> a binary but through cabal. So it adds an extra layer of complexity. Using 
> the `brew` analogy again, `brew cask` (the simple tool) won't work in the 
> later case, which forces one to use something like `brew` directly which is 
> complicated (and even `brew` don't install cabal's packages. It allows the 
> user to install cabal-install and hand over the package managing part to 
> cabal). 
>
> On Thursday, October 13, 2016 at 2:23:00 AM UTC-7, BP Jonsson wrote:
>>
>> Den 2016-10-12 kl. 20:29, skrev Sergio Correia: 
>> > As long as it's something easy to parse by a script, I'm fine. One 
>> > advantage of just having a list of repos is that whenever I add a new 
>> > package in my repo I don't have to go back and add the entry for the 
>> > package, and that you can auto-populate a table of packages based on 
>> the 
>> > yaml files of each repo. 
>>
>> My approach to this is a commit hook which extracts an abstract 
>> from the documentation of each individual script/filter and builds 
>> the repo README with a boilerplate introduction and a heading for 
>> each script name followed by a link to the script file and the 
>> abstract. 
>>
>> Just my two worthless coins... 
>>
>> /bpj 
>>
> ​

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/2081f946-878c-48af-8450-9e0dcb48f852%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[-- Attachment #1.2: Type: text/html, Size: 12101 bytes --]

Re: Building pandoc "ecosystems": package manger, "central gallery", "wikibook", etc.?

['Jakob Voss' via pandoc-discuss]

[-- Attachment #1.1: Type: text/plain, Size: 1396 bytes --]

Hi Kolen,

thanks for the effort. You wrote:

> Going back to the idea about "lapandoc". I'm thinking if it is doable, 
and convenient, to have a big class of filters that combine 
> multiple filters together, so that installing one such filter equals to 
installing a number of them.
> This solve part of the problem of having the filters being decentralized. 
Essentially this is like creating a document class in LaTeX.

Yes. I'd call this a "package" but nevermind.

> It doesn't seem to work to incorporate different filters from different 
languages.

Unless the filters are written in scripting languages that don't need to be 
compiled. We just need to ensure that the right environment (language 
version, dependencies...) for each language is available by directing to 
pip, composer, npm, cpanm...
 
Cheers
Jakob

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/0c8d515f-1fbb-4c78-9d09-ac42799d44db%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[-- Attachment #1.2: Type: text/html, Size: 1914 bytes --]

Re: Building pandoc "ecosystems": package manger, "central gallery", "wikibook", etc.?

[John MacFarlane]

Something else to keep in mind:  filters can go stale.

Though we've kept it pretty stable over time, pandoc's AST
and its JSON representation can change.  In fact, in the
soon to be released pandoc 1.17.3, the JSON representation
changes quite a bit:

    + The toplevel JSON format is now `{"pandoc-api-version" :
      [MAJ, MIN, REV], "meta" : META, "blocks": BLOCKS}`
      instead of `[{"unMeta": META}, [BLOCKS]]`.
    + Leaf nodes no longer have an empty array for their "c" value.
      Thus, for example, a `Space` is encoded as `{"t":"Space"}`
      rather than `{"t":"Space","c":[]}` as before.

I've modified the pandocfilters (Python) library so that it
works with both the older and the newer JSON format.  But
other libraries will also need to be updated.

For Haskell filters, the situation is a bit different.
A filter compiled against pandoc-types 1.17.x will only
work with pandoc compiled against pandoc-types 1.17.x.
But the source code shouldn't need any changes (unless
it involves the new LineBlock element).  With Haskell
filters you'll get a runtime error if there is a mismatch
of versions between pandoc and the filter itself.

Re: Building pandoc "ecosystems": package manger, "central gallery", "wikibook", etc.?

[Kolen Cheung]

[-- Attachment #1.1: Type: text/plain, Size: 2383 bytes --]

Thanks for the info, especially the point about Haskell filter. Why does 
they need to match? On one hand the filter can be just any executables 
(meaning it doesn't seem to care what it is), but when that executable is a 
haskell filter than it "suddenly" care so much?

For example, if I use pandoc to output to JSON, then run the JSON through 
the filter, then pass the file back to pandoc, either through a pipe, or if 
it doesn't work, through a temp file. Would this help to get around the 
problem?

On Tuesday, October 25, 2016 at 5:58:30 AM UTC-7, John MacFarlane wrote:
>
> Something else to keep in mind:  filters can go stale. 
>
> Though we've kept it pretty stable over time, pandoc's AST 
> and its JSON representation can change.  In fact, in the 
> soon to be released pandoc 1.17.3, the JSON representation 
> changes quite a bit: 
>
>     + The toplevel JSON format is now `{"pandoc-api-version" : 
>       [MAJ, MIN, REV], "meta" : META, "blocks": BLOCKS}` 
>       instead of `[{"unMeta": META}, [BLOCKS]]`. 
>     + Leaf nodes no longer have an empty array for their "c" value. 
>       Thus, for example, a `Space` is encoded as `{"t":"Space"}` 
>       rather than `{"t":"Space","c":[]}` as before. 
>
> I've modified the pandocfilters (Python) library so that it 
> works with both the older and the newer JSON format.  But 
> other libraries will also need to be updated. 
>
> For Haskell filters, the situation is a bit different. 
> A filter compiled against pandoc-types 1.17.x will only 
> work with pandoc compiled against pandoc-types 1.17.x. 
> But the source code shouldn't need any changes (unless 
> it involves the new LineBlock element).  With Haskell 
> filters you'll get a runtime error if there is a mismatch 
> of versions between pandoc and the filter itself. 
>
>
>

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/acaf9b4b-24b0-4674-b8e3-21a978ad71dc%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[-- Attachment #1.2: Type: text/html, Size: 3201 bytes --]

Re: Building pandoc "ecosystems": package manger, "central gallery", "wikibook", etc.?

[John MacFarlane]

+++ Kolen Cheung [Oct 25 16 09:19 ]:
>   Thanks for the info, especially the point about Haskell filter. Why
>   does they need to match? On one hand the filter can be just any
>   executables (meaning it doesn't seem to care what it is), but when that
>   executable is a haskell filter than it "suddenly" care so much?

Because JSON encoding/decoding is done automatically by the
pandoc-types library.  If your filter is compiled against a
different version of pandoc-types than pandoc, then the JSON
your filter expects and produces may be different from the
JSON pandoc expects and produces.  With the version number
checking, we can issue an informative error message that
tells the user what the problem is, rather than a cryptic one.

Re: Building pandoc "ecosystems": package manger, "central gallery", "wikibook", etc.?

['Jakob Voss' via pandoc-discuss]

[-- Attachment #1.1: Type: text/plain, Size: 1397 bytes --]

John MacFarlane wrote:

Something else to keep in mind:  filters can go stale. 
>
> Though we've kept it pretty stable over time, pandoc's AST 
> and its JSON representation can change.  In fact, in the 
> soon to be released pandoc 1.17.3, the JSON representation 
> changes quite a bit.
>

How can filters know which AST version to use? Parsing the AST is less a 
problem with a liberal parser that can understand any version of the JSON 
format, but which JSON version should a filter emit? The solution I use in 
my Perl library is to check an environment variable `PANDOC_VERSION`. It 
would help if pandoc sets such variable before executing filters or pass 
the version number as additional argument/option. I hope that pandoc 1.17.3 
includes any such solution to let filters work with different versions of 
pandoc.

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/0d50a3ae-dc78-4442-a129-9d80925f1cd3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[-- Attachment #1.2: Type: text/html, Size: 2006 bytes --]

Re: Building pandoc "ecosystems": package manger, "central gallery", "wikibook", etc.?

[Sergio Correia]

[-- Attachment #1.1: Type: text/plain, Size: 2023 bytes --]

New version representations will have the following form

{
* "pandoc-api-version" : [MAJ, MIN, REV],*
 "meta" : META,
 "blocks": BLOCKS
}

Thus, you can base your output based on the pandoc-api-version key (and 
whether it exists or not).

One possible problem that I need to think about is what happens if you 
concatenate multiple filters. EG: from what I understand pandocfilters will 
still output the old *"c":[]* syntax, which might create a failure on 
filters that expect something else based on the API version.


On Wednesday, October 26, 2016 at 4:31:32 PM UTC-4, Jakob Voss wrote:
>
> John MacFarlane wrote:
>
> Something else to keep in mind:  filters can go stale. 
>>
>> Though we've kept it pretty stable over time, pandoc's AST 
>> and its JSON representation can change.  In fact, in the 
>> soon to be released pandoc 1.17.3, the JSON representation 
>> changes quite a bit.
>>
>
> How can filters know which AST version to use? Parsing the AST is less a 
> problem with a liberal parser that can understand any version of the JSON 
> format, but which JSON version should a filter emit? The solution I use in 
> my Perl library is to check an environment variable `PANDOC_VERSION`. It 
> would help if pandoc sets such variable before executing filters or pass 
> the version number as additional argument/option. I hope that pandoc 1.17.3 
> includes any such solution to let filters work with different versions of 
> pandoc.
>

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/d5a59501-45cb-440f-88f4-b055ecb971b2%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[-- Attachment #1.2: Type: text/html, Size: 2940 bytes --]

Re: Building pandoc "ecosystems": package manger, "central gallery", "wikibook", etc.?

[John MacFarlane]

+++ Sergio Correia [Oct 26 16 13:38 ]:
>   One possible problem that I need to think about is what happens if you
>   concatenate multiple filters. EG: from what I understand pandocfilters
>   will still output the old "c":[] syntax, which might create a failure
>   on filters that expect something else based on the API version.

This is a bit of a hack -- ideally pandocfilters would emit
{"t":"Space"} instead of {"t":"Space","c":[]} when
processing JSON with the most recent format.  Changing it
to do this would have required a lot of structural changes,
and in fact pandoc will just ignore the superfluous
"c":[] in reading JSON, so I left this.

In general I think it makes sense to be forgiving in this
way in parsing JSON, i.e. not failing if there's an extra
pair in an object, as long as everything that is required
is there.

Anyway, while I'm still minimally maintaining pandocfilters,
I'd recommend that people use panflute instead -- from what
I can see, it is better designed.  (Python is not really
my thing!)

Re: Building pandoc "ecosystems": package manger, "central gallery", "wikibook", etc.?

[Bruce D'Arcus]

[-- Attachment #1.1: Type: text/plain, Size: 1555 bytes --]

Couldn't you include a version variable in the output?

On Wednesday, October 26, 2016 at 4:31:32 PM UTC-4, Jakob Voss wrote:
>
> John MacFarlane wrote:
>
> Something else to keep in mind:  filters can go stale. 
>>
>> Though we've kept it pretty stable over time, pandoc's AST 
>> and its JSON representation can change.  In fact, in the 
>> soon to be released pandoc 1.17.3, the JSON representation 
>> changes quite a bit.
>>
>
> How can filters know which AST version to use? Parsing the AST is less a 
> problem with a liberal parser that can understand any version of the JSON 
> format, but which JSON version should a filter emit? The solution I use in 
> my Perl library is to check an environment variable `PANDOC_VERSION`. It 
> would help if pandoc sets such variable before executing filters or pass 
> the version number as additional argument/option. I hope that pandoc 1.17.3 
> includes any such solution to let filters work with different versions of 
> pandoc.
>

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/dff5854d-173b-4851-a10c-9d47c9d6737d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[-- Attachment #1.2: Type: text/html, Size: 2271 bytes --]

Re: Building pandoc "ecosystems": package manger, "central gallery", "wikibook", etc.?

[John MacFarlane]

+++ Bruce D'Arcus [Oct 26 16 13:39 ]:
>   Couldn't you include a version variable in the output?

That is in fact what we've done in the latest release.

Re: Building pandoc "ecosystems": package manger, "central gallery", "wikibook", etc.?

[John MacFarlane]

+++ 'Jakob Voss' via pandoc-discuss [Oct 26 16 13:31 ]:
>   How can filters know which AST version to use?

Since this information is now encoded in the input JSON,
the filter can just look there.

>   Parsing the AST is less
>   a problem with a liberal parser that can understand any version of the
>   JSON format, but which JSON version should a filter emit?

The same version it receives, since that's what pandoc will
be expecting to get.

Re: Building pandoc "ecosystems": package manger, "central gallery", "wikibook", etc.?

[Kolen Cheung]

[-- Attachment #1: Type: text/plain, Size: 508 bytes --]

>The centralized repo is already there at https://github.com/jgm/pandoc/wiki/Pandoc-Filters, at least for filters. The page just needs some cleanup and may better be split into description of filters but there is no need to create another centralized repo.

I tried to make a pull request there, but @jgm specifically said it's primary purpose is for examples, not package manager. And even if we change that to serve as a package manager, it can only contains filters written with the pandocfilter package.

Re: Building pandoc "ecosystems": package manger, "central gallery", "wikibook", etc.?

['Jakob Voss' via pandoc-discuss]

[-- Attachment #1.1: Type: text/plain, Size: 1752 bytes --]

Kolen Cheung wrote:
 

> But the problem about GitHub's wiki is it is kind of like out of the 
> spotlight. It is just not intuitive to go there and find "tips and tricks". 
> And while GitHub's html output is better than plain text, the styling is 
> still, arguably, not very presentable (compare, say, wikibook's LaTeX page 
> in https://en.wikibooks.org/wiki/LaTeX and pandoc extras and how they 
> looks.)
>

We could create a nice looking-read-only version of (parts of) the Wiki 
hosted at <http://pandoc.org/wiki/>, <http://pandoc.org/book/> or similar 
and point back to the Wiki for editing. The wiki is just a git repository 
with GitHub-flavoured Markdown files. See http://librecat.org/Catmandu/ for 
documentation of another project build from another GitHub wiki. I just 
cloned that Wiki to better show how it works, see the scripts to convert 
Wiki to HTML at 
<https://github.com/nichtich/Catmandu-Wiki/tree/master/book>. Similar 
scripts can be used to extract and combine parts of the Pandoc GitHub Wiki 
and parts of the Pandoc manual 
(<https://github.com/jgm/pandoc/blob/master/MANUAL.txt>) with Pandoc 
filters and templates into a Pandoc textbook.

Jakob

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/f7057fc0-1579-4d1b-bb99-c0885129cd1d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[-- Attachment #1.2: Type: text/html, Size: 2818 bytes --]

Re: Building pandoc "ecosystems": package manger, "central gallery", "wikibook", etc.?

[Kolen Cheung]

[-- Attachment #1.1: Type: text/plain, Size: 2828 bytes --]



On Wednesday, October 12, 2016 at 3:24:10 AM UTC-7, Jakob Voss wrote:
>
> Kolen Cheung wrote:
>  
>
>> But the problem about GitHub's wiki is it is kind of like out of the 
>> spotlight. It is just not intuitive to go there and find "tips and tricks". 
>> And while GitHub's html output is better than plain text, the styling is 
>> still, arguably, not very presentable (compare, say, wikibook's LaTeX page 
>> in https://en.wikibooks.org/wiki/LaTeX and pandoc extras and how they 
>> looks.)
>>
>
> We could create a nice looking-read-only version of (parts of) the Wiki 
> hosted at <http://pandoc.org/wiki/>, <http://pandoc.org/book/> or similar 
> and point back to the Wiki for editing. The wiki is just a git repository 
> with GitHub-flavoured Markdown files. See http://librecat.org/Catmandu/ 
> for documentation of another project build from another GitHub wiki. I just 
> cloned that Wiki to better show how it works, see the scripts to convert 
> Wiki to HTML at <
> https://github.com/nichtich/Catmandu-Wiki/tree/master/book>. Similar 
> scripts can be used to extract and combine parts of the Pandoc GitHub Wiki 
> and parts of the Pandoc manual (<
> https://github.com/jgm/pandoc/blob/master/MANUAL.txt>) with Pandoc 
> filters and templates into a Pandoc textbook.
>
> Jakob
>

I see a lot of documentation has those little link symbol point back to the 
source for editing. But I suggest if we bother to do that, a separate 
GitHub repo (or may be just pandoc-website itself?) is better than the 
GitHub pandoc wiki. While it is also a git, it is not as easy to manage 
(than a GitHub repo).

And I also want to see if GitIt would be an option. Since it is also @jgm's 
project and uses pandoc, a link similar to "Try pandoc" can be put in 
pandoc.org to point to a GitIt wiki, that not only becomes the wiki of 
pandoc, but also a showcase of GitIt. So that wiki is also kind of a "Try 
GitIt". Since the GitIt also uses a git, it can even be pushed to another 
GitHub repo (say, pandoc-wiki).

I personally did exactly that for a personal wiki. I liked GitIt a lot but 
sees that it seems very under-utilized (and the pandoc version embedded is 
old). Putting it front and center in the already successful pandoc could 
make a boost.

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/20789bf3-cca6-4b5b-8f5b-aab2eede66bb%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[-- Attachment #1.2: Type: text/html, Size: 6477 bytes --]

Re: Building pandoc "ecosystems": package manger, "central gallery", "wikibook", etc.?

[Simon Michael]

On 10/12/16 3:36 AM, Kolen Cheung wrote:
> And I also want to see if GitIt would be an option. Since it is also @jgm's
> project and uses pandoc, a link similar to "Try pandoc" can be put in
> pandoc.org to point to a GitIt wiki, that not only becomes the wiki of
> pandoc, but also a showcase of GitIt. So that wiki is also kind of a "Try
> GitIt". Since the GitIt also uses a git, it can even be pushed to another
> GitHub repo (say, pandoc-wiki).
>
> I personally did exactly that for a personal wiki. I liked GitIt a lot but
> sees that it seems very under-utilized (and the pandoc version embedded is
> old). Putting it front and center in the already successful pandoc could
> make a boost.

Kolen - just a +1 from me, that seems like good thinking.

Re: Building pandoc "ecosystems": package manger, "central gallery", "wikibook", etc.?

[John MacFarlane]

+++ Simon Michael [Oct 12 16 16:17 ]:
>On 10/12/16 3:36 AM, Kolen Cheung wrote:
>>And I also want to see if GitIt would be an option. Since it is also @jgm's
>>project and uses pandoc, a link similar to "Try pandoc" can be put in
>>pandoc.org to point to a GitIt wiki, that not only becomes the wiki of
>>pandoc, but also a showcase of GitIt. So that wiki is also kind of a "Try
>>GitIt". Since the GitIt also uses a git, it can even be pushed to another
>>GitHub repo (say, pandoc-wiki).
>>
>>I personally did exactly that for a personal wiki. I liked GitIt a lot but
>>sees that it seems very under-utilized (and the pandoc version embedded is
>>old). Putting it front and center in the already successful pandoc could
>>make a boost.

I had a gitit demo running at http://gitit.net, but I had to
take it down because of a very persistent spammer.

Even changing to use GitHub authorization didn't help; the
spammer had junk GitHub accounts to use.

I'm not sure this is a problem with gitit itself, but it
makes me reluctant to try again!

Re: Building pandoc "ecosystems": package manger, "central gallery", "wikibook", etc.?

[Simon Michael]

On 10/14/16 6:35 AM, John MacFarlane wrote:
> +++ Simon Michael [Oct 12 16 16:17 ]:
>> On 10/12/16 3:36 AM, Kolen Cheung wrote:
>>> And I also want to see if GitIt would be an option. Since it is also @jgm's
>>> project and uses pandoc, a link similar to "Try pandoc" can be put in
>>> pandoc.org to point to a GitIt wiki, that not only becomes the wiki of
>>> pandoc, but also a showcase of GitIt. So that wiki is also kind of a "Try
>>> GitIt". Since the GitIt also uses a git, it can even be pushed to another
>>> GitHub repo (say, pandoc-wiki).
>>>
>>> I personally did exactly that for a personal wiki. I liked GitIt a lot but
>>> sees that it seems very under-utilized (and the pandoc version embedded is
>>> old). Putting it front and center in the already successful pandoc could
>>> make a boost.
>
> I had a gitit demo running at http://gitit.net, but I had to
> take it down because of a very persistent spammer.
>
> Even changing to use GitHub authorization didn't help; the
> spammer had junk GitHub accounts to use.

Wow, that's pretty bad. What's the next step ? I guess it would need 
admin approval of new accounts/new account edits.

Re: Building pandoc "ecosystems": package manger, "central gallery", "wikibook", etc.?

[Kolen Cheung]

[-- Attachment #1: Type: text/plain, Size: 1394 bytes --]

Oh, my God! How come!

I wonder what makes it different from the GitHub wiki; or the "Try pandoc online" or 'BabelMark" since it involves a server as well and can be spammed by people to waste resources. Perhaps a wiki a good place to spread information? And are there any hint on who the spammer is, say, if (s)he is a pandoc hater, or a student of you getting a bad grade?

On the other hand, handling a wiki will soon or later need to face these kind of problem.

I'm just brainstorming, would it be possible to have a static version of GitIt that basically use ghpages and pull request to handle the generation of contents? There's a collorative Jekyll blog like this (mentioned in the original post in this thread). Javis CI will then be used to build the page, JSON+javascript will be used for search.

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/f4a2117c-5d79-46db-a49e-85e64511d1a5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Building pandoc "ecosystems": package manger, "central gallery", "wikibook", etc.?

[John MacFarlane]

+++ Kolen Cheung [Oct 14 16 15:07 ]:
>Oh, my God! How come!
>
>I wonder what makes it different from the GitHub wiki; or the "Try pandoc online" or 'BabelMark" since it involves a server as well and can be spammed by people to waste resources. Perhaps a wiki a good place to spread information? And are there any hint on who the spammer is, say, if (s)he is a pandoc hater, or a student of you getting a bad grade?

No hints, multiple IP addresses and accounts were used.
The content was fairly random and it was generated using
content from someone else's site (the support site from
some commercial software). They eventually complained, and
after several attempts to stop the spamming failed, I
figured it wasn't worth the effort to keep the demo alive.

It's hard to know if it would ever be a problem again; I
had that site running for at least five years with no
problems.

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/20161016201400.GE19592%40Johns-MBP.home.
For more options, visit https://groups.google.com/d/optout.

Re: Building pandoc "ecosystems": package manger, "central gallery", "wikibook", etc.?

[Kolen Cheung]

[-- Attachment #1.1: Type: text/plain, Size: 2869 bytes --]

In a sense, these kind of attack/spam can always happens in a platform like 
wiki/forum, etc. May be we should assume this will happen and see how GitIt 
could be improved to deal with that kind of situation? And by the way, it 
also related to the status of GitIt and how much effort will be pour in it. 
It seems to me GitIt is very under-utilized.

And even if we settled for the current wiki in GitHub, which as of now has 
a dozen pages, and say we are going to take that seriously and expand the 
quality and quantity of articles there, there's still a risk of potential 
spam. As a GitHub user (who do not have any admin rights in pandoc repo), I 
can go into that pandoc github wiki to delete posts without any sort of 
administration (I did delete some "junk" posts in the past if you look at 
the history. Fortunately I am not spamming but to only delete junks.). For 
example, this can be used to attack the pandoc-extra links from pandoc.org, 
since it links directly to GitHub's pandoc wiki page, pandoc Extras. Either 
deleting that page, or spamming the content of that page would be kind of 
disastrous.

Another idea is to make pull request to jgm/pandoc-website. A section might 
be named "Community Articles" or whatever that serves the purpose of the 
current wiki.

On Sunday, October 16, 2016 at 1:14:14 PM UTC-7, John MacFarlane wrote:
>
> +++ Kolen Cheung [Oct 14 16 15:07 ]: 
> >Oh, my God! How come! 
> > 
> >I wonder what makes it different from the GitHub wiki; or the "Try pandoc 
> online" or 'BabelMark" since it involves a server as well and can be 
> spammed by people to waste resources. Perhaps a wiki a good place to spread 
> information? And are there any hint on who the spammer is, say, if (s)he is 
> a pandoc hater, or a student of you getting a bad grade? 
>
> No hints, multiple IP addresses and accounts were used. 
> The content was fairly random and it was generated using 
> content from someone else's site (the support site from 
> some commercial software). They eventually complained, and 
> after several attempts to stop the spamming failed, I 
> figured it wasn't worth the effort to keep the demo alive. 
>
> It's hard to know if it would ever be a problem again; I 
> had that site running for at least five years with no 
> problems. 
>
>

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/d9623b52-01e2-44ae-a7f8-2598404553b5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[-- Attachment #1.2: Type: text/html, Size: 3545 bytes --]

Re: Building pandoc "ecosystems": package manger, "central gallery", "wikibook", etc.?

[Kolen Cheung]

[-- Attachment #1.1: Type: text/plain, Size: 3723 bytes --]



Hi,
<#>pandoc package manager: not a popular idea? 

Not much people has edited the Google Sheet. So I guess probably it 
reflects pandoc filters package manager or even a filter gallery is not a 
very popular idea?
<#>Questions: how to collaborate while using pandoc filters 

May be I should instead ask these questions: for those who has used pandoc 
filters, do you use it personally only or collaboratively as well? If 
pandoc filter(s) are used in a collaborative project, how would you make 
sure every other people has the filter (and installed correctly), possibly 
across different OSes (adding difficulties to include the executables into 
the repository)?
<#>“Lapandoc” 

And I’ve been giving much thought above the “lapandoc” concept (didn’t 
quite figure out the etymology on how to call that yet):

I don’t have a stat yet, but it seems to me the majority of “useful and 
general” pandoc filters are in Haskell/Python, because they are the 
*official* way of writing pandoc filters. The Haskell ones are the 
“trouble-maker” since the ones you want to collaborate with most likely do 
not have cabal, and for the reason mentioned by @jgm (about matching 
versions in JSON), it is quite impossible to distribute binaries for 
filters to ease the installation. So it seems the only remaining way to 
ease the installation of haskell pandoc filters is actually include the 
pandoc in the filter itself. In other words, a “lapandoc”, which contains 
pandoc, but comes with popular/useful/general purpose filters (with a set 
of rules to trigger them).

(And for the filters that wrote in Python, since Python is almost 
“ubiquitous”, its (and pip’s) existence on the host systems can be assumed 
(just ask the collaborators to install it). And then either the script is 
included as a submodule, or a simple instruction using pip is provided. 
Other options include porting popular python pandoc filters to haskell and 
included in “lapandoc”, or includes those python scripts directly in 
“lapandoc”.)

Another fancy thing “lapandoc” could do is some sort of Rmarkdown-style 
notebooks. Since quite a few of pandoc filters “hijack” the code-block for 
something else, “lapandoc” could defines it in a very general way, that 
covers the case of existing use while extending it for arbitrary code 
execution.

But of course there’s an obvious danger about “lapandoc”, it might creates 
yet another unnecessary divide. Part of the solution I think lies in the 
naming. I don’t quite like that Rmarkdown or scholdoc/scholarlymarkdown 
doesn’t have pandoc in its name (and I can never remember the name of the 
later one. Should it be scholarlydoc or scholdown or else). (Similarly, 
Octopress keep emphasizing it is part of Jekyll, but people keep confused 
and consider it as an alternative.) I hope the naming is kind of clear: 
lapandoc to pandoc is like LaTeX to TeX, it doesn’t replace it but extends 
it.

P.S. I’m not saying I’m going to do this, but it is a possible path that we 
can explore.
​

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/a48d8355-f92d-440d-88a5-c20d14285a6e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[-- Attachment #1.2: Type: text/html, Size: 10701 bytes --]

Re: Building pandoc "ecosystems": package manger, "central gallery", "wikibook", etc.?

[Kolen Cheung]

[-- Attachment #1.1: Type: text/plain, Size: 1172 bytes --]



Just a quick update: If anyone is interested in having a centralized 
panflute filters gallery, we are discussing it in Misc. ideas · Issue #12 · 
sergiocorreia/panflute <https://github.com/sergiocorreia/panflute/issues/12>. 
If the idea talked about over there is realized, it will be the “lapandoc” 
that I visualized (without the embedding-pandoc part). For other filters, a 
separate gallery (possibly under the same umbrella if GitHub organization 
is used) might be created that documents the instruction to install as well 
as descriptions and possibly examples.
​

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/4a63c5a3-5f64-4df7-97ca-224ae766a293%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[-- Attachment #1.2: Type: text/html, Size: 2830 bytes --]

Re: Building pandoc "ecosystems": package manger, "central gallery", "wikibook", etc.?

[Kolen Cheung]

[-- Attachment #1.1: Type: text/plain, Size: 1338 bytes --]



An suggestion on the name of a centralized pandoc filter organization, with 
possibly a mechanism to declare the filters used in YAML?

   - lapandoc: a word play on LaTeX from TeX, but la probably stands for 
   Lamport, the creator of LaTeX. 
   - pandocx: x stands for extra, but people might think pan-docx rather 
   than pan-doc-x 
   - pandoc-extras: boring but clear 
   - panfill: my new favorite: word-play between pandoc and polyfill. “pan” 
   kind of have a meaning of “poly”, and the reason behind making an similarly 
   with polyfill is because a pandoc filters typically is a kind of 
   polyfill/shim to define a new syntax you wish it is there. It also sounds 
   like a short form of “pandoc filters”. 

​

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/1a0e355a-9093-40dd-897a-f97699a5121f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[-- Attachment #1.2: Type: text/html, Size: 6552 bytes --]

Re: Building pandoc "ecosystems": package manger, "central gallery", "wikibook", etc.?

[Albert Krewinkel]

I didn't follow the discussion closely, sorry if that's been discussed
before: How about you create a PIP 503 compliant repo for the filters,
provide a simple way to ship/install both pandoc and panflute and call
the result panhandle (like the SF park)?

Kolen Cheung <christian.kolen-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> writes:

> An suggestion on the name of a centralized pandoc filter organization, with
> possibly a mechanism to declare the filters used in YAML?
>
> * lapandoc: a word play on LaTeX from TeX, but la probably stands for Lamport,
>   the creator of LaTeX. 
> * pandocx: x stands for extra, but people might think pan-docx rather than
>   pan-doc-x 
> * pandoc-extras: boring but clear 
> * panfill: my new favorite: word-play between pandoc and polyfill. “pan” kind of
>   have a meaning of “poly”, and the reason behind making an similarly with
>   polyfill is because a pandoc filters typically is a kind of polyfill/shim to
>   define a new syntax you wish it is there. It also sounds like a short form of
>   “pandoc filters”. 
>
> ​ 

-- 
Albert Krewinkel
GPG: 8eed e3e2 e8c5 6f18 81fe  e836 388d c0b2 1f63 1124

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/871sy8gypu.fsf%40espresso.zeitkraut.de.
For more options, visit https://groups.google.com/d/optout.

Re: Building pandoc "ecosystems": package manger, "central gallery", "wikibook", etc.?

[Sergio Correia]

[-- Attachment #1.1: Type: text/plain, Size: 2233 bytes --]

The idea seems interesting, but that would leave out non-python filters 
(node, perl, and more importantly haskell), as well as non-filters 
(templates, css, csl, etc.)

Also, it would mean someone has to actually host the repo, instead of just 
pointing to the creators' repos)

On Friday, November 18, 2016 at 6:22:47 PM UTC-5, Albert Krewinkel wrote:
>
> I didn't follow the discussion closely, sorry if that's been discussed 
> before: How about you create a PIP 503 compliant repo for the filters, 
> provide a simple way to ship/install both pandoc and panflute and call 
> the result panhandle (like the SF park)? 
>
> Kolen Cheung <christi...-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org <javascript:>> writes: 
>
> > An suggestion on the name of a centralized pandoc filter organization, 
> with 
> > possibly a mechanism to declare the filters used in YAML? 
> > 
> > * lapandoc: a word play on LaTeX from TeX, but la probably stands for 
> Lamport, 
> >   the creator of LaTeX. 
> > * pandocx: x stands for extra, but people might think pan-docx rather 
> than 
> >   pan-doc-x 
> > * pandoc-extras: boring but clear 
> > * panfill: my new favorite: word-play between pandoc and polyfill. “pan” 
> kind of 
> >   have a meaning of “poly”, and the reason behind making an similarly 
> with 
> >   polyfill is because a pandoc filters typically is a kind of 
> polyfill/shim to 
> >   define a new syntax you wish it is there. It also sounds like a short 
> form of 
> >   “pandoc filters”. 
> > 
> > ​ 
>
> -- 
> Albert Krewinkel 
> GPG: 8eed e3e2 e8c5 6f18 81fe  e836 388d c0b2 1f63 1124 
>

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/085c9a3a-5370-4e6a-b943-5bff6d6d5a57%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[-- Attachment #1.2: Type: text/html, Size: 3054 bytes --]

Re: Building pandoc "ecosystems": package manger, "central gallery", "wikibook", etc.?

[Albert Krewinkel]

Sergio Correia <sergio.correia-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> writes:

> The idea seems interesting, but that would leave out non-python filters (node,
> perl, and more importantly haskell), as well as non-filters (templates, css,
> csl, etc.)
>
> Also, it would mean someone has to actually host the repo, instead of just
> pointing to the creators' repos)

My understanding is tht pip allows the creation of binary
wheels/packages, so it should be possible to ship haskell filters as
well. However, setting up the infrastructure to support this would
indeed be very involved to the point of being unfeasible

I'd love to be able to point people to a single download that provides
them with all the parts required to use advanced pandoc features without
much knowledge of programming or system administration. IMHO, having
that would by far outweight the restriction of all easily available
filters having to be written in Python.

I won't be complaining no matter what you decide to do. I'm just happy
this effort exists at all :)

> On Friday, November 18, 2016 at 6:22:47 PM UTC-5, Albert Krewinkel wrote:
>
>>  didn't follow the discussion closely, sorry if that's been discussed 
>>  before: How about you create a PIP 503 compliant repo for the filters, 
>>  provide a simple way to ship/install both pandoc and panflute and call 
>>  the result panhandle (like the SF park)? 

s/PIP/PEP/, s/pandoc and panfulte/pandoc, pip, and panflute/

-- 
Albert Krewinkel
GPG: 8eed e3e2 e8c5 6f18 81fe  e836 388d c0b2 1f63 1124

Re: Building pandoc "ecosystems": package manger, "central gallery", "wikibook", etc.?

[John MacFarlane]

Just a note on this.  Pandoc already embeds a lua
interpreter, for custom writers.  So I've sometimes thought
that it would be a good idea to write a lua library for
constructing filters, and build it into pandoc.  That way
lua filters could be run by anyone with the pandoc
executable, without the need for any external Python or
Haskell interpreter at all.  That would make the filters
maximally portable.

I think there'd even be a way to do this that would insulate
the lua library from the details of the JSON encoding, but
I'm less sure about that.  Ideally the details of tree
traversal could be handled entirely by Text.Pandoc.Walk.

I'd have to think about how this could be done.

Re: Building pandoc "ecosystems": package manger, "central gallery", "wikibook", etc.?

['Jakob Voss' via pandoc-discuss]

[-- Attachment #1.1: Type: text/plain, Size: 1182 bytes --]

John MacFarlane wrote:

That way lua filters could be run by anyone with the pandoc 
> executable, without the need for any external Python or 
> Haskell interpreter at all. That would make the filters 
> maximally portable. 
>

I doubt that users will agree on any language and I consider this a 
feature: based on JSON filters can be implemented in any programming 
language This reminds me that an officiall mapping of pandoc-types to XML 
would be nice to also support XSLT filters. Lua filters directly runnable 
by pandoc executable would also be nice but they would not end this 
discussion ;-) 

Jakob

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/06a9ff7a-3dc9-44c0-b4c9-9a7117371481%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[-- Attachment #1.2: Type: text/html, Size: 1757 bytes --]

Re: Building pandoc "ecosystems": package manger, "central gallery", "wikibook", etc.?

[BP Jonsson]

[-- Attachment #1: Type: text/plain, Size: 2903 bytes --]

I just wanted to give my +1 to this. While I realize that we are the Perl
faction speaking, but the fact that filters can be written in any language,
or at least any language which can read/write JSON, is not just a feature
but an important feature. Not only do different languages suit different
people (or perhaps different generations :-) but they also have different
strengths, in particular different libraries. Perl for example has many
modules for natural language and text processing and superb Unicode support
which is very important to me. I still do most of my text spans which need
special handling as Code elements which I modify with Perl string handling
routines in my filters, eventually converting them into Str elements with
or without wrapping Span elements. Other people need different features and
libraries which may be better developed elsewhere.

/bpj

Den 21 nov 2016 20:26 skrev "'Jakob Voss' via pandoc-discuss" <
pandoc-discuss-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org>:

John MacFarlane wrote:

That way lua filters could be run by anyone with the pandoc
> executable, without the need for any external Python or
> Haskell interpreter at all. That would make the filters
> maximally portable.
>

I doubt that users will agree on any language and I consider this a
feature: based on JSON filters can be implemented in any programming
language This reminds me that an officiall mapping of pandoc-types to XML
would be nice to also support XSLT filters. Lua filters directly runnable
by pandoc executable would also be nice but they would not end this
discussion ;-)

Jakob

-- 
You received this message because you are subscribed to the Google Groups
"pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/
msgid/pandoc-discuss/06a9ff7a-3dc9-44c0-b4c9-9a7117371481%40googlegroups.com
<https://groups.google.com/d/msgid/pandoc-discuss/06a9ff7a-3dc9-44c0-b4c9-9a7117371481%40googlegroups.com?utm_medium=email&utm_source=footer>
.

For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/CAFC_yuT0UXz7s2ryZ_LUdnerO1pMoKxgac4oy%3DpFe0EAw7PHWw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[-- Attachment #2: Type: text/html, Size: 4211 bytes --]

Re: Building pandoc "ecosystems": package manger, "central gallery", "wikibook", etc.?

[Kolen Cheung]

[-- Attachment #1: Type: text/plain, Size: 3598 bytes --]

@bpj: do you mean -1?

Bruce: could you elaborate?

On one hand, I'm kind of surprised that embedding lua interpreter seems ok while Python's becomes so not ok... but on the other hand I can see (from my limited search on how to embed Python) it can increase complexity and might requires much more time to mantain it (e.g. what if some filters requires some other libraries not bundled?), and it might set a dangerous precedence that people will ask for more interpeters (perl, javascript, etc.).

Another note is that the original moltivation is for easier filter distribution, which does not say that "now everyone write your filter in Python" but "you can write filters in any languages but we included this in the binary".

Anyway, from the reaction, what I worry more is not if pandoc will embed Python ("clearly" it shouldn't and it's really not all that important), what I worry is how to eventually get to the "lapandoc"/"panfill" thing I talked about. What I had in mind is that a bunch of fitlers becomes a kind of "document class", where all filters in a particular class are (reasonably) expected to be written in 1 language only, but different classes can be in different language.

Before I proceed, let's point out that classes like memoir class borrows ideas from other packages but he wrote his own code. Applying that to the "lapandoc" concept, it means a filter might be written in any languages, but when they are put in a class, they will be modified/rewritten in *a* certain language.

The current problem of pandoc is, there's no *a* language (besides lua, which is not for filter for the moment). So that a pandoc with "document classes"/"a bunch of filters" cannot be self contained.

The problem of not being self-contained when filters is involved means that using filters can never break the barriers of "personal use"/"hacking pandoc documents". I'm sure one might think given instructions on installing all the dependencies will be enough. It's not (at least in my real world situation: I even wrote a script that auto-install everything, in the end, I am the one who ran the script on my colleage's computer.). An analogy will be LaTeX again, for many people including me, the first time they use LaTeX is to install a *big* binary and call it for a day. For first time users, they might have never visited command lines, don't know how to use tlmgr, etc.

To conclude, I guess the biggest question for me is if there's a possible way to eventually get to "lapandoc"/"panfill", that laymen can easily use *some* "document class", and collaborative projects can safely includes filters in production without worrying any of the potential "content" contributors cannot install the neccessary component. From today's reaction, I guess the only path to that is via haskell/lua. Haskell is is in the list because it seems it is not impossible to modify pandoc's binary build process to includes some other haskell filters/"document classes" and call it "lapandoc"/"panfill".

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/2fdfe79c-5c05-4996-a2f1-f427fa2e3ade%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Building pandoc "ecosystems": package manger, "central gallery", "wikibook", etc.?

[daniel]

On 2016-11-26 18:56, Kolen Cheung wrote:

> On one hand, I'm kind of surprised that embedding lua interpreter
> seems ok while Python's becomes so not ok... but on the other hand I
> can see (from my limited search on how to embed Python) it can
> increase complexity and might requires much more time to mantain it
> (e.g. what if some filters requires some other libraries not
> bundled?), and it might set a dangerous precedence that people will
> ask for more interpeters (perl, javascript, etc.).

Two main things to think about on this, in my opinion:

   - Lua is designed to be embedded.  It's lightweight and easy to embed.
     Python is not; it's designed to be run by a stand-alone interpreter.

   - Lua is already being used by Pandoc internally already: 'embedding a 
lua
     interpreter' basically means exposing the existing one for filters 
to use.
     Adding Python or another language basically means adding in a 
significant
     codebase just for filters to use.

Daniel T. Staal

---------------------------------------------------------------
This email copyright the author.  Unless otherwise noted, you
are expressly allowed to retransmit, quote, or otherwise use
the contents for non-commercial purposes.  This copyright will
expire 5 years after the author's death, or in 30 years,
whichever is longer, unless such a period is in excess of
local copyright law.
---------------------------------------------------------------

Re: Building pandoc "ecosystems": package manger, "central gallery", "wikibook", etc.?

[BP Jonsson]

[-- Attachment #1: Type: text/plain, Size: 4909 bytes --]

No I meant that I agreed with Jakob's post which I was replying to. Please
look at the quoted text. Of course it follows that I think it's unnecessary
or even harmful (for the reasons Sergio gave, at least), to embed python.
Not that I think that John would want to do that.

BTW, to make the embedded Lua interpreter really useful a Lua Unicode/UTF-8
library should also be embedded. Out of the box Lua has no Unicode support
at all, which makes it practically worthless for work with most natural
languages.

/bpj

Den 27 nov 2016 00:56 skrev "Kolen Cheung" <christian.kolen-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>:

> @bpj: do you mean -1?
>
> Bruce: could you elaborate?
>
> On one hand, I'm kind of surprised that embedding lua interpreter seems ok
> while Python's becomes so not ok... but on the other hand I can see (from
> my limited search on how to embed Python) it can increase complexity and
> might requires much more time to mantain it (e.g. what if some filters
> requires some other libraries not bundled?), and it might set a dangerous
> precedence that people will ask for more interpeters (perl, javascript,
> etc.).
>
> Another note is that the original moltivation is for easier filter
> distribution, which does not say that "now everyone write your filter in
> Python" but "you can write filters in any languages but we included this in
> the binary".
>
> Anyway, from the reaction, what I worry more is not if pandoc will embed
> Python ("clearly" it shouldn't and it's really not all that important),
> what I worry is how to eventually get to the "lapandoc"/"panfill" thing I
> talked about. What I had in mind is that a bunch of fitlers becomes a kind
> of "document class", where all filters in a particular class are
> (reasonably) expected to be written in 1 language only, but different
> classes can be in different language.
>
> Before I proceed, let's point out that classes like memoir class borrows
> ideas from other packages but he wrote his own code. Applying that to the
> "lapandoc" concept, it means a filter might be written in any languages,
> but when they are put in a class, they will be modified/rewritten in *a*
> certain language.
>
> The current problem of pandoc is, there's no *a* language (besides lua,
> which is not for filter for the moment). So that a pandoc with "document
> classes"/"a bunch of filters" cannot be self contained.
>
> The problem of not being self-contained when filters is involved means
> that using filters can never break the barriers of "personal use"/"hacking
> pandoc documents". I'm sure one might think given instructions on
> installing all the dependencies will be enough. It's not (at least in my
> real world situation: I even wrote a script that auto-install everything,
> in the end, I am the one who ran the script on my colleage's computer.). An
> analogy will be LaTeX again, for many people including me, the first time
> they use LaTeX is to install a *big* binary and call it for a day. For
> first time users, they might have never visited command lines, don't know
> how to use tlmgr, etc.
>
> To conclude, I guess the biggest question for me is if there's a possible
> way to eventually get to "lapandoc"/"panfill", that laymen can easily use
> *some* "document class", and collaborative projects can safely includes
> filters in production without worrying any of the potential "content"
> contributors cannot install the neccessary component. From today's
> reaction, I guess the only path to that is via haskell/lua. Haskell is is
> in the list because it seems it is not impossible to modify pandoc's binary
> build process to includes some other haskell filters/"document classes" and
> call it "lapandoc"/"panfill".
>
> --
> You received this message because you are subscribed to the Google Groups
> "pandoc-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
> To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
> To view this discussion on the web visit https://groups.google.com/d/
> msgid/pandoc-discuss/2fdfe79c-5c05-4996-a2f1-f427fa2e3ade%
> 40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/CAFC_yuTVjfGBs7H6%2Bm%2BYqerih0XJ53AqXPOiPVe8A4kRSppLTw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[-- Attachment #2: Type: text/html, Size: 6220 bytes --]

Re: Building pandoc "ecosystems": package manger, "central gallery", "wikibook", etc.?

[Mohammed Haris Minai]

[-- Attachment #1.1: Type: text/plain, Size: 2623 bytes --]

Let me chime in by saying that I prefer pandoc the way it is. Currently it 
is a generic doc converter, with binaries for the common platforms, and 
great extensibility.

For a lapandoc kind of thing, why not build on top of that, using the *NIX 
philosophy of one tool for an action which is really good at that and can 
be easily interfaced to others. For instance RStudio has used pandoc very 
well in their rmarkdown (http://rmarkdown.rstudio.com) to create a system 
for writing documents with embedded R code.

Assuming we have panflute as a starting point. Let panflute have a 
dependency on pandoc, create its own executable, called maybe panflute, and 
have a selectable list of filters that can be applied by default. This is 
what panzer has done. It has its own executable called "panzer", this 
builds the pandoc command line depending upon the configuration (given as 
styles) and requires that pandoc be installed on the system. I had tried 
something like that to use pandoc specifically for academic papers, but 
panzer's flexibility allowed me to simply tune a few parameters, download 
some specific filters (these can be in any language as long as you have the 
interpreter for that language) and get what I wanted.

Once we have that, we could have examples, wikibook, custom templates, the 
whole 9 yards.

So instead of embedding anything in pandoc, we embed pandoc in something 
else to get our lapandoc.

Regards,
MHMinai

On Sunday, November 27, 2016 at 5:26:41 AM UTC+5:30, Kolen Cheung wrote:
>
> @bpj: do you mean -1?
>
> Bruce: could you elaborate?
>
> On one hand, I'm kind of surprised that embedding lua interpreter seems ok 
> while Python's becomes so not ok... but on the other hand I can see (from 
> my limited search on how to embed Python) it can increase complexity and 
> might requires much more time to mantain it (e.g. what if some filters 
> requires some other libraries not bundled?), and it might set a dangerous 
> precedence that people will ask for more interpeters (perl, javascript, 
> etc.).
>
>

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/eee47f80-604b-4166-9f13-be2afd307911%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[-- Attachment #1.2: Type: text/html, Size: 3331 bytes --]

Re: Building pandoc "ecosystems": package manger, "central gallery", "wikibook", etc.?

[Bruce D'Arcus]

[-- Attachment #1: Type: text/plain, Size: 4574 bytes --]

I meant what Sergio and other subsequently said. Was in a hurry :-)

On Sat, Nov 26, 2016, 6:56 PM Kolen Cheung <christian.kolen-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
wrote:

> @bpj: do you mean -1?
>
> Bruce: could you elaborate?
>
> On one hand, I'm kind of surprised that embedding lua interpreter seems ok
> while Python's becomes so not ok... but on the other hand I can see (from
> my limited search on how to embed Python) it can increase complexity and
> might requires much more time to mantain it (e.g. what if some filters
> requires some other libraries not bundled?), and it might set a dangerous
> precedence that people will ask for more interpeters (perl, javascript,
> etc.).
>
> Another note is that the original moltivation is for easier filter
> distribution, which does not say that "now everyone write your filter in
> Python" but "you can write filters in any languages but we included this in
> the binary".
>
> Anyway, from the reaction, what I worry more is not if pandoc will embed
> Python ("clearly" it shouldn't and it's really not all that important),
> what I worry is how to eventually get to the "lapandoc"/"panfill" thing I
> talked about. What I had in mind is that a bunch of fitlers becomes a kind
> of "document class", where all filters in a particular class are
> (reasonably) expected to be written in 1 language only, but different
> classes can be in different language.
>
> Before I proceed, let's point out that classes like memoir class borrows
> ideas from other packages but he wrote his own code. Applying that to the
> "lapandoc" concept, it means a filter might be written in any languages,
> but when they are put in a class, they will be modified/rewritten in *a*
> certain language.
>
> The current problem of pandoc is, there's no *a* language (besides lua,
> which is not for filter for the moment). So that a pandoc with "document
> classes"/"a bunch of filters" cannot be self contained.
>
> The problem of not being self-contained when filters is involved means
> that using filters can never break the barriers of "personal use"/"hacking
> pandoc documents". I'm sure one might think given instructions on
> installing all the dependencies will be enough. It's not (at least in my
> real world situation: I even wrote a script that auto-install everything,
> in the end, I am the one who ran the script on my colleage's computer.). An
> analogy will be LaTeX again, for many people including me, the first time
> they use LaTeX is to install a *big* binary and call it for a day. For
> first time users, they might have never visited command lines, don't know
> how to use tlmgr, etc.
>
> To conclude, I guess the biggest question for me is if there's a possible
> way to eventually get to "lapandoc"/"panfill", that laymen can easily use
> *some* "document class", and collaborative projects can safely includes
> filters in production without worrying any of the potential "content"
> contributors cannot install the neccessary component. From today's
> reaction, I guess the only path to that is via haskell/lua. Haskell is is
> in the list because it seems it is not impossible to modify pandoc's binary
> build process to includes some other haskell filters/"document classes" and
> call it "lapandoc"/"panfill".
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "pandoc-discuss" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/pandoc-discuss/wbebx65e1Nk/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
> To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/pandoc-discuss/2fdfe79c-5c05-4996-a2f1-f427fa2e3ade%40googlegroups.com
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/CAF-FPGPHrtCuGtVgG1ey0Ho78FA7YV1ZJ66Jwrb-1kaMRv0PDw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[-- Attachment #2: Type: text/html, Size: 6472 bytes --]

Re: Building pandoc "ecosystems": package manger, "central gallery", "wikibook", etc.?

[Kolen Cheung]

[-- Attachment #1: Type: text/plain, Size: 1733 bytes --]

I've thought about that in the last couple of hours, may be the easiest way is to package a big binary per language, and then each of them verioned by the pandoc's major-minor version since this is what's required on the JSON. e.g. panzer, panflute, pandocpm, etc. can be bundled in one package (called something else, e.g. panfill-python) with verison 1.18.1. All it does is providing all the neccessary binaries (possibly excluding pandoc) and guarantee to work on pandoc 1.18.*. So for n languages there could be n such binaries, and if the admin of a certain projects want to have absolute fool-proof way of distibuting it, another big archive including all of them can be setup.
(This is kind of ugly though, if language-barrier is not a problem, a much nicer approach would by (re)writing all filters a certain "document class" needed in Haskell and pandoc can be easily included in that binary. This does not diminish filters in other language however: Not all filters are general-purpose enough to be beneficial in a "document-class", and filters in other language can form a fast-prototyping for what's ultimately needed.)
	

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/26FBD4A4AF4DCEF5.B6CC2491-656C-4EDB-9957-E1A280D22379%40mail.outlook.com.
For more options, visit https://groups.google.com/d/optout.

[-- Attachment #2: Type: text/html, Size: 2373 bytes --]

Re: Building pandoc "ecosystems": package manger, "central gallery", "wikibook", etc.?

[Sergio Correia]

[-- Attachment #1.1: Type: text/plain, Size: 2776 bytes --]

I don't think that the current alternative is that bad:

   1. Install Pandoc
   2. Install Python
   3. pip install panzer, panflute, pandocpm

Then,

   - User panzer to setup styles, preprocessing commands, etc. (through 
   style files)
   - Use panflute (or pandocfilters, or a node/haskell/lua/perl script) to 
   apply filters to the JSON
   - Use pandocpm to download/update filters, templates, csl files, and 
   even the panzer style files

This setup feels nice, and also kinda follows the "unix approach" (one tool 
to do one thing correctly). That said, if you want to implement a bundle, 
it would be easy to create through a python package that just sets up the 
above tools as requirements:

   - pypandoc <https://github.com/bebraw/pypandoc> already embeds pandoc
   - then add panzer/panflute/pandocfilters/pandocpm

IMHO, doing this would be by far the easiest way to install the entire 
python-related ecosystem.


Cheers,
Sergio


On Sunday, November 27, 2016 at 9:49:03 AM UTC-5, Kolen Cheung wrote:
>
> I've thought about that in the last couple of hours, may be the easiest 
> way is to package a big binary per language, and then each of them verioned 
> by the pandoc's major-minor version since this is what's required on the 
> JSON. e.g. panzer, panflute, pandocpm, etc. can be bundled in one package 
> (called something else, e.g. panfill-python) with verison 1.18.1. All it 
> does is providing all the neccessary binaries (possibly excluding pandoc) 
> and guarantee to work on pandoc 1.18.*. So for n languages there could be n 
> such binaries, and if the admin of a certain projects want to have absolute 
> fool-proof way of distibuting it, another big archive including all of them 
> can be setup.
> (This is kind of ugly though, if language-barrier is not a problem, a much 
> nicer approach would by (re)writing all filters a certain "document class" 
> needed in Haskell and pandoc can be easily included in that binary. This 
> does not diminish filters in other language however: Not all filters are 
> general-purpose enough to be beneficial in a "document-class", and filters 
> in other language can form a fast-prototyping for what's ultimately needed.)
>

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/4ed6cb21-bf87-40b3-b5cd-8c2d04008f2e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[-- Attachment #1.2: Type: text/html, Size: 3645 bytes --]

Re: Building pandoc "ecosystems": package manger, "central gallery", "wikibook", etc.?

[Kolen Cheung]

[-- Attachment #1.1: Type: text/plain, Size: 2569 bytes --]



Hi, all,

There’s some sort of pandoc package manager in the working (so far for 
panflute filters only, should be easy to include pandocfilters, and might 
be possible to include any others in the future (e.g. through running 
cabal... by the manager)). It is intended for easier filters (or even 
templates) discovery, installation, and applying the filters through YAML. 
I want to hear your opinion on these:

   - 
   
   In general, do you think a pandoc package manager would be helpful, and 
   what kind of (realistic) features do you expect?
   - 
   
   This part concerns *auto-install* only from markdown’s YAML front matter:
   - 
      
      In principle, it can auto-install filters specified in the markdown 
      source’s YAML (as long as it is on our index). However, there might be a 
      security concern. Do you think it should have a tighter control on that?
      - 
      
      If so, do you think a centralized GitHub organization can solves the 
      problem? i.e. in principle, the organization is being overseen. Then any 
      filters in this organization is trusted and will be installed by the 
      package manager automatically. To be a solution, people will want to submit 
      to their. So as filters writers, would you like to submit/migrate to this 
      organization or will it be too troublesome? (you will still be managing 
      your own filters.)
      - 
   
   Name:
   - pandocpm 
      - pdmgr: PanDoc ManaGeR, from tlmgr as TeXLive Manager 
      - pacdoc: wordplay 
      - packdoc: variant of the previous one 
      - pif: Pif Installs Filters, from pip as PIP Installs Packages 
      - pandocbrew: from homebrew 
      - panbrew: variant of the previous one 
      - panfill: wordplay on pandoc and polyfill 
   
Sidenote: in principle, the package manager (written in Python) can be 
packaged together with panflute and pandocfilters and distribute it as 
binary.
​

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/30f2467b-f440-433f-a579-cb90061b57b4%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[-- Attachment #1.2: Type: text/html, Size: 13001 bytes --]

Re: Building pandoc "ecosystems": package manger, "central gallery", "wikibook", etc.?

[Sergio Correia]

[-- Attachment #1.1: Type: text/plain, Size: 4017 bytes --]

To expand a bit:


   1. The idea is to support all of the extras, including csl files, css, 
   etc.
   2. There are 3 problems involved with package managers: hosting the 
   packages, hosting the index, and writing the program that download/updates. 
   We don't want to host the packages (better to let everyone host them as 
   they want). Instead, we would just maintain the program and a smal index 
   comprised by a .yaml file 
   <https://github.com/pandoc-extras/packages/blob/master/filters.yaml>, or 
   perhaps with one file per package
   3. There are two install scenarios: simple (single-file 
   filters/templates), and complex (installs with dependencies, etc.)
   4. Single-file extras would just involve copying files to the 
   corresponding $datadir subfolder
   5. To deal with complex extras, we could defer the install to 
   pip/cabal/etc 
   6. Complicated commands such as "update all" are for now not 
   implemented. Same for versioning, hashes, etc. (at least until we get the 
   basics)

I don't have a strong opinion on the name of the package manager (I picked 
pandocpm but that was more of a placeholder). That said, it would be better 
if whatever name is easy to remember and doesn't clash with other names.


On Tuesday, November 22, 2016 at 2:42:02 AM UTC-5, Kolen Cheung wrote:
>
> Hi, all,
>
> There’s some sort of pandoc package manager in the working (so far for 
> panflute filters only, should be easy to include pandocfilters, and might 
> be possible to include any others in the future (e.g. through running 
> cabal... by the manager)). It is intended for easier filters (or even 
> templates) discovery, installation, and applying the filters through YAML. 
> I want to hear your opinion on these:
>
>    - 
>    
>    In general, do you think a pandoc package manager would be helpful, 
>    and what kind of (realistic) features do you expect?
>    - 
>    
>    This part concerns *auto-install* only from markdown’s YAML front 
>    matter:
>    - 
>       
>       In principle, it can auto-install filters specified in the markdown 
>       source’s YAML (as long as it is on our index). However, there might be a 
>       security concern. Do you think it should have a tighter control on that?
>       - 
>       
>       If so, do you think a centralized GitHub organization can solves 
>       the problem? i.e. in principle, the organization is being overseen. Then 
>       any filters in this organization is trusted and will be installed by the 
>       package manager automatically. To be a solution, people will want to submit 
>       to their. So as filters writers, would you like to submit/migrate to this 
>       organization or will it be too troublesome? (you will still be managing 
>       your own filters.)
>       - 
>    
>    Name:
>    - pandocpm 
>       - pdmgr: PanDoc ManaGeR, from tlmgr as TeXLive Manager 
>       - pacdoc: wordplay 
>       - packdoc: variant of the previous one 
>       - pif: Pif Installs Filters, from pip as PIP Installs Packages 
>       - pandocbrew: from homebrew 
>       - panbrew: variant of the previous one 
>       - panfill: wordplay on pandoc and polyfill 
>    
> Sidenote: in principle, the package manager (written in Python) can be 
> packaged together with panflute and pandocfilters and distribute it as 
> binary.
> ​
>

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/326b26e9-2a8b-4a60-ad59-512205311483%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[-- Attachment #1.2: Type: text/html, Size: 13964 bytes --]

Re: Building pandoc "ecosystems": package manger, "central gallery", "wikibook", etc.?

[Mohammed Haris Minai]

[-- Attachment #1.1: Type: text/plain, Size: 4681 bytes --]

I do believe that a mechanism to bundle pandoc with its related "helpers" 
would be beneficial. For my purposes I have been using panzer 
(https://github.com/msprev/panzer)  to good effect for this. It allows me 
to collect all my csl, css, latex templates etc. in one place and have 
multiple targets in a styles.xml file. It might serve as a good starting 
point.

It was discussed on this group here: http://bit.ly/2gFF0A3

Regards,
Mohammad Haris Minai

On Tuesday, November 22, 2016 at 2:09:39 PM UTC+5:30, Sergio Correia wrote:
>
> To expand a bit:
>
>
>    1. The idea is to support all of the extras, including csl files, css, 
>    etc.
>    2. There are 3 problems involved with package managers: hosting the 
>    packages, hosting the index, and writing the program that download/updates. 
>    We don't want to host the packages (better to let everyone host them as 
>    they want). Instead, we would just maintain the program and a smal index 
>    comprised by a .yaml file 
>    <https://github.com/pandoc-extras/packages/blob/master/filters.yaml>, 
>    or perhaps with one file per package
>    3. There are two install scenarios: simple (single-file 
>    filters/templates), and complex (installs with dependencies, etc.)
>    4. Single-file extras would just involve copying files to the 
>    corresponding $datadir subfolder
>    5. To deal with complex extras, we could defer the install to 
>    pip/cabal/etc 
>    6. Complicated commands such as "update all" are for now not 
>    implemented. Same for versioning, hashes, etc. (at least until we get the 
>    basics)
>
> I don't have a strong opinion on the name of the package manager (I picked 
> pandocpm but that was more of a placeholder). That said, it would be better 
> if whatever name is easy to remember and doesn't clash with other names.
>
>
> On Tuesday, November 22, 2016 at 2:42:02 AM UTC-5, Kolen Cheung wrote:
>>
>> Hi, all,
>>
>> There’s some sort of pandoc package manager in the working (so far for 
>> panflute filters only, should be easy to include pandocfilters, and might 
>> be possible to include any others in the future (e.g. through running 
>> cabal... by the manager)). It is intended for easier filters (or even 
>> templates) discovery, installation, and applying the filters through YAML. 
>> I want to hear your opinion on these:
>>
>>    - 
>>    
>>    In general, do you think a pandoc package manager would be helpful, 
>>    and what kind of (realistic) features do you expect?
>>    - 
>>    
>>    This part concerns *auto-install* only from markdown’s YAML front 
>>    matter:
>>    - 
>>       
>>       In principle, it can auto-install filters specified in the 
>>       markdown source’s YAML (as long as it is on our index). However, there 
>>       might be a security concern. Do you think it should have a tighter control 
>>       on that?
>>       - 
>>       
>>       If so, do you think a centralized GitHub organization can solves 
>>       the problem? i.e. in principle, the organization is being overseen. Then 
>>       any filters in this organization is trusted and will be installed by the 
>>       package manager automatically. To be a solution, people will want to submit 
>>       to their. So as filters writers, would you like to submit/migrate to this 
>>       organization or will it be too troublesome? (you will still be managing 
>>       your own filters.)
>>       - 
>>    
>>    Name:
>>    - pandocpm 
>>       - pdmgr: PanDoc ManaGeR, from tlmgr as TeXLive Manager 
>>       - pacdoc: wordplay 
>>       - packdoc: variant of the previous one 
>>       - pif: Pif Installs Filters, from pip as PIP Installs Packages 
>>       - pandocbrew: from homebrew 
>>       - panbrew: variant of the previous one 
>>       - panfill: wordplay on pandoc and polyfill 
>>    
>> Sidenote: in principle, the package manager (written in Python) can be 
>> packaged together with panflute and pandocfilters and distribute it as 
>> binary.
>> ​
>>
>

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/7d6f6311-b9ab-4a0b-8ba0-5a2c7a68416d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[-- Attachment #1.2: Type: text/html, Size: 15202 bytes --]

Re: Building pandoc "ecosystems": package manger, "central gallery", "wikibook", etc.?

[Kolen Cheung]

[-- Attachment #1: Type: text/plain, Size: 1820 bytes --]

By the way, your link to me links to <https://groups.google.com/forum/m/#!overview>. Is it intended?

Speaking of panzer, I only knew about it recently. Before that I wrote something like it for personal use, but much more restrictive (effectively my script assign 1 "style" per input/output format). While one can say the info is already out there (in pandoc extra), it is almost impossible to discover but buried deep. For what it seems to do (I didn't try yet, it's very difficult to change a workflow once created), *every* pandoc users should use it. But looking at the amount of stars it has, I guess not much people know/use it.

This kind of problems is the motivation for this thread and whatever I hope to build: a more organized and systematic way of sharing worflow around pandoc. One benefit of the package manager is the "formula" can be reuse for building a "central gallery". And then for the "wikibook" thing, I guess it would be easier to start as a centralized blog with guest posts sharing on their workflow.

Going back to panzer, I'll ask Sergio if it is possble to integrate them together (or how deep can the integration goes, they can certainly be orthogonal), especially when they are all written in Python.

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/e90cf932-6e51-4c7c-b299-71cd8ab0c74d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Building pandoc "ecosystems": package manger, "central gallery", "wikibook", etc.?

[Kolen Cheung]

[-- Attachment #1: Type: text/plain, Size: 255 bytes --]

By the way, currently pandoc.org has a link to pandoc-extras that is external (on github wiki). Would @jgm be interested in linking it to an external website, that includes all the info in github wiki pandoc extras and filters, and potentially much more?

Re: Building pandoc "ecosystems": package manger, "central gallery", "wikibook", etc.?

[Mohammed Haris Minai]

[-- Attachment #1.1: Type: text/plain, Size: 2039 bytes --]

It was supposed to go 
here: https://groups.google.com/forum/#!topic/pandoc-discuss/5zvWi4rOOhQ

On Tuesday, November 22, 2016 at 4:30:41 PM UTC+5:30, Kolen Cheung wrote:
>
> By the way, your link to me links to <
> https://groups.google.com/forum/m/#!overview>. Is it intended?
>
> Speaking of panzer, I only knew about it recently. Before that I wrote 
> something like it for personal use, but much more restrictive (effectively 
> my script assign 1 "style" per input/output format). While one can say the 
> info is already out there (in pandoc extra), it is almost impossible to 
> discover but buried deep. For what it seems to do (I didn't try yet, it's 
> very difficult to change a workflow once created), *every* pandoc users 
> should use it. But looking at the amount of stars it has, I guess not much 
> people know/use it.
>
> This kind of problems is the motivation for this thread and whatever I 
> hope to build: a more organized and systematic way of sharing worflow 
> around pandoc. One benefit of the package manager is the "formula" can be 
> reuse for building a "central gallery". And then for the "wikibook" thing, 
> I guess it would be easier to start as a centralized blog with guest posts 
> sharing on their workflow.
>
> Going back to panzer, I'll ask Sergio if it is possble to integrate them 
> together (or how deep can the integration goes, they can certainly be 
> orthogonal), especially when they are all written in Python.
>
>

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/8fc300c8-cb4a-4fac-9c02-264d7fd4154e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[-- Attachment #1.2: Type: text/html, Size: 2885 bytes --]

Re: Building pandoc "ecosystems": package manger, "central gallery", "wikibook", etc.?

[Albert Krewinkel]

Hi,

Kolen Cheung <christian.kolen-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> writes:

> There’s some sort of pandoc package manager in the working (so far for panflute
> filters only, should be easy to include pandocfilters, and might be possible to
> include any others in the future (e.g. through running cabal... by the
> manager)). It is intended for easier filters (or even templates) discovery,
> installation, and applying the filters through YAML. I want to hear your opinion
> on these:
>
> *   In general, do you think a pandoc package manager would be helpful, and what
>   kind of (realistic) features do you expect?

I think a package manager would definitely be helpful! So despite my
excitement, let me be cautious and raise some security concerns:
Third-party code is downloaded and executed with user permissions. A
rouge filter could submit the document elsewhere or even simply take
over the user's computer. Security should not be taken lightly.

Some examples of what should be consider here: Who controls the package
lists and the servers? Are the servers secure? Are the computers used to
control the server secure? Are filters reviewed? Can authors change the
filter after they have been reviewed? Do authors control the download
location and could they ship different versions to different people?
Will the filter-download be tamper-proof? SSL/TLS and/or hashes and
cryptographic signatures? How will signatures and certificates be
validated? How are filter-updates handled?

Some of those questions are addressed by the centralized GitHub idea. I
like that idea for its pragmatism and simplicity. Best-practice dictates
to reduce risks by reusing as many existing solutions as possible.

Again, apologies if this has been discussed before, but with a central
repo like that, one could forgo most of the above issues and ship the
complete repository as a single package (without the manager). That
would also make it possible to ship compiled haskell filters and might
even reduce filter compatibility issues.


-- 
Albert Krewinkel
GPG: 8eed e3e2 e8c5 6f18 81fe  e836 388d c0b2 1f63 1124

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/87wpfve3im.fsf%40espresso.zeitkraut.de.
For more options, visit https://groups.google.com/d/optout.

Re: Building pandoc "ecosystems": package manger, "central gallery", "wikibook", etc.?

[Kolen Cheung]

[-- Attachment #1: Type: text/plain, Size: 2244 bytes --]

Defintely there can be serious security concern. It is one thing we spent most time on discussing about. In short, I'm trying to borrow the ideas from homebrew. I think for auto-installation the "formula" that tell pandocpm what to do should be host centrally, pointing to a fix version of filter (rather than latest, e.g. by pointing to a particular commit), and contain a SHA-256 sum of the filter, and then https is required for all urls.

So at the moment the "formula" of the filter is submitted to the central repository, the fitler would be reviewed by the maintainer, and since there's an SHA-256 sum, in principle the filter cannot be modified without noticed.

The biggest debate on this is how to balance the security and among of workload on the maintainers. Filters usually being short will help. Other tell is if the filter writer are "real person", if it can be traced back to a real person who have reputation in some form, then it can be easier to trust (since they can be held responsible).

Another point is, the "formula" need not be submitted by the author of the filter, it can be anyone (just like brew).

For filters that has package manager setup already, say, a python filter using pip, potentially a script can be written to automatically parse the setup.py to a formula pandocpm need. In this case, the sanitizing part is defered to the said package manager (in a sense it means we are holding pip/cabal/... to be responsible).

You can go to <https://github.com/pandoc-extras/pandocpm/issues/2> for giving ideas.

By the way, about the name: it is not settled yet. Feel free to make suggestion. e.g. I kind of like pacdoc.

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/4bc19147-da0f-4198-8dd8-9d27c09534f4%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Building pandoc "ecosystems": package manger, "central gallery", "wikibook", etc.?

[Sergio Correia]

[-- Attachment #1.1: Type: text/plain, Size: 3146 bytes --]

I agree. Note though that the security problem is also present for almost 
*all* of the language-specific package managers (npm, pip, etc.): 
ultimately you trust the package authors not to replace the code by 
something like "rm -rf".

That said, would this work?

There would be 3 ways of installing packages:

   1. Simple filters can be kept as single files in one main repo that just 
   need to be copied to $datadir. So authors just submit a PR to the repo and 
   the filter gets added. This is hopefully safer as you would only need to 
   trust the pandoc-extras admins instead of the authors of every filter you 
   try.
   2. Alternatively, you could copy the files directly from the authors 
   repo/website with an *--unsafe* flag (*pandocpm install csv-tables 
   --unsafe*). *(unsafe is probably not the best name though)*
   3. Finally, large filters with a build process can be installed by 
   calling "pip install xyz" (or cabal install/etc). In that case, you would 
   still be kinda trusting the author (but with a higher bar than in #2). 

Now, I kinda get why we need hash checks in general (to prevent MITM 
attacks or tampering of the files?), but I'm not sure if they are actually 
useful for us, given that we would be keeping both the files and hashes in 
the same repo (in #1 at least), and github forces all downloads through 
https. 

About your other questions:


> Who controls the package lists and the servers?

Whoever we trust to manage the repo and approve the pull requests

> Are the servers secure? Are the computers used to  control the server 
secure?

As you said, github covers this

> Are filters reviewed?

In #1, they are briefly reviewed by the repo admin when reviewing the PRs.

In #2, no review. In #3 you have the level of safety given by pip/cabal/etc 
(which is sadly less than you would think)

> Can authors change the  filter after they have been reviewed?

In #2 they can change it at any time; in #1 they can only change it after 
review by the repo admin(s)

> Do authors control the download  location and could they ship different 
versions to different people? 
 
Only with #2

> Will the filter-download be tamper-proof? SSL/TLS and/or hashes and 
cryptographic signatures? How will signatures and certificates be 
validated?

Still an open question, I'm not an expert in security so I don't know if 
github hosting is fully tamperproof.

> How are filter-updates handled? 

>
For #1 and #2 I would just check version numbers and then 
uninstall+install. For #3 I would just call "pip --update" or similar.

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/86ae71c1-9c1e-48e5-9308-103fb1357435%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[-- Attachment #1.2: Type: text/html, Size: 4257 bytes --]

Re: Building pandoc "ecosystems": package manger, "central gallery", "wikibook", etc.?

[Albert Krewinkel]

Thank you for your prompt replies and for alleviating my concerns.
Knowing that security had already been considered as part of the
planning gives me great peace of mind.

I'm very much looking forward to see the results of this :)


Sergio Correia <sergio.correia-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> writes:

> I agree. Note though that the security problem is also present for almost *all*
> of the language-specific package managers (npm, pip, etc.): ultimately you trust
> the package authors not to replace the code by something like "rm -rf".
>
> That said, would this work?
>
> There would be 3 ways of installing packages:
>
> 1 Simple filters can be kept as single files in one main repo that just need to
>   be copied to $datadir. So authors just submit a PR to the repo and the filter
>   gets added. This is hopefully safer as you would only need to trust the
>   pandoc-extras admins instead of the authors of every filter you try.
> 2 Alternatively, you could copy the files directly from the authors repo/website
>   with an --unsafe flag (pandocpm install csv-tables --unsafe). (unsafe is
>   probably not the best name though)
> 3 Finally, large filters with a build process can be installed by calling "pip
>   install xyz" (or cabal install/etc). In that case, you would still be kinda
>   trusting the author (but with a higher bar than in #2). 
>
> Now, I kinda get why we need hash checks in general (to prevent MITM attacks or
> tampering of the files?), but I'm not sure if they are actually useful for us,
> given that we would be keeping both the files and hashes in the same repo (in #1
> at least), and github forces all downloads through https. 
>
> About your other questions:
>
>> Who controls the package lists and the servers?
>
> Whoever we trust to manage the repo and approve the pull requests
>
>> Are the servers secure? Are the computers used to control the server secure?
>
> As you said, github covers this
>
>> Are filters reviewed?
>
> In #1, they are briefly reviewed by the repo admin when reviewing the PRs.
>
> In #2, no review. In #3 you have the level of safety given by pip/cabal/etc
> (which is sadly less than you would think)
>
>> Can authors change the filter after they have been reviewed?
>
> In #2 they can change it at any time; in #1 they can only change it after review
> by the repo admin(s)
>
>> Do authors control the download location and could they ship different
> versions to different people? 
> Only with #2
>
>> Will the filter-download be tamper-proof? SSL/TLS and/or hashes and 
> cryptographic signatures? How will signatures and certificates be 
> validated?
>
> Still an open question, I'm not an expert in security so I don't know if github
> hosting is fully tamperproof.
>
>> How are filter-updates handled? 
>
>         
>
>         
>
>     
>
> For #1 and #2 I would just check version numbers and then uninstall+install. For
> #3 I would just call "pip --update" or similar.

-- 
Albert Krewinkel
GPG: 8eed e3e2 e8c5 6f18 81fe  e836 388d c0b2 1f63 1124

Re: Building pandoc "ecosystems": package manger, "central gallery", "wikibook", etc.?

[Kolen Cheung]

[-- Attachment #1: Type: text/plain, Size: 903 bytes --]

I've been thinking about the fact that pandoc already embedded lua interpreter. How about embedding Python interpreter (and possibly including pandocfilters and panflute)? This might bloated the size of pandoc a bit, and I can't find a number on the size, but there are projects aimming at using Python on emdding systems.

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/523cf1e2-af4c-41b7-b288-a458238faddd%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Building pandoc "ecosystems": package manger, "central gallery", "wikibook", etc.?

[Bruce D'Arcus]

[-- Attachment #1: Type: text/plain, Size: 1751 bytes --]

Please: no!

On Sat, Nov 26, 2016 at 9:55 AM Kolen Cheung <christian.kolen-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
wrote:

> I've been thinking about the fact that pandoc already embedded lua
> interpreter. How about embedding Python interpreter (and possibly including
> pandocfilters and panflute)? This might bloated the size of pandoc a bit,
> and I can't find a number on the size, but there are projects aimming at
> using Python on emdding systems.
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "pandoc-discuss" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/pandoc-discuss/wbebx65e1Nk/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
> To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/pandoc-discuss/523cf1e2-af4c-41b7-b288-a458238faddd%40googlegroups.com
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/CAF-FPGM%2Bdqpj6ZaWyruVSsQ3JXygoTB8NEN207_BYn0-P3JgyA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[-- Attachment #2: Type: text/html, Size: 3198 bytes --]

Re: Building pandoc "ecosystems": package manger, "central gallery", "wikibook", etc.?

[Sergio Correia]

[-- Attachment #1.1: Type: text/plain, Size: 2250 bytes --]

To elaborate on Bruce's reply:

I personally don't care about *size* per se. I care that if Pandoc becomes 
too complex, bad things will happen. And one of the fastest ways to 
increase complexity would be a bundle like this. I've had projects where 
progress stopped completely just because it became too complex for me to 
grapple. After that, I became way more conservative about this.


On Saturday, November 26, 2016 at 10:42:59 AM UTC-5, Bruce D'Arcus wrote:
>
> Please: no!
>
> On Sat, Nov 26, 2016 at 9:55 AM Kolen Cheung <christi...-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org 
> <javascript:>> wrote:
>
>> I've been thinking about the fact that pandoc already embedded lua 
>> interpreter. How about embedding Python interpreter (and possibly including 
>> pandocfilters and panflute)? This might bloated the size of pandoc a bit, 
>> and I can't find a number on the size, but there are projects aimming at 
>> using Python on emdding systems.
>>
>> --
>> You received this message because you are subscribed to a topic in the 
>> Google Groups "pandoc-discuss" group.
>> To unsubscribe from this topic, visit 
>> https://groups.google.com/d/topic/pandoc-discuss/wbebx65e1Nk/unsubscribe.
>> To unsubscribe from this group and all its topics, send an email to 
>> pandoc-discus...-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org <javascript:>.
>> To post to this group, send email to pandoc-...-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org 
>> <javascript:>.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/pandoc-discuss/523cf1e2-af4c-41b7-b288-a458238faddd%40googlegroups.com
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/09b65a9e-920c-43df-ba59-46f62e0fcaa7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[-- Attachment #1.2: Type: text/html, Size: 4661 bytes --]

Re: Building pandoc "ecosystems": package manger, "central gallery", "wikibook", etc.?

[Kolen Cheung]

[-- Attachment #1.1: Type: text/plain, Size: 1306 bytes --]

I have another question not directly related to the recent discussions: 
license.

I see that usually filters or other tools built around pandoc choose a more 
permissive license. But for those cannot work without pandoc, it doesn't 
seem to make a difference since pandoc's license is more restrictive. And 
then when they are of a different license with pandoc, it might makes it 
more difficult to package them together. (I am trying to figure out in the 
long term, if something like "texlive-full" can be built around pandoc.)

Sidenote, there's podoc which is under BSD-3-Clause license. In principle 
pandoc filters can be made working with that kind of tools. In this case 
there might be a need of a more permissive license.

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/40c5e715-294a-4ad9-a561-7f37a9cdd187%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[-- Attachment #1.2: Type: text/html, Size: 1781 bytes --]

Re: Building pandoc "ecosystems": package manger, "central gallery", "wikibook", etc.?

[Sergio Correia]

[-- Attachment #1.1: Type: text/plain, Size: 1701 bytes --]

I would just let authors pick whatever license they want. If at some point 
# years from now someone wants to distribute a version of pandoc with 
filters included (and with python, etc.), it would be easy to just select 
those filters with a permissive license (MIT, BSD, Apache2, CC0, etc)

On Tuesday, November 22, 2016 at 7:49:43 PM UTC-5, Kolen Cheung wrote:
>
> I have another question not directly related to the recent discussions: 
> license.
>
> I see that usually filters or other tools built around pandoc choose a 
> more permissive license. But for those cannot work without pandoc, it 
> doesn't seem to make a difference since pandoc's license is more 
> restrictive. And then when they are of a different license with pandoc, it 
> might makes it more difficult to package them together. (I am trying to 
> figure out in the long term, if something like "texlive-full" can be built 
> around pandoc.)
>
> Sidenote, there's podoc which is under BSD-3-Clause license. In principle 
> pandoc filters can be made working with that kind of tools. In this case 
> there might be a need of a more permissive license.
>

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/ba3ee05b-d27f-4790-b92b-7d617c4b72f2%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[-- Attachment #1.2: Type: text/html, Size: 2297 bytes --]

Re: Building pandoc "ecosystems": package manger, "central gallery", "wikibook", etc.?

[Kolen Cheung]

[-- Attachment #1.1: Type: text/plain, Size: 2888 bytes --]



Concerning the “wikibook” part:

Since the idea of using gitit doesn’t work, I’m now thinking about using 
ghpages for this task. The main points are:

   - 
   
   blog: 
   - 
      
      rather than wiki, it will be more like a blog
      - 
      
      the blog posts will be “heavily” tagged and categorized, for easier 
      discoveries and better organization. e.g. each filter currently documented 
      in pandoc’s wiki might turned into a blog post will a bunch of tags, and 
      the category “filters”. Note that this is independent with the “gallery” 
      idea since that would probably be auto-generated.
      - 
      
      ghpages is chosen so it is easier for pull request, the only thing a 
      guest post needed is the content in GFM (no pandoc markdown, however), and 
      a pull request (inspired by DesignOpen/designopen.github.io 
      <https://github.com/DesignOpen/designopen.github.io>)
      - 
   
   hosting: GitHub. Also see if @jgm is interested to link from 
   pandoc.org’s current “Extras” on the left, or even use the pandoc.org 
   domain (e.g. users seeing pandoc.org/extras/index.html, which is actually 
   from pandoc-extras.github.io/index.html). This might be controversial, but 
   note that currently pandoc.org’s “Extras” linking to GitHub’s wiki is 
   opened to “attack” for any GitHub users.
   - 
   
   info from pandoc-extras & pandoc-filters in pandoc’s current wiki can 
   either be ported/mirrored, depending on the last point.
   - 
   
   license: open for discussion. CC? (seems good for collaboration and 
   guest posts. Alternatively, each guest can just hold their own copyright. I 
   don’t know which one gives more incentives for guest posts.) Another point 
   to note: I probably will put on a tip jar.
   - 
   
   search: Google search is much easier to setup, especially for static 
   site. But for sites that has low traffics (which probably is going to be 
   true for this site), Google’s search might take a long time to update. An 
   alternative will be lunr.js. It won’t be complied on pull request however. 
   (In principle Travis can be used, but pure ghpages might be easier for this 
   small site.)
   
​

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/d09ff929-8807-44ac-91db-35f1d9b70491%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[-- Attachment #1.2: Type: text/html, Size: 7896 bytes --]

Re: Building pandoc "ecosystems": package manger, "central gallery", "wikibook", etc.?

[Kolen Cheung]

[-- Attachment #1.1: Type: text/plain, Size: 772 bytes --]

Is the name "pacdoc" as the pandoc package manager ok with you?

I like how it sounds and similarity with pandoc. But I don't know if anyone 
would find it confusing since it differs 1 alphabet only.

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/c2a90568-1c74-421a-ae2a-98296bbff4e8%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[-- Attachment #1.2: Type: text/html, Size: 1221 bytes --]

Re: Building pandoc "ecosystems": package manger, "central gallery", "wikibook", etc.?

[Kolen Cheung]

[-- Attachment #1.1: Type: text/plain, Size: 1544 bytes --]



Regarding an official pandoc GitHub Organization, mentioned as a digression 
in pandoc-discuss/KOMA-script specific LaTeX template 
<https://groups.google.com/d/msg/pandoc-discuss/r55_FPn68xg/Tm3bHZFPBwAJ>:

Effort has been made to try to get the github.com/pandoc. But long story 
short, it’s a dead end.

@jgm has mentioned he could register the pandoc-org name instead.

I think having one such organization will have many benefits, e.g. the 
current pandoc-extras organization *could* fall into the same organization. 
i.e. all pandoc related tools/stuffs *can* be centered around this 
organization, official or 3rd party. (But there might be reasons to split 
it at least into 2, pandoc-org for official, pandoc-extras for 3rd party.)

But a migration like this could be troublesome (e.g. Travis builds pointing 
to the releases URL under jgm/pandoc). May be this change should be aligned 
with pandoc 2.0 to lessen the troubles?
​

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/ec25b6f9-bcca-4443-81f7-39301d8a5c05%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[-- Attachment #1.2: Type: text/html, Size: 3985 bytes --]

Re: Building pandoc "ecosystems": package manger, "central gallery", "wikibook", etc.?

[Václav Haisman]

[-- Attachment #1: Type: text/plain, Size: 2143 bytes --]

On 7 January 2017 at 19:41, Kolen Cheung <christian.kolen-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:

Regarding an official pandoc GitHub Organization, mentioned as a digression
> in pandoc-discuss/KOMA-script specific LaTeX template
> <https://groups.google.com/d/msg/pandoc-discuss/r55_FPn68xg/Tm3bHZFPBwAJ>:
>
> Effort has been made to try to get the github.com/pandoc. But long story
> short, it’s a dead end.
>
> @jgm has mentioned he could register the pandoc-org name instead.
>
> I think having one such organization will have many benefits, e.g. the
> current pandoc-extras organization *could* fall into the same
> organization. i.e. all pandoc related tools/stuffs *can* be centered
> around this organization, official or 3rd party. (But there might be
> reasons to split it at least into 2, pandoc-org for official, pandoc-extras
> for 3rd party.)
>
> But a migration like this could be troublesome (e.g. Travis builds
> pointing to the releases URL under jgm/pandoc). May be this change should
> be aligned with pandoc 2.0 to lessen the troubles?
> ​
>
> ​It seems that this might not be a problem. From page “About repository
transfers - User Documentation
<https://help.github.com/articles/about-repository-transfers/>“:​

*All links to the previous repository location are automatically redirected
to the new location.* When you use git clone, git fetch, or git push on a
transferred repository, these commands will redirect to the new repository
location or URL. (…)

​
-- 
VH

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/CAKw7uViZN4Q0t0OpGCe5175x8145JEt1DkhFRAc1xnO3jL7FOw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[-- Attachment #2: Type: text/html, Size: 14071 bytes --]

Re: Building pandoc "ecosystems": package manger, "central gallery", "wikibook", etc.?

[Kolen Cheung]

[-- Attachment #1: Type: text/plain, Size: 455 bytes --]

An update on pandocpm, the upcoming pandoc package manager:

We're finalizing the details of pandocpm in https://github.com/pandoc-extras/pandocpm/issues , in particular, there was some discussion here about the securities features. The lastest proposal is in https://github.com/pandoc-extras/pandocpm/issues/5 .

Feel free to make suggestions there. We probably will finalize the basic features within this month and start building an auto-gallery soon.

Re: Building pandoc "ecosystems": package manger, "central gallery", "wikibook", etc.?

[mb21]

[-- Attachment #1.1: Type: text/plain, Size: 812 bytes --]

I haven't been following this discussion thoroughly, but GitHub seems to 
just have introduced tags, so why not have all pandoc filters tagged with 
"pandoc-filter", e.g. 
https://github.com/search?q=topic%3Apandoc-filter&type=Repositories

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/07ccbc08-e55d-4df7-afab-7a0a54c9b56f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[-- Attachment #1.2: Type: text/html, Size: 1326 bytes --]

Re: Building pandoc "ecosystems": package manger, "central gallery", "wikibook", etc.?

[Kolen Cheung]

[-- Attachment #1: Type: text/plain, Size: 507 bytes --]

Ya, it is a new feature just launched in the previous few days. That would be helpful.

But for some filters that is not actively maintained, they won't be tagged (but on the other hand they will becomes obsolete, so it might not be important at all).

I think it will be great for us to come up with a convension of the name of the tags (best if done officially, may be even put in the MANUAL), so that people are not doing it sllightly differently (e.g. pandoc-filter or pandocfilters or pandoc-filters).