lua filter to process github wiki links ?

lua filter to process github wiki links ?

[Simon Michael]

Hi all,

I'm working towards using github wiki as a content management system, but rendering the wiki pages myself for better integration with the rest of the website. Github's wiki links look like [[Page]] or [[Link text|Page]]. I can convert them with sed or haskell, but I'm already using pandoc's excellent lua filter feature, and I'd like to do the wiki linking that way too if feasible. Would anyone have a script for this, or advice on how to tackle it ?

Here's an example link: 

[[Convert CSV files]]  

and native format:

,Para [Str "[[Convert",Space,Str "CSV",Space,Str "files]]"

in which I'm not sure to detect the links robustly.

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/E4517694-C440-4902-9EBC-9413F199BCE5%40joyful.com.
For more options, visit https://groups.google.com/d/optout.

Re: lua filter to process github wiki links ?

[Simon Michael]

[-- Attachment #1: Type: text/plain, Size: 3717 bytes --]



> On Feb 8, 2019, at 3:39 PM, BP Jonsson <bpjonsson-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> 
> Den 2019-02-08 kl. 16:47, skrev Simon Michael:
>> Hi all,
>> I'm working towards using github wiki as a content management system, but rendering the wiki pages myself for better integration with the rest of the website. Github's wiki links look like [[Page]] or [[Link text|Page]]. I can convert them with sed or haskell, but I'm already using pandoc's excellent lua filter feature, and I'd like to do the wiki linking that way too if feasible. Would anyone have a script for this, or advice on how to tackle it ?
> 
> In my experience it is actually easier to filter the source to convert wiki links to Pandoc links.  You just have to take care that you don't do the change inside code spans and code blocks --- typically by capturing code at a higher priority than actual wiki links and just pass them through.
> 
> To convert wiki links to Pandoc links I actually use this:

Thanks for this. I agree, this kind of transformation seems more easily done at the text level than at the pandoc AST level, in the typical case where absolute robustness is not needed. I wonder why pandoc's native AST breaks up strings at spaces.

Your scripts are quite interesting, and worth further study for me. Since I want to be cross-platform and reduce external dependencies, and I'm in a haskell shake script, I have moved the wiki linking into haskell. For interest, here's my current shake rule for generating html from markdown:

    -- render one website page (from main or wiki repos) as html,
    -- and save in sites/_site/.
    -- We first preprocess wiki content (linking github wikilinks etc.)
    -- in haskell, and then convert to html by running pandoc.
    "site/_site//*.html" %> \out -> do
        let filename = takeBaseName out
            pagename = fileNameToPageName filename
            iswikipage = filename `elem` wikipagefilenames
            source
              | iswikipage = "wiki" </> filename <.> "md"
              | otherwise  = "site" </> filename <.> "md"
            template = "site/site.tmpl"
            siteRoot = if "site/_site/doc//*" ?== out then "../.." else "."
        need [source, template]
        -- read markdown source, link any wikilinks, pipe it to pandoc, 
        -- write html out
        Stdin . wikify (if iswikipage then Just (fileNameToPageName filename) else Nothing) <$> (readFile' source) >>=
          (cmd Shell pandoc fromsrcmd "-t html"
                           "--template" template
                           ("--metadata=siteRoot:" ++ siteRoot)
                           ("--metadata=\"title:" ++ pagename ++ "\"")
                           "--lua-filter=tools/pandoc-site.lua"
                           "-o" out )

This is all a bit complicated right now but I think it's paying off in robustness and power. It has allowed me to integrate github wiki content in the http://hledger.org <http://hledger.org/> website (see Cookbook), which I'm very pleased about. Pandoc continues to be a backbone of hledger's docs. Thanks Pandoc!


-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/3F388E0A-32D1-4B0D-B7CB-EBDBCFCFA42D%40joyful.com.
For more options, visit https://groups.google.com/d/optout.

[-- Attachment #2: Type: text/html, Size: 6139 bytes --]