Extracting hyperlinks with Pandoc API

Extracting hyperlinks with Pandoc API

[Gwern Branwen]

In order to feed my archiver tool
(http://hackage.haskell.org/package/archiver), I've written a small
tool to extract external links from markdown files. This might be
useful for other purposes (checking whether they exist, for example).
It's a short program:

import System.Environment (getArgs)
import Text.Pandoc (defaultParserState, processWithM, readMarkdown,
Inline(Link), Pandoc)

main :: IO ()
main = getArgs >>= mapM readFile >>= mapM_ analyzePage

analyzePage :: String -> IO Pandoc
analyzePage x = processWithM printLinks (readMarkdown defaultParserState x)

printLinks :: Inline -> IO Inline
printLinks (Link _ (x, _)) = putStrLn x >> return undefined
printLinks x                   = return x

You would use it something like this:

$ find . -name "*.page" -exec link-extractor {} \; | sort | grep -v
'\!' >> ~/.urls.txt

(The sort is there for aesthetics, and the grep is there to filter out
the interwiki links. My archiver program runs on the specified file.)

-- 
gwern
http://www.gwern.net

Re: Extracting hyperlinks with Pandoc API

[John MacFarlane]

You could also use queryWith for this (untested, but
should give the basic idea):

qLink :: Inline -> [String]
qLink (Link _ (x,_)) = [x]
qLink _              = []

findLinks :: Pandoc -> [String]
findLinks = sort . queryWith qLink

John

+++ Gwern Branwen [Nov 30 10 15:15 ]:
> In order to feed my archiver tool
> (http://hackage.haskell.org/package/archiver), I've written a small
> tool to extract external links from markdown files. This might be
> useful for other purposes (checking whether they exist, for example).
> It's a short program:
> 
> import System.Environment (getArgs)
> import Text.Pandoc (defaultParserState, processWithM, readMarkdown,
> Inline(Link), Pandoc)
> 
> main :: IO ()
> main = getArgs >>= mapM readFile >>= mapM_ analyzePage
> 
> analyzePage :: String -> IO Pandoc
> analyzePage x = processWithM printLinks (readMarkdown defaultParserState x)
> 
> printLinks :: Inline -> IO Inline
> printLinks (Link _ (x, _)) = putStrLn x >> return undefined
> printLinks x                   = return x
> 
> You would use it something like this:
> 
> $ find . -name "*.page" -exec link-extractor {} \; | sort | grep -v
> '\!' >> ~/.urls.txt
> 
> (The sort is there for aesthetics, and the grep is there to filter out
> the interwiki links. My archiver program runs on the specified file.)
> 
> -- 
> gwern
> http://www.gwern.net
> 
> -- 
> You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
> To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
> To unsubscribe from this group, send email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
> For more options, visit this group at http://groups.google.com/group/pandoc-discuss?hl=en.
>