* Lua filter reading data from metadata or defaults file @ 2020-09-25 9:36 krulis....-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org [not found] ` <8a242488-066f-4c2a-aa0a-992788fa9e93n-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org> 0 siblings, 1 reply; 5+ messages in thread From: krulis....-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org @ 2020-09-25 9:36 UTC (permalink / raw) To: pandoc-discuss [-- Attachment #1.1: Type: text/plain, Size: 1503 bytes --] I would like to ask, whether it is possible for a lua filter to read data from metadata block (or metadata file) or from defaults file? I looked around website documentation, also into `lua filter repository`, but found no filter accessing metadata values in `defaults file` fields. I am working on a custom lua filter, for which I want to specify a metadata field, which would point to external file with required definitions, very similar to python filter `pandoc-mustache` github source here : <https://github.com/michaelstepner/pandoc-mustache/blob/master/pandoc_mustache/pandoc_mustache.py> `defaults.yaml`: ``` ... mustache: - ./src/varDefinitions.yaml ... ``` *Motivation:* Plans for this filter are to provide functionalities of LaTeX `glossaries` package to `pandoc`. There are already some filters, which take theyre own approach to the issue, but none of them gives full acronym support specific to given format. Long-term plans would be to also provide support for term definitions in `glossaries`-like style. Thanks for any guidance in this issue. Regards, Tomas -- You received this message because you are subscribed to the Google Groups "pandoc-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/8a242488-066f-4c2a-aa0a-992788fa9e93n%40googlegroups.com. [-- Attachment #1.2: Type: text/html, Size: 2069 bytes --] ^ permalink raw reply [flat|nested] 5+ messages in thread
[parent not found: <8a242488-066f-4c2a-aa0a-992788fa9e93n-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org>]
* Re: Lua filter reading data from metadata or defaults file [not found] ` <8a242488-066f-4c2a-aa0a-992788fa9e93n-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org> @ 2020-09-25 10:06 ` Albert Krewinkel [not found] ` <87v9g2jhm4.fsf-9EawChwDxG8hFhg+JK9F0w@public.gmane.org> 0 siblings, 1 reply; 5+ messages in thread From: Albert Krewinkel @ 2020-09-25 10:06 UTC (permalink / raw) To: pandoc-discuss-/JYPxA39Uh5TLH3MbocFFw You should be able to access `mustache` as a normal metadata field if it is nested under the field `metadata` in defaults.yaml: --- metadata: mustache: ./src/varDefinitions.yaml --- krulis....-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org writes: > I would like to ask, whether it is possible for a lua filter to read data > from metadata block (or metadata file) or from defaults file? I looked > around website documentation, also into `lua filter repository`, but found > no filter accessing metadata values in `defaults file` fields. > I am working on a custom lua filter, for which I want to specify a metadata > field, which would point to external file with required definitions, very > similar to python filter `pandoc-mustache` github source here : > <https://github.com/michaelstepner/pandoc-mustache/blob/master/pandoc_mustache/pandoc_mustache.py> > > `defaults.yaml`: > > ``` > ... > > mustache: > - ./src/varDefinitions.yaml > > ... > ``` > > *Motivation:* Plans for this filter are to provide functionalities of LaTeX > `glossaries` package to `pandoc`. There are already some filters, which > take theyre own approach to the issue, but none of them gives full acronym > support specific to given format. > Long-term plans would be to also provide support for term definitions in > `glossaries`-like style. > > Thanks for any guidance in this issue. > Regards, Tomas -- Albert Krewinkel GPG: 8eed e3e2 e8c5 6f18 81fe e836 388d c0b2 1f63 1124 ^ permalink raw reply [flat|nested] 5+ messages in thread
[parent not found: <87v9g2jhm4.fsf-9EawChwDxG8hFhg+JK9F0w@public.gmane.org>]
* Re: Lua filter reading data from metadata or defaults file [not found] ` <87v9g2jhm4.fsf-9EawChwDxG8hFhg+JK9F0w@public.gmane.org> @ 2020-09-25 11:36 ` krulis....-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org [not found] ` <9dbbc23d-1cde-4c0b-898a-ecbe4679d614n-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org> 0 siblings, 1 reply; 5+ messages in thread From: krulis....-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org @ 2020-09-25 11:36 UTC (permalink / raw) To: pandoc-discuss [-- Attachment #1.1: Type: text/plain, Size: 2644 bytes --] Those are good news! But my question might have been a little confusing. As a follow-up, how can load and access data provided in arbitraty yaml or json file with lua filter? Is there any helper function for that, or I should just use simple `open` or something ... ? I am looking at lua filters part in pandoc docs; example: "Replacing placeholders with their metadata value;" which kinda similar to my case, but here are just flat values. I would like to specify a filepath to yaml file with actual definitions. Dne pátek 25. září 2020 v 12:06:24 UTC+2 uživatel Albert Krewinkel napsal: > You should be able to access `mustache` as a normal metadata field if it > is nested under the field `metadata` in defaults.yaml: > > --- > metadata: > mustache: ./src/varDefinitions.yaml > --- > > > krulis....-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org writes: > > > I would like to ask, whether it is possible for a lua filter to read data > > from metadata block (or metadata file) or from defaults file? I looked > > around website documentation, also into `lua filter repository`, but > found > > no filter accessing metadata values in `defaults file` fields. > > I am working on a custom lua filter, for which I want to specify a > metadata > > field, which would point to external file with required definitions, very > > similar to python filter `pandoc-mustache` github source here : > > < > https://github.com/michaelstepner/pandoc-mustache/blob/master/pandoc_mustache/pandoc_mustache.py > > > > > > `defaults.yaml`: > > > > ``` > > ... > > > > mustache: > > - ./src/varDefinitions.yaml > > > > ... > > ``` > > > > *Motivation:* Plans for this filter are to provide functionalities of > LaTeX > > `glossaries` package to `pandoc`. There are already some filters, which > > take theyre own approach to the issue, but none of them gives full > acronym > > support specific to given format. > > Long-term plans would be to also provide support for term definitions in > > `glossaries`-like style. > > > > Thanks for any guidance in this issue. > > Regards, Tomas > > > -- > Albert Krewinkel > GPG: 8eed e3e2 e8c5 6f18 81fe e836 388d c0b2 1f63 1124 > -- You received this message because you are subscribed to the Google Groups "pandoc-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/9dbbc23d-1cde-4c0b-898a-ecbe4679d614n%40googlegroups.com. [-- Attachment #1.2: Type: text/html, Size: 3809 bytes --] ^ permalink raw reply [flat|nested] 5+ messages in thread
[parent not found: <9dbbc23d-1cde-4c0b-898a-ecbe4679d614n-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org>]
* Re: Lua filter reading data from metadata or defaults file [not found] ` <9dbbc23d-1cde-4c0b-898a-ecbe4679d614n-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org> @ 2020-09-25 13:06 ` BPJ [not found] ` <CADAJKhBkrFo1pT6UWE6qu=aAaOoMsxvNPXwHGiGoauyF-2XPnw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org> 0 siblings, 1 reply; 5+ messages in thread From: BPJ @ 2020-09-25 13:06 UTC (permalink / raw) To: pandoc-discuss [-- Attachment #1: Type: text/plain, Size: 5194 bytes --] This function is a hack which takes a file name, reads the file, makes sure as well as it can that Pandoc will see the text as a YAML block, parses the text with pandoc and returns the metadata of the result. ``````lua local function load_yaml (file_name) local file = assert(io.open(file_name)) local text = assert(file:read('a')) file:close() -- remove leading/trailing whitespace and existing delimiters text = text:gsub('^%s+',"") :gsub('%s+$',"") :gsub('^%-%-%-',"") :gsub('%-%-%-$',"") :gsub('%.%.%.$',"") -- insert delimiters text = "---\n" .. text .. "\n---\n" -- parse the YAML return pandoc.read(text, 'markdown').meta end `````` Note that this returns a metadata object, not a plain data data structure. You will need another function to convert the metadata into a plain data structure: ``````lua -- Turn Pandoc metadata into "plain" data: -- a table for Meta, MetaList, MetaMap; -- stringify all other elements; -- return anything else as-is. -- Either the whole metadata or a metadata item works. local meta_to_data do meta_to_data = function (meta) if 'table' == type(meta) then local tag = meta.tag if not(tag) or 'Meta' == tag or 'MetaList' == tag or 'MetaMap' == tag then local data = {} for k,v in pairs(meta) do data[k] = meta_to_data(v) end return data else return pandoc.utils.stringify(meta) end else return meta end end end `````` Note: (1) pandoc.utils.stringify doesn't handle block content very well. (2) I just rewrote meta_to_data from memory but it should work OK. -- Better --help|less than helpless Den fre 25 sep. 2020 13:37krulis....-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org <krulis.tomas.tk-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> skrev: > Those are good news! > > But my question might have been a little confusing. > As a follow-up, how can load and access data provided in arbitraty yaml or > json file with lua filter? Is there any helper function for that, or I > should just use simple `open` or something ... ? > I am looking at lua filters part in pandoc docs; example: "Replacing > placeholders with their metadata value;" which kinda similar to my case, > but here are just flat values. I would like to specify a filepath to yaml > file with actual definitions. > > Dne pátek 25. září 2020 v 12:06:24 UTC+2 uživatel Albert Krewinkel napsal: > >> You should be able to access `mustache` as a normal metadata field if it >> is nested under the field `metadata` in defaults.yaml: >> >> --- >> metadata: >> mustache: ./src/varDefinitions.yaml >> --- >> >> >> krulis....-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org writes: >> >> > I would like to ask, whether it is possible for a lua filter to read >> data >> > from metadata block (or metadata file) or from defaults file? I looked >> > around website documentation, also into `lua filter repository`, but >> found >> > no filter accessing metadata values in `defaults file` fields. >> > I am working on a custom lua filter, for which I want to specify a >> metadata >> > field, which would point to external file with required definitions, >> very >> > similar to python filter `pandoc-mustache` github source here : >> > < >> https://github.com/michaelstepner/pandoc-mustache/blob/master/pandoc_mustache/pandoc_mustache.py> >> >> > >> > `defaults.yaml`: >> > >> > ``` >> > ... >> > >> > mustache: >> > - ./src/varDefinitions.yaml >> > >> > ... >> > ``` >> > >> > *Motivation:* Plans for this filter are to provide functionalities of >> LaTeX >> > `glossaries` package to `pandoc`. There are already some filters, which >> > take theyre own approach to the issue, but none of them gives full >> acronym >> > support specific to given format. >> > Long-term plans would be to also provide support for term definitions >> in >> > `glossaries`-like style. >> > >> > Thanks for any guidance in this issue. >> > Regards, Tomas >> >> >> -- >> Albert Krewinkel >> GPG: 8eed e3e2 e8c5 6f18 81fe e836 388d c0b2 1f63 1124 >> > -- > You received this message because you are subscribed to the Google Groups > "pandoc-discuss" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org > To view this discussion on the web visit > https://groups.google.com/d/msgid/pandoc-discuss/9dbbc23d-1cde-4c0b-898a-ecbe4679d614n%40googlegroups.com > <https://groups.google.com/d/msgid/pandoc-discuss/9dbbc23d-1cde-4c0b-898a-ecbe4679d614n%40googlegroups.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "pandoc-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/CADAJKhBkrFo1pT6UWE6qu%3DaAaOoMsxvNPXwHGiGoauyF-2XPnw%40mail.gmail.com. [-- Attachment #2: Type: text/html, Size: 8026 bytes --] ^ permalink raw reply [flat|nested] 5+ messages in thread
[parent not found: <CADAJKhBkrFo1pT6UWE6qu=aAaOoMsxvNPXwHGiGoauyF-2XPnw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>]
* Re: Lua filter reading data from metadata or defaults file [not found] ` <CADAJKhBkrFo1pT6UWE6qu=aAaOoMsxvNPXwHGiGoauyF-2XPnw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org> @ 2020-09-25 17:23 ` krulis....-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org 0 siblings, 0 replies; 5+ messages in thread From: krulis....-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org @ 2020-09-25 17:23 UTC (permalink / raw) To: pandoc-discuss [-- Attachment #1.1: Type: text/plain, Size: 9021 bytes --] Thank you for your help @BPJ , I test it as soon as possible with the other parts of my code and report back. But I must say it is a little hazy to me (I have very small knowledge of lua) - the "data" is table consisting of key-value pairs of string values in yaml file? Maybe I have started with project that is beyond my skills now ... Would it be (a lot) simpler if I would use json format for data structure? I am not limited in that regard ... I have actually decided to describe my whole idea/plan here, since to continue from this point I will need help or guidance from someone more versed in lua than me. The whole idea is to have in markdown source hooks like this: ``````markdown Lets use a few acronyms, such as [!html] and [!xml]. `````` In the output, if the format would be .tex, acronym sequences would be translated to appropriate glossaries cmmands: ``````tex Lets use a few acronyms, such as \gls{html} and \gls{xml}. `````` For other formats it would do string replacement, say, for docx: ``````docx Lets use a few acronyms, such as HTML and XML. `````` Acronym definitions should be in an external file, in json or yaml format; example yaml: ``````yaml xml: - short: XML - long: extensible markup language `````` Now I would love to be able to reference this yaml file in defaults file, read data within and create some data structure out of it, ideally a dictionary (does lua even have that?). I have now gotten into a point, in which I dont know how to load external data with pandoc lua filter, hence this question. How far have I got (Still highly work in progress, certainly there are a lot of errors or unfinished bussinesses): ```````glossaries.lua [[ Acronym sequence identifiers - beginning and end ]] local acronymBegin = "[!" local acronymEnd = "]" [[ Acronym modifiers, for explicitly substituing long form, short form, or full - meaning long (short) ]] local short = "<" local long = ">" local full = "*" [[ Definition of `status` variables - those hold status information for acronym replacement (by command or plain text) ]] local shortStatus = false local longStatus = false local fullStatus = false [[ Matching function for each acronym command modifier. The idea is to store match results to `status` variables. ]] local function matchShort(acronymSequence) if acronymSequence.match(short) return true end end local function matchLong(acronymSequence) if acronymSequence.match(long) return true end end local function matchFull(acronymSequence) if acronymSequence.match(full) return true end end [[ Acronym detection - matching for acronym pattern in Str element ]] local function matchAcronym(acronymSequence) if string.match(acronymSequence, acronymBegin..'[<>*]?'..'(%a+)'..acronymEnd) return true, key -- "key" should hold acronym identifier, I hope ... else return false, nil end end [[ Core filter functions ]] -- Match for every identifier and return command accordingly: not finished local function processAcronym(acronymSequence) return command end -- Function connected to pandoc types, dealing with return values local function Str(elem) matchAcronym(elem.text) = isAcronym, key if isAcronym = true then processAcronym(elem.text) = -- again, not finished else --not acronym, dont change the Str element return nil end end ``````` Dne pátek 25. září 2020 v 15:06:34 UTC+2 uživatel BPJ napsal: > This function is a hack which takes a file name, reads the file, makes > sure as well as it can that Pandoc will see the text as a YAML block, > parses the text with pandoc and returns the metadata of the result. > > ``````lua > local function load_yaml (file_name) > local file = assert(io.open(file_name)) > local text = assert(file:read('a')) > file:close() > -- remove leading/trailing whitespace and existing delimiters > text = text:gsub('^%s+',"") > :gsub('%s+$',"") > :gsub('^%-%-%-',"") > :gsub('%-%-%-$',"") > :gsub('%.%.%.$',"") > -- insert delimiters > text = "---\n" .. text .. "\n---\n" > -- parse the YAML > return pandoc.read(text, 'markdown').meta > end > `````` > > Note that this returns a metadata object, not a plain data data structure. > You will need another function to convert the metadata into a plain data > structure: > > ``````lua > -- Turn Pandoc metadata into "plain" data: > -- a table for Meta, MetaList, MetaMap; > -- stringify all other elements; > -- return anything else as-is. > -- Either the whole metadata or a metadata item works. > local meta_to_data > do > meta_to_data = function (meta) > if 'table' == type(meta) then > local tag = meta.tag > if not(tag) or 'Meta' == tag or 'MetaList' == tag or 'MetaMap' == > tag then > local data = {} > for k,v in pairs(meta) do > data[k] = meta_to_data(v) > end > return data > else > return pandoc.utils.stringify(meta) > end > else > return meta > end > end > end > `````` > > Note: > (1) pandoc.utils.stringify doesn't handle block content very well. > (2) I just rewrote meta_to_data from memory but it should work OK. > > > -- > Better --help|less than helpless > > Den fre 25 sep. 2020 13:37krulis....-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org <krulis....-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> > skrev: > >> Those are good news! >> >> But my question might have been a little confusing. >> As a follow-up, how can load and access data provided in arbitraty yaml >> or json file with lua filter? Is there any helper function for that, or I >> should just use simple `open` or something ... ? >> I am looking at lua filters part in pandoc docs; example: "Replacing >> placeholders with their metadata value;" which kinda similar to my case, >> but here are just flat values. I would like to specify a filepath to yaml >> file with actual definitions. >> >> Dne pátek 25. září 2020 v 12:06:24 UTC+2 uživatel Albert Krewinkel napsal: >> >>> You should be able to access `mustache` as a normal metadata field if it >>> is nested under the field `metadata` in defaults.yaml: >>> >>> --- >>> metadata: >>> mustache: ./src/varDefinitions.yaml >>> --- >>> >>> >>> krulis....-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org writes: >>> >>> > I would like to ask, whether it is possible for a lua filter to read >>> data >>> > from metadata block (or metadata file) or from defaults file? I looked >>> > around website documentation, also into `lua filter repository`, but >>> found >>> > no filter accessing metadata values in `defaults file` fields. >>> > I am working on a custom lua filter, for which I want to specify a >>> metadata >>> > field, which would point to external file with required definitions, >>> very >>> > similar to python filter `pandoc-mustache` github source here : >>> > < >>> https://github.com/michaelstepner/pandoc-mustache/blob/master/pandoc_mustache/pandoc_mustache.py> >>> >>> > >>> > `defaults.yaml`: >>> > >>> > ``` >>> > ... >>> > >>> > mustache: >>> > - ./src/varDefinitions.yaml >>> > >>> > ... >>> > ``` >>> > >>> > *Motivation:* Plans for this filter are to provide functionalities of >>> LaTeX >>> > `glossaries` package to `pandoc`. There are already some filters, >>> which >>> > take theyre own approach to the issue, but none of them gives full >>> acronym >>> > support specific to given format. >>> > Long-term plans would be to also provide support for term definitions >>> in >>> > `glossaries`-like style. >>> > >>> > Thanks for any guidance in this issue. >>> > Regards, Tomas >>> >>> >>> -- >>> Albert Krewinkel >>> GPG: 8eed e3e2 e8c5 6f18 81fe e836 388d c0b2 1f63 1124 >>> >> -- >> You received this message because you are subscribed to the Google Groups >> "pandoc-discuss" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to pandoc-discus...-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/pandoc-discuss/9dbbc23d-1cde-4c0b-898a-ecbe4679d614n%40googlegroups.com >> <https://groups.google.com/d/msgid/pandoc-discuss/9dbbc23d-1cde-4c0b-898a-ecbe4679d614n%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> > -- You received this message because you are subscribed to the Google Groups "pandoc-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/8cf956a4-8a5e-4b6d-a817-8d4a6bcbb272n%40googlegroups.com. [-- Attachment #1.2: Type: text/html, Size: 13814 bytes --] ^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2020-09-25 17:23 UTC | newest] Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2020-09-25 9:36 Lua filter reading data from metadata or defaults file krulis....-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org [not found] ` <8a242488-066f-4c2a-aa0a-992788fa9e93n-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org> 2020-09-25 10:06 ` Albert Krewinkel [not found] ` <87v9g2jhm4.fsf-9EawChwDxG8hFhg+JK9F0w@public.gmane.org> 2020-09-25 11:36 ` krulis....-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org [not found] ` <9dbbc23d-1cde-4c0b-898a-ecbe4679d614n-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org> 2020-09-25 13:06 ` BPJ [not found] ` <CADAJKhBkrFo1pT6UWE6qu=aAaOoMsxvNPXwHGiGoauyF-2XPnw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org> 2020-09-25 17:23 ` krulis....-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).