Re: Lua filter to change macro for figure caption md -> latex

["'William Lupton' via pandoc-discuss" <pandoc-discuss-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org>]

[-- Attachment #1: Type: text/plain, Size: 20550 bytes --]

The caption looks like this (pasted from earlier).

    caption: {

      long: Blocks[1] {

        [1] Plain {

          content: Inlines[7] {

            [1] Str "caption"

            [2] Space

            [3] Str "to"

            [4] Space

            [5] Str "an"

            [6] Space

            [7] Str "image"

          }

        }

      }

    }



It's of type Caption (https://pandoc.org/lua-filters.html#type-caption) and
I believe that you can assume that the long caption is present (because the
docs say "The caption ... with an optional short caption") but perhaps you
should check.

So, in your code, fig.caption.long will be of type Blocks (
https://pandoc.org/lua-filters.html#type-blocks).

The caption could contain formatting but if you don't care about this (you
could always circle back later) you can just stringify it,
i.e., pandoc.utils.stringify(figure.caption.long) (
https://pandoc.org/lua-filters.html#pandoc.utils.stringify). The docs say
that you can only stringify an element but it seems to work with Blocks
(probably via some type coercion).

My logging output was just running pandoc with the (default) HTML output.
It wasn't running your filter.

On Mon, 11 Dec 2023 at 22:04, 'lukeflo' via pandoc-discuss <
pandoc-discuss-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org> wrote:

> Dear William, dear Pablo,
>
> thanks for your replies. It does indeed help me to understand the way such
> writers/filters work. But unfortunatley, I can't produce a working filter
> on my own.
>
> @William: I tried using 'fig.caption' instead of 'fig.content.caption[1]'.
> But then I get a different error message: *Error running Lua: attempt to
> concatenate a table value (field 'caption')*.
> Thus, I'm a little bit confused, since the html output at the end of your
> logging output looks like most fields are processed correctly.
>
> @Pablo: Your filters are really interesting. The float code gave me some
> usefule insights. I also tried to adapt your Latex code:
>
> ``` lua
> function Figure(elem)
>   raw_elem = '\\begin{figure*}\n' ..
>               '\\centering\n' ..
>               '\\includegraphics[width=0.9\\textwidth]{' .. elem.content
> .. '}\n' ..
>               '\\caption{' .. elem.caption .. '}\n' ..
>               '\\end{figure*}'
>   return pandoc.RawBlock('latex', raw_elem)
> end
> ```
> I run it as filter instead as writer. But get the same error as above.
>
> Generally, my biggest problem is that I don't fully understand how to
> access the values of the different AST fields. How, for example, can I use
> the values of the different fields of a Pandoc Figure Block
> <https://pandoc.org/lua-filters.html#type-figure>? Caption is there defined
> as key-value pair <https://pandoc.org/lua-filters.html#type-caption> for
> long and short captions, but as mentioned 'fig.caption' oder 'elem.caption'
> throws the table value error. Might that be due to the key-value form? And
> is there a difference accessing these things for a filter instead of a
> writer?
>
> If someone has some informations on that, it would be great; even better
> with a (short) framework or example that shows how to access a value and
> put it in a framework with latex-code surrounded. The docs are very
> detailed, but I don't find an explanation of these basics which makes sense
> to me...
>
> Thanks again and all the best!
>
> Pablo Serrati schrieb am Montag, 11. Dezember 2023 um 15:53:38 UTC+1:
>
>> Hello Lukeflo
>>
>> I have been working for a few months on a project to enable the
>> publication of scientific journals (https://github.com/estedeahora/guri).
>> The project has a workflow that takes `docx` and generates the final files
>> (`pdf`, `xml` and `html`), using an `markdown` (md) as an intermediate
>> file. One of the issues I had to address was the floating elements and in
>> particular the use of subcaptions for "sources" and "notes". While the
>> project is still in the development stage, I think some of the filters I
>> use may help you (see `files/filters/`).
>>
>> So far we are using the proposal in the journal [Quid 16](
>> https://publicaciones.sociales.uba.ar/index.php/quid16/issue/current) )
>> if you are interested in seeing the results. My plan is to generate a
>> stable release that can be used by other journals in the coming months. I
>> am open to new ideas and contributions to improve the project.
>>
>> Best,
>>
>> Pablo
>>
>> El lun, 11 dic 2023 a la(s) 08:30, 'William Lupton' via pandoc-discuss (
>> pandoc-...-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org) escribió:
>>
>>> Luke,
>>>
>>> Filter functions take a single argument (the element in question) so you
>>> need the second form: Figure(fig).
>>>
>>> I think the crash is because the caption is in fig.caption (not in
>>> fig.content[1].caption). See
>>> https://pandoc.org/lua-filters.html#type-figure.
>>>
>>> Perhaps I can put in a plug for https://github.com/pandoc-ext/logging.
>>> I think you might find this helpful for gaining insight into the element
>>> structure. See example below.
>>>
>>> Hope this helps.
>>>
>>> Cheers,
>>> William
>>>
>>> --------
>>>
>>> With this filter in rep.lua:
>>>
>>> local logging = require 'logging'
>>>
>>> function Pandoc(pandoc)
>>>
>>>     if logging.loglevel > 0 then
>>>
>>>         logging.temp('meta', pandoc.meta)
>>>
>>>     end
>>>
>>>     logging.temp('blocks', pandoc.blocks)
>>>
>>> end
>>>
>>> ...your input gives this:
>>>
>>> *% *pandoc luke.md -L rep.lua
>>>
>>> (#) blocks Blocks[3] {
>>>
>>>   [1] Header {
>>>
>>>     attr: Attr {
>>>
>>>       attributes: AttributeList {}
>>>
>>>       classes: List {}
>>>
>>>       identifier: "headline"
>>>
>>>     }
>>>
>>>     content: Inlines[1] {
>>>
>>>       [1] Str "Headline"
>>>
>>>     }
>>>
>>>     level: 1
>>>
>>>   }
>>>
>>>   [2] Para {
>>>
>>>     content: Inlines[3] {
>>>
>>>       [1] Str "Some"
>>>
>>>       [2] Space
>>>
>>>       [3] Str "text"
>>>
>>>     }
>>>
>>>   }
>>>
>>>   [3] Figure {
>>>
>>>     attr: Attr {
>>>
>>>       attributes: AttributeList {}
>>>
>>>       classes: List {}
>>>
>>>       identifier: ""
>>>
>>>     }
>>>
>>>     caption: {
>>>
>>>       long: Blocks[1] {
>>>
>>>         [1] Plain {
>>>
>>>           content: Inlines[7] {
>>>
>>>             [1] Str "caption"
>>>
>>>             [2] Space
>>>
>>>             [3] Str "to"
>>>
>>>             [4] Space
>>>
>>>             [5] Str "an"
>>>
>>>             [6] Space
>>>
>>>             [7] Str "image"
>>>
>>>           }
>>>
>>>         }
>>>
>>>       }
>>>
>>>     }
>>>
>>>     content: Blocks[1] {
>>>
>>>       [1] Plain {
>>>
>>>         content: Inlines[1] {
>>>
>>>           [1] Image {
>>>
>>>             attr: Attr {
>>>
>>>               attributes: AttributeList {}
>>>
>>>               classes: List {}
>>>
>>>               identifier: ""
>>>
>>>             }
>>>
>>>             caption: Inlines[7] {
>>>
>>>               [1] Str "caption"
>>>
>>>               [2] Space
>>>
>>>               [3] Str "to"
>>>
>>>               [4] Space
>>>
>>>               [5] Str "an"
>>>
>>>               [6] Space
>>>
>>>               [7] Str "image"
>>>
>>>             }
>>>
>>>             src: "counter_plot_new_periods.png"
>>>
>>>             title: ""
>>>
>>>           }
>>>
>>>         }
>>>
>>>       }
>>>
>>>     }
>>>
>>>   }
>>>
>>> }
>>>
>>> <h1 id="headline">Headline</h1>
>>>
>>> <p>Some text</p>
>>>
>>> <figure>
>>>
>>> <img src="counter_plot_new_periods.png" alt="caption to an image" />
>>>
>>> <figcaption aria-hidden="true">caption to an image</figcaption>
>>>
>>> </figure>
>>>
>>> On Mon, 11 Dec 2023 at 10:36, 'lukeflo' via pandoc-discuss <
>>> pandoc-...-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org> wrote:
>>>
>>>> PS: I know I'm using two very different approaches calling the Figure
>>>> function with arguments. Thats due to the fact that I'm not sure which way
>>>> is the right one...
>>>>
>>>> lukeflo schrieb am Montag, 11. Dezember 2023 um 11:33:21 UTC+1:
>>>>
>>>>> So far, I took the markdwon writer example from the pandoc docs
>>>>> <https://pandoc.org/custom-writers.html#example-modified-markdown-writer>
>>>>> to try out the general function of writers. It works and I think that I
>>>>> understand the general usage.
>>>>>
>>>>> But especially figures (in latex writer and presuambley in general)
>>>>> are relativley complex. Here are two things I tried out so far but always
>>>>> got an error:
>>>>>
>>>>> ``` lua
>>>>> function Writer (doc, opts)
>>>>>    local filter = {
>>>>>       function Figure (caption, image, attr)
>>>>>  local figcap = '\sidecaption{' .. caption .. '}'
>>>>>  return '\\begin{figure}\n' .. image .. '\n' .. figcap .. '\n'
>>>>> '\\end{figure}\n'
>>>>>       end
>>>>>    }
>>>>>    return pandoc.write(doc:walk(filter), 'latex', opts)
>>>>> end
>>>>> ```
>>>>> If I run this writer with my custom template from the CLI using *pandoc
>>>>> --template=../custom.template -t test-writer.lua ast-test.txt -o
>>>>> ast-test.tex* I get
>>>>> *Error running Lua:test-writer.lua:27: '(' expected near 'Figure'*.
>>>>>
>>>>> Furthermore, I tried running the following code just to understand how
>>>>> those writer work. Here I just wanted to replace {figure} with the starred
>>>>> version {figure*} (not sidecaption):
>>>>>
>>>>> ``` lua
>>>>> function Writer (doc, opts)
>>>>>   local filter = {
>>>>>      Figure = function (fig)
>>>>> local tester = '\\begin{figure*}\n' ..
>>>>> fig.content[1].caption[1].attributes[1] .. '\\end{figure*}\n'
>>>>> return pandoc.RawBlock('latex', tester)
>>>>>     end
>>>>>   }
>>>>>   return pandoc.write(doc:walk(filter), 'latex', opts)
>>>>> end
>>>>> ```
>>>>> But also got an error:
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> *Error running Lua:test-writer.lua:28: attempt to index a nil value
>>>>> (field 'caption')stack traceback: [C]: in ? [C]: in method 'walk'
>>>>> test-writer.lua:32: in function 'Writer'stack traceback:
>>>>> test-writer.lua:32: in function 'Writer'*
>>>>>
>>>>> I'm aware that I might be missing something very basic and maybe even
>>>>> very simple. But I'm kind of getting lost a little bit inside all
>>>>> functions, modules etc. as well as the general framework of such writers.
>>>>>
>>>>> Thus, any help explaining my errors and maybe suggesting some better
>>>>> code is very appreciated!
>>>>>
>>>>> The test file in both cases is very simple:
>>>>>
>>>>> ``` markdown
>>>>> ---
>>>>> title: A title
>>>>> ---
>>>>>
>>>>> # Headline
>>>>>
>>>>> Some text
>>>>>
>>>>> ![caption to an image](counter_plot_new_periods.png)
>>>>> ```
>>>>>
>>>>> Thanks in advance!
>>>>> lukeflo schrieb am Freitag, 8. Dezember 2023 um 08:35:48 UTC+1:
>>>>>
>>>>>> Hello Julien,
>>>>>>
>>>>>> thanks for the reply. Unfortunatley, as mentioned in the
>>>>>> stackoverflow post, your suggested LaTeX code won't work.
>>>>>>
>>>>>> The \caption macro is very complex in the backend and cannot be
>>>>>> copied on the fly via \let, \NewCommandCopy or something similar.
>>>>>> Even after doing so with e.g. \NewCommandCopy{\oldcaption}{\caption}
>>>>>> and then setting \RenewDocumentCommand{\caption}{o
>>>>>> m}{\sidecaption[#1]{#2}} nothing changes and the definition of
>>>>>> \caption, checked with \meaning or something similar, stays the same
>>>>>> as before (even \DeclareDocumentCommand doesn't work).
>>>>>>
>>>>>> In the end, it might be possible to somehow change the \caption
>>>>>> macro itself. But the effort might not be worth the result (and its more of
>>>>>> a question for TeX.SE).
>>>>>>
>>>>>> Using a custom writer for building Latex figures and replace the
>>>>>> \caption string inside would be a great solution. I read through the writer
>>>>>> manual, but didn't really understand how the AST works and which values
>>>>>> have to be used in such a writer. Furthermore, I'm using a a custom Latex
>>>>>> template for exporting (based on the default.template.latex) which has to
>>>>>> be integrated with such a writer.
>>>>>>
>>>>>> Therefore, I really woud appreciate a Lua framework to understand
>>>>>> which functions have to be edited etc. to accomplish the substitution.
>>>>>>
>>>>>> Best
>>>>>>
>>>>>> Julien Dutant schrieb am Dienstag, 5. Dezember 2023 um 17:09:19 UTC+1:
>>>>>>
>>>>>>> Lua filters only change Pandoc's AST representation of your
>>>>>>> document, i.e. before it is then converted to LaTeX. A Raw block filter
>>>>>>> will not act on Pandoc's LaTeX output, but only on Raw LaTeX blocks that
>>>>>>> are in the markdown itself.
>>>>>>>
>>>>>>> A Pandoc solution would be to write a custom Lua *writer*
>>>>>>> <https://pandoc.org/custom-writers.html>. The writer would use
>>>>>>> pandoc.write to generate Pandoc's own LaTeX output (body only) and modify
>>>>>>> it with regular expressions or Lua patterns. To replace just a command name
>>>>>>> this is fairly easy, though longer than the third solution below.
>>>>>>>
>>>>>>> A LaTeX solution is to redefine \caption as \sidecaption:
>>>>>>> \renewcommand{\caption}{\sidecaption}
>>>>>>>
>>>>>>> You can keep this enclosed in groups ({...}) to ensure that the
>>>>>>> redefinition only applies locally.
>>>>>>>
>>>>>>> A hybrid Pandoc/LaTeX solution is a Lua filter that insert LaTeX
>>>>>>> code to redefine \caption around figures:
>>>>>>>
>>>>>>> ``` lua
>>>>>>> if FORMAT:match 'latex' then
>>>>>>>     function Figure (elem) return {
>>>>>>>         pandoc.RawBlock('latex',
>>>>>>> '{\\renewcommand{\\caption}{\\subcaption}'),
>>>>>>>          elem,
>>>>>>>          pandoc.RawBlock('latex','}')
>>>>>>>        }
>>>>>>>    end
>>>>>>> end
>>>>>>>
>>>>>>> ```
>>>>>>>
>>>>>>> This replaces any 'Figure' block element by a list (succession) of
>>>>>>> three raw LaTeX blocks. The output should look like:
>>>>>>> {\renewcommand{\caption}{\subcaption}
>>>>>>> ... Pandoc's LaTeX for the figure ...
>>>>>>> }
>>>>>>>
>>>>>>> Reposted from
>>>>>>> https://stackoverflow.com/questions/77504584/pandoc-md-latex-write-lua-filter-to-change-latex-macro-used-for-caption/77607636#77607636
>>>>>>>
>>>>>>> On Monday, November 20, 2023 at 7:06:57 AM UTC+11 lukeflo wrote:
>>>>>>>
>>>>>>>> Hi everybody,
>>>>>>>>
>>>>>>>> I have written a custom latex `.cls' file to establish a typesetting
>>>>>>>> workflow for the scientific journals of my research institute. The
>>>>>>>> texts
>>>>>>>> should be written in Markdown and then be processed with `pandoc' to
>>>>>>>> LaTeX.
>>>>>>>>
>>>>>>>> I already have an elaborated pandoc template to produce the LaTeX
>>>>>>>> preambel etc. So far its working great.
>>>>>>>>
>>>>>>>> But for the figures I need the caption from the Markdown file to be
>>>>>>>> set
>>>>>>>> with `\sidecaption' instead of `\caption' in LaTeX, as well as with
>>>>>>>> an
>>>>>>>> optional argument (short-caption) for the image attribution in the
>>>>>>>> list
>>>>>>>> of figures.
>>>>>>>>
>>>>>>>> To get the latter working I use the following template from a GitHub
>>>>>>>> discussion in the [pandoc repo]:
>>>>>>>>
>>>>>>>> ┌────
>>>>>>>> │ PANDOC_VERSION:must_be_at_least '3.1'
>>>>>>>> │
>>>>>>>> │ if FORMAT:match 'latex' then
>>>>>>>> │   function Figure(f)
>>>>>>>> │     local short =
>>>>>>>> f.content[1].content[1].attributes['short-caption']
>>>>>>>> │     if short and not f.caption.short then
>>>>>>>> │       f.caption.short = pandoc.Inlines(short)
>>>>>>>> │     end
>>>>>>>> │     return f
>>>>>>>> │   end
>>>>>>>> │ end
>>>>>>>> └────
>>>>>>>>
>>>>>>>> That works without any flaws.
>>>>>>>>
>>>>>>>> But now I need to figure out how to change the LaTeX macro used for
>>>>>>>> the
>>>>>>>> caption. The older [approach of pre pandoc version 3.0 posted] by
>>>>>>>> tarleb
>>>>>>>> is really intuitive and I could have easily adapted it to my needs.
>>>>>>>> But
>>>>>>>> since pandoc 3.0 there is the new [/complex figures/] approach and,
>>>>>>>> so
>>>>>>>> far, I couldn't figure out how to change the LaTeX macro used for
>>>>>>>> the
>>>>>>>> captions with this new behaviour.
>>>>>>>>
>>>>>>>> I tried something like that (Adapted from [here]:
>>>>>>>>
>>>>>>>> ┌────
>>>>>>>> │ if FORMAT:match 'latex' then
>>>>>>>> │   function RawBlock (raw)
>>>>>>>> │     local caption = raw.text:match('\\caption')
>>>>>>>> │     if caption then
>>>>>>>> │        raw:gsub('\\caption', '\\sidecaption')
>>>>>>>> │     end
>>>>>>>> │     return raw
>>>>>>>> │   end
>>>>>>>> │ end
>>>>>>>> └────
>>>>>>>>
>>>>>>>> But nothing happened.
>>>>>>>>
>>>>>>>> The main challenge for me are my more-or-less non-existing lua
>>>>>>>> skills. I
>>>>>>>> just never had to use it for my daily tasks. I thought about using
>>>>>>>> `awk'
>>>>>>>> or `sed' to edit the `.tex' file itself using a regex-substitution,
>>>>>>>> but
>>>>>>>> that should remain an absolute stopgap, since it makes the whole
>>>>>>>> workflow less portable.
>>>>>>>>
>>>>>>>> Thus, I'm hoping for a hint/a solution in form of a pandoc-lua
>>>>>>>> script
>>>>>>>> which 1. helps me to achieve the goal, and 2. improve my
>>>>>>>> understanding
>>>>>>>> of lua and the /complex figures/ approach for similar future tasks.
>>>>>>>>
>>>>>>>> I appreciate any tipp!
>>>>>>>>
>>>>>>>> Best,
>>>>>>>> Lukeflo
>>>>>>>>
>>>>>>>> This question is also posted on StackOverFlow:
>>>>>>>> https://stackoverflow.com/q/77504584/19647155
>>>>>>>>
>>>>>>>> [pandoc repo]
>>>>>>>> <https://github.com/jgm/pandoc/issues/7915#issuecomment-1427113349>
>>>>>>>>
>>>>>>>> [approach of pre pandoc version 3.0 posted]
>>>>>>>> <https://github.com/jgm/pandoc/issues/7915#issuecomment-1039370851>
>>>>>>>>
>>>>>>>> [/complex figures/] <https://github.com/jgm/pandoc/releases?page=2>
>>>>>>>>
>>>>>>>> [here] <https://stackoverflow.com/a/71296595/19647155>
>>>>>>>>
>>>>>>> --
>>>> You received this message because you are subscribed to the Google
>>>> Groups "pandoc-discuss" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>> an email to pandoc-discus...-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
>>>> To view this discussion on the web visit
>>>> https://groups.google.com/d/msgid/pandoc-discuss/32dfe8eb-98ac-40ee-92d7-162528add367n%40googlegroups.com
>>>> <https://groups.google.com/d/msgid/pandoc-discuss/32dfe8eb-98ac-40ee-92d7-162528add367n%40googlegroups.com?utm_medium=email&utm_source=footer>
>>>> .
>>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "pandoc-discuss" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to pandoc-discus...-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
>>>
>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/pandoc-discuss/CAEe_xxikobOS_G9x71nxtz0dr99VVhgBV8in%3DxKzXgh7JMaRcw%40mail.gmail.com
>>> <https://groups.google.com/d/msgid/pandoc-discuss/CAEe_xxikobOS_G9x71nxtz0dr99VVhgBV8in%3DxKzXgh7JMaRcw%40mail.gmail.com?utm_medium=email&utm_source=footer>
>>> .
>>>
>> --
> You received this message because you are subscribed to the Google Groups
> "pandoc-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/pandoc-discuss/78503784-88bd-4d34-b5f5-a8634d667ba0n%40googlegroups.com
> <https://groups.google.com/d/msgid/pandoc-discuss/78503784-88bd-4d34-b5f5-a8634d667ba0n%40googlegroups.com?utm_medium=email&utm_source=footer>
> .
>

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/CAEe_xxj7N7897P6r_kUzwjbG3WDko38_2O178%2BWtZhQXNjuiPA%40mail.gmail.com.

[-- Attachment #2: Type: text/html, Size: 60610 bytes --]