Adding own elements to pandoc AST

Adding own elements to pandoc AST

[A A]

[-- Attachment #1: Type: text/plain, Size: 1149 bytes --]

Dear all,

I have a use case where I would like to be able to add my own element to
the pandoc AST such that I can use it in a panflute filter. This would be
an inline element emulating LaTeX’s \gls{term} command from the
glossaries.sty package.

I envision the markdown to look something like this:

mentioning a gls:term from the glossary.

or something like this:

mentioning a gls(term) from the glossary

So the syntax would be very similar to LaTeX, but a bit more lightweight.

Is there a way to do this in pandoc? Inheriting from Block or Inline and
setting up custom syntax such that it can be processed by a Lua or Python
filter?

Thanks for your consideration.

Regards,

Amine

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/CAMwawgNgACJDXLyFAcx26iy4-71yOUD%2BHccx67%2BRgJB132D0_A%40mail.gmail.com.

[-- Attachment #2: Type: text/html, Size: 5644 bytes --]

Re: Adding own elements to pandoc AST

[Bastien DUMONT]

Is the content of the \gls macro always a one-word string (without spaces)? If so, using the markup in your sample, you will end up with Str objects that you can handle like this (in Lua, I don't speak Python but it should not be difficult to translate):

```
function Str(s)
  if string.match(s.text, '^gls:') then
    -- code to handle the string
  end
  return s
end
```

Le Wednesday 13 July 2022 à 11:05:33PM, A A a écrit :
> Dear all,
> 
> I have a use case where I would like to be able to add my own element to the
> pandoc AST such that I can use it in a panflute filter. This would be an inline
> element emulating LaTeX’s \gls{term} command from the glossaries.sty package.
> 
> I envision the markdown to look something like this:
> 
> mentioning a gls:term from the glossary.
> 
> or something like this:
> 
> mentioning a gls(term) from the glossary
> 
> So the syntax would be very similar to LaTeX, but a bit more lightweight.
> 
> Is there a way to do this in pandoc? Inheriting from Block or Inline and
> setting up custom syntax such that it can be processed by a Lua or Python
> filter?
> 
> Thanks for your consideration.
> 
> Regards,
> 
> Amine
> 
> --
> You received this message because you are subscribed to the Google Groups
> "pandoc-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an email
> to [1]pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
> To view this discussion on the web visit [2]https://groups.google.com/d/msgid/
> pandoc-discuss/
> CAMwawgNgACJDXLyFAcx26iy4-71yOUD%2BHccx67%2BRgJB132D0_A%40mail.gmail.com.
> 
> References:
> 
> [1] mailto:pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
> [2] https://groups.google.com/d/msgid/pandoc-discuss/CAMwawgNgACJDXLyFAcx26iy4-71yOUD%2BHccx67%2BRgJB132D0_A%40mail.gmail.com?utm_medium=email&utm_source=footer

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/Ys9IXXrcD4Nwmu6q%40localhost.

Re: Adding own elements to pandoc AST

[A A]

[-- Attachment #1: Type: text/plain, Size: 3641 bytes --]

Dear Bastien,

Yes, in python I could also do a regex match on string elements, but you
can imagine this is a bit less ideal than creating a new AST element
altogether. Having an AST element would reduce the complexity of my code on
the filter side. I would only have to add an appropriately named function
in Lua, rather than have multiple if string.match then statements inside
the Str object for every new syntax I’d like to add. I’m trying to see how
difficult/feasible that would be…

Some extensions already add syntax elements, consider fenced_divs, how do
those extensions do that?

Regards,

Amine

On Thu, 14 Jul 2022 at 00:34, Bastien DUMONT <bastien.dumont-VwIFZPTo/vqsTnJN9+BGXg@public.gmane.org>
wrote:

> Is the content of the \gls macro always a one-word string (without
> spaces)? If so, using the markup in your sample, you will end up with Str
> objects that you can handle like this (in Lua, I don't speak Python but it
> should not be difficult to translate):
>
> ```
> function Str(s)
>   if string.match(s.text, '^gls:') then
>     -- code to handle the string
>   end
>   return s
> end
> ```
>
> Le Wednesday 13 July 2022 à 11:05:33PM, A A a écrit :
> > Dear all,
> >
> > I have a use case where I would like to be able to add my own element to
> the
> > pandoc AST such that I can use it in a panflute filter. This would be an
> inline
> > element emulating LaTeX’s \gls{term} command from the glossaries.sty
> package.
> >
> > I envision the markdown to look something like this:
> >
> > mentioning a gls:term from the glossary.
> >
> > or something like this:
> >
> > mentioning a gls(term) from the glossary
> >
> > So the syntax would be very similar to LaTeX, but a bit more lightweight.
> >
> > Is there a way to do this in pandoc? Inheriting from Block or Inline and
> > setting up custom syntax such that it can be processed by a Lua or Python
> > filter?
> >
> > Thanks for your consideration.
> >
> > Regards,
> >
> > Amine
> >
> > --
> > You received this message because you are subscribed to the Google Groups
> > "pandoc-discuss" group.
> > To unsubscribe from this group and stop receiving emails from it, send
> an email
> > to [1]pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
> > To view this discussion on the web visit [2]
> https://groups.google.com/d/msgid/
> > pandoc-discuss/
> > CAMwawgNgACJDXLyFAcx26iy4-71yOUD%2BHccx67%2BRgJB132D0_A%40mail.gmail.com
> .
> >
> > References:
> >
> > [1] mailto:pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
> > [2]
> https://groups.google.com/d/msgid/pandoc-discuss/CAMwawgNgACJDXLyFAcx26iy4-71yOUD%2BHccx67%2BRgJB132D0_A%40mail.gmail.com?utm_medium=email&utm_source=footer
>
> --
> You received this message because you are subscribed to the Google Groups
> "pandoc-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/pandoc-discuss/Ys9IXXrcD4Nwmu6q%40localhost
> .
>

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/CAMwawgNLuqw-NM_RYCHurBdg0EH20sjeAiYGOepmbgQLW9MD_A%40mail.gmail.com.

[-- Attachment #2: Type: text/html, Size: 7114 bytes --]

Re: Adding own elements to pandoc AST

[Bastien DUMONT]

Your intent is twofold: you want to add a new type to the data structure of Pandoc and to implement a new syntax corresponding to this type.

You cannot do the former unless you modify the sources of Pandoc and build it yourself, so you will have to emulate your "gls" type from another type. From the samples you provide, Str would be the most straightforward, but you can consider Span too, especially if the content can be composed of more than one word (e.g. [the term]{.gls}).

If you want to stick to panflute, I can't think of another way to implement the new syntactic rule than what I proposed. However, you can also use a preprocessor such as pp, which would enable you to use the !gls(term) syntax and transform it to [term]{.gls} before passing it to Pandoc. You can write your own script for preprocessing too. But in the end, you will always end up with a regular Pandoc object that you will need to post-process using pattern matching in your filter.

Le Thursday 14 July 2022 à 05:17:26AM, A A a écrit :
> Dear Bastien,
> 
> Yes, in python I could also do a regex match on string elements, but you can
> imagine this is a bit less ideal than creating a new AST element altogether.
> Having an AST element would reduce the complexity of my code on the filter
> side. I would only have to add an appropriately named function in Lua, rather
> than have multiple if string.match then statements inside the Str object for
> every new syntax I’d like to add. I’m trying to see how difficult/feasible that
> would be…
> 
> Some extensions already add syntax elements, consider fenced_divs, how do those
> extensions do that?
> 
> Regards,
> 
> Amine
> 
> 
> On Thu, 14 Jul 2022 at 00:34, Bastien DUMONT <[1]bastien.dumont-VwIFZPTo/vqzQB+pC5nmwQ@public.gmane.orgt>
> wrote:
> 
>     Is the content of the \gls macro always a one-word string (without spaces)?
>     If so, using the markup in your sample, you will end up with Str objects
>     that you can handle like this (in Lua, I don't speak Python but it should
>     not be difficult to translate):
> 
>     ```
>     function Str(s)
>       if string.match(s.text, '^gls:') then
>         -- code to handle the string
>       end
>       return s
>     end
>     ```
> 
>     Le Wednesday 13 July 2022 à 11:05:33PM, A A a écrit :
>     > Dear all,
>     >
>     > I have a use case where I would like to be able to add my own element to
>     the
>     > pandoc AST such that I can use it in a panflute filter. This would be an
>     inline
>     > element emulating LaTeX’s \gls{term} command from the glossaries.sty
>     package.
>     >
>     > I envision the markdown to look something like this:
>     >
>     > mentioning a gls:term from the glossary.
>     >
>     > or something like this:
>     >
>     > mentioning a gls(term) from the glossary
>     >
>     > So the syntax would be very similar to LaTeX, but a bit more lightweight.
>     >
>     > Is there a way to do this in pandoc? Inheriting from Block or Inline and
>     > setting up custom syntax such that it can be processed by a Lua or Python
>     > filter?
>     >
>     > Thanks for your consideration.
>     >
>     > Regards,
>     >
>     > Amine
>     >
>     > --
>     > You received this message because you are subscribed to the Google Groups
>     > "pandoc-discuss" group.
>     > To unsubscribe from this group and stop receiving emails from it, send an
>     email
>     > to [1][2]pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
>     > To view this discussion on the web visit [2][3]https://groups.google.com/
>     d/msgid/
>     > pandoc-discuss/
>     > CAMwawgNgACJDXLyFAcx26iy4-71yOUD%2BHccx67%2BRgJB132D0_A%[4]
>     40mail.gmail.com.
>     >
>     > References:
>     >
>     > [1] mailto:[5]pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
>     > [2] [6]https://groups.google.com/d/msgid/pandoc-discuss/
>     CAMwawgNgACJDXLyFAcx26iy4-71yOUD%2BHccx67%2BRgJB132D0_A%40mail.gmail.com?
>     utm_medium=email&utm_source=footer
> 
>     --
>     You received this message because you are subscribed to the Google Groups
>     "pandoc-discuss" group.
>     To unsubscribe from this group and stop receiving emails from it, send an
>     email to [7]pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
>     To view this discussion on the web visit [8]https://groups.google.com/d/
>     msgid/pandoc-discuss/Ys9IXXrcD4Nwmu6q%40localhost.
> 
> --
> You received this message because you are subscribed to the Google Groups
> "pandoc-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an email
> to [9]pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
> To view this discussion on the web visit [10]https://groups.google.com/d/msgid/
> pandoc-discuss/
> CAMwawgNLuqw-NM_RYCHurBdg0EH20sjeAiYGOepmbgQLW9MD_A%40mail.gmail.com.
> 
> References:
> 
> [1] mailto:bastien.dumont-VwIFZPTo/vqsTnJN9+BGXg@public.gmane.org
> [2] mailto:pandoc-discuss%2Bunsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
> [3] https://groups.google.com/d/msgid/
> [4] http://40mail.gmail.com/
> [5] mailto:pandoc-discuss%2Bunsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
> [6] https://groups.google.com/d/msgid/pandoc-discuss/CAMwawgNgACJDXLyFAcx26iy4-71yOUD%2BHccx67%2BRgJB132D0_A%40mail.gmail.com?utm_medium=email&utm_source=footer
> [7] mailto:pandoc-discuss%2Bunsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
> [8] https://groups.google.com/d/msgid/pandoc-discuss/Ys9IXXrcD4Nwmu6q%40localhost
> [9] mailto:pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
> [10] https://groups.google.com/d/msgid/pandoc-discuss/CAMwawgNLuqw-NM_RYCHurBdg0EH20sjeAiYGOepmbgQLW9MD_A%40mail.gmail.com?utm_medium=email&utm_source=footer

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/Ys/Jvo/Zf1AW12Vk%40localhost.

Re: Adding own elements to pandoc AST

[John MacFarlane]

You can't add new AST elements.  But instead of just using a string, you could use a Span with attributes.  That would be more convenient to match on in a filter.

> On Jul 14, 2022, at 5:17 AM, A A <amine.aboufirass-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> 
> Dear Bastien,
> 
> Yes, in python I could also do a regex match on string elements, but you can imagine this is a bit less ideal than creating a new AST element altogether. Having an AST element would reduce the complexity of my code on the filter side. I would only have to add an appropriately named function in Lua, rather than have multiple if string.match then statements inside the Str object for every new syntax I’d like to add. I’m trying to see how difficult/feasible that would be…
> 
> Some extensions already add syntax elements, consider fenced_divs, how do those extensions do that?
> 
> Regards,
> 
> Amine
> 
> 
> On Thu, 14 Jul 2022 at 00:34, Bastien DUMONT <bastien.dumont-VwIFZPTo/vqsTnJN9+BGXg@public.gmane.org> wrote:
> Is the content of the \gls macro always a one-word string (without spaces)? If so, using the markup in your sample, you will end up with Str objects that you can handle like this (in Lua, I don't speak Python but it should not be difficult to translate):
> 
> ```
> function Str(s)
>   if string.match(s.text, '^gls:') then
>     -- code to handle the string
>   end
>   return s
> end
> ```
> 
> Le Wednesday 13 July 2022 à 11:05:33PM, A A a écrit :
> > Dear all,
> > 
> > I have a use case where I would like to be able to add my own element to the
> > pandoc AST such that I can use it in a panflute filter. This would be an inline
> > element emulating LaTeX’s \gls{term} command from the glossaries.sty package.
> > 
> > I envision the markdown to look something like this:
> > 
> > mentioning a gls:term from the glossary.
> > 
> > or something like this:
> > 
> > mentioning a gls(term) from the glossary
> > 
> > So the syntax would be very similar to LaTeX, but a bit more lightweight.
> > 
> > Is there a way to do this in pandoc? Inheriting from Block or Inline and
> > setting up custom syntax such that it can be processed by a Lua or Python
> > filter?
> > 
> > Thanks for your consideration.
> > 
> > Regards,
> > 
> > Amine
> > 
> > --
> > You received this message because you are subscribed to the Google Groups
> > "pandoc-discuss" group.
> > To unsubscribe from this group and stop receiving emails from it, send an email
> > to [1]pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
> > To view this discussion on the web visit [2]https://groups.google.com/d/msgid/
> > pandoc-discuss/
> > CAMwawgNgACJDXLyFAcx26iy4-71yOUD%2BHccx67%2BRgJB132D0_A%40mail.gmail.com.
> > 
> > References:
> > 
> > [1] mailto:pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
> > [2] https://groups.google.com/d/msgid/pandoc-discuss/CAMwawgNgACJDXLyFAcx26iy4-71yOUD%2BHccx67%2BRgJB132D0_A%40mail.gmail.com?utm_medium=email&utm_source=footer
> 
> -- 
> You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
> To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/Ys9IXXrcD4Nwmu6q%40localhost.
> 
> -- 
> You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
> To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/CAMwawgNLuqw-NM_RYCHurBdg0EH20sjeAiYGOepmbgQLW9MD_A%40mail.gmail.com.

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/6BC0E81E-3DB4-4DA1-AA2B-3FCE12F77E52%40gmail.com.