Lua filters: Extensions with pandoc.read?

Lua filters: Extensions with pandoc.read?

[BPJ]

[-- Attachment #1: Type: text/plain, Size: 705 bytes --]

What is the proper way to include (reader) extensions with `pandoc.read`?
Specifically `-tex_math_dollars`, `-shortcut_reference_links` and
`+four_space_rule`.

I have read the docu but I don't get it, so an example (here and in the
docu) would be much appreciated!

TIA,

/bpj

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/CADAJKhCqSXXZBPvXRfr%2Bh4ady_yCzO6s%3DVDKY_Tt54dcLtDR5w%40mail.gmail.com.

[-- Attachment #2: Type: text/html, Size: 1196 bytes --]

Re: Lua filters: Extensions with pandoc.read?

[Albert Krewinkel]

BPJ <melroch-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> writes:

> What is the proper way to include (reader) extensions with
> `pandoc.read`? Specifically `-tex_math_dollars`,
> `-shortcut_reference_links` and `+four_space_rule`.
>
> I have read the docu but I don't get it, so an example (here and in
> the docu) would be much appreciated!

Extensions have to be passed as part of the format string, e.g.,

    pandoc.read(source, 'markdown-tex_math_dollars')

I'm not entirely happy with this, as it means that any extensions passed
via a ReaderOptions object will be ignored.

    -- this may seem like it should disable all extensions,
    -- but the ReaderOptions `extensions` field is ignored.
    -- The default 'markdown' extensions will be used instead.
    pandoc.read(src, 'markdown', {extensions = {}})

Ideas to improve this would be most welcome.


-- 
Albert Krewinkel
GPG: 8eed e3e2 e8c5 6f18 81fe  e836 388d c0b2 1f63 1124

Re: Lua filters: Extensions with pandoc.read?

[BPJ]

[-- Attachment #1: Type: text/plain, Size: 2294 bytes --]

Den mån 19 sep. 2022 14:51Albert Krewinkel <albert+pandoc-9EawChwDxG8hFhg+JK9F0w@public.gmane.org>
skrev:

>
> BPJ <melroch-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> writes:
>
> > What is the proper way to include (reader) extensions with
> > `pandoc.read`? Specifically `-tex_math_dollars`,
> > `-shortcut_reference_links` and `+four_space_rule`.
> >
> > I have read the docu but I don't get it, so an example (here and in
> > the docu) would be much appreciated!
>
> Extensions have to be passed as part of the format string, e.g.,
>
>     pandoc.read(source, 'markdown-tex_math_dollars')
>
> I'm not entirely happy with this, as it means that any extensions passed
> via a ReaderOptions object will be ignored.
>
>     -- this may seem like it should disable all extensions,
>     -- but the ReaderOptions `extensions` field is ignored.
>     -- The default 'markdown' extensions will be used instead.
>     pandoc.read(src, 'markdown', {extensions = {}})
>
> Ideas to improve this would be most welcome.
>

I would just change it to not ignore reader options, and throw an error if
the format string and the reader options contradict each other (except for
empty table = no extensions and no extensions in string = default
extensions). And better hurry before someone starts to rely on reader
options being ignored! :-)


>
> --
> Albert Krewinkel
> GPG: 8eed e3e2 e8c5 6f18 81fe  e836 388d c0b2 1f63 1124
>
> --
> You received this message because you are subscribed to the Google Groups
> "pandoc-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/pandoc-discuss/87czbrfrzr.fsf%40zeitkraut.de
> .
>

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/CADAJKhApJjnCMghJ61rg6H2U9_zOP3Q%3DRJxHc9E2dFUH0Py%2BSg%40mail.gmail.com.

[-- Attachment #2: Type: text/html, Size: 3599 bytes --]

Re: Lua filters: Extensions with pandoc.read?

[Albert Krewinkel]

BPJ <bpj-J3H7GcXPSITLoDKTGw+V6w@public.gmane.org> writes:

> Den mån 19 sep. 2022 14:51 Albert Krewinkel <
>> albert+pandoc-9EawChwDxG8hFhg+JK9F0w@public.gmane.org> skrev:
>>
>> Extensions have to be passed as part of the format string, e.g.,
>>
>>     pandoc.read(source, 'markdown-tex_math_dollars')
>>
>> I'm not entirely happy with this, as it means that any extensions
>> passed  via a ReaderOptions object will be ignored.
>>
>>     -- this may seem like it should disable all extensions,
>>     -- but the ReaderOptions `extensions` field is ignored.
>>     -- The default 'markdown' extensions will be used instead.
>>     pandoc.read(src, 'markdown', {extensions = {}})
>>
>> Ideas to improve this would be most welcome.
>
> I would just change it to not ignore reader options, and throw an
> error if the format string and the reader options contradict each
> other (except for empty table = no extensions and no extensions in
> string = default extensions). And better hurry before someone starts
> to rely on reader options being ignored! :-)

I believe that this has already become the expected behavior, so I'd be
very hesitant to break it. How about this approach: The format specifier
can currently only be a string. We change this to accept either a
string, or a table with the extensions being given in separate fields.
E.g.

    -- extensions relative to default for format
    { format = 'markdown',
    , disable = {'markdown_in_html_blocks'},
    , enable = {'emoji'},
    }

or

    -- absolute set of extensions
    { format = 'docx',
    , extensions = {'auto_identifiers', 'styles', 'citations'}
    }

If the function is called with either a string or a table that has
explicit info on extensions, then the ReaderOptions extensions are
overridden. If the table has no info about extensions, then we use the
ReaderOptions.

It might still be a little confusing, as

    pandoc.read(src, 'markdown', opts)  -- default Markdown extensions

will behave differently than

    pandoc.read(src, {format: 'markdown'}, opts) -- extensions in opts

Extensions are closer to formats than to reader options, conceptually.
It seems sensible to tie them to formats. But maybe there is a better
way?

-- 
Albert Krewinkel
GPG: 8eed e3e2 e8c5 6f18 81fe  e836 388d c0b2 1f63 1124

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/87v8oz3ohm.fsf%40zeitkraut.de.

Re: Lua filters: Extensions with pandoc.read?

[BPJ]

[-- Attachment #1: Type: text/plain, Size: 4070 bytes --]

Den tis 4 okt. 2022 11:52Albert Krewinkel <albert+pandoc-9EawChwDxG8hFhg+JK9F0w@public.gmane.org>
skrev:

>
> BPJ <bpj-J3H7GcXPSITLoDKTGw+V6w@public.gmane.org> writes:
>
> > Den mån 19 sep. 2022 14:51 Albert Krewinkel <
> >> albert+pandoc-9EawChwDxG8hFhg+JK9F0w@public.gmane.org> skrev:
> >>
> >> Extensions have to be passed as part of the format string, e.g.,
> >>
> >>     pandoc.read(source, 'markdown-tex_math_dollars')
> >>
> >> I'm not entirely happy with this, as it means that any extensions
> >> passed  via a ReaderOptions object will be ignored.
> >>
> >>     -- this may seem like it should disable all extensions,
> >>     -- but the ReaderOptions `extensions` field is ignored.
> >>     -- The default 'markdown' extensions will be used instead.
> >>     pandoc.read(src, 'markdown', {extensions = {}})
> >>
> >> Ideas to improve this would be most welcome.
> >
> > I would just change it to not ignore reader options, and throw an
> > error if the format string and the reader options contradict each
> > other (except for empty table = no extensions and no extensions in
> > string = default extensions). And better hurry before someone starts
> > to rely on reader options being ignored! :-)
>
> I believe that this has already become the expected behavior, so I'd be
> very hesitant to break it. How about this approach: The format specifier
> can currently only be a string. We change this to accept either a
> string, or a table with the extensions being given in separate fields.
> E.g.
>
>     -- extensions relative to default for format
>     { format = 'markdown',
>     , disable = {'markdown_in_html_blocks'},
>     , enable = {'emoji'},
>     }
>

I like this!


> or
>
>     -- absolute set of extensions
>     { format = 'docx',
>     , extensions = {'auto_identifiers', 'styles', 'citations'}
>     }
>

This `extensions` field would be identical to the `enable` field, wouldn't
it, or would it mean that all default extensions not explicitly mentioned
in it would be disabled? And what would happen if someone uses it together
with `disable` and/or `enable`; which would be ignored, or would it be an
error condition?


> If the function is called with either a string or a table that has
> explicit info on extensions, then the ReaderOptions extensions are
> overridden. If the table has no info about extensions, then we use the
> ReaderOptions.
>

If the format can be a table as described I think it is better to always
ignore the format/extension info in the ReaderOptions — and say so clearly
in the documentation!


> It might still be a little confusing, as
>
>     pandoc.read(src, 'markdown', opts)  -- default Markdown extensions
>
> will behave differently than
>
>     pandoc.read(src, {format: 'markdown'}, opts) -- extensions in opts
>
> Extensions are closer to formats than to reader options, conceptually.
> It seems sensible to tie them to formats.


Agreed.

But maybe there is a better
> way?
>

I think this would be fine.

Also I assume that `pandoc.write()` would behave analogously?


> --
> Albert Krewinkel
> GPG: 8eed e3e2 e8c5 6f18 81fe  e836 388d c0b2 1f63 1124
>
> --
> You received this message because you are subscribed to the Google Groups
> "pandoc-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/pandoc-discuss/87v8oz3ohm.fsf%40zeitkraut.de
> .
>

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/CADAJKhCg3rMk%2B0fJBOcte32BVh2c9tYQHa57DEGHdyAz2u7%2BkQ%40mail.gmail.com.

[-- Attachment #2: Type: text/html, Size: 6740 bytes --]

Re: Lua filters: Extensions with pandoc.read?

[Albert Krewinkel]

BPJ <melroch-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> writes:

> Den tis 4 okt. 2022 11:52Albert Krewinkel <albert+pandoc-9EawChwDxG8hFhg+JK9F0w@public.gmane.org> skrev:
>>    -- absolute set of extensions
>>    { format = 'docx',
>>    , extensions = {'auto_identifiers', 'styles', 'citations'}
>>    }
>
> This `extensions` field would be identical to the `enable` field,
> wouldn't it, or would it mean that all default extensions not
> explicitly mentioned in it would be disabled? And what would happen if
> someone uses it together with `disable` and/or `enable`; which would
> be ignored, or would it be an error condition?

Mmmh, good point. My initial idea was to use the format's default
extension when no `extensions` field is given. But that's confusing,
because now omitting the `extensions` field can have multiple meanings
("fall back to reader options" or "use default extensions"), depending
on which other fields are set. Not a good interface.

Maybe extensions should be Lua objects with some convenient methods:

``` lua
{ format: 'markdown',
  extensions: pandoc.extensions.default('markdown'):disable{'smart'}
}
```

Bit more typing, but much cleaner.

> If the format can be a table as described I think it is better to
> always ignore the format/extension info in the ReaderOptions — and
> say so clearly in the documentation!

Yes, I think you are right.

> Also I assume that `pandoc.write()` would behave analogously?

Definitely, `read` and `write` should stay symmetric.

-- 
Albert Krewinkel
GPG: 8eed e3e2 e8c5 6f18 81fe  e836 388d c0b2 1f63 1124

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/87r0zn3dff.fsf%40zeitkraut.de.

Re: Lua filters: Extensions with pandoc.read?

[John MacFarlane]

> On Oct 4, 2022, at 6:31 AM, Albert Krewinkel <albert+pandoc-9EawChwDxG8hFhg+JK9F0w@public.gmane.org> wrote:
> 
> Mmmh, good point. My initial idea was to use the format's default
> extension when no `extensions` field is given. But that's confusing,
> because now omitting the `extensions` field can have multiple meanings
> ("fall back to reader options" or "use default extensions"), depending
> on which other fields are set. Not a good interface.


Not sure about this, but what about?

- if no `extensions` field is present, then default extensions for that format are used, as possibly modified by `+` or `-` in the format specifier
- if `extensions` is present, then those extensions will be used, instead of the defaults; if you want no extensions, just use `ejxtensions = {}`
- if `extensions` is present *and* modifiers to format, then start with `extensions` and apply the modifiers

Re: Lua filters: Extensions with pandoc.read?

[BPJ]

[-- Attachment #1: Type: text/plain, Size: 2021 bytes --]

Den tis 4 okt. 2022 19:26John MacFarlane <fiddlosopher-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> skrev:

>
>
> > On Oct 4, 2022, at 6:31 AM, Albert Krewinkel <albert+pandoc-9EawChwDxG8hFhg+JK9F0w@public.gmane.org>
> wrote:
> >
> > Mmmh, good point. My initial idea was to use the format's default
> > extension when no `extensions` field is given. But that's confusing,
> > because now omitting the `extensions` field can have multiple meanings
> > ("fall back to reader options" or "use default extensions"), depending
> > on which other fields are set. Not a good interface.
>
>
> Not sure about this, but what about?
>
> - if no `extensions` field is present, then default extensions for that
> format are used, as possibly modified by `+` or `-` in the format specifier
>

Wouldn't it be better if `+`/`-` in the `format` field be an error
condition if the format argument is a table? Least confusing IMO.


- if `extensions` is present, then those extensions will be used, instead
> of the defaults; if you want no extensions, just use `ejxtensions = {}`
> - if `extensions` is present *and* modifiers to format, then start with
> `extensions` and apply the modifiers
>
> --
> You received this message because you are subscribed to the Google Groups
> "pandoc-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/pandoc-discuss/5AC38771-6A87-47C2-80C7-5C7B389EDD1B%40gmail.com
> .
>

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/CADAJKhCan6ec3ruV5r08ynTFPVLUtOpc%3Dh%2BJt7-geYNUcJb6QA%40mail.gmail.com.

[-- Attachment #2: Type: text/html, Size: 3374 bytes --]

Re: Lua filters: Extensions with pandoc.read?

[BPJ]

[-- Attachment #1: Type: text/plain, Size: 2312 bytes --]

If this change is done would it be possible to allow a similar format in
defaults files? It does get a bit unwieldy when you have a long list of
extensions.

Den tors 6 okt. 2022 13:11BPJ <melroch-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> skrev:

>
>
> Den tis 4 okt. 2022 19:26John MacFarlane <fiddlosopher-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> skrev:
>
>>
>>
>> > On Oct 4, 2022, at 6:31 AM, Albert Krewinkel <
>> albert+pandoc-9EawChwDxG8hFhg+JK9F0w@public.gmane.org> wrote:
>> >
>> > Mmmh, good point. My initial idea was to use the format's default
>> > extension when no `extensions` field is given. But that's confusing,
>> > because now omitting the `extensions` field can have multiple meanings
>> > ("fall back to reader options" or "use default extensions"), depending
>> > on which other fields are set. Not a good interface.
>>
>>
>> Not sure about this, but what about?
>>
>> - if no `extensions` field is present, then default extensions for that
>> format are used, as possibly modified by `+` or `-` in the format specifier
>>
>
> Wouldn't it be better if `+`/`-` in the `format` field be an error
> condition if the format argument is a table? Least confusing IMO.
>
>
> - if `extensions` is present, then those extensions will be used, instead
>> of the defaults; if you want no extensions, just use `ejxtensions = {}`
>> - if `extensions` is present *and* modifiers to format, then start with
>> `extensions` and apply the modifiers
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "pandoc-discuss" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/pandoc-discuss/5AC38771-6A87-47C2-80C7-5C7B389EDD1B%40gmail.com
>> .
>>
>

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/CADAJKhAj02V6aSzbD1N4kRzWbuyHncyRcLktkHNu7k2FW3qB6A%40mail.gmail.com.

[-- Attachment #2: Type: text/html, Size: 4007 bytes --]