Convert \index{true@`true`} to \index{true@\texttt{true}} ?

Convert \index{true@`true`} to \index{true@\texttt{true}} ?

[Axel Rauschmayer]

[-- Attachment #1.1: Type: text/plain, Size: 977 bytes --]

What is the best way to convert \index{true@`true`} to 
\index{true@\texttt{true}} via a filter?

   - I can parse what comes after @ into an AST via read(), but I don’t 
   know how to do either of the following two things:
      - wrap raw LaTeX around the AST (not possible, AFAICT)
      - convert the AST to a string with raw LaTeX (to be wrapped in 
      \index{true@···})
   

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/2d229904-88ac-452d-90da-d27d54528779%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[-- Attachment #1.2: Type: text/html, Size: 1546 bytes --]

Re: Convert \index{true@`true`} to \index{true@\texttt{true}} ?

[BP Jonsson]

> Ämne: Convert \index{true@`true`} to \index{true@\texttt{true}} ?
> Från: Axel Rauschmayer <rauschma@...>
> Datum: 2019-03-19 17:31
> Till: pandoc-discuss <pandoc-discuss-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org>

> What is the best way to convert \index{true@`true`} to 
> \index{true@\texttt{true}} via a filter?

For starters the entire ``\index{true@`true`}`` will show up as a RawInline or RawBlock element, even the `` `true` `` since it is inside a TeX argument, so you have to look for Raw... elements.

>    - I can parse what comes after @ into an AST via read(), but I don’t 
>    know how to do either of the following two things:
>       - wrap raw LaTeX around the AST (not possible, AFAICT)

Yes it is, if you put RawInline elements before and/or after the content list of the Para in the AST and return that list:

````lua
local function fix_index (elem)
    if elem.format:match('tex$') then -- gross but sufficient
        local before, text = elem.text:match('^(\\index%{%S-%@)(.-)%}$')
        -- no match: do nothing
        if nil == before then return nil end
        -- match
        local doc = pandoc.read(text)
        if doc then -- read success
            local blocks = doc.blocks
            -- check that we got the expected kind of data and proceed if so
            if #blocks == 1 and 'Para' == blocks[1].t then
                local content = blocks[1].content
                -- turn the `\\index{SORTKEY@` part into a RawInline
                before = pandoc.RawInline('latex', before)
                -- create a RawInline with the closing brace
                local after = pandoc.RawInline('latex', '}')
                -- prepend the before and append the after to the content
                -- insert(table, index, inserted)
                table.insert(content, 1, before)
                -- two args: index defaults to #content+1
                table.insert(content, after)
                -- if we got a block wrap the content in a block
                if 'RawBlock' == elem.t then
                    content = pandoc.Para(content)
                end
                return content
            end
        end
    end
    return nil -- default case, do nothing
end

return { { RawBlock = fix_index, RawInline = fix_index } }
````

A more effective strategy where you do not need to rely on Lua patterns or `read()` is to use a Span or Code with an attribute in your source:

````markdown
[actual text]{idx="SORTKEY"}

`true`{idx=true}
````

````Lua
local function fix_index (elem)
    -- get the value of the `idx` attribute if any
    local sortkey = elem.attributes.idx
    -- no such attribute: do nothing
    if nil == sortkey then return nil end
    elem.attributes.idx = nil -- remove the attribute just in case
    -- create a RawInline with the `\\index{SORTKEY@` part
    local before = pandoc.RawInline('latex', '\\index{' .. sortkey .. '@')
    -- create a RawInline with the closing brace
    local after = pandoc.RawInline('latex', '}')
    -- just return the two pieces of raw LaTeX with the element between them
    return {before, elem, after}
end


return { { Code = fix_index, Span = fix_index } }
````


>       - convert the AST to a string with raw LaTeX (to be wrapped in 
>       \index{true@···})

You don't need to, just return a list with raw inlines with LaTeX before and after the content.

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/2e883e9d-f7c8-f7b7-952d-41945ef038aa%40gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Convert \index{true@`true`} to \index{true@\texttt{true}} ?

[Axel Rauschmayer]

[-- Attachment #1.1: Type: text/plain, Size: 1143 bytes --]

Good stuff, thank you!

IME, the LaTeX approach of \index{} being completely invisible works better 
than annotating visible text.

In principle, you could do the same via [actual text]{idx="SORTKEY"}, but 
(to me) it still implies that what’s in brackets is visible. As far as I 
know, Pandoc does not have something that is closer to a pure (potentially 
invisible) command (which is OK, because you can always resort to 
LaTeX-style commands).

Other than that, this approach is nice – I especially like that attributes 
have descriptive keys.

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/2ac8ec4b-51f8-4121-89a5-06ebc6090ffa%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[-- Attachment #1.2: Type: text/html, Size: 1611 bytes --]

Re: Convert \index{true@`true`} to \index{true@\texttt{true}} ?

[BPJ]

[-- Attachment #1: Type: text/plain, Size: 3300 bytes --]

You can always use a(nother) filter to remove the spans if you produce
another format. If you return an empty table from a filter function the
original element will be removed from the AST.

The problem with using a Lua pattern to parse a string is that it requires
rather crude heuristics compared to more full-fledged regexes which support
alternations and quantified groups: the Raw… element must contain exactly
one index entry and nothing else, the first `@` char in the string must be
the separator.

I have a (perl) "filter" which collects such spans and produces a YAML file
keyed on sort keys with a file path and a serialized perl span object as
values and then another script which takes one or more such files and
produces a rudimentary index for a website -- just a sorted Pandoc
definition list with links to occurrences in files. Usually the anchor text
and the index text are the same so visibility is no problem. Otherwise I
have a span within the span with an `.idx` class which is used as the index
text and removed from the AST.

I even have a dormant project for a Unicode-aware drop-in makeindex
replacement using the Perl Sort::ArbBiLex (which allows easy on-the-fly
sort order definitions) or Unicode::Collate modules.


Den ons 20 mars 2019 17:37Axel Rauschmayer <rauschma-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> skrev:

> Good stuff, thank you!
>
> IME, the LaTeX approach of \index{} being completely invisible works
> better than annotating visible text.
>
> In principle, you could do the same via [actual text]{idx="SORTKEY"}, but
> (to me) it still implies that what’s in brackets is visible. As far as I
> know, Pandoc does not have something that is closer to a pure (potentially
> invisible) command (which is OK, because you can always resort to
> LaTeX-style commands).
>
> Other than that, this approach is nice – I especially like that attributes
> have descriptive keys.
>
> --
> You received this message because you are subscribed to the Google Groups
> "pandoc-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
> To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/pandoc-discuss/2ac8ec4b-51f8-4121-89a5-06ebc6090ffa%40googlegroups.com
> <https://groups.google.com/d/msgid/pandoc-discuss/2ac8ec4b-51f8-4121-89a5-06ebc6090ffa%40googlegroups.com?utm_medium=email&utm_source=footer>
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/CADAJKhAG5w8dCPSNbSTrNf4ZgUFk_EG1ri5R2UNHzCxDfZMvXA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[-- Attachment #2: Type: text/html, Size: 4519 bytes --]

Re: Convert \index{true@`true`} to \index{true@\texttt{true}} ?

[Axel Rauschmayer]

[-- Attachment #1.1: Type: text/plain, Size: 690 bytes --]

True. I’ll stick with \index{} for now. Thankfully, it’s easy to switch 
later, should I change my mind.

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/a81728b5-0e06-454e-a5eb-8a3515fd2f33%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[-- Attachment #1.2: Type: text/html, Size: 1092 bytes --]

Re: Convert \index{true@`true`} to \index{true@\texttt{true}} ?

[John MacFarlane]

> The problem with using a Lua pattern to parse a string is that it requires
> rather crude heuristics compared to more full-fledged regexes which support
> alternations and quantified groups: the Raw… element must contain exactly
> one index entry and nothing else, the first `@` char in the string must be
> the separator.

I agree about the superiority of the span approach.

But note, if you have more complex parsing needs in a
lua filter, you could use lpeg.

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/yh480kh8bw6t3b.fsf%40johnmacfarlane.net.
For more options, visit https://groups.google.com/d/optout.

Re: Convert \index{true@`true`} to \index{true@\texttt{true}} ?

[Axel Rauschmayer]

[-- Attachment #1.1: Type: text/plain, Size: 871 bytes --]

Long term, it may make sense for Pandoc to provide a utility function for 
Lua filters that parses LaTeX commands (including attribute lists).

Rationale: Some advanced things can be passed directly to LaTeX, but need 
to be parsed and processed for, e.g., HTML. Having help there would be 
useful.

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/4c44b992-efd3-4a3c-8f25-eb21212af0a1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[-- Attachment #1.2: Type: text/html, Size: 1304 bytes --]

Re: Convert \index{true@`true`} to \index{true@\texttt{true}} ?

[BPJ]

[-- Attachment #1: Type: text/plain, Size: 2388 bytes --]

I have been working on such a thing, as a helper module for filter authors
or as a filter itself. It has a function which will turn any unparsed
command into a Span with a
`custom-style="<user-defined-prefix><command-name-minus-backslash>"`
attribute and each argument into a nested span with a similar attribute
with an `-arg-N` suffix, where N is 0 for an optional argument and 1
through 9 for a mandatory argument, using `pandoc.read()` to convert each
argument into an AST, naturally also walking the AST calling the conversion
function on any nested unparsed commands. If this sounds interesting I'll
try to get it into a presentable form ASAP.

Den tors 4 apr. 2019 15:32Axel Rauschmayer <rauschma-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> skrev:

> Long term, it may make sense for Pandoc to provide a utility function for
> Lua filters that parses LaTeX commands (including attribute lists).
>
> Rationale: Some advanced things can be passed directly to LaTeX, but need
> to be parsed and processed for, e.g., HTML. Having help there would be
> useful.
>
> --
> You received this message because you are subscribed to the Google Groups
> "pandoc-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
> To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/pandoc-discuss/4c44b992-efd3-4a3c-8f25-eb21212af0a1%40googlegroups.com
> <https://groups.google.com/d/msgid/pandoc-discuss/4c44b992-efd3-4a3c-8f25-eb21212af0a1%40googlegroups.com?utm_medium=email&utm_source=footer>
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/CADAJKhAkq_SYCvz94%2BRq8HSA30kUywKE%2Bkoc4f6Rq2Xk785L_A%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[-- Attachment #2: Type: text/html, Size: 3471 bytes --]

Re: Convert \index{true@`true`} to \index{true@\texttt{true}} ?

[Axel Rauschmayer]

[-- Attachment #1.1: Type: text/plain, Size: 1024 bytes --]


>
> If this sounds interesting I'll try to get it into a presentable form ASAP.


I don’t have an immediate need for it right now, but it definitely sounds 
interesting in general!

Wouldn’t it make more sense as follows, though?

INPUT:

\dpcgraphics[height=4in, width=3in]{foo}{bar}

RESULT:

{
  command = "dpcgraphics",
  keywordargs = {
    height = "4in",
    width = "3in"
  },
  posargs = { "foo", "bar" }
}

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/03de54da-e7f9-459d-a579-b4fed3964ee4%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[-- Attachment #1.2: Type: text/html, Size: 2264 bytes --]

Re: Convert \index{true@`true`} to \index{true@\texttt{true}} ?

[BPJ]

[-- Attachment #1: Type: text/plain, Size: 3540 bytes --]

My thinking is that when converting it into spans you can get it directly
into the output if you want/need to but you can also fairly easily dive
down into the structure in the filter if you need to. There should probably
be a `cmd=COMMANDNAME` or `env=ENVIRONMENTNAME` on the Span/Div to make
diving easier, and there should probably be an `arg=NUM` attribute on each
argument span. On environments it probably would make sense to turn the
arguments into attributes as they seldom will be textual. As for key=value
options you cannot just assume that an optional argument contains them, nor
that a mandatory argument does not, nor that an argument matching
`^%s*(%a+)%s*%=%s([^,]*)*` contains them, much less that an argument
matching `^%s*(%a+)%s*,` contains them. It would be highly nontrivial to
parse LaTeX style keyvals with Lua patterns even if you assume that a key
is always one or more alphabetics and a value is always a single brace
group or zero or more non-commas. You would still have to write your own
inchworming parser and loop through several patterns until you find a match
at each turn, since Lua patterns lack both alternations and quantified
groups, as well as named captures. I have written such a parser, even a
module which lets you implement such a parser as an object reusing the
boilerplate, but I don't fancy sticking all that code into a filter. The
point of Lua filters is that you don't need any external libraries.  If you
do need them you can as well use a JSON filter in some more powerful
language with more powerful regular expressions in the first place.

Den mån 8 apr. 2019 13:40Axel Rauschmayer <rauschma-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> skrev:

> If this sounds interesting I'll try to get it into a presentable form ASAP.
>
>
> I don’t have an immediate need for it right now, but it definitely sounds
> interesting in general!
>
> Wouldn’t it make more sense as follows, though?
>
> INPUT:
>
> \dpcgraphics[height=4in, width=3in]{foo}{bar}
>
> RESULT:
>
> {
>   command = "dpcgraphics",
>   keywordargs = {
>     height = "4in",
>     width = "3in"
>   },
>   posargs = { "foo", "bar" }
> }
>
> --
> You received this message because you are subscribed to the Google Groups
> "pandoc-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
> To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/pandoc-discuss/03de54da-e7f9-459d-a579-b4fed3964ee4%40googlegroups.com
> <https://groups.google.com/d/msgid/pandoc-discuss/03de54da-e7f9-459d-a579-b4fed3964ee4%40googlegroups.com?utm_medium=email&utm_source=footer>
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/CADAJKhD1Y5tmXv3HyfESc-OYtv78nAAiYOiyZhOhm_7GCXN1Tw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[-- Attachment #2: Type: text/html, Size: 5338 bytes --]