"'William Lupton' via pandoc-discuss" <pandoc-discuss@googlegroups.com> writes:

> pandoc.utils.type() return values seem to be a little inconsistent.
> For example, it can return:
>
>     'List' ... for pandoc.List (note: logging.lua leaves this alone)
>     'pandoc Xxx' ... for pandoc.Row etc. (note: logging.lua replaces
>     the space with a dot)
>     'HsLua.JSON.array' ... presumably for arrays that have come via a
>     JSON library

Thank you for digging and asking! It's great to have more eyeballs in
this part of the code.

A bit of background: all types have a corresponding entry in the Lua
"registry", a global table that contains all sorts of info. The Lua
manual states "you should use as key a string containing your library
name", hence the "pandoc" and "HsLua" prefixes. Additional
inconsistencies are cause by the fact that I didn't follow the Lua
manual's advice in the beginning, as name clashes with other packages
where not really a concern of mine.

> I was toying with not indicating HsLua.JSON.array (i.e., showing it
> like any other table) but perhaps it's better just to report things
> as they are. Can any other HsLua.JSON.xxx values be returned, e.g.,
> why not HsLua.JSON.object or indeed HsLua.JSON.
> {boolean,number,string,null}?

There's a technical reason for this: we must be able to round-trip
from JSON to Lua to JSON. Both JSON arrays and objects are represented
as Lua tables, but how can we know whether an empty Lua table `{}`
represents an empty array or an empty object? We solve this by assigning
a metatable/type to array values.

I guess we could also do this for JSON objects, but the advantage is
less clear in that case: being able to add metamethods like `insert`
makes sense for lists, but not for objects: String fields can shadow any
method, which would most likely lead to subtle bugs. E.g.,
`obj:merge(obj2)` would stop working if `obj` has a key `merge`.
Adding a metatable to objects would therefore be little more than
performance degrading overhead.

The other JSON types, boolean, numbers, and strings, map directly to the
respective Lua types of the same name, and pandoc.utils.type will report
the Lua type name in those cases.

Anyway, please do open issues for any inconsistencies you find. We may
want to keep some of those old names to keep error messages more
readable, but consistency would be nice to have, too.

> On Fri, 20 Jan 2023 at 16:33, William Lupton <
> wlupton-QSt+ys/nuMyEUIsrzH9SikB+6BGkLq7r@public.gmane.org> wrote:
>
>     Update: Until today, logging.lua assumed that pairs() would work
>     on all userdata. Option 2 (my favourite) should work well if
>     there are ever any other types of userdata (not sure whether
>     there ever will be).
>   
>     On Fri, 20 Jan 2023 at 16:29, William Lupton <
>     wlupton-QSt+ys/nuMyEUIsrzH9SikB+6BGkLq7r@public.gmane.org> wrote:
>   
>         Thanks Albert. I expect I'll raise an issue. I assume
>         that this is the only use of light userdata?
>       
>         Looking more closely (why didn't I do this before?) I see
>         that tostring() returns 'userdata: 0x0' for these values, so
>         I think it makes sense to report them as one of:
>             Whatever tostring() returns
>             As above with the leading 'userdata: ' removed           
>                    <--- probably my favourite
>             As above with '0x0' reported as 'nil' (not sure about
>             this)
>       
>         On Fri, 20 Jan 2023 at 16:05, ChrisD <
>         cd34-gg-4SSc53hpTiu9TMao6EloiEEOCMrvLtNR@public.gmane.org> wrote:
>       
>             On 1/20/2023 2:54 AM, 'William Lupton' via pandoc-discuss
>             wrote:
>             > I investigated, and parts of the writer options are
>             "light userdata" (I hadn't heard of that). I've committed
>             and merged a fix that will report such items as "
>             <pointer>". You can now list most of the writer options
>             (just a few colors show as <pointer>).
>           
>             Your update works well. (I didn't know about light
>             userdata either.) Thanks!

"'William Lupton' via pandoc-discuss" <pandoc-discuss-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org> writes:

> Re this:
>
>> Also, I tried to print the value of PANDOC_WRITER_OPTIONS, but
> wlupton's logging.lua errors on it (bad argument #1 to 'for iterator'
> (table expected, got light userdata))
>
> I investigated, and parts of the writer options are "light userdata"
> (I hadn't heard of that). I've committed and merged a fix that will
> report such items as "<pointer>". You can now list most of the writer
> options (just a few colors show as <pointer>).
>
> I'm not sure whether the use of light userdata is intentional here
> (this isn't a 3.0 thing; it was already the case in previous
> versions). Albert?

Let's call it semi-intentional: what happens is that we go via JSON to
create the Lua representation of the writer options. The problem is
that, if a JSON value is `null`, we can't use `nil` in Lua because that
would make the key "disappear", i.e., it would not show up when
iterating through the resulting table with `pairs`. So, in an effort to
gain compatibility with the very popular Lua library "cjson", we
represent `null` values as C null pointers. Hence the "light userdata".

The one solution I can think of is to set a custon `__pairs` metamethod
on PANDOC_WRITER_OPTIONS. That would allow us to set the missing values
to `nil`, while still having `pairs(PANDOC_WRITER_OPTIONS)` list all
supported keys. Please feel free to raise an issue for this on GitHub.