logging.lua and 'light userdata'

["'William Lupton' via pandoc-discuss" <pandoc-discuss-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org>]

[-- Attachment #1: Type: text/plain, Size: 8100 bytes --]

(Creating a new thread with cleaned-up history. Sorry for hijacking your
thread ChrisD!)

I've merged a small update to logging.lua (just removing the leading
'userdata: ' when reporting light userdata values).

Thanks for the explanation Albert. I don't suppose that you really want to
change the behaviour of pandoc.utils.type(), so I'm not really inclined to
create an issue suggesting that you should do so. However (if you like) I
could create an issue suggesting that it be slightly more fully documented
(e.g., giving some examples of what it would return for various pandoc.xxx
types).

As for your other earlier suggestion (creating an issue to suggest a custom
`__pairs` metamethod on PANDOC_WRITER_OPTIONS), would that fix the issue?
The 0x0 values aren't associated with top-level keys of
PANDOC_WRITER_OPTIONS, they're (for example) within highlight_style (see
below). I can't help feeling that we should either do nothing or else do
something that would work for arbitrary structures (not just for
PANDOC_WRITER_OPTIONS). Given that I've worked around the problem in
logging.lua I'd be quite happy to do nothing.

A specific point about the items that have 0x0 values: these are all items
that have no value, so is it the end of the world if they are simply
omitted? Mightn't that be more whatever-the-lua-equivalent-of-pythonic-is?
Yes pairs() won't visit them, but an attempt to access them will return nil
as expected.

WriterOptions {
  cite_method: "citeproc"
  ...
  highlight_style: {
    background-color: 0x0
    line-number-background-color: 0x0
    line-number-color: "#aaaaaa"
    text-color: 0x0
    ...

On Sat, 21 Jan 2023 at 13:18, Albert Krewinkel <albert+pandoc-9EawChwDxG8hFhg+JK9F0w@public.gmane.org>
wrote:

>
> "'William Lupton' via pandoc-discuss" <pandoc-discuss-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org>
> writes:
>
> > pandoc.utils.type() return values seem to be a little inconsistent.
> > For example, it can return:
> >
> >     'List' ... for pandoc.List (note: logging.lua leaves this alone)
> >     'pandoc Xxx' ... for pandoc.Row etc. (note: logging.lua replaces
> >     the space with a dot)
> >     'HsLua.JSON.array' ... presumably for arrays that have come via a
> >     JSON library
>
> Thank you for digging and asking! It's great to have more eyeballs in
> this part of the code.
>
> A bit of background: all types have a corresponding entry in the Lua
> "registry", a global table that contains all sorts of info. The Lua
> manual states "you should use as key a string containing your library
> name", hence the "pandoc" and "HsLua" prefixes. Additional
> inconsistencies are cause by the fact that I didn't follow the Lua
> manual's advice in the beginning, as name clashes with other packages
> where not really a concern of mine.
>
> > I was toying with not indicating HsLua.JSON.array (i.e., showing it
> > like any other table) but perhaps it's better just to report things
> > as they are. Can any other HsLua.JSON.xxx values be returned, e.g.,
> > why not HsLua.JSON.object or indeed HsLua.JSON.
> > {boolean,number,string,null}?
>
> There's a technical reason for this: we must be able to round-trip
> from JSON to Lua to JSON. Both JSON arrays and objects are represented
> as Lua tables, but how can we know whether an empty Lua table `{}`
> represents an empty array or an empty object? We solve this by assigning
> a metatable/type to array values.
>
> I guess we could also do this for JSON objects, but the advantage is
> less clear in that case: being able to add metamethods like `insert`
> makes sense for lists, but not for objects: String fields can shadow any
> method, which would most likely lead to subtle bugs. E.g.,
> `obj:merge(obj2)` would stop working if `obj` has a key `merge`.
> Adding a metatable to objects would therefore be little more than
> performance degrading overhead.
>
> The other JSON types, boolean, numbers, and strings, map directly to the
> respective Lua types of the same name, and pandoc.utils.type will report
> the Lua type name in those cases.
>
> Anyway, please do open issues for any inconsistencies you find. We may
> want to keep some of those old names to keep error messages more
> readable, but consistency would be nice to have, too.
>
> > On Fri, 20 Jan 2023 at 16:33, William Lupton <
> > wlupton-QSt+ys/nuMyEUIsrzH9SikB+6BGkLq7r@public.gmane.org> wrote:
> >
> >     Update: Until today, logging.lua assumed that pairs() would work
> >     on all userdata. Option 2 (my favourite) should work well if
> >     there are ever any other types of userdata (not sure whether
> >     there ever will be).
> >
> >     On Fri, 20 Jan 2023 at 16:29, William Lupton <
> >     wlupton-QSt+ys/nuMyEUIsrzH9SikB+6BGkLq7r@public.gmane.org> wrote:
> >
> >         Thanks Albert. I expect I'll raise an issue. I assume
> >         that this is the only use of light userdata?
> >
> >         Looking more closely (why didn't I do this before?) I see
> >         that tostring() returns 'userdata: 0x0' for these values, so
> >         I think it makes sense to report them as one of:
> >             Whatever tostring() returns
> >             As above with the leading 'userdata: ' removed
> >                    <--- probably my favourite
> >             As above with '0x0' reported as 'nil' (not sure about
> >             this)
> >
> >         On Fri, 20 Jan 2023 at 16:05, ChrisD <
> >         cd34-gg-4SSc53hpTiu9TMao6EloiEEOCMrvLtNR@public.gmane.org> wrote:
> >
> >             On 1/20/2023 2:54 AM, 'William Lupton' via pandoc-discuss
> >             wrote:
> >             > I investigated, and parts of the writer options are
> >             "light userdata" (I hadn't heard of that). I've committed
> >             and merged a fix that will report such items as "
> >             <pointer>". You can now list most of the writer options
> >             (just a few colors show as <pointer>).
> >
> >             Your update works well. (I didn't know about light
> >             userdata either.) Thanks!
>

On Fri, 20 Jan 2023 at 15:59, Albert Krewinkel <albert+pandoc-9EawChwDxG8hFhg+JK9F0w@public.gmane.org>
wrote:

>
> "'William Lupton' via pandoc-discuss" <pandoc-discuss-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org>
> writes:
>
> > Re this:
> >
> >> Also, I tried to print the value of PANDOC_WRITER_OPTIONS, but
> > wlupton's logging.lua errors on it (bad argument #1 to 'for iterator'
> > (table expected, got light userdata))
> >
> > I investigated, and parts of the writer options are "light userdata"
> > (I hadn't heard of that). I've committed and merged a fix that will
> > report such items as "<pointer>". You can now list most of the writer
> > options (just a few colors show as <pointer>).
> >
> > I'm not sure whether the use of light userdata is intentional here
> > (this isn't a 3.0 thing; it was already the case in previous
> > versions). Albert?
>
> Let's call it semi-intentional: what happens is that we go via JSON to
> create the Lua representation of the writer options. The problem is
> that, if a JSON value is `null`, we can't use `nil` in Lua because that
> would make the key "disappear", i.e., it would not show up when
> iterating through the resulting table with `pairs`. So, in an effort to
> gain compatibility with the very popular Lua library "cjson", we
> represent `null` values as C null pointers. Hence the "light userdata".
>
> The one solution I can think of is to set a custon `__pairs` metamethod
> on PANDOC_WRITER_OPTIONS. That would allow us to set the missing values
> to `nil`, while still having `pairs(PANDOC_WRITER_OPTIONS)` list all
> supported keys. Please feel free to raise an issue for this on GitHub.

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/CAEe_xxjsmxwx4B_f1j4buVT8P%2BcdVdVw4q8dbtxiuJNoXsNweQ%40mail.gmail.com.

[-- Attachment #2: Type: text/html, Size: 10840 bytes --]