From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.io/gmane.text.pandoc/32065 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Albert Krewinkel Newsgroups: gmane.text.pandoc Subject: Re: logging.lua and 'light userdata' Date: Mon, 23 Jan 2023 11:49:54 +0100 Message-ID: <87bkmpldx5.fsf@zeitkraut.de> References: <1D22B433-211B-4033-8A63-F637F52B2008@gmail.com> <87fsc4koud.fsf@zeitkraut.de> Reply-To: pandoc-discuss-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="18708"; mail-complaints-to="usenet@ciao.gmane.io" To: pandoc-discuss-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org Original-X-From: pandoc-discuss+bncBCZJF7XJTILRBT6OXGPAMGQEI2KNURI-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org Mon Jan 23 11:54:11 2023 Return-path: Envelope-to: gtp-pandoc-discuss@m.gmane-mx.org Original-Received: from mail-wm1-f56.google.com ([209.85.128.56]) by ciao.gmane.io with esmtps (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1pJuSp-0004gN-PO for gtp-pandoc-discuss@m.gmane-mx.org; Mon, 23 Jan 2023 11:54:11 +0100 Original-Received: by mail-wm1-f56.google.com with SMTP id p1-20020a05600c1d8100b003daff82f5edsf7300697wms.8 for ; Mon, 23 Jan 2023 02:54:11 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1674471251; cv=pass; d=google.com; s=arc-20160816; b=oREcMXSRkNxlX9PqNykiziduVCmvpo9r3VlhPAazpf3fjnw+tM6bbUEbPyP3kDDG1U bkxpAwshvsuNdAEfU/ejGV9J8qKiI1J1RLCTufGhHJRUfI9QAWQcCnH+B6rwVqJ7LeYD Pucgkq3gBEMekfeULE20/azEPBKtSEejuNAmoFe9I9Gjebcs9uORRCd/QBiEfFmfkwmD nbNTAOChIbgnntXF7tE1FJCL3iBbhAARKB4AXh+WThHvkfpfCUeGN1P+1DrRR5j0nxtJ kNGgJKlbeb86knpwxd9irnLBsUdalGnnIVXcQbeZ0EnoUEPIuH1LcTMCgPxIXhz7DG6k l1WA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:reply-to:content-transfer-encoding :mime-version:message-id:in-reply-to:date:subject:to:from:references :sender:dkim-signature; bh=33oIXt3IJs2PMEDFyqEe/zxxc9DyZ0uMtB6XHeSRk8Y=; b=bb8IDY6ohj50id5fr5rdN+vIUBLix456wpKyrOJcIIG68ypx2YrWAoWTRDayNwq8LA e8XJ2Lwf7Tpp1eeOvYWW5sVxt7l8KisgVH09U8mOHjLgqD3O6y4+NQRkduWvTv/irdkS +z0TCb6dy/d+as/VYiOVco79J4IjzUk2VBBEvUQ3++rhQoGoDKYurCXIsS3N8TGSxpUs DHElZyAhEEnmtgxkLasxB2e/mtsNWA6fUZ+QdwGVVjTj8i6MTOsq+J0jjKWf/uxXNL2i cIpkpDgQ6Eeo2w4ea5mZx4FbA0YoGA27t1qw5MDKedMujcQfmxH1adAhd5E42tC5nBh0 RvUQ== ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of albert+pandoc-9EawChwDxG8hFhg+JK9F0w@public.gmane.org designates 80.241.56.171 as permitted sender) smtp.mailfrom=albert+pandoc-9EawChwDxG8hFhg+JK9F0w@public.gmane.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20210112; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:reply-to :x-original-authentication-results:x-original-sender :content-transfer-encoding:mime-version:message-id:in-reply-to:date :subject:to:from:references:sender:from:to:cc:subject:date :message-id:reply-to; bh=33oIXt3IJs2PMEDFyqEe/zxxc9DyZ0uMtB6XHeSRk8Y=; b=i0FgiBRi7kGyTnfMSdkhNbNXTXXNu+Y/5OCmjBkmYRAXa5G512gDohQQdFNNnkEbvY Lu4Py0SYQFVfZxesN1I/ErgY5N2X0ZvmlA/HXZrM+bgWCvZqkuEcX838BQiWshZNKoKj 3Ot7QDhFWRrve1XNLWl0tMKfJ2fl6l2PovYJOe4DGKZ0tMyZc4zVjAZNjvD1CTKqlC+A GNmr85lM2BJXnOhnrQZzyYWKNpay5dKBu0NZAU+TfMTJHFrKqxleYfNzjf0GQJc7S/1y NzwNbK4f6AiAYzeZaAA/6tSGW5xGU68w7fSZ5yQc/gEniqlJr43F4K1aqide+mgdF X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence:reply-to :x-original-authentication-results:x-original-sender :content-transfer-encoding:mime-version:message-id:in-reply-to:date :subject:to:from:references:x-gm-message-state:sender:from:to:cc :subject:date:message-id:reply-to; bh=33oIXt3IJs2PMEDFyqEe/zxxc9DyZ0uMtB6XHeSRk8Y=; b=ZkSp2YAjdItQBjbvsEM2P6zc1LuzVpM2T5onJlawKFFj+lxI6814BB+VdINDZv3e+U imQXK6k2T19SQmM79Gsq119RPnOBjxMSmnp/GQ4T56A6QKwOERb+wLaivonsUAzpNwBJ +Csb8ncWcejEhTyqaQQF+gbV5T8kDJF6/vGB4c71FBoAdcT9HneAHz4nFXAZ2y/MY4+t BLpjEwHEKYGK/ZFVSZi7JquXjM3w0TTJHulgHtBJJVkCskHvkbxftD/LfiBmfx61jkJh WRq1O8uwcO6D/r1Qbiq6 Original-Sender: pandoc-discuss-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org X-Gm-Message-State: AFqh2kqwCwf58U3g+8ZL2YBIl2OZp0CvHflBDnZi0rKmLhu2RpR9K6eU AyjUrbV2k3op+qBsQK+jJZg= X-Google-Smtp-Source: AMrXdXvZNznOlIDAbLEsWCUcPMy3vBmM4a8X5MSVRRf+P026cPq6Uhyv4xHfo8w89/U7VQ8emtFRCA== X-Received: by 2002:a05:600c:3d9a:b0:3db:1d86:1ec6 with SMTP id bi26-20020a05600c3d9a00b003db1d861ec6mr1252550wmb.45.1674471251439; Mon, 23 Jan 2023 02:54:11 -0800 (PST) X-BeenThere: pandoc-discuss-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org Original-Received: by 2002:a5d:47a9:0:b0:298:bd4a:4dd9 with SMTP id 9-20020a5d47a9000000b00298bd4a4dd9ls10295511wrb.1.-pod-prod-gmail; Mon, 23 Jan 2023 02:54:06 -0800 (PST) X-Received: by 2002:adf:eec6:0:b0:2be:57b9:f569 with SMTP id a6-20020adfeec6000000b002be57b9f569mr9866775wrp.23.1674471246721; Mon, 23 Jan 2023 02:54:06 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1674471246; cv=none; d=google.com; s=arc-20160816; b=SgVgmT3ffLrdX6cN2eyuFLOylTjZR0dTv/iwP51sje/v1fS9O2k/FILT3om6rgpU42 mxj6iayxscsjTf7HlM95n2cZ6ygqe2nFr5TWXrh6ygb9qXAMYbKMLApof0M2mzZi/iUz kffzehbQHvA6Oi/2kToqKdrmo2QNIGYSkcKhvu92whRLgl5cvc52x9X1bnNuC5sdziDa l2N/PguHxnyq6SRfFrNFqJs4moOecJwsgKUkCdEPEAyTm97LHOII4eQv9BIShDfWzphl Bc2FrWxPlao+8mkSCgQTJeB8ia2MzMZ72uoRI2ucYGdYa956uvu6/B538ISL0xS59uwz 9XLQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:message-id:in-reply-to:date :subject:to:from:references; bh=S8HHv4OZKfl/lfA3zBqQy8yX7dYyAGPVXl6fuGi1a5E=; b=vzcpUTebABhgLFBlSaD8Sm70nybMaesLapoi5WkTGqAWT/X89Uclk4ETbxi4SMX7bB 1VikGI+AnZ4YMucT+0BirQN2Was36rEFEjOAuih8YDagpLo4/MbB4HP8vdmHZh/+QMRm tN3kgNeXPK3sKWJUKmBEET/zH5R6QzNCi4zty/eTstSHggJ+ohXPW0L1To7mrGvCuO/X YZmpaWDVtnlo2Lb8wWgqh70i6Szy58FSMZBkvPlENCAx8wUtjVR8I+R3jfDONj/tOFvt WAI0EPnD8ScXf3VDtX5Bn0VhYKbsnnO31S8xVLCP4vk1GM9k56R47PwDML40P1XBlmNw lMIg== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of albert+pandoc-9EawChwDxG8hFhg+JK9F0w@public.gmane.org designates 80.241.56.171 as permitted sender) smtp.mailfrom=albert+pandoc-9EawChwDxG8hFhg+JK9F0w@public.gmane.org Original-Received: from mout-p-201.mailbox.org (mout-p-201.mailbox.org. [80.241.56.171]) by gmr-mx.google.com with ESMTPS id by11-20020a056000098b00b00241d0141fbcsi650027wrb.8.2023.01.23.02.54.06 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Jan 2023 02:54:06 -0800 (PST) Received-SPF: pass (google.com: domain of albert+pandoc-9EawChwDxG8hFhg+JK9F0w@public.gmane.org designates 80.241.56.171 as permitted sender) client-ip=80.241.56.171; Original-Received: from smtp1.mailbox.org (smtp1.mailbox.org [10.196.197.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-384) server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mout-p-201.mailbox.org (Postfix) with ESMTPS id 4P0n6020y8z9sZ1 for ; Mon, 23 Jan 2023 11:54:00 +0100 (CET) In-reply-to: X-Original-Sender: albert+pandoc-9EawChwDxG8hFhg+JK9F0w@public.gmane.org X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of albert+pandoc-9EawChwDxG8hFhg+JK9F0w@public.gmane.org designates 80.241.56.171 as permitted sender) smtp.mailfrom=albert+pandoc-9EawChwDxG8hFhg+JK9F0w@public.gmane.org Precedence: list Mailing-list: list pandoc-discuss-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org; contact pandoc-discuss+owners-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org List-ID: X-Google-Group-Id: 1007024079513 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , Xref: news.gmane.io gmane.text.pandoc:32065 Archived-At: I've opened https://github.com/jgm/pandoc/issues/8574 for this. We can't really change the return values of `pandoc.utils.types` for Inlines and Blocks, as that would break a lot of filters. But we can still strive to make the output more consistent. "'William Lupton' via pandoc-discuss" wri= tes: > (Creating a new thread with cleaned-up=C2=A0history. Sorry for hijacking > your thread ChrisD!) > > I've merged a small update to logging.lua (just removing the leading > 'userdata: ' when reporting light userdata values). > > Thanks for the explanation Albert. I don't suppose that you really > want to change the behaviour of pandoc.utils.type(), so I'm not > really inclined to create an issue suggesting that you should do=C2=A0so. > However (if you like) I could create an issue suggesting that it be > slightly more fully documented (e.g., giving some examples of what it > would return for various pandoc.xxx types). > > As for your other earlier suggestion (creating an issue to suggest a > custom `__pairs` metamethod on PANDOC_WRITER_OPTIONS), would that fix > the issue? The 0x0 values aren't associated with top-level keys of > PANDOC_WRITER_OPTIONS, they're (for example) within highlight_style > (see below). I can't help feeling that we should either do nothing or > else do something that would work for arbitrary structures (not just > for PANDOC_WRITER_OPTIONS). Given that I've worked around the problem > in logging.lua I'd be quite happy to do nothing. > > A specific point about the items that have 0x0 values: these are all > items that have no value, so is it the end of the world if they are > simply omitted? Mightn't that be more > whatever-the-lua-equivalent-of-pythonic-is? Yes pairs() won't visit > them, but an attempt to access them will return nil as expected. > > WriterOptions { > =C2=A0 cite_method: "citeproc" > =C2=A0 ... > =C2=A0 highlight_style: { > =C2=A0 =C2=A0 background-color: 0x0 > =C2=A0 =C2=A0 line-number-background-color: 0x0 > =C2=A0 =C2=A0 line-number-color: "#aaaaaa" > =C2=A0 =C2=A0 text-color: 0x0 > =C2=A0 =C2=A0 ... > > On Sat, 21 Jan 2023 at 13:18, Albert Krewinkel < > albert+pandoc-9EawChwDxG8hFhg+JK9F0w@public.gmane.org> wrote: > > =20 > "'William Lupton' via pandoc-discuss" < > pandoc-discuss-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org> writes: > =20 > > pandoc.utils.type() return values seem to be a little > inconsistent. > > For example, it can return: > > > >=C2=A0 =C2=A0 =C2=A0'List' ... for pandoc.List (note: logging.lua le= aves this > alone) > >=C2=A0 =C2=A0 =C2=A0'pandoc Xxx' ... for pandoc.Row etc. (note: logg= ing.lua > replaces > >=C2=A0 =C2=A0 =C2=A0the space with a dot) > >=C2=A0 =C2=A0 =C2=A0'HsLua.JSON.array' ... presumably for arrays tha= t have come > via a > >=C2=A0 =C2=A0 =C2=A0JSON library > =20 > Thank you for digging and asking! It's great to have more > eyeballs in > this part of the code. > =20 > A bit of background: all types have a corresponding entry in the > Lua > "registry", a global table that contains all sorts of info. The > Lua > manual states "you should use as key a string containing your > library > name", hence the "pandoc" and "HsLua" prefixes. Additional > inconsistencies are cause by the fact that I didn't follow the > Lua > manual's advice in the beginning, as name clashes with other > packages > where not really a concern of mine. > =20 > > I was toying with not indicating HsLua.JSON.array (i.e., > showing it > > like any other table) but perhaps it's better just to report > things > > as they are. Can any other HsLua.JSON.xxx values be returned, > e.g., > > why not HsLua.JSON.object or indeed HsLua.JSON. > > {boolean,number,string,null}? > =20 > There's a technical reason for this: we must be able to > round-trip > from JSON to Lua to JSON. Both JSON arrays and objects are > represented > as Lua tables, but how can we know whether an empty Lua table `{} > ` > represents an empty array or an empty object? We solve this by > assigning > a metatable/type to array values. > =20 > I guess we could also do this for JSON objects, but the advantage > is > less clear in that case: being able to add metamethods like > `insert` > makes sense for lists, but not for objects: String fields can > shadow any > method, which would most likely lead to subtle bugs. E.g., > `obj:merge(obj2)` would stop working if `obj` has a key `merge`. > Adding a metatable to objects would therefore be little more than > performance degrading overhead. > =20 > The other JSON types, boolean, numbers, and strings, map directly > to the > respective Lua types of the same name, and pandoc.utils.type will > report > the Lua type name in those cases. > =20 > Anyway, please do open issues for any inconsistencies you find. > We may > want to keep some of those old names to keep error messages more > readable, but consistency would be nice to have, too. > =20 > > On Fri, 20 Jan 2023 at 16:33, William Lupton < > > wlupton-QSt+ys/nuMyEUIsrzH9SikB+6BGkLq7r@public.gmane.org> wrote: > > > >=C2=A0 =C2=A0 =C2=A0Update: Until today, logging.lua assumed that pa= irs() would > work > >=C2=A0 =C2=A0 =C2=A0on all userdata. Option 2 (my favourite) should = work well > if > >=C2=A0 =C2=A0 =C2=A0there are ever any other types of userdata (not = sure > whether > >=C2=A0 =C2=A0 =C2=A0there ever will be). > >=C2=A0 =C2=A0 > >=C2=A0 =C2=A0 =C2=A0On Fri, 20 Jan 2023 at 16:29, William Lupton < > >=C2=A0 =C2=A0 =C2=A0wlupton-QSt+ys/nuMyEUIsrzH9SikB+6BGkLq7r@public.gmane.org> wrote: > >=C2=A0 =C2=A0 > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Thanks Albert. I expect I'll raise= an issue. I assume > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0that=C2=A0this is the only use of = light userdata? > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Looking more closely (why didn't I= do this before?) I > see > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0that tostring() returns 'userdata:= 0x0' for these > values, so > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0I think it makes sense to report t= hem as one of: > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Whatever tostring() = returns > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0As above=C2=A0with= =C2=A0the leading 'userdata: ' removed=C2=A0 =C2=A0 =C2=A0 > =C2=A0 =C2=A0 =C2=A0 > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0=C2=A0 =C2=A0 =C2=A0= =C2=A0<--- probably my favourite > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0As above with '0x0' = reported as 'nil' (not sure > about > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0this) > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0On Fri, 20 Jan 2023 at 16:05, Chri= sD < > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0cd34-gg-4SSc53hpTiu9TMao6EloiEEOCMrvLtNR@public.gmane.org> wrote= : > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0On 1/20/2023 2:54 AM= , 'William Lupton' via > pandoc-discuss > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0wrote: > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0> I investigated, an= d parts of the writer options > are > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0"light userdata" (I = hadn't heard of that). I've > committed > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0and merged a fix tha= t will report such items as " > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0". You can = now list most of the writer > options > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0(just a few colors s= how as ). > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Your update works we= ll. (I didn't know about light > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0userdata either.) Th= anks! > > > On Fri, 20 Jan 2023 at 15:59, Albert Krewinkel < > albert+pandoc-9EawChwDxG8hFhg+JK9F0w@public.gmane.org> wrote: > > =20 > "'William Lupton' via pandoc-discuss" < > pandoc-discuss-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org> writes: > =20 > > Re this: > > > >> Also, I tried to print the value of PANDOC_WRITER_OPTIONS, but > > wlupton's logging.lua errors on it (bad argument #1 to 'for > iterator' > > (table expected, got light userdata)) > > > > I investigated, and parts of the writer options are "light > userdata" > > (I hadn't heard of that). I've committed and merged a fix that > will > > report such items as "". You can now list most of the > writer > > options (just a few colors show as ). > > > > I'm not sure whether the use of light userdata is intentional > here > > (this isn't a 3.0 thing; it was already the case in previous > > versions). Albert? > =20 > Let's call it semi-intentional: what happens is that we go via > JSON to > create the Lua representation of the writer options. The problem > is > that, if a JSON value is `null`, we can't use `nil` in Lua > because that > would make the key "disappear", i.e., it would not show up when > iterating through the resulting table with `pairs`. So, in an > effort to > gain compatibility with the very popular Lua library "cjson", we > represent `null` values as C null pointers. Hence the "light > userdata". > =20 > The one solution I can think of is to set a custon `__pairs` > metamethod > on PANDOC_WRITER_OPTIONS. That would allow us to set the missing > values > to `nil`, while still having `pairs(PANDOC_WRITER_OPTIONS)` list > all > supported keys. Please feel free to raise an issue for this on > GitHub. > > =C2=A0 --=20 Albert Krewinkel GPG: 8eed e3e2 e8c5 6f18 81fe e836 388d c0b2 1f63 1124 --=20 You received this message because you are subscribed to the Google Groups "= pandoc-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an e= mail to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To view this discussion on the web visit https://groups.google.com/d/msgid/= pandoc-discuss/87bkmpldx5.fsf%40zeitkraut.de.