From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.io/gmane.text.pandoc/32572 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Stephan Meijer Newsgroups: gmane.text.pandoc Subject: Re: Open Discussion: Features of Templating Date: Tue, 9 May 2023 12:16:53 -0700 (PDT) Message-ID: <0297ad79-799e-4ed4-aa5b-943c124f7f55n@googlegroups.com> References: Reply-To: pandoc-discuss-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_Part_2242_309595536.1683659813908" Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="36110"; mail-complaints-to="usenet@ciao.gmane.io" To: pandoc-discuss Original-X-From: pandoc-discuss+bncBCYOPL5A34MBBJ5Y5KRAMGQE5Q4NONI-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org Tue May 09 21:16:58 2023 Return-path: Envelope-to: gtp-pandoc-discuss@m.gmane-mx.org Original-Received: from mail-oo1-f55.google.com ([209.85.161.55]) by ciao.gmane.io with esmtps (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1pwSpW-0009EW-NG for gtp-pandoc-discuss@m.gmane-mx.org; Tue, 09 May 2023 21:16:58 +0200 Original-Received: by mail-oo1-f55.google.com with SMTP id 006d021491bc7-54f89215b64sf691577eaf.3 for ; Tue, 09 May 2023 12:16:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20221208; t=1683659817; x=1686251817; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:reply-to:x-original-sender :mime-version:subject:references:in-reply-to:message-id:to:from:date :sender:from:to:cc:subject:date:message-id:reply-to; bh=S6h45bRYJ2Sr3gIvhqGN2E33V6aMUJapmbO0NQEMZT4=; b=P7j88TjJV0ZJVAKF7oF47pzwQIG2IIaN2q1TjLbJjjZLXDoSTQ5EoVkpvMwLVs+Rgc rw07mjMHpD2oEvuXvizpNbFClx+vgH6VdpZVFvBJnilDdzjjurWGHE1BW5S4fEj5Bw5j G0xdBYQnuafJZaIdZ5A3br/oxFlWWSv5EtLzm9q9xk/6g456LFC6/GOkvbbnTjfLunuK MF0wsn+LJp6UwxEYbL43wqC4D/Nu/ZO9mjf5RNlNhBxmZ4Lz4Sqk0Qy2843tZp2NqI9d Ta2Mao8+mMGKOkv5w+GJNYSMaDtsBQw6L9/3rN690aGiYzxY4yeHqcC8FPknD576i/fk kmcQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=stephanmeijer-com.20221208.gappssmtp.com; s=20221208; t=1683659817; x=1686251817; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:reply-to:x-original-sender :mime-version:subject:references:in-reply-to:message-id:to:from:date :from:to:cc:subject:date:message-id:reply-to; bh=S6h45bRYJ2Sr3gIvhqGN2E33V6aMUJapmbO0NQEMZT4=; b=buyvb/2qtV5qob3eR/MAX8oidVh44SGNFbK7SSrSBXxyyHQHRpHoxk6z7WxCJBf3sG Vd6KpiZk8cr+6eKw94espma0KP6enpsbmteV9B4BuV6hsQjobhOoXRHnf7gi3YB5Zyyq XrOeBu0mQiooso7J4sdV+Z9kcojqD3eAyYJrACgESo66FCbllHj+2GazBanZdjzsDIiu 367AT/pn+wlt37ns1CpnO65KBYWuQk1uE67vaBhYhJecZaJ9LMwQfsPX5EMBPeej9ivf 6fif9qCFj+PHTW6PbVf0mzTg+Q/NlP2T6r2P/23IuWzswdbPTSUzzLOrYAetXDzhoplQ Um9g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1683659817; x=1686251817; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence:reply-to :x-original-sender:mime-version:subject:references:in-reply-to :message-id:to:from:date:x-beenthere:x-gm-message-state:sender:from :to:cc:subject:date:message-id:reply-to; bh=S6h45bRYJ2Sr3gIvhqGN2E33V6aMUJapmbO0NQEMZT4=; b=YL65NgHUIT+9ut+yPHduEGWi0iL7gLPdz5O76t4oIPCAu88dnKHcFx7lB7VujlO+bm c1Na69wprmA6AFfWP+UBbYiP4rK5vIn9PpCdaZzB/gCTF0M/SgBEr4czOEKHT2I4yHS7 9SQqEopCr/IIAY9zJJvPDN3lRuZjZ/hL3TKsl4lL7/jj5R4qJRd8rLWb9mdmt6mE/GnY 3JTT2n+VKtKPSnsSOBI8rDZ9aLzivyUf3kUoQ2vvNZFBoPaCeJBdGuWzMoUH/egztc2h gKq/bYzRzsqs+dSebiS/tFZ9ODL8ApTNLa//JzVUxo6xvpjPN1 Original-Sender: pandoc-discuss-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org X-Gm-Message-State: AC+VfDyWAzow0qUPSPiBOH2S1gklnAi9bRR+b71i4BqVlGXyU0bdxS8s qPt3SptyhQHrjfpVSZaREuM= X-Google-Smtp-Source: ACHHUZ74CTG6DBQTO8XgLt9JH30kvtTWGPn0090nDFvJXfGgTvRbMIK84vh7nhkgT7bUg3x/RiC7Jg== X-Received: by 2002:a4a:e709:0:b0:547:150a:fd1a with SMTP id y9-20020a4ae709000000b00547150afd1amr1113137oou.0.1683659817483; Tue, 09 May 2023 12:16:57 -0700 (PDT) X-BeenThere: pandoc-discuss-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org Original-Received: by 2002:a4a:c90a:0:b0:54f:b75a:2175 with SMTP id v10-20020a4ac90a000000b0054fb75a2175ls380657ooq.0.-pod-prod-07-us; Tue, 09 May 2023 12:16:54 -0700 (PDT) X-Received: by 2002:aca:dfc1:0:b0:38e:2135:6981 with SMTP id w184-20020acadfc1000000b0038e21356981mr1030415oig.3.1683659814505; Tue, 09 May 2023 12:16:54 -0700 (PDT) In-Reply-To: X-Original-Sender: me-nPKYAObcRdo6Blr+0TYHagC/G2K4zDHf@public.gmane.org Precedence: list Mailing-list: list pandoc-discuss-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org; contact pandoc-discuss+owners-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org List-ID: X-Google-Group-Id: 1007024079513 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , Xref: news.gmane.io gmane.text.pandoc:32572 Archived-At: ------=_Part_2242_309595536.1683659813908 Content-Type: multipart/alternative; boundary="----=_Part_2243_2110363049.1683659813908" ------=_Part_2243_2110363049.1683659813908 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Maybe at this stage it'd even be easier to be able to decouple the=20 templating engine from Pandoc, so people can use their own templating=20 engine. On Tuesday, 9 May 2023 at 19:35:18 UTC+2 BPJ wrote: > There are several things which have to be decided for something as simple= =20 > as equality comparison. I have no idea how hard or easy it would be to=20 > implement it in any form for Pandoc templates, but will point to some=20 > decisions which might need to be made. > > Pandoc variables/metadata can be of several types: strings, booleans,=20 > complex data structures =E2=80=94 lists and key=E2=80=94value mappings wh= ich in turn can=20 > contain any type. I=E2=80=99m not sure if Pandoc treats numbers as distin= ct from=20 > strings; metadata seems not to. This raises several questions: What happe= ns=20 > when the compared objects are of different types: are they silently unequ= al=20 > or is there a warning or an error? How do complex data types compare to= =20 > each other: content or object equality, same keys or also same values, or= =20 > same number of keys for mappings? Same items or same number of items for= =20 > lists? You probably end up needing a length operator! > > If there are going to be size comparison (less/greater (or equal)) how is= =20 > size determined. I touched on that above for complex data types, but ther= e=20 > are problems for strings too: do we compare byte length or (UTF-8)=20 > character length? (I hope the latter!) I don=E2=80=99t know to what exten= t metadata=20 > contains AST-like objects at the template processing stage, but if they d= o=20 > how do they compare: similar to complex data data structures and/or text= =20 > content equality? If both there may be need for an operator which=20 > stringifies its operand. Shall it be possible to regex match strings? > > It may be possible to mimic the way Pandoc=E2=80=99s Lua API does these t= hings,=20 > perhaps even possible to reuse code but pretty soon you will start to hav= e=20 > features similar to a general programming language. Where do you draw the= =20 > line? You could even allow embedding for example Lua, but that may clash= =20 > with more core templating features, there may be security concerns, and h= ow=20 > do you pass what to Lua-land? > > I have thought a lot about this kind of thing, having put quite some work= =20 > into creating a small template-like (pre)processor =E2=80=94 I call it an= =20 > interpolation engine because it can=E2=80=99t really handle formatting of= =20 > multi-line blocks. Two things which I needed to decide were whether to=20 > support complex data structures and whether it should be possible to assi= gn=20 > values to variables. For now the answer is yes to the first =E2=80=94 wit= h the=20 > limitation that you can only compare string and/or number xor boolean lea= f=20 > values =E2=80=94 and no to the second. In core only comparison, pattern m= atching=20 > and true/false =E2=80=94 Lua style where only `false` and `nil` evaluate = as false =E2=80=94=20 > or empty/nonempty =E2=80=94 empty string and 0 also evaluate as false =E2= =80=94 boolean=20 > evaluation with something like a ternary operator syntax is in core but= =20 > there is extensibility by letting the program using the engine supply=20 > functions to do arbitrary things, with a syntax for calling and passing= =20 > arguments to such functions. There is also a mechanism for inserting=20 > characters by codepoint or entity name. The thing is that even with=20 > comparison etc. built in from scratch and with the limitations on what ca= n=20 > be compared a lot of decisions and different operators had to be made. I= =20 > ended up with several things which made the language not-so-small anymore= : > > - Distinct logic operators for if-true/if-false and=20 > if-non-empty/if-empty (they are modular `??` vs. `?!` and `&?` vs. `&!` b= ut=20 > anyway! > > - Distinct operators for numeric (e.g. `=3D=3D#`) and case-folded (e.g.= =20 > `=3D=3D?`) comparison in addition to straight string comparison (e.g. `= =3D=3D`).=20 > Added complication: there is (of course) no numeric pattern matching, but= =20 > the parser has to take special note of attempts to use the would-be=20 > operator! > > - Distinct =E2=80=9Ctext=E2=80=9D =E2=80=94 parenthesis delimited, with= interpolation =E2=80=94 and=20 > =E2=80=9Cstring=E2=80=9D =E2=80=94 quotes delimited without interpolation= . > > - Distinct string-variable/function/character-by-id/math interpolation= =20 > syntaxes (and math is basically an additional DSL which isn=E2=80=99t ful= ly=20 > implemented yet!) > > - Distinct string and math variable spaces. > > - Support methods for doing expansion-like things with function=20 > arguments. > > - There are all of for/while/until style loops (although limited to the= =20 > kinds of comparisons described above). > > The outcome: I have switched to a simpler syntax which basically is=20 > sprintf on not quite as much steroids: > > (1) Arguments are specified as variable names inside the string. > > (2) Custom conversions implemented as functions passed when instantiating= =20 > the formatter object. > > (3) Custom conversions can have almost arbitrary names, including=20 > multi-character names. > > (4) Multiple arguments per format specifier are possible: > > (a) by default this means =E2=80=9Cuse the value of the first existin= g=20 > variable corresponding to a name in the argument list=E2=80=9D, but > (b) custom conversions can use the arguments (all strings) however=20 > they please, including using them as embedded format strings. > > Nothing more, but (2), (3) and (4)(b) make a wide variety of extensibilit= y=20 > possible =E2=80=94 including everything the more complex syntax described= above=20 > does =E2=80=94, so inevitably I wrote a =E2=80=9Cmicro=E2=80=9D implement= ation which leaves any=20 > non-standard argument parsing to custom conversion functions and in core= =20 > supports only single variable name arguments and standard sprintf=20 > conversions. > > /bpj > > > Den tis 9 maj 2023 16:08Stephan Meijer skrev: > >> I would like to create an open discussion around this. >> ------------------------------ >> >> Hi all, >> >> As I understand, the templating engine Pandoc uses is built-in. This is= =20 >> all good, but I feel like there are some missing features. >> >> One feature I personally miss: >> If-statement conditions >> >> What I miss most, is being able to create conditions in if-statments. Fo= r=20 >> example: >> >> %if(pagetitle =3D=3D "null")% >> ... >> %endif% >> >> Currently this seems not possible. >> >> Other users that have noticed this: >> >> -=20 >> https://stackoverflow.com/questions/45404607/if-conditionals-in-pando= c-templates-depending-on-value-of-a-variable >> >> ------------------------------ >> >> Feel free to add your own points. >> >> --=20 >> You received this message because you are subscribed to the Google Group= s=20 >> "pandoc-discuss" group. >> To unsubscribe from this group and stop receiving emails from it, send a= n=20 >> email to pandoc-discus...-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org >> To view this discussion on the web visit=20 >> https://groups.google.com/d/msgid/pandoc-discuss/dbe4c60c-1178-406f-b794= -ce8c9807c2c1n%40googlegroups.com=20 >> >> . >> > --=20 You received this message because you are subscribed to the Google Groups "= pandoc-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an e= mail to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To view this discussion on the web visit https://groups.google.com/d/msgid/= pandoc-discuss/0297ad79-799e-4ed4-aa5b-943c124f7f55n%40googlegroups.com. ------=_Part_2243_2110363049.1683659813908 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Maybe at this stage it'd even be easier to be able to decouple the templati= ng engine from Pandoc, so people can use their own templating engine.
=
On = Tuesday, 9 May 2023 at 19:35:18 UTC+2 BPJ wrote:
The= re are several things which have to be decided for something as simple as e= quality comparison. I have no idea how hard or easy it would be to implemen= t it in any form for Pandoc templates, but will point to some decisions whi= ch might need to be made.

Pandoc variables/metadata can be of several types: strings, booleans,= complex data structures =E2=80=94 lists and key=E2=80=94value mappings whi= ch in turn can contain any type. I=E2=80=99m not sure if Pandoc treats numb= ers as distinct from strings; metadata seems not to. This raises several qu= estions: What happens when the compared objects are of different types: are= they silently unequal or is there a warning or an error? How do complex da= ta types compare to each other: content or object equality, same keys or al= so same values, or same number of keys for mappings? Same items or same num= ber of items for lists? You probably end up needing a length operator!

If there are going to be siz= e comparison (less/greater (or equal)) how is size determined. I touched on= that above for complex data types, but there are problems for strings too:= do we compare byte length or (UTF-8) character length? (I hope the latter!= ) I don=E2=80=99t know to what extent metadata contains AST-like objects at= the template processing stage, but if they do how do they compare: similar= to complex data data structures and/or text content equality? If both ther= e may be need for an operator which stringifies its operand. Shall it be po= ssible to regex match strings?

It may be possible to mimic the way Pandoc=E2=80=99s Lua API does th= ese things, perhaps even possible to reuse code but pretty soon you will st= art to have features similar to a general programming language. Where do yo= u draw the line? You could even allow embedding for example Lua, but that m= ay clash with more core templating features, there may be security concerns= , and how do you pass what to Lua-land?

I have thought a lot about this kind of thing, having put q= uite some work into creating a small template-like (pre)processor =E2=80=94= I call it an interpolation engine because it can=E2=80=99t really handle f= ormatting of multi-line blocks. Two things which I needed to decide were wh= ether to support complex data structures and whether it should be possible = to assign values to variables. For now the answer is yes to the first =E2= =80=94 with the limitation that you can only compare string and/or number x= or boolean leaf values =E2=80=94 and no to the second. In core only compari= son, pattern matching and true/false =E2=80=94 Lua style where only `false`= and `nil` evaluate as false =E2=80=94 or empty/nonempty =E2=80=94 empty st= ring and 0 also evaluate as false =E2=80=94 boolean evaluation with somethi= ng like a ternary operator syntax is in core but there is extensibility by = letting the program using the engine supply functions to do arbitrary thing= s, with a syntax for calling and passing arguments to such functions. There= is also a mechanism for inserting characters by codepoint or entity name. = The thing is that even with comparison etc. built in from scratch and with = the limitations on what can be compared a lot of decisions and different op= erators had to be made. I ended up with several things which made the langu= age not-so-small anymore:

-=C2=A0 =C2=A0Distinct logic operators for if-true/if-false and if-non-em= pty/if-empty (they are modular `??` vs. `?!` and `&?` vs. `&!` but = anyway!

-=C2=A0 =C2=A0Di= stinct operators for numeric (e.g. `=3D=3D#`) and case-folded (e.g. `=3D=3D= ?`) comparison in addition to straight string comparison (e.g. `=3D=3D`). A= dded complication: there is (of course) no numeric pattern matching, but th= e parser has to take special note of attempts to use the would-be operator!=

-=C2=A0 =C2=A0Distinct = =E2=80=9Ctext=E2=80=9D =E2=80=94 parenthesis delimited, with interpolation = =E2=80=94 and =E2=80=9Cstring=E2=80=9D =E2=80=94 quotes delimited without i= nterpolation.

-=C2=A0 = =C2=A0Distinct string-variable/function/character-by-id/math interpolation = syntaxes (and math is basically an additional DSL which isn=E2=80=99t fully= implemented yet!)

-=C2= =A0 =C2=A0Distinct string and math variable spaces.
=
-=C2=A0 =C2=A0Support methods for doing expansi= on-like things with function arguments.

-=C2=A0 =C2=A0There are all of for/while/until style loops = (although limited to the kinds of comparisons described above).

The outcome: I have switched to a s= impler syntax which basically is sprintf on not quite as much steroids:

(1) Arguments are specified= as variable names inside the string.

(2) Custom conversions implemented as functions passed when i= nstantiating the formatter object.

(3) Custom conversions can have almost arbitrary names, includin= g multi-character names.

(4) Multiple arguments per format specifier are possible:

=C2=A0 =C2=A0 (a) by default this means = =E2=80=9Cuse the value of the first existing variable corresponding to a na= me in the argument list=E2=80=9D, but
=C2=A0 =C2=A0 = (b) custom conversions can use the arguments (all strings) however they ple= ase, including using them as embedded format strings.

Nothing more, but (2), (3) and (4)(b) make a = wide variety of extensibility possible =E2=80=94 including everything the m= ore complex syntax described above does =E2=80=94, so inevitably I wrote a = =E2=80=9Cmicro=E2=80=9D implementation which leaves any non-standard argume= nt parsing to custom conversion functions and in core supports only single = variable name arguments and standard sprintf conversions.

/bpj


Den tis 9 maj 2023 16:08Stephan Meijer <m...-nPKYAObcRdo6Blr+0TYHagC/G2K4zDHf@public.gmane.org> skr= ev:
I would like to create an open discussion arou= nd this.

Hi all,

As I un= derstand, the templating engine Pandoc uses is built-in. This is all good, = but I feel like there are some missing features.

One feature I personally miss:

If-statement conditions

= What I miss most, is being able to create conditions i= n if-statments. For example:

%if(pagetitle =3D=3D "null")%
...
%endif%=

Currently this s= eems not possible.

Other users that have noticed this:

Feel free to add your own points.

--
You received this message because you are subscribed to the Google Groups &= quot;pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an e= mail to pandoc-discus= ...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discus= s/dbe4c60c-1178-406f-b794-ce8c9807c2c1n%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups &= quot;pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an e= mail to pand= oc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org.
To view this discussion on the web visit https://groups.google.com/d= /msgid/pandoc-discuss/0297ad79-799e-4ed4-aa5b-943c124f7f55n%40googlegroups.= com.
------=_Part_2243_2110363049.1683659813908-- ------=_Part_2242_309595536.1683659813908--