Type of the `filter' argument to traversal functions

Type of the `filter' argument to traversal functions

[Anton Shepelev]

Hello, all

I  am  trying to write a Lua filter.  The documenta-
tion to walk_block() and walk_filter()  states  that
the  second  parameter  is  "a  Lua filter (table of
functions) to be applied within the block  element".
When, however, I pass a filter table consisting of a
single row:

   pandoc.walk_block( elem, {{ Span = OnSpan }} )

it is ignored. By experiment I found that the  func-
tions  expect not a table of functions, but a single
row of it:

   pandoc.walk_block( elem, { Span = OnSpan } )

Now it works. Is the documentation incorrect or do I
misinterpret  it?  For aught I know, {Span = OnSpan}
is not a "table of functions".

-- 
Please, do not forward replies to the list to my e-mail.

Re: Type of the `filter' argument to traversal functions

[John MacFarlane]

Anton Shepelev <anton.txt-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> writes:


> misinterpret  it?  For aught I know, {Span = OnSpan}
> is not a "table of functions".

Isn't it?

{ } makes a table.  OnSpan is a function.
So, a table of functions.

Re: Type of the `filter' argument to traversal functions

[Anton Shepelev]

John MacFarlane to Anton Shepelev:

> > The  documentation to walk_block() and walk_fil-
> > ter() states that the second parameter is "a Lua
> > filter (table of functions) to be applied within
> > the block element".
> > [...]
> > Is the documentation incorrect or do I misinter-
> > pret  it?  For  aught I know, {Span = OnSpan} is
> > not a "table of functions".
>
> { } makes a table.  OnSpan is a function.   So,  a
> table of functions.

Please,  bear with me. A table in Lua is the one-to-
rule-them-all complex type, that can represent  many
sturctures,  including  array, associative array, or
structure.  On the other hand,  the  generally  used
meaning of `table' in programming and mathematics is
a two-dimensional sturcture  with  columns  and  any
number of rows from zero up, which, too, can be rep-
resented as a table in Lua, although it would  be  a
compound one.

That  said,  the  simplest literal interpretation of
"table of functions" is a linear array:

              { Func1, Func2, Func3 }

Is it a table? In Lua terms, yes.  Is it a table  of
functions -- most  certainly,  for  the  elements of
this table are functions (if you believe  the  names
of their identifiers :-)

But  since the parameter is described as "a Lua fil-
ter" and "a table of functions", the only way to in-
terpret it is as describing the structure of the Lua
filter itself, which is a table of functions in both
Lua and general terminology:

                { { Id1 = Func1 }
                  { Id2 = Func2 }
                  { Id3 = Func3 } }

For  the  reader  was acquainted with the concept of
`Lua filter' at the beginning of the document.

Futhermore, "a table of functions" has `function' in
the  plural,  so the phrase cannot apply to a struc-
ture that contains exactly one  function -- no  more
and no fewer.

What the traversal funcions accept as the second ar-
gument is a row from the table of functions  that  a
Lua filter is.

Re: Type of the `filter' argument to traversal functions

[EBkysko]

[-- Attachment #1.1: Type: text/plain, Size: 1186 bytes --]

From the Lua filter page:

Lua filters are tables with element names as keys and values consisting of 
> functions acting on those elements.
>

`{Span = OnSpan}` is a table in which `Span` is the key, `OnSpan` is the 
value, a function acting on `Span` element.

You could put more key-values in the filter you send for a that walk_block:

`{ Span = OnSpan, BulletList = MyBulletList, Para = HerPara }`

which is a table of functions, each function being key-value "indexed" so 
to say by a key which is necessarily one Pandoc element. This would be an 
example of a filter containing more than one function.

Independently of pandoc, `{ Id1 = Func1, Id2 = Func2, Id3 = Func3 }`, to 
take your last example, is a table of functions, each accessible by their 
respective key.

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/ab7d1128-cb98-466c-9f8c-35cd6b63422bo%40googlegroups.com.

[-- Attachment #1.2: Type: text/html, Size: 1736 bytes --]

Re: Type of the `filter' argument to traversal functions

[Anton Shepelev]

EBkysko:

> From the Lua filter page:
>
> > Lua  filters  are  tables  with element names as
> > keys and values consisting of  functions  acting
> > on those elements.
>
>
> `{Span  =  OnSpan}`  is a table in which `Span` is
> the key, `OnSpan` is the value, a function  acting
> on `Span` element.

OK.

> You  could  put  more key-values in the filter you
> send for a that walk_block:
>
> `{ Span = OnSpan, BulletList = MyBulletList, Para = HerPara }`
>
> which is a table of functions, each function being
> key-value  "indexed"  so  to say by a key which is
> necessarily one Pandoc element. This would  be  an
> example of a filter containing more than one func-
> tion.

Thank you for the explanation, I was wrong.

> Independently of pandoc, `{ Id1  =  Func1,  Id2  =
> Func2,  Id3 = Func3 }`, to take your last example,
> is a table of functions, each accessible by  their
> respective key.

No, it is not my example. My example was an array of
tables:
                { { Id1 = Func1 }
                  { Id2 = Func2 }
                  { Id3 = Func3 } }

A Lua filter is such an array of tables, whereas the
`filter' argument to the `walk_' functions is a sin-
gle table as in your examples above. They  are  dif-
ferent  and  incompatible stuctures. The entire .lua
script must return the one whereas the `walk_'  fun-
cions accept the other.

For examle, when a filter end with:

   return { Header=OnHeader }

it does not work. But when it ends with

   return { { Header=OnHeader } }

it  works.  This  is  the difference between the two
structures.

Re: Type of the `filter' argument to traversal functions

[EBkysko]

[-- Attachment #1.1: Type: text/plain, Size: 1365 bytes --]

True, it was not literally your example, but what a "table of function" 
would be using your example, I worded incorrectly. (and by the way, your 
example would need commas to separate the inner tables :) )

A Lua filter is a table of functions as described in the official page, it 
is not an array (ie a numerical indexed table).

It is true though that a .lua script has to return an *array of filters*, 
as said in the guide:

Pandoc expects each Lua file to return a list of filters
>

So:
- a Lua filter is a table of functions.
- walk_block (and walk_inline) take a Lua filter (table of functions) as 
second argument)
- a script must return an array of filters (i.e. an array of tables of 
functions)
- a script can return many filters, such as:

return {
 { Meta = meta_begin },
 { Pandoc = pandoc_preprocess }.
 { Div = div_preprocess, Para = parasol, Span = doSpan },
 { Inlines = doInlines, Blocks = doBlocks},
 { BulletList = whatever },
}


-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/996602a6-8c95-406c-b569-f381f517af31o%40googlegroups.com.

[-- Attachment #1.2: Type: text/html, Size: 2111 bytes --]

Re: Type of the `filter' argument to traversal functions

[EBkysko]

[-- Attachment #1.1: Type: text/plain, Size: 707 bytes --]

correcting a typo in the preceding, replacing period with comma (can't edit 
on Google Groups) :

return {
 { Meta = meta_begin },
 { Pandoc = pandoc_preprocess },
 { Div = div_preprocess, Para = parasol, Span = doSpan },
 { Inlines = doInlines, Blocks = doBlocks},
 { BulletList = whatever },
}

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/d4492670-e41b-4806-8d99-de4ea11615d3o%40googlegroups.com.

[-- Attachment #1.2: Type: text/html, Size: 1139 bytes --]

Re: Type of the `filter' argument to traversal functions

[John MacFarlane]

Anton Shepelev <anton.txt-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> writes:

> John MacFarlane to Anton Shepelev:
>
>> > The  documentation to walk_block() and walk_fil-
>> > ter() states that the second parameter is "a Lua
>> > filter (table of functions) to be applied within
>> > the block element".
>> > [...]
>> > Is the documentation incorrect or do I misinter-
>> > pret  it?  For  aught I know, {Span = OnSpan} is
>> > not a "table of functions".
>>
>> { } makes a table.  OnSpan is a function.   So,  a
>> table of functions.
>
> Please,  bear with me. A table in Lua is the one-to-
> rule-them-all complex type, that can represent  many
> sturctures,  including  array, associative array, or
> structure.  On the other hand,  the  generally  used
> meaning of `table' in programming and mathematics is
> a two-dimensional sturcture  with  columns  and  any
> number of rows from zero up, which, too, can be rep-
> resented as a table in Lua, although it would  be  a
> compound one.

I would have thought that in the context of documenting a Lua
function 'table' would be naturally interpreted as 'Lua table'.

> But  since the parameter is described as "a Lua fil-
> ter" and "a table of functions", the only way to in-
> terpret it is as describing the structure of the Lua
> filter itself, which is a table of functions in both
> Lua and general terminology:
>
>                 { { Id1 = Func1 }
>                   { Id2 = Func2 }
>                   { Id3 = Func3 } }

According to the docs, "Lua filters are tables with element names
as keys and values consisting of functions acting on those
elements." So yes, the parameter to walk_block is a Lua filter as
defined in the documentation.

If you had

  { { Id1 = Funct1,
      Id2 = Funct2 } }

that would be an array of Lua filters. But now, I see where you
may be getting confused. Reading on in the docs: "Pandoc expects
each Lua file to return a list of filters. The filters in that
list are called sequentially, each on the result of the previous
filter." It may be confusing that one often uses the term
"lua filter" for the file itself. But the file can return
a list (technically an array, implemented as a Lua table) of
filters.  I suspect you are confusing an array of Lua filters --
which can be defined in a single Lua file -- with a Lua filter.

Re: Type of the `filter' argument to traversal functions

[BPJ]

[-- Attachment #1: Type: text/plain, Size: 3952 bytes --]

I suppose that it might be worth pointing out that the reason why it is
useful that a filter script can return an array of filters is explained
here:

https://pandoc.org/lua-filters.html#execution-order

A note on terminology: Since "table" on its own is rather ambiguous in Lua may
be a good idea to clarify what kind of data a it given table is supposed to
contain by using the terms "array table" and "map table" and possibly
"mixed table" for a table with both array and map data, which are sometimes
convenient in Lua code, e.g. to tag an array.

-- 
Better --help|less than helpless

Den ons 12 aug. 2020 02:27John MacFarlane <jgm-TVLZxgkOlNX2fBVCVOL8/A@public.gmane.org> skrev:

> Anton Shepelev <anton.txt-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> writes:
>
> > John MacFarlane to Anton Shepelev:
> >
> >> > The  documentation to walk_block() and walk_fil-
> >> > ter() states that the second parameter is "a Lua
> >> > filter (table of functions) to be applied within
> >> > the block element".
> >> > [...]
> >> > Is the documentation incorrect or do I misinter-
> >> > pret  it?  For  aught I know, {Span = OnSpan} is
> >> > not a "table of functions".
> >>
> >> { } makes a table.  OnSpan is a function.   So,  a
> >> table of functions.
> >
> > Please,  bear with me. A table in Lua is the one-to-
> > rule-them-all complex type, that can represent  many
> > sturctures,  including  array, associative array, or
> > structure.  On the other hand,  the  generally  used
> > meaning of `table' in programming and mathematics is
> > a two-dimensional sturcture  with  columns  and  any
> > number of rows from zero up, which, too, can be rep-
> > resented as a table in Lua, although it would  be  a
> > compound one.
>
> I would have thought that in the context of documenting a Lua
> function 'table' would be naturally interpreted as 'Lua table'.
>
> > But  since the parameter is described as "a Lua fil-
> > ter" and "a table of functions", the only way to in-
> > terpret it is as describing the structure of the Lua
> > filter itself, which is a table of functions in both
> > Lua and general terminology:
> >
> >                 { { Id1 = Func1 }
> >                   { Id2 = Func2 }
> >                   { Id3 = Func3 } }
>
> According to the docs, "Lua filters are tables with element names
> as keys and values consisting of functions acting on those
> elements." So yes, the parameter to walk_block is a Lua filter as
> defined in the documentation.
>
> If you had
>
>   { { Id1 = Funct1,
>       Id2 = Funct2 } }
>
> that would be an array of Lua filters. But now, I see where you
> may be getting confused. Reading on in the docs: "Pandoc expects
> each Lua file to return a list of filters. The filters in that
> list are called sequentially, each on the result of the previous
> filter." It may be confusing that one often uses the term
> "lua filter" for the file itself. But the file can return
> a list (technically an array, implemented as a Lua table) of
> filters.  I suspect you are confusing an array of Lua filters --
> which can be defined in a single Lua file -- with a Lua filter.
>
> --
> You received this message because you are subscribed to the Google Groups
> "pandoc-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/pandoc-discuss/m2o8nghfs7.fsf%40johnmacfarlane.net
> .
>

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/CADAJKhCR8ZQwWhEW2x_b9sUAyMikaFEySXnQCFVtFWq6VOdRKQ%40mail.gmail.com.

[-- Attachment #2: Type: text/html, Size: 5807 bytes --]

Re: Type of the `filter' argument to traversal functions

[BPJ]

[-- Attachment #1: Type: text/plain, Size: 4335 bytes --]

I wrote:

> why it is useful that a filter script can return an array of filters

That should obviously have been "must return"! Sorry for any confusion!

-- 
Better --help|less than helpless

Den ons 12 aug. 2020 10:19BPJ <melroch-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> skrev:

> I suppose that it might be worth pointing out that the reason why it is
> useful that a filter script can return an array of filters is explained
> here:
>
> https://pandoc.org/lua-filters.html#execution-order
>
> A note on terminology: Since "table" on its own is rather ambiguous in Lua may
> be a good idea to clarify what kind of data a it given table is supposed
> to contain by using the terms "array table" and "map table" and possibly
> "mixed table" for a table with both array and map data, which are sometimes
> convenient in Lua code, e.g. to tag an array.
>
> --
> Better --help|less than helpless
>
> Den ons 12 aug. 2020 02:27John MacFarlane <jgm-TVLZxgkOlNX2fBVCVOL8/A@public.gmane.org> skrev:
>
>> Anton Shepelev <anton.txt-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> writes:
>>
>> > John MacFarlane to Anton Shepelev:
>> >
>> >> > The  documentation to walk_block() and walk_fil-
>> >> > ter() states that the second parameter is "a Lua
>> >> > filter (table of functions) to be applied within
>> >> > the block element".
>> >> > [...]
>> >> > Is the documentation incorrect or do I misinter-
>> >> > pret  it?  For  aught I know, {Span = OnSpan} is
>> >> > not a "table of functions".
>> >>
>> >> { } makes a table.  OnSpan is a function.   So,  a
>> >> table of functions.
>> >
>> > Please,  bear with me. A table in Lua is the one-to-
>> > rule-them-all complex type, that can represent  many
>> > sturctures,  including  array, associative array, or
>> > structure.  On the other hand,  the  generally  used
>> > meaning of `table' in programming and mathematics is
>> > a two-dimensional sturcture  with  columns  and  any
>> > number of rows from zero up, which, too, can be rep-
>> > resented as a table in Lua, although it would  be  a
>> > compound one.
>>
>> I would have thought that in the context of documenting a Lua
>> function 'table' would be naturally interpreted as 'Lua table'.
>>
>> > But  since the parameter is described as "a Lua fil-
>> > ter" and "a table of functions", the only way to in-
>> > terpret it is as describing the structure of the Lua
>> > filter itself, which is a table of functions in both
>> > Lua and general terminology:
>> >
>> >                 { { Id1 = Func1 }
>> >                   { Id2 = Func2 }
>> >                   { Id3 = Func3 } }
>>
>> According to the docs, "Lua filters are tables with element names
>> as keys and values consisting of functions acting on those
>> elements." So yes, the parameter to walk_block is a Lua filter as
>> defined in the documentation.
>>
>> If you had
>>
>>   { { Id1 = Funct1,
>>       Id2 = Funct2 } }
>>
>> that would be an array of Lua filters. But now, I see where you
>> may be getting confused. Reading on in the docs: "Pandoc expects
>> each Lua file to return a list of filters. The filters in that
>> list are called sequentially, each on the result of the previous
>> filter." It may be confusing that one often uses the term
>> "lua filter" for the file itself. But the file can return
>> a list (technically an array, implemented as a Lua table) of
>> filters.  I suspect you are confusing an array of Lua filters --
>> which can be defined in a single Lua file -- with a Lua filter.
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "pandoc-discuss" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/pandoc-discuss/m2o8nghfs7.fsf%40johnmacfarlane.net
>> .
>>
>

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/CADAJKhDhPnbCr-_qyb%3Dn3odtCneVnhsgPHo1wJXXm1nOuF%3DSsg%40mail.gmail.com.

[-- Attachment #2: Type: text/html, Size: 6683 bytes --]

Re: Type of the `filter' argument to traversal functions

[Anton Shepelev]

BPJ:

> A note on terminology: Since "table" on its own is
> rather ambiguous in Lua may  be  a  good  idea  to
> clarify what kind of data a it given table is sup-
> posed to contain by using the terms "array  table"
> and  "map  table" and possibly "mixed table" for a
> table with both array  and  map  data,  which  are
> sometimes  convenient  in Lua code, e.g. to tag an
> array.

That would certanly help such newcomers as  I.   Has
there no standard terminology been developed for the
various kinds of Lua table?  If not,  then  I  agree
than  the Pandoc manual should introduce some simple
internal terminology, perhaps dependent on usage:

   array,list: an integer-indexed table
   map,record: a string-indexed table

It makes a Lua filter a map  of  elements  names  to
processing  functions, and the return type of a fil-
ter script an array of such maps, the `Header'  ele-
ment  a record with fields `level', `content', &c. I
have not encountered what you call  "mixed  tables",
but  I  think they would be records with two fields:
one an array and the other a map...

-- 
Please, do not forward replies to the list to my e-mail.

Re: Type of the `filter' argument to traversal functions

[EBkysko]

[-- Attachment #1.1: Type: text/plain, Size: 629 bytes --]

See the official online Lua book (a bit outdated :) ) :

*2.5 – Tables <https://www.lua.org/pil/2.5.html>*

*3.6 – Table Constructors <https://www.lua.org/pil/3.6.html>*

it'll have useful information.

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/83e15321-1098-4d01-8746-cb1d6da3be79o%40googlegroups.com.

[-- Attachment #1.2: Type: text/html, Size: 1019 bytes --]

Re: Type of the `filter' argument to traversal functions

[Anton Shepelev]

John MacFarlane to Anton Shepelev:

> [...]
> But  now, I see where you may be getting confused.
> Reading on in the docs: "Pandoc expects  each  Lua
> file  to  return a list of filters. The filters in
> that list are called sequentially, each on the re-
> sult  of the previous filter." It may be confusing
> that one often uses the term "lua filter" for  the
> file itself. But the file can return a list (tech-
> nically an array, implemented as a Lua  table)  of
> filters.   I suspect you are confusing an array of
> Lua filters -- which can be defined  in  a  single
> Lua file -- with a Lua filter.

I think that was it.  Thank you for the explanation,
John.

-- 
Please, do not forward replies to the list to my e-mail.