help filtering languages

help filtering languages

[Pablo Rodríguez]

Dear list,

I would like to add (natural languages) to attributes, such as in:

  _Deutsch_{:de}

A filter would help to filter the language attribute defined as
everything after a colon (preceded by a space or at the start of the
attributes) and before a space (or the end of the attributes).

If I’m not wrong, this would filter:

  _Straße_{:de-1901}

but it would avoid:

  # Title {#first:title}

I think the filtering rule is pretty simple, but I don’t know how to
write the filter.

Many thanks in advance,

Pablo

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/591336ef-c61e-31a8-3f44-2625551f7e07%40web.de.

Re: help filtering languages

[Bastien DUMONT]

{:de} will be parsed as a string, not as an attribute list. Attributes are of the form key=value. In this case, you can write (-f commonmark+attributes):

_Deutsch_{lang=de}

What you propose is a syntax extension, and it would be quite difficult to write a filter taking into account multiple attributes and cases where the {...} string is immediately followed by other characters. Maybe a custom reader would be better suited. But it would be easier to stick to existing conventions!

Le Monday 13 March 2023 à 08:25:15PM, Pablo Rodríguez a écrit :
> Dear list,
> 
> I would like to add (natural languages) to attributes, such as in:
> 
>   _Deutsch_{:de}
> 
> A filter would help to filter the language attribute defined as
> everything after a colon (preceded by a space or at the start of the
> attributes) and before a space (or the end of the attributes).
> 
> If I’m not wrong, this would filter:
> 
>   _Straße_{:de-1901}
> 
> but it would avoid:
> 
>   # Title {#first:title}
> 
> I think the filtering rule is pretty simple, but I don’t know how to
> write the filter.
> 
> Many thanks in advance,
> 
> Pablo
> 
> -- 
> You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
> To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/591336ef-c61e-31a8-3f44-2625551f7e07%40web.de.

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/ZA98zxHsNXsGRE9c%40localhost.

Re: help filtering languages

[Pablo Rodríguez]

On 3/13/23 20:43, Bastien DUMONT wrote:
> {:de} will be parsed as a string, not as an attribute list.
> Attributes are of the form key=value. In this case, you can write
> (-f commonmark+attributes):
>
> _Deutsch_{lang=de}
>
> What you propose is a syntax extension

Many thanks for your reply, Bastien.

I know there is a way to full attribute specification, since the
original proposal is almost a decade old
(https://github.com/jgm/pandoc/issues/895 [which was superseded by
https://github.com/jgm/pandoc/issues/3451]).

> and it would be quite difficult to write a filter taking into
> account multiple attributes and cases where the {...} string is
> immediately followed by other characters.

Well, I see what I was missing here.

> Maybe a custom reader would be better suited.

I wouldn’t mind to add this to (a custom version of) djot. But I clearly
lack the knowledge to do that.

> But it would be easier to stick to existing conventions!

I’m used to the convention (although I‘d rather avoid it, since it tend
to mix languages a lot), but when explaining CommonMark/Markdown to
newcomers, this is one important exception to lightweight markup.

And believe me, it is not easy to explain the key="value" attribute
syntax (it doesn’t make it easier to ignore the quotes) to people with
practically no computer background at all. Because this goes on top of a
bunch of new things for them (which would make anyone explode).

Many thanks for your help,

Pablo

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/103a971c-503b-e15e-8e1d-2b9700099138%40web.de.

Re: help filtering languages

[BPJ]

[-- Attachment #1: Type: text/plain, Size: 3116 bytes --]

At least you can abbreviate `[rien]{lang=fr}` to `[rien]{l=fr}` --- a
reduction from five to two chars instead of from five to one is at least
better than none! (And Pandoc allows you to omit the quotes around the
attribute value if the value is a valid name.)

``````lua
local function handler(elem)
  if elem.attributes.l then
    elem.attributes.lang = elem.attributes.l
    elem.attributes.l = nil
    return elem
  end
  return nil
end

return {
  {
    Div = handler,
    Span = handler,
    Link = handler,
    Header = handler,
  }
}
``````

Den mån 13 mars 2023 21:47Pablo Rodríguez <oinos-S0/GAf8tV78@public.gmane.org> skrev:

> On 3/13/23 20:43, Bastien DUMONT wrote:
> > {:de} will be parsed as a string, not as an attribute list.
> > Attributes are of the form key=value. In this case, you can write
> > (-f commonmark+attributes):
> >
> > _Deutsch_{lang=de}
> >
> > What you propose is a syntax extension
>
> Many thanks for your reply, Bastien.
>
> I know there is a way to full attribute specification, since the
> original proposal is almost a decade old
> (https://github.com/jgm/pandoc/issues/895 [which was superseded by
> https://github.com/jgm/pandoc/issues/3451]).
>
> > and it would be quite difficult to write a filter taking into
> > account multiple attributes and cases where the {...} string is
> > immediately followed by other characters.
>
> Well, I see what I was missing here.
>
> > Maybe a custom reader would be better suited.
>
> I wouldn’t mind to add this to (a custom version of) djot. But I clearly
> lack the knowledge to do that.
>
> > But it would be easier to stick to existing conventions!
>
> I’m used to the convention (although I‘d rather avoid it, since it tend
> to mix languages a lot), but when explaining CommonMark/Markdown to
> newcomers, this is one important exception to lightweight markup.
>
> And believe me, it is not easy to explain the key="value" attribute
> syntax (it doesn’t make it easier to ignore the quotes) to people with
> practically no computer background at all. Because this goes on top of a
> bunch of new things for them (which would make anyone explode).
>
> Many thanks for your help,
>
> Pablo
>
> --
> You received this message because you are subscribed to the Google Groups
> "pandoc-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/pandoc-discuss/103a971c-503b-e15e-8e1d-2b9700099138%40web.de
> .
>

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/CADAJKhDcpBdJxQO7qCbnj2Tk%2BkuxuVrh%2BHGS7eg8JoUoGSuC0w%40mail.gmail.com.

[-- Attachment #2: Type: text/html, Size: 4720 bytes --]

Re: help filtering languages

[Pablo Rodríguez]

On 3/15/23 09:32, BPJ wrote:
> At least you can abbreviate `[rien]{lang=fr}` to `[rien]{l=fr}` --- a
> reduction from five to two chars instead of from five to one is at least
> better than none!

Many thanks for your reply, BPJ.

I’m afraid these kinds of abbreviations make the source almost unreadable.

If there is no lightweight syntax for an attribute, I guess complete key
names are better (I mean, `class="first"` is more readable than
`c="first"`).

> (And Pandoc allows you to omit the quotes around the
> attribute value if the value is a valid name.)

I know, this would be great if it would be standard XML practice.

Otherwise, it isn‘t the best habit when you have to write complete XML code.

Many thanks for your help,

Pablo

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/f5687eba-f82c-64c2-21b8-94093a1158f5%40web.de.

Re: help filtering languages

[Julien Dutant]

[-- Attachment #1.1: Type: text/plain, Size: 2062 bytes --]

Hi Pablo,

If simple markdown is key here, why not hack the link syntax? Nobody ever 
links to 'fr' or 'en-gb' so we could write:

[savoir-faire](fr)

and use a filter that picks up links with 'xx' or 'xx-xx' targets and 
returns whatever you want (probably a Span with lang attribute):

-- langTags.lua
function Link (l)
if l.target:match('^%a%a$') or l.target:match('$^%a%a%-%a%a$') then
return pandoc.Span(l.content, { lang = l.target})
end
end

Example

$> echo '[savoir-faire](fr)' | pandoc -L lanTags.lua 
<p><span lang="fr">savoir-faire</span></p>

$> echo '[savoir-faire](fr)' | pandoc -L langTags.lua -t native
[ Para
    [ Span
        ( "" , [] , [ ( "lang" , "fr" ) ] ) [ Str "savoir-faire" ]
    ]
]

Best,
J

On Wednesday, March 15, 2023 at 7:45:33 PM UTC Pablo Rodríguez wrote:

> On 3/15/23 09:32, BPJ wrote:
> > At least you can abbreviate `[rien]{lang=fr}` to `[rien]{l=fr}` --- a
> > reduction from five to two chars instead of from five to one is at least
> > better than none!
>
> Many thanks for your reply, BPJ.
>
> I’m afraid these kinds of abbreviations make the source almost unreadable.
>
> If there is no lightweight syntax for an attribute, I guess complete key
> names are better (I mean, `class="first"` is more readable than
> `c="first"`).
>
> > (And Pandoc allows you to omit the quotes around the
> > attribute value if the value is a valid name.)
>
> I know, this would be great if it would be standard XML practice.
>
> Otherwise, it isn‘t the best habit when you have to write complete XML 
> code.
>
> Many thanks for your help,
>
> Pablo
>

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/49e8c09d-513a-47e5-a85e-c4cf9ca01d5en%40googlegroups.com.

[-- Attachment #1.2: Type: text/html, Size: 3092 bytes --]

Re: help filtering languages

[Pablo Rodríguez]

On 3/16/23 10:32, Julien Dutant wrote:
> Hi Pablo,
>
> If simple markdown is key here, why not hack the link syntax? Nobody
> ever links to 'fr' or 'en-gb' so we could write:
>
> [savoir-faire](fr)

Hi Julien,

I really appreciate your help, but I’m afraid that lightweight markup
doesn’t allow such an inconsistency.

Sorry, I thought that language was a basic attribute (such as identifier
and class). But I seem to be

This is why I thought I would make sense to have a lightweight syntax.

I apologize again, imagine explaining that language goes in link
destination to an audience with absolutely no coding background. This
would give anyone a hard time trying to justify it.

Really appreciate you help, but as Bastien suggested, my proposal for
lightweight language syntax seems to require a custom reader.

Many thanks for your help again,

Pablo

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/c54cca0e-6eaf-273b-caaa-882f5d9e1c1e%40web.de.

Re: help filtering languages

[Julien Dutant]

[-- Attachment #1.1: Type: text/plain, Size: 2834 bytes --]

Ok I think I understand better: you want to add attributes to inlines, as 
in djot. 
https://htmlpreview.github.io/?https://github.com/jgm/djot/blob/master/doc/syntax.html#inline-attributes 
. The problem, then, isn't just that language isn't a basic attribute. It's 
also that Pandoc's internal representation of documents don't allow 
attributes *at all* on arbitrary inline elements. See the specification of 
the AST elements here: https://pandoc.org/lua-filters.html#type-inline . 
Code, Span, Image have attributes but Emph doesn't, for instance. 

So your custom reader would need to convert whatever syntax you use to Span 
elements with a "lang=xx" attribute. 

Re the link hack, you could require a colon, i.e. tag languages with 
[...](:de):

-- langTags.lua
function Link (l)
if l.target:match('^%a%a$') or l.target:match('$:^%a%a%-%a%a$') then
return pandoc.Span(l.content, { lang = l.target})
end
end
 
Your newbie audience would need to understand that:

[this one](https://www.example.org)  is a link
[that one](:en) is language markup

I don't think it's difficult to master for authoring. Of course it the 
point is rather to teach about markup, the ambiguity isn't great. (Though 
you could consider that unambiguous, e.g. declare that all [...](:..) is 
markup. There's also the issue that you can't combine this language 
attribute with others.

J

On Thursday, March 16, 2023 at 5:46:05 PM UTC Pablo Rodríguez wrote:

> On 3/16/23 10:32, Julien Dutant wrote:
> > Hi Pablo,
> >
> > If simple markdown is key here, why not hack the link syntax? Nobody
> > ever links to 'fr' or 'en-gb' so we could write:
> >
> > [savoir-faire](fr)
>
> Hi Julien,
>
> I really appreciate your help, but I’m afraid that lightweight markup
> doesn’t allow such an inconsistency.
>
> Sorry, I thought that language was a basic attribute (such as identifier
> and class). But I seem to be
>
> This is why I thought I would make sense to have a lightweight syntax.
>
> I apologize again, imagine explaining that language goes in link
> destination to an audience with absolutely no coding background. This
> would give anyone a hard time trying to justify it.
>
> Really appreciate you help, but as Bastien suggested, my proposal for
> lightweight language syntax seems to require a custom reader.
>
> Many thanks for your help again,
>
> Pablo
>

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/c238d079-4ba5-4f39-8171-c17e1119d4c5n%40googlegroups.com.

[-- Attachment #1.2: Type: text/html, Size: 3767 bytes --]

Re: help filtering languages

[Pablo Rodríguez]

On 3/16/23 22:28, Julien Dutant wrote:
> Ok I think I understand better: you want to add attributes to inlines,
> as in djot [...] Pandoc's internal representation of documents
> doesn't allow attributes *at all* on arbitrary (inline) elements.

Hi Julien,

I wanted attributes for all elements
(https://yewtu.be/watch?v=hFDcoX7s6rE, previously
https://github.com/jgm/pandoc/issues/1966).

AFAIK, these are available for CommonMark with extensions (and for djot?).

I also knew that internally pandoc cannot deal with this natively.

> Your newbie audience would need to understand that:
>
> [this one](https://www.example.org)  is a link
> [that one](:en) is language markup
>
> I don't think it's difficult to master for authoring. Of course it the
> point is rather to teach about markup, the ambiguity isn't great.
> (Though you could consider that unambiguous, e.g. declare that all
> [...](:..) is markup. There's also the issue that you can't combine this
> language attribute with others.

I really appreciate your help, but it seems that a compromise is
required for lightweight language markup. In that case, I‘m going to
wait to explain this.

Many thanks for your help again,

Pablo

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/68de3cab-a729-7728-2652-774a8ff599f4%40web.de.

Re: help filtering languages

[Julien Dutant]

[-- Attachment #1.1: Type: text/plain, Size: 2002 bytes --]

Actually (if I finally understood correctly) there's an open issue for it 
on the Pandoc repo: https://github.com/jgm/pandoc/issues/7921
Looks like the idea would be to make it an "extension" of Pandoc's 
markdown. 

On Monday, March 20, 2023 at 5:20:40 PM UTC Pablo Rodríguez wrote:

> On 3/16/23 22:28, Julien Dutant wrote:
> > Ok I think I understand better: you want to add attributes to inlines,
> > as in djot [...] Pandoc's internal representation of documents
> > doesn't allow attributes *at all* on arbitrary (inline) elements.
>
> Hi Julien,
>
> I wanted attributes for all elements
> (https://yewtu.be/watch?v=hFDcoX7s6rE, previously
> https://github.com/jgm/pandoc/issues/1966).
>
> AFAIK, these are available for CommonMark with extensions (and for djot?).
>
> I also knew that internally pandoc cannot deal with this natively.
>
> > Your newbie audience would need to understand that:
> >
> > [this one](https://www.example.org)  is a link
> > [that one](:en) is language markup
> >
> > I don't think it's difficult to master for authoring. Of course it the
> > point is rather to teach about markup, the ambiguity isn't great.
> > (Though you could consider that unambiguous, e.g. declare that all
> > [...](:..) is markup. There's also the issue that you can't combine this
> > language attribute with others.
>
> I really appreciate your help, but it seems that a compromise is
> required for lightweight language markup. In that case, I‘m going to
> wait to explain this.
>
> Many thanks for your help again,
>
> Pablo
>

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/bb779d38-8c39-478c-a358-27dc1ec86bf4n%40googlegroups.com.

[-- Attachment #1.2: Type: text/html, Size: 3421 bytes --]