Re: Syntax highlighting for English as a natural language

[BPJ <melroch-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>]

[-- Attachment #1: Type: text/plain, Size: 3383 bytes --]

That would mean natural language parsing and natural language parsing is
hard, not to mention CPU and memory intensive, and needs a huge database (a
tagged dictionary). You would need a largish external program for that,
unless you are content with highlighting a few closed (i
.e. having a smallish finite set of members) word classes like pronouns and
prepositions. Telling nouns, adjectives and verbs apart in English is very
hard since often the same form can be two or three of these depending on
context, so you basically need to do a semantic parse of the whole sentence
with grammatical syntax, which is non-trivial. If you try to translate from
English into a language which clearly distinguishes two or all three of
verbs/nouns/adjectives you will see that it often fails. Also just knowing
the word class of a word is of limited usefulness. You will generally want
to include syntactic information like what is the subject, direct object,
indirect object, main verb, auxiliary verb etc. Not easy. Been there, done
that. Admittedly it was nearly two decades ago and the techniques have
probably at least partly changed/improved since then, but that does not
necessarily mean that they have become less resource intensive. There is a
reason why tasks like translation are commonly done server-side rather than
client-side. There is also a reason why computer programs are written in
unambiguous artificial languages rather than ambiguous natural languages
and all natural languages are ambiguous; human brains handle ambiguity well
while computers basically suck at it. Natural language processing is 90%
disambiguation.

Sorry if this is disappointing but such is the complexity of the task.

-- 
Better --help|less than helpless

Den tis 8 dec. 2020 05:54R (Chandra) Chandrasekhar <chyavana-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
skrev:

> I am looking at highlighting parts of speech in English within a
> suitably demarcated code block in Markdown.
>
> For example:
>
> ```{english}
> I use Pandoc.
>
> ```
>
> where:
>
> _I_ is given a unique colour for pronouns
> _use_ is given a unique colour for verbs
> _Pandoc_ is given a unique colour for nouns.
>
> I fear that something that does this generically might require natural
> language processing (NLP) capabilities.
>
> Since my application is small, I am happy to hand-code individual
> colours for individual words.
>
> How would I accomplish this in a fashion that allows me to use the same
> source for HTML5 and XeLaTeX/PDF outputs?
>
> Thanks.
>
> Chandra
>
> --
> You received this message because you are subscribed to the Google Groups
> "pandoc-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/pandoc-discuss/9a7f850d-7335-074b-1527-8a578ed13741%40gmail.com
> .
>

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/CADAJKhAK%3DEWS0gTgqL%3DWpXTpWEK-AAKKASfxwgU4MSLgXtU%3Dig%40mail.gmail.com.

[-- Attachment #2: Type: text/html, Size: 4467 bytes --]