From: Paul Mazaitis via ntg-context <ntg-context@ntg.nl>
To: ntg-context@ntg.nl
Cc: Paul Mazaitis <paul@mazaitis.org>
Subject: New Tool: tree-sitter-context_en Parser
Date: Fri, 22 Apr 2022 20:58:52 -0400 [thread overview]
Message-ID: <03FD8A27-1E53-463C-A9B7-0175E2B8C50C@mazaitis.org> (raw)
Hi, everyone!
I've been working on an updated tree-sitter parser for the ConTeXt file format.
The parser has pretty good support for parsing a ConTeXt .tex file and producing a syntax tree for that file (and, optionally, highlighting).
The parser also has support for using appropriate parsers for areas of the document that embed other languages or formats. For example: a tree-sitter parser for Lua code already exists, and this parser can call out to that parser to handle any properly demarcated Lua code (\startluacode ... \stopluacode) present in a ConTeXt document. A full list of supported inclusions (with or without available parsers), along with additional notes, are in the Garden:
https://wiki.contextgarden.net/pm39_tree_sitter_parser
...and the repo for the tree-sitter-context_en parser itself is here:
https://github.com/pmazaitis/tree-sitter-context_en
Tree-sitter parsers support lots of language bindings with which to build tools; I've put together a very badly engineered format-aware spellchecker as a demo (may have trouble with Windows EOL encoding):
https://github.com/pmazaitis/contextpyspell-demo
Please, please do not use this demo in production. :)
A massive caveat regarding OS support:
So far this has been all developed and tested under MacOS, and seems to work pretty well in that environment. I've also tested running the parser on Linux (Ubuntu), and that was also fairly straight forward. In either case, Rust and a C compiler are required for directly using the parser (and you'll also need node/npm to rebuild the grammar itself, if you want to make any changes).
I have a Windows 10 machine, and the development environment on it is a complete puzzle to me. I am learning, but: running this parser under Windows isn't something I've been able to get working. If you have Windows development experience, I'd be happy for any advice on how to support this software on that platform.
A second, massive caveat for Windows support: this parser should be agnostic about how line endings are encoded, and I've done minimal testing, but it should have a lot more.
Looking forward:
- This parser is...lightly tested. (More testing of more domains would be helpful!)
- This tooling should be useful for small utilities: a linter, a formatter, a better spellchecker?
- This tooling should be useful for LSP support?
Onward!
-Paul
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!
maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage : http://www.pragma-ade.nl / http://context.aanhet.net
archive : https://bitbucket.org/phg/context-mirror/commits/
wiki : http://contextgarden.net
___________________________________________________________________________________
reply other threads:[~2022-04-23 0:58 UTC|newest]
Thread overview: [no followups] expand[flat|nested] mbox.gz Atom feed
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=03FD8A27-1E53-463C-A9B7-0175E2B8C50C@mazaitis.org \
--to=ntg-context@ntg.nl \
--cc=paul@mazaitis.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).