New Tool: tree-sitter-context_en Parser

New Tool: tree-sitter-context_en Parser

[Paul Mazaitis via ntg-context]

Hi, everyone!

I've been working on an updated tree-sitter parser for the ConTeXt file format.

The parser has pretty good support for parsing a ConTeXt .tex file and producing a syntax tree for that file (and, optionally, highlighting).

The parser also has support for using appropriate parsers for areas of the document that embed other languages or formats. For example: a tree-sitter parser for Lua code already exists, and this parser can call out to that parser to handle any properly demarcated Lua code (\startluacode ... \stopluacode) present in a ConTeXt document. A full list of supported inclusions (with or without available parsers), along with additional notes, are in the Garden:

    https://wiki.contextgarden.net/pm39_tree_sitter_parser

...and the repo for the tree-sitter-context_en parser itself is here:

	https://github.com/pmazaitis/tree-sitter-context_en

Tree-sitter parsers support lots of language bindings with which to build tools; I've put together a very badly engineered format-aware spellchecker as a demo (may have trouble with Windows EOL encoding):

	https://github.com/pmazaitis/contextpyspell-demo

Please, please do not use this demo in production. :)

A massive caveat regarding OS support:

So far this has been all developed and tested under MacOS, and seems to work pretty well in that environment. I've also tested running the parser on Linux (Ubuntu), and that was also fairly straight forward. In either case, Rust and a C compiler are required for directly using the parser  (and you'll also need node/npm to rebuild the grammar itself, if you want to make any changes).

I have a Windows 10 machine, and the development environment on it is a complete puzzle to me. I am learning, but: running this parser under Windows isn't something I've been able to get working. If you have Windows development experience, I'd be happy for any advice on how to support this software on that platform.

A second, massive caveat for Windows support: this parser should be agnostic about how line endings are encoded, and I've done minimal testing, but it should have a lot more.

Looking forward:

- This parser is...lightly tested. (More testing of more domains would be helpful!)
- This tooling should be useful for small utilities: a linter, a formatter, a better spellchecker?
- This tooling should be useful for LSP support?

Onward!

	-Paul
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki     : http://contextgarden.net
___________________________________________________________________________________