How to approach a ConTeXt language server?

How to approach a ConTeXt language server?

[Philipp A.]

[-- Attachment #1.1: Type: text/plain, Size: 2211 bytes --]

Language servers are the new big deal in editor and IDE development:
https://langserver.org/

It would be cool to have a ConTeXt one for autocompletion (for ConTeXt:
command names, \cite IDs, labels, named parameters, …), go-to-definition,
hover information (docs about a command) and so on.
The way it works is that you have a server process that is the source of
truth for all this information, and the editor passes requests to it.
The editor tells the server when it opens/closes files and when the user
requests something of the above.

The way I’d implement it in ConTeXt is to keep a list of open ConTeXt
projects in the server (obtained by following \include, \component,
\product, \environment, \project).
Now my questions begin:

   1. Then I’d make context load the project without compiling it to a PDF
   but make it execute some Lua (how do I do this?)
   2. I’d need a way to get all available commands with their signatures
   into Lua. I assume this is done here, but how?
   http://www.pragma-ade.nl/general/qrcs/setup-en.pdf
   3. Optimally, for hover information and completion, I’d want some
   help/doc text for each command that has some. Is there a way to get it?
   4. Optimally, for label autocompletion, I’d also like a list of defined
   labels. Since I played around with bibliographies I already know how to
   query the bibliography DB from Lua.
   5. Optimally I’d also want some parse tree of each document, but I
   assume the way macros work, this doesn’t exist? This would make things
   easier that I’d otherwise have to (imperfectly) parse out of the document
   (due to things like catcode changes, but I guess I can pretend they don’t
   exist and \unprotect is always on)
   6. Optimally, for go-to-definition, I’d also want a list of files
   ConTeXt loaded so I can find definitions in it.

Can anyone help me, especially with 1-2? To get me started, it would be
great to have an example script and a command line to invoke it, which
makes ConTeXt load a main tex file, execute some Lua, and exit without
creating a document or writing anything else to the channel Lua writes to
(stdout?).

Best, Philipp

[-- Attachment #1.2: Type: text/html, Size: 2410 bytes --]

[-- Attachment #2: Type: text/plain, Size: 493 bytes --]

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: How to approach a ConTeXt language server?

[Henning Hraban Ramm]

> Am 2020-02-04 um 16:45 schrieb Philipp A. <flying-sheep@web.de>:
> 
> Language servers are the new big deal in editor and IDE development: https://langserver.org/
> 
> It would be cool to have a ConTeXt one for autocompletion (for ConTeXt: command names, \cite IDs, labels, named parameters, …), go-to-definition, hover information (docs about a command) and so on.
> The way it works is that you have a server process that is the source of truth for all this information, and the editor passes requests to it.
> The editor tells the server when it opens/closes files and when the user requests something of the above.
> 
> The way I’d implement it in ConTeXt is to keep a list of open ConTeXt projects in the server (obtained by following \include, \component, \product, \environment, \project).
> Now my questions begin:
> 	• Then I’d make context load the project without compiling it to a PDF but make it execute some Lua (how do I do this?)
> 	• I’d need a way to get all available commands with their signatures into Lua. I assume this is done here, but how? http://www.pragma-ade.nl/general/qrcs/setup-en.pdf

Look for the interface files i-*.xml

> 	• Optimally, for hover information and completion, I’d want some help/doc text for each command that has some. Is there a way to get it?

No, there isn’t. It could be in the interface files if someone would put in the work.

> 	• Optimally, for label autocompletion, I’d also like a list of defined labels. Since I played around with bibliographies I already know how to query the bibliography DB from Lua.
> 	• Optimally I’d also want some parse tree of each document, but I assume the way macros work, this doesn’t exist? This would make things easier that I’d otherwise have to (imperfectly) parse out of the document (due to things like catcode changes, but I guess I can pretend they don’t exist and \unprotect is always on)

You could run ConTeXt and use the export XML.

> 	• Optimally, for go-to-definition, I’d also want a list of files ConTeXt loaded so I can find definitions in it.
> Can anyone help me, especially with 1-2? To get me started, it would be great to have an example script and a command line to invoke it, which makes ConTeXt load a main tex file, execute some Lua, and exit without creating a document or writing anything else to the channel Lua writes to (stdout?).

Look at the .tuc file that’s created in a ConTeXt run, it’s a Lua table and contains “all“ the information about the project.


Greetlings, Hraban
---
https://www.fiee.net
http://wiki.contextgarden.net
https://www.dreiviertelhaus.de
GPG Key ID 1C9B22FD

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: How to approach a ConTeXt language server?

[Philipp A.]

[-- Attachment #1.1: Type: text/plain, Size: 3549 bytes --]

Thank you, that’s great information!

What’s the “export XML”?

Am Di., 4. Feb. 2020 um 19:09 Uhr schrieb Henning Hraban Ramm <
texml@fiee.net>:

>
> > Am 2020-02-04 um 16:45 schrieb Philipp A. <flying-sheep@web.de>:
> >
> > Language servers are the new big deal in editor and IDE development:
> https://langserver.org/
> >
> > It would be cool to have a ConTeXt one for autocompletion (for ConTeXt:
> command names, \cite IDs, labels, named parameters, …), go-to-definition,
> hover information (docs about a command) and so on.
> > The way it works is that you have a server process that is the source of
> truth for all this information, and the editor passes requests to it.
> > The editor tells the server when it opens/closes files and when the user
> requests something of the above.
> >
> > The way I’d implement it in ConTeXt is to keep a list of open ConTeXt
> projects in the server (obtained by following \include, \component,
> \product, \environment, \project).
> > Now my questions begin:
> >       • Then I’d make context load the project without compiling it to a
> PDF but make it execute some Lua (how do I do this?)
> >       • I’d need a way to get all available commands with their
> signatures into Lua. I assume this is done here, but how?
> http://www.pragma-ade.nl/general/qrcs/setup-en.pdf
>
> Look for the interface files i-*.xml
>
> >       • Optimally, for hover information and completion, I’d want some
> help/doc text for each command that has some. Is there a way to get it?
>
> No, there isn’t. It could be in the interface files if someone would put
> in the work.
>
> >       • Optimally, for label autocompletion, I’d also like a list of
> defined labels. Since I played around with bibliographies I already know
> how to query the bibliography DB from Lua.
> >       • Optimally I’d also want some parse tree of each document, but I
> assume the way macros work, this doesn’t exist? This would make things
> easier that I’d otherwise have to (imperfectly) parse out of the document
> (due to things like catcode changes, but I guess I can pretend they don’t
> exist and \unprotect is always on)
>
> You could run ConTeXt and use the export XML.
>
> >       • Optimally, for go-to-definition, I’d also want a list of files
> ConTeXt loaded so I can find definitions in it.
> > Can anyone help me, especially with 1-2? To get me started, it would be
> great to have an example script and a command line to invoke it, which
> makes ConTeXt load a main tex file, execute some Lua, and exit without
> creating a document or writing anything else to the channel Lua writes to
> (stdout?).
>
> Look at the .tuc file that’s created in a ConTeXt run, it’s a Lua table
> and contains “all“ the information about the project.
>
>
> Greetlings, Hraban
> ---
> https://www.fiee.net
> http://wiki.contextgarden.net
> https://www.dreiviertelhaus.de
> GPG Key ID 1C9B22FD
>
>
> ___________________________________________________________________________________
> If your question is of interest to others as well, please add an entry to
> the Wiki!
>
> maillist : ntg-context@ntg.nl /
> http://www.ntg.nl/mailman/listinfo/ntg-context
> webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
> archive  : https://bitbucket.org/phg/context-mirror/commits/
> wiki     : http://contextgarden.net
>
> ___________________________________________________________________________________
>

[-- Attachment #1.2: Type: text/html, Size: 4859 bytes --]

[-- Attachment #2: Type: text/plain, Size: 493 bytes --]

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: How to approach a ConTeXt language server?

[Henning Hraban Ramm]

> Am 2020-02-05 um 10:42 schrieb Philipp A. <flying-sheep@web.de>:
> 
> Thank you, that’s great information!
> 
> What’s the “export XML”?

Should I google that for you?

https://wiki.contextgarden.net/Export


Grüßlinge, Hraban
---
https://www.fiee.net
http://wiki.contextgarden.net
https://www.dreiviertelhaus.de
GPG Key ID 1C9B22FD

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: How to approach a ConTeXt language server?

[Hans Hagen]

On 2/4/2020 4:45 PM, Philipp A. wrote:
> Language servers are the new big deal in editor and IDE development: 
> https://langserver.org/
> 
> It would be cool to have a ConTeXt one for autocompletion (for ConTeXt: 
> command names, \cite IDs, labels, named parameters, …), 
> go-to-definition, hover information (docs about a command) and so on.
> The way it works is that you have a server process that is the source of 
> truth for all this information, and the editor passes requests to it.
> The editor tells the server when it opens/closes files and when the user 
> requests something of the above.
> 
> The way I’d implement it in ConTeXt is to keep a list of open ConTeXt 
> projects in the server (obtained by following \include, \component, 
> \product, \environment, \project).
> Now my questions begin:
> 
>  1. Then I’d make context load the project without compiling it to a PDF
>     but make it execute some Lua (how do I do this?)
>  2. I’d need a way to get all available commands with their signatures
>     into Lua. I assume this is done here, but how?
>     http://www.pragma-ade.nl/general/qrcs/setup-en.pdf
>  3. Optimally, for hover information and completion, I’d want some
>     help/doc text for each command that has some. Is there a way to get it?
>  4. Optimally, for label autocompletion, I’d also like a list of defined
>     labels. Since I played around with bibliographies I already know how
>     to query the bibliography DB from Lua.
>  5. Optimally I’d also want some parse tree of each document, but I
>     assume the way macros work, this doesn’t exist? This would make
>     things easier that I’d otherwise have to (imperfectly) parse out of
>     the document (due to things like catcode changes, but I guess I can
>     pretend they don’t exist and \unprotect is always on)
>  6. Optimally, for go-to-definition, I’d also want a list of files
>     ConTeXt loaded so I can find definitions in it.
> 
> Can anyone help me, especially with 1-2? To get me started, it would be 
> great to have an example script and a command line to invoke it, which 
> makes ConTeXt load a main tex file, execute some Lua, and exit without 
> creating a document or writing anything else to the channel Lua writes 
> to (stdout?).
There is already some stuff in the distribution (for quite a while so i 
always need to check if it still works, and it seems to do), like:

-- start a help server:

mtxrun --script server --auto --start

-- pop up some help:

mtxrun --gethelp 
--url="http://localhost:8088/mtx-server-ctx-help.lua?command=framed"

which i hook into for instahce F1 in the editor (then it takes the 
command below the cursor).

so basically it's no big deal to add more queries, or instead pipe some 
info to a file or ...

Hans



-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
        tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki     : http://contextgarden.net
___________________________________________________________________________________