etexshow and generation of cont-en.xlm (cont-XX.xml)?

etexshow and generation of cont-en.xlm (cont-XX.xml)?

[Jonathan Barchi]

Hi all,

I'm using the emacs package etexshow to provide a quick ConTeXt
command reference in EMACS. This utility parses the interface file
cont-en.xml (for english, in my case) and uses the contents to provide
a basic dictionary of available commands.

In the documentation for etexshow, there is a reference to being able
to generate a complete interface description (the aforementioned xml
file) like so:

";; There is still an xml-file shipped with this code. Usually you would
;; generate the xml-file with 'texexec'ing the file setupe.tex. Then you
;; will get the cont-en.xml file that can (could) be used as an input for
;; this etexshow. But for now, this won't work. It will work rsn."

It is not clear whether that file (setupe.tex or setup.tex) is/was
supposed to be shipped with etexshow or was supposed to be a part of
context. I've searched and found a few references to this problem, but
never a definitive answer, so if anyone knows the answer to these
questions I would love to know!!

1. Is the referenced setup(e).tex a file that was previously shipped
with ConTeXt - presumably mkii since the author refers to texexec - or
would this be something the author would have included with etexshow
itself?

2. Is the interface file, e.g. cont-en.xml, auto-generated from all of
the defined macros in ConTeXt, or is that file hand-written as
metadata to accompany the source-code definitions?

3. Since self-documentation is a goal of the project, is it possible
to generate something akin to the interface file that presents a
snapshot of all commands known to context at a given time, for use in
things like etexshow or other tools? If so, is it possible to do this
with modules loaded to see what they additionally define?

Thanks in advance for any help! I'm hoping to use any information I
get to update etexshow, if possible, and maybe update the ConTeXt
support in AUCTEX as well. I think it would be really cool if AUCTEX
could reach out at compile or run time and pull in macro definitions
from the installed environment, for example.

Best,
Jon
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: etexshow and generation of cont-en.xlm (cont-XX.xml)?

[Hans Hagen]

On 3/18/2013 11:35 PM, Jonathan Barchi wrote:
> Hi all,
>
> I'm using the emacs package etexshow to provide a quick ConTeXt
> command reference in EMACS. This utility parses the interface file
> cont-en.xml (for english, in my case) and uses the contents to provide
> a basic dictionary of available commands.
>
> In the documentation for etexshow, there is a reference to being able
> to generate a complete interface description (the aforementioned xml
> file) like so:
>
> ";; There is still an xml-file shipped with this code. Usually you would
> ;; generate the xml-file with 'texexec'ing the file setupe.tex. Then you
> ;; will get the cont-en.xml file that can (could) be used as an input for
> ;; this etexshow. But for now, this won't work. It will work rsn."
>
> It is not clear whether that file (setupe.tex or setup.tex) is/was
> supposed to be shipped with etexshow or was supposed to be a part of
> context. I've searched and found a few references to this problem, but
> never a definitive answer, so if anyone knows the answer to these
> questions I would love to know!!
>
> 1. Is the referenced setup(e).tex a file that was previously shipped
> with ConTeXt - presumably mkii since the author refers to texexec - or
> would this be something the author would have included with etexshow
> itself?
>
> 2. Is the interface file, e.g. cont-en.xml, auto-generated from all of
> the defined macros in ConTeXt, or is that file hand-written as
> metadata to accompany the source-code definitions?

the xml file is the master file (but I admit that it's a bit behind)

if you only need the commands then the mult-def.lua files are better, 
also because they contain low level commands and primitives (it's what I 
use to generate files for scite where we have a rather advanced lexer)

> 3. Since self-documentation is a goal of the project, is it possible
> to generate something akin to the interface file that presents a
> snapshot of all commands known to context at a given time, for use in
> things like etexshow or other tools? If so, is it possible to do this
> with modules loaded to see what they additionally define?

we can add a --emacs to mtx-interfaces.lua if something other than xml 
is needed .. so, what does emacs need?

> Thanks in advance for any help! I'm hoping to use any information I
> get to update etexshow, if possible, and maybe update the ConTeXt
> support in AUCTEX as well. I think it would be really cool if AUCTEX
> could reach out at compile or run time and pull in macro definitions
> from the installed environment, for example.

Hans

-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
     tel: 038 477 53 69 | voip: 087 875 68 74 | www.pragma-ade.com
                                              | www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: etexshow and generation of cont-en.xlm (cont-XX.xml)?

[hwitloc]

That is a wonderful idea and one that would be very, very helpful, and much appreciated.

Best

Jonathan Barchi <barchij@gmail.com> wrote:
> Hi all,
> 
> I'm using the emacs package etexshow to provide a quick ConTeXt
> command reference in EMACS. This utility parses the interface file
> cont-en.xml (for english, in my case) and uses the contents to provide
> a basic dictionary of available commands.
> 
> In the documentation for etexshow, there is a reference to being able
> to generate a complete interface description (the aforementioned xml
> file) like so:
> 
> ";; There is still an xml-file shipped with this code. Usually you would
> ;; generate the xml-file with 'texexec'ing the file setupe.tex. Then you
> ;; will get the cont-en.xml file that can (could) be used as an input for
> ;; this etexshow. But for now, this won't work. It will work rsn."
> 
> It is not clear whether that file (setupe.tex or setup.tex) is/was
> supposed to be shipped with etexshow or was supposed to be a part of
> context. I've searched and found a few references to this problem, but
> never a definitive answer, so if anyone knows the answer to these
> questions I would love to know!!
> 
> 1. Is the referenced setup(e).tex a file that was previously shipped
> with ConTeXt - presumably mkii since the author refers to texexec - or
> would this be something the author would have included with etexshow
> itself?
> 
> 2. Is the interface file, e.g. cont-en.xml, auto-generated from all of
> the defined macros in ConTeXt, or is that file hand-written as
> metadata to accompany the source-code definitions?
> 
> 3. Since self-documentation is a goal of the project, is it possible
> to generate something akin to the interface file that presents a
> snapshot of all commands known to context at a given time, for use in
> things like etexshow or other tools? If so, is it possible to do this
> with modules loaded to see what they additionally define?
> 
> Thanks in advance for any help! I'm hoping to use any information I
> get to update etexshow, if possible, and maybe update the ConTeXt
> support in AUCTEX as well. I think it would be really cool if AUCTEX
> could reach out at compile or run time and pull in macro definitions
> from the installed environment, for example.
> 
> Best,
> Jon
> ___________________________________________________________________________________
> If your question is of interest to others as well, please add an entry to the Wiki!
> 
> maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
> webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
> archive  : http://foundry.supelec.fr/projects/contextrev/
> wiki     : http://contextgarden.net
> ___________________________________________________________________________________
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: etexshow and generation of cont-en.xlm (cont-XX.xml)?

[Jonathan Barchi]

On Mon, Mar 18, 2013 at 7:16 PM, Hans Hagen <pragma@wxs.nl> wrote:
> On 3/18/2013 11:35 PM, Jonathan Barchi wrote:
>>
>> Hi all,
>>
>> I'm using the emacs package etexshow to provide a quick ConTeXt
>> command reference in EMACS. This utility parses the interface file
>> cont-en.xml (for english, in my case) and uses the contents to provide
>> a basic dictionary of available commands.
>>
>> In the documentation for etexshow, there is a reference to being able
>> to generate a complete interface description (the aforementioned xml
>> file) like so:
>>
>> ";; There is still an xml-file shipped with this code. Usually you would
>> ;; generate the xml-file with 'texexec'ing the file setupe.tex. Then you
>> ;; will get the cont-en.xml file that can (could) be used as an input for
>> ;; this etexshow. But for now, this won't work. It will work rsn."
>>
>> It is not clear whether that file (setupe.tex or setup.tex) is/was
>> supposed to be shipped with etexshow or was supposed to be a part of
>> context. I've searched and found a few references to this problem, but
>> never a definitive answer, so if anyone knows the answer to these
>> questions I would love to know!!
>>
>> 1. Is the referenced setup(e).tex a file that was previously shipped
>> with ConTeXt - presumably mkii since the author refers to texexec - or
>> would this be something the author would have included with etexshow
>> itself?
>>
>> 2. Is the interface file, e.g. cont-en.xml, auto-generated from all of
>> the defined macros in ConTeXt, or is that file hand-written as
>> metadata to accompany the source-code definitions?
>
>
> the xml file is the master file (but I admit that it's a bit behind)
>
> if you only need the commands then the mult-def.lua files are better, also
> because they contain low level commands and primitives (it's what I use to
> generate files for scite where we have a rather advanced lexer)

I'll take a look at those files, thanks!

Is that scite code free software? I'd love to take a look at how you generate
the files for it, and/or how its lexer is implemented.

>> 3. Since self-documentation is a goal of the project, is it possible
>> to generate something akin to the interface file that presents a
>> snapshot of all commands known to context at a given time, for use in
>> things like etexshow or other tools? If so, is it possible to do this
>> with modules loaded to see what they additionally define?
>
>
> we can add a --emacs to mtx-interfaces.lua if something other than xml is
> needed .. so, what does emacs need?

I'm still figuring this part out.  Emacs is quite modular, so I'm not
sure what the
best approach is yet. AUCTEX implements syntax highlighting internally, in
a sub-package called font-latex (that does work reasonably well for ConTeXt
too), but there is also more sophisticated support for LaTeX that doesn't cross
over to context.

There are more general solutions for lexing, like semantic, which might be
the better way to go. Semantic has an internal port of bison that can use BNF
grammar descriptions, but some reading suggests that this can't be done for
*TeX. It can also be fed tags by a regexp parser or an external
utility like ctags.

I'm quite new to all this, so if anyone out on the mailing list with more
experience wants to jump in with advice or corrections I'm all ears! I'm also
working on my dissertation, so time is limited (at least for the next couple
months).

- Jon

>> Thanks in advance for any help! I'm hoping to use any information I
>> get to update etexshow, if possible, and maybe update the ConTeXt
>> support in AUCTEX as well. I think it would be really cool if AUCTEX
>> could reach out at compile or run time and pull in macro definitions
>> from the installed environment, for example.
>
>
> Hans
>
> -----------------------------------------------------------------
>                                           Hans Hagen | PRAGMA ADE
>               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
>     tel: 038 477 53 69 | voip: 087 875 68 74 | www.pragma-ade.com
>                                              | www.pragma-pod.nl
> -----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: etexshow and generation of cont-en.xlm (cont-XX.xml)?

[Marcin Borkowski]

Dnia 2013-03-20, o godz. 12:22:55
Jonathan Barchi <barchij@gmail.com> napisał(a):

> BNF grammar descriptions, but some reading suggests that this can't
> be done for *TeX.

Don't listen to that.  While it's technically true, (a) most users don't
redefine catcodes or internal macros anyway (and imho there are fewer
reasons to do such hacking in ConTeXt than in LaTeX) and (b) for the
sake of syntax highlighting/parsing by external tools, you have to
assume some sanity anyway (so that you don't have to reimplement full
TeX instead of writing some parser).

That said, I know little about regular languages, BNF notations
etc. (with the exception of some _practical_ experience with regexen),
so I'm not sure about what I've written above.  But I would guess that
if you make some (reasonable) assumptions about catcodes and command
syntax, writing a BNF style grammar for (at least some part of)
ConTeXt might be pretty doable.

Regards,

-- 
Marcin Borkowski
http://octd.wmi.amu.edu.pl/en/Marcin_Borkowski
Adam Mickiewicz University
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: etexshow and generation of cont-en.xlm (cont-XX.xml)?

[Hans Hagen]

On 3/20/2013 5:22 PM, Jonathan Barchi wrote:

> Is that scite code free software? I'd love to take a look at how you generate
> the files for it, and/or how its lexer is implemented.

it's all in context/data/scite (if only because I use it on a daily basis)

> I'm still figuring this part out.  Emacs is quite modular, so I'm not
> sure what the
> best approach is yet. AUCTEX implements syntax highlighting internally, in
> a sub-package called font-latex (that does work reasonably well for ConTeXt
> too), but there is also more sophisticated support for LaTeX that doesn't cross
> over to context.

best start from scratch .. too many differences .. the easy way is just 
to color symbols special

red: {}$

green: \. \....

blue: []()#=<>"

yellow: %^_&-+/`|

that should be doable i guess

next figure out how to read in list with known \commands

Hans


-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
     tel: 038 477 53 69 | voip: 087 875 68 74 | www.pragma-ade.com
                                              | www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________