[***SPAM***] OT: Generating Command reference from source code

[***SPAM***] OT: Generating Command reference from source code

[Jan Tosovsky]

Dear All,

in another thread I was dreaming ... to have always the complete and up to
date command/macro reference with basic descriptions and list all of the
parameter/options...

'Spoiled' by goodies offered by various Java APIs (based on
http://en.wikipedia.org/wiki/Javadoc ) I am missing this a lot.

Example of the result:
http://docs.oracle.com/javase/7/docs/api/index.html?overview-summary.html

In Java there is the following structure (simplified):
1) 'packages' defining the hierarchy
2) 'classes' defining objects
3) 'properties' and 'methods' for every particular 'class'

I've roughly scanned the ConTeXt source structure and bringing the following
idea:

1) There is no 'packages' counterpart in ConTeXt, but it would be handy to
think of any logical structure to group commands to logical groups (even
nested). Maybe the filename title prefix could be taken as a starting point
for this.
2) 'classes' = all individual .mkiv files
3) 'methods' = all public \def, \define etc in mkiv files (+maybe other
contructs I am not aware)

Supposing that ...
a) every .mkiv file is enhanced by 'package' annotation
b) every public command/macro contains a description with the list of all
available params
(sensible subset of JavaDoc annotations)

... I can imagine to extract this info and:

A) build the static command reference website with all the hyperlinks
B) eventually generate PDF variant of the same
C) export it into the form which can be imported into the WIKI (with
read-only mode)

If every change of behaviour was reflected immediately in the description,
if any newly added parameter was added into the parameter list, the main
goal would be achieved.

But I understand there is lot of to do in the beginning... with disputable
effect in the end.

I can help with some coding, transforming ... 

Jan

Btw, would it be sufficient to scan texmf-context\tex\context\base\*.mkiv
files or there are other parts somewhere else? There are approx 500 files
now. It is huge number, but finite :-)


___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: [***SPAM***] OT: Generating Command reference from source code

[Wolfgang Schuster]

[-- Attachment #1.1: Type: text/plain, Size: 1469 bytes --]


Am 04.12.2013 um 20:11 schrieb Jan Tosovsky <j.tosovsky@email.cz>:

> On 2013-12-04 Wolfgang Schuster wrote: 
>> Am 04.12.2013 um 18:56 schrieb Jan Tosovsky <j.tosovsky@email.cz>:
>>> On 2013-12-04 Hans Hagen wrote:
>>>> 
>>>> there is a (somewhat incomplete) reference definition in xml format
>>>> so your (b) is in principle covered;
>>> 
>>> where is it stored?
>> 
>> The english interface is stored in the file cont-en.xml, the path for
>> the file on my system is
>> 
>>    /Users/wolf/context/beta/tex/texmf-
>> context/tex/context/interface/cont-en.xml
>> 
>> A PDF with all commands from the files is available here:
>> http://pragma-ade.com/show-man-10.htm
> 
> Thanks. Interesting to see what can be found on my PC :-)

You can produce the PDF yourself with "context --global x-set-12.mkiv“.

> Unfortunately there isn't even minimal description what the particular
> command does :-(

The file shows only the syntax of the command, when you want a description of the command
you have to writer this yourself. To show which keys and values are available for each command
you can load a single one of them from the big file.

> And what is the purpose of those localized versions?

ConTeXt has a multilingual interface where you can use dutch, french etc. command names, e.g.

    \starttext
    …
    \stoptext

is written as

    \starttekst
    …
    \stoptekst

when you use the dutch interface.

Wolfgang

[-- Attachment #1.2: Type: text/html, Size: 2454 bytes --]

[-- Attachment #2: Type: text/plain, Size: 485 bytes --]

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: [***SPAM***] OT: Generating Command reference from source code

[Jan Tosovsky]

On 2013-12-04 Wolfgang Schuster wrote: 
> Am 04.12.2013 um 18:56 schrieb Jan Tosovsky <j.tosovsky@email.cz>:
> > On 2013-12-04 Hans Hagen wrote:
> >>
> >> there is a (somewhat incomplete) reference definition in xml format
> >> so your (b) is in principle covered;
> >
> > where is it stored?
> 
> The english interface is stored in the file cont-en.xml, the path for
> the file on my system is
> 
>     /Users/wolf/context/beta/tex/texmf-
> context/tex/context/interface/cont-en.xml
> 
> A PDF with all commands from the files is available here:
> http://pragma-ade.com/show-man-10.htm

Thanks. Interesting to see what can be found on my PC :-)

Unfortunately there isn't even minimal description what the particular
command does :-(

And what is the purpose of those localized versions?

Jan

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: [***SPAM***] OT: Generating Command reference from source code

[Wolfgang Schuster]

Am 04.12.2013 um 18:56 schrieb Jan Tosovsky <j.tosovsky@email.cz>:

> On 2013-12-04 Hans Hagen wrote:
>> On 12/3/2013 10:59 PM, Jan Tosovsky wrote:
>>> 
>>> Supposing that ...
>>> a) every .mkiv file is enhanced by 'package' annotation
>>> b) every public command/macro contains a description with the list of
>>>   all available params (sensible subset of JavaDoc annotations)
>>> 
>>> .... I can imagine to extract this info and:
>>> 
>>> A) build the static command reference website with all the hyperlinks
>>> B) eventually generate PDF variant of the same
>>> C) export it into the form which can be imported into the WIKI (with
>>> read-only mode)
>>> 
>>> If every change of behaviour was reflected immediately in the
>>> description, if any newly added parameter was added into the parameter 
>>> list, the main goal would be achieved.
>> 
>> there is a (somewhat incomplete) reference definition in xml format so
>> your (b) is in principle covered; 
> 
> where is it stored?

The english interface is stored in the file cont-en.xml, the path for the file on my system is

    /Users/wolf/context/beta/tex/texmf-context/tex/context/interface/cont-en.xml

A PDF with all commands from the files is available here: http://pragma-ade.com/show-man-10.htm

Wolfgang


___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: [***SPAM***] OT: Generating Command reference from source code

[Jan Tosovsky]

On 2013-12-04 Hans Hagen wrote:
> On 12/3/2013 10:59 PM, Jan Tosovsky wrote:
> >
> > Supposing that ...
> > a) every .mkiv file is enhanced by 'package' annotation
> > b) every public command/macro contains a description with the list of
> >    all available params (sensible subset of JavaDoc annotations)
> >
> > .... I can imagine to extract this info and:
> >
> > A) build the static command reference website with all the hyperlinks
> > B) eventually generate PDF variant of the same
> > C) export it into the form which can be imported into the WIKI (with
> > read-only mode)
> >
> > If every change of behaviour was reflected immediately in the
> > description, if any newly added parameter was added into the parameter 
> > list, the main goal would be achieved.
> 
> there is a (somewhat incomplete) reference definition in xml format so
> your (b) is in principle covered; 

where is it stored?

> these definitions used to be in the source (more than a decade ago) but
> i moved them out because some functionality is stepwise provided 
> (not in one place) and also because many commands are generated 
> (you will not find their names by interpreting the source)

I was afraid of something like this :-(

In this case all the idea seems to be unrealistic.
 
> for documentation several methods are used:
> 
> - examples in the test suite
> - some explanation in the source
> - explanations on the wiki
> - dedicated manuals
> - articles by users (+examples)
> 

A nice overview. I would add the ConTeXt mail archive to this list ;-)

Thanks for your exhaustive insight to this topic!

Jan

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: [***SPAM***] OT: Generating Command reference from source code

[Hans Hagen]

On 12/4/2013 9:59 AM, Keith J. Schultz wrote:

> I do have a request: would it be possible to finish the section on
> overlays and stacks in the manual or at least
> start on it. I need that information because I need to use stacks inside
> of frames and these are then stacked inside of
> others. The examples I have found are for margins, pages and inside of
> tables, but are not helpful for the task for me
> at hand.

Well, I have to do all those kind of things in company time (where it 
doesn't pay off at all) and spare time (in which case it has to be fun 
doing), so some patience is needed.

Hans

-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
     tel: 038 477 53 69 | voip: 087 875 68 74 | www.pragma-ade.com
                                              | www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: [***SPAM***] OT: Generating Command reference from source code

[Keith J. Schultz]

[-- Attachment #1.1: Type: text/plain, Size: 1728 bytes --]

Hi Hans,


Am 04.12.2013 um 00:44 schrieb Hans Hagen <pragma@wxs.nl>:
[snip, snip]
> 
> 
> the problem with tex (and therefore context) is that we are dealing with quite some interaction between unpredictable user input, macro (expansions), tex-internals, resources like fonts, patterns and whatever, etc. and this is why you often need to play with mechanisms: small test files and see how it looks (tex is a visual tool)
> 
> it's often not as simple as "you feed it this" end "get back that"
	I would disagree with you on a philosophical level, but no need to go into that!
> 
> for documentation several methods are used:
> 
> - examples in the test suite
> - some explanation in the source
> - explanations on the wiki
> - dedicated manuals
> - articles by users (+examples)
> 
> although i sometimes run into impressive auto-documented documentation for languages or os commands, in practice i find myself looking for examples and/or experimenting and/or going into sources
> 
We could need some more dedicated manuals especially in the area of side effects of parameters and thier
interactions inside of context.

Most examples focus on just one or two parameters which is good as a starting point, yet many posts here
show the lack of examples for the interplay of parameters locally and globally. 

I do have a request: would it be possible to finish the section on overlays and stacks in the manual or at least
start on it. I need that information because I need to use stacks inside of frames and these are then stacked inside of
others. The examples I have found are for margins, pages and inside of tables, but are not helpful for the task for me
at hand. 

regards
	Keith.




[-- Attachment #1.2: Type: text/html, Size: 3232 bytes --]

[-- Attachment #2: Type: text/plain, Size: 485 bytes --]

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: [***SPAM***] OT: Generating Command reference from source code

[Hans Hagen]

On 12/3/2013 10:59 PM, Jan Tosovsky wrote:
> Dear All,
>
> in another thread I was dreaming ... to have always the complete and up to
> date command/macro reference with basic descriptions and list all of the
> parameter/options...
>
> 'Spoiled' by goodies offered by various Java APIs (based on
> http://en.wikipedia.org/wiki/Javadoc ) I am missing this a lot.
>
> Example of the result:
> http://docs.oracle.com/javase/7/docs/api/index.html?overview-summary.html
>
> In Java there is the following structure (simplified):
> 1) 'packages' defining the hierarchy
> 2) 'classes' defining objects
> 3) 'properties' and 'methods' for every particular 'class'
>
> I've roughly scanned the ConTeXt source structure and bringing the following
> idea:
>
> 1) There is no 'packages' counterpart in ConTeXt, but it would be handy to
> think of any logical structure to group commands to logical groups (even
> nested). Maybe the filename title prefix could be taken as a starting point
> for this.
> 2) 'classes' = all individual .mkiv files
> 3) 'methods' = all public \def, \define etc in mkiv files (+maybe other
> contructs I am not aware)
>
> Supposing that ...
> a) every .mkiv file is enhanced by 'package' annotation
> b) every public command/macro contains a description with the list of all
> available params
> (sensible subset of JavaDoc annotations)
>
> .... I can imagine to extract this info and:
>
> A) build the static command reference website with all the hyperlinks
> B) eventually generate PDF variant of the same
> C) export it into the form which can be imported into the WIKI (with
> read-only mode)
>
> If every change of behaviour was reflected immediately in the description,
> if any newly added parameter was added into the parameter list, the main
> goal would be achieved.

keep in mind that quite some functionality is the result of user 
requests and for a while can be in flux and i consider it then the users 
responsibility to wrap up on the wiki (for completely new mechanisms i 
often write a dedicated manual or an article for a user group journal)

> But I understand there is lot of to do in the beginning... with disputable
> effect in the end.
>
> I can help with some coding, transforming ...
>
> Jan
>
> Btw, would it be sufficient to scan texmf-context\tex\context\base\*.mkiv
> files or there are other parts somewhere else? There are approx 500 files
> now. It is huge number, but finite :-)

there is a (somewhat incomplete) reference definition in xml format so 
your (b) is in principle covered; these definitions used to be in the 
source (more than a decade ago) but i moved them out because some 
functionality is stepwise provided (not in one place) and also because 
many commands are generated (you will not find their names by 
interpreting the source)

the problem with tex (and therefore context) is that we are dealing with 
quite some interaction between unpredictable user input, macro 
(expansions), tex-internals, resources like fonts, patterns and 
whatever, etc. and this is why you often need to play with mechanisms: 
small test files and see how it looks (tex is a visual tool)

it's often not as simple as "you feed it this" end "get back that"

for documentation several methods are used:

- examples in the test suite
- some explanation in the source
- explanations on the wiki
- dedicated manuals
- articles by users (+examples)

although i sometimes run into impressive auto-documented documentation 
for languages or os commands, in practice i find myself looking for 
examples and/or experimenting and/or going into sources

Hans


-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
     tel: 038 477 53 69 | voip: 087 875 68 74 | www.pragma-ade.com
                                              | www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________