ConTeXt source browser website update

ConTeXt source browser website update

[Taco Hoekwater]

Hi all,

I just replaced the software of https://source.contextgarden.net/ by a Lua implementation that uses Hans Hagen’s SciTE lexers for syntax highlighting and a Lua-generated search index.

There is no official documentation yet (because I may want to change some stuff still) but here is some relevant information:

* The search field uses a Lua string pattern to match a single word (no multi-word search as of yet). 

  See https://www.lua.org/manual/5.3/manual.html#6.4.1 for the syntax of lua string patterns.

* The search index only contains 7-bit ASCII ‘words’ that contain at least one of 'a-zA-Z’ (writing an LPEG for unicode matching is a tricky and probably would be quite slow even if I could make it work). 

  Still, this should be ok-ish because most of the ConTeXt source is ASCII, except for the interface files.

* The files context-en.xml, mtxrun{.lua}, and all of the files in the tex/context/patterns subtree are skipped in the search index. Also skipped are words shorter than 3 bytes, lua keywords, and words having more than (approx) 1500 hits

* On the plus side, because of the SciTE lexer, “contextversion’ and “\contextversion” are separate entries in the search index. And you can search for “\@@kldirection”.

* For text file types that have no SciTE lexer there is no syntax highlighting, and binary files are ignored altogether.

The rest of the interface should be self-explanatory.

Suggestions for improvements are welcome.


Index statistics:

4085 files processed, 178 skipped, 3744 with matches
120733 words found
119154 words exported to word.idx
37928 comment words found
36880 words exported to comment.idx
26756 documentation words found
26018 words exported to docs.idx

The word.idx is roughly 11 megabytes.

Have fun,

Taco




___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki     : http://contextgarden.net
___________________________________________________________________________________

wiki update - status

[Garulfo]

Hi all,

New source browser is nice ! Thanks a lot.

After level 1 (Main Page / Download and Install / Documentation / 
Mailing lists), I mostly finished a review of level 2 (and some level 3) 
pages.

Several tweaks all over the place like:

- Fonts: full update for newcomers,
- Try to be responsive "enough" for smartphone
- More homogeneous style / theme
- Old Content category and tag pages, update dead links
- Layout:
   - do the todo, like merging content of paper setup + paper sizes pages
   - complete the flow with imposition
   - rework the "Further reading section", add PDF links and so on
- Programming & DB:
   - Gather "Calling Lua from TeX" and "Calling TeX from Lua"
     (section +links) in a same page
- Tools:
   - segregate "core tools" and "additional tools"
   - highlight existence of mtxrun and context scripts pages
- Remove former "manually made links bar" at the top of the pages
   current structure should be enough now have an easier understanding of
   the wiki structure.

Thanks for your help Taco, Hans, Aditya, Hraban, Tomáš.

I hope that this will help to improve the reception of users and their 
willingness to contribute.

Bye.
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: wiki update - status

[Hans Hagen]

On 6/8/2020 12:05 AM, Garulfo wrote:
> Hi all,
> 
> New source browser is nice ! Thanks a lot.
> 
> After level 1 (Main Page / Download and Install / Documentation / 
> Mailing lists), I mostly finished a review of level 2 (and some level 3) 
> pages.
> 
> Several tweaks all over the place like:
> 
> - Fonts: full update for newcomers,
> - Try to be responsive "enough" for smartphone
> - More homogeneous style / theme
> - Old Content category and tag pages, update dead links
> - Layout:
>    - do the todo, like merging content of paper setup + paper sizes pages
>    - complete the flow with imposition
>    - rework the "Further reading section", add PDF links and so on
> - Programming & DB:
>    - Gather "Calling Lua from TeX" and "Calling TeX from Lua"
>      (section +links) in a same page
> - Tools:
>    - segregate "core tools" and "additional tools"
>    - highlight existence of mtxrun and context scripts pages
> - Remove former "manually made links bar" at the top of the pages
>    current structure should be enough now have an easier understanding of
>    the wiki structure.
> 
> Thanks for your help Taco, Hans, Aditya, Hraban, Tomáš.
> 
> I hope that this will help to improve the reception of users and their 
> willingness to contribute.
I'm really impressed by all the progress that has been made!

Hans

-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
        tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: wiki update - status

[Marcus Vinicius Mesquita]

[-- Attachment #1.1: Type: text/plain, Size: 1948 bytes --]

So much better! Thank you very much for the effort!

Greetings

Marcus Vinicius

On Sun, Jun 7, 2020, 19:05 Garulfo <garulfo@azules.eu> wrote:

> Hi all,
>
> New source browser is nice ! Thanks a lot.
>
> After level 1 (Main Page / Download and Install / Documentation /
> Mailing lists), I mostly finished a review of level 2 (and some level 3)
> pages.
>
> Several tweaks all over the place like:
>
> - Fonts: full update for newcomers,
> - Try to be responsive "enough" for smartphone
> - More homogeneous style / theme
> - Old Content category and tag pages, update dead links
> - Layout:
>    - do the todo, like merging content of paper setup + paper sizes pages
>    - complete the flow with imposition
>    - rework the "Further reading section", add PDF links and so on
> - Programming & DB:
>    - Gather "Calling Lua from TeX" and "Calling TeX from Lua"
>      (section +links) in a same page
> - Tools:
>    - segregate "core tools" and "additional tools"
>    - highlight existence of mtxrun and context scripts pages
> - Remove former "manually made links bar" at the top of the pages
>    current structure should be enough now have an easier understanding of
>    the wiki structure.
>
> Thanks for your help Taco, Hans, Aditya, Hraban, Tomáš.
>
> I hope that this will help to improve the reception of users and their
> willingness to contribute.
>
> Bye.
>
> ___________________________________________________________________________________
> If your question is of interest to others as well, please add an entry to
> the Wiki!
>
> maillist : ntg-context@ntg.nl /
> http://www.ntg.nl/mailman/listinfo/ntg-context
> webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
> archive  : https://bitbucket.org/phg/context-mirror/commits/
> wiki     : http://contextgarden.net
>
> ___________________________________________________________________________________
>

[-- Attachment #1.2: Type: text/html, Size: 3009 bytes --]

[-- Attachment #2: Type: text/plain, Size: 493 bytes --]

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: wiki update - status

[Aditya Mahajan]

On Mon, 8 Jun 2020, Garulfo wrote:

> - Remove former "manually made links bar" at the top of the pages
>   current structure should be enough now have an easier understanding of
>   the wiki structure.

Along that line, we should also considering fixing the getting started navbox or removing it altogether. It used to have a nice visual layout, but that was lost when upgrading the wiki. In the current form, it uses too much vertical space and it might be redundant with the new wiki structure.

Aditya
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: ConTeXt source browser website update

[Willi Egger]

This is really a nice interface! Thank you so much for all this work!

Willi

> On 7 Jun 2020, at 16:25, Taco Hoekwater <taco@elvenkind.com> wrote:
> 
> Hi all,
> 
> I just replaced the software of https://source.contextgarden.net/ by a Lua implementation that uses Hans Hagen’s SciTE lexers for syntax highlighting and a Lua-generated search index.
> 
> There is no official documentation yet (because I may want to change some stuff still) but here is some relevant information:
> 
> * The search field uses a Lua string pattern to match a single word (no multi-word search as of yet). 
> 
> See https://www.lua.org/manual/5.3/manual.html#6.4.1 for the syntax of lua string patterns.
> 
> * The search index only contains 7-bit ASCII ‘words’ that contain at least one of 'a-zA-Z’ (writing an LPEG for unicode matching is a tricky and probably would be quite slow even if I could make it work). 
> 
> Still, this should be ok-ish because most of the ConTeXt source is ASCII, except for the interface files.
> 
> * The files context-en.xml, mtxrun{.lua}, and all of the files in the tex/context/patterns subtree are skipped in the search index. Also skipped are words shorter than 3 bytes, lua keywords, and words having more than (approx) 1500 hits
> 
> * On the plus side, because of the SciTE lexer, “contextversion’ and “\contextversion” are separate entries in the search index. And you can search for “\@@kldirection”.
> 
> * For text file types that have no SciTE lexer there is no syntax highlighting, and binary files are ignored altogether.
> 
> The rest of the interface should be self-explanatory.
> 
> Suggestions for improvements are welcome.
> 
> 
> Index statistics:
> 
> 4085 files processed, 178 skipped, 3744 with matches
> 120733 words found
> 119154 words exported to word.idx
> 37928 comment words found
> 36880 words exported to comment.idx
> 26756 documentation words found
> 26018 words exported to docs.idx
> 
> The word.idx is roughly 11 megabytes.
> 
> Have fun,
> 
> Taco
> 
> 
> 
> 
> ___________________________________________________________________________________
> If your question is of interest to others as well, please add an entry to the Wiki!
> 
> maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
> webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
> archive  : https://bitbucket.org/phg/context-mirror/commits/
> wiki     : http://contextgarden.net
> ___________________________________________________________________________________

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: ConTeXt source browser website update

[Henning Hraban Ramm]

> Am 07.06.2020 um 16:25 schrieb Taco Hoekwater <taco@elvenkind.com>:
> 
> 
> * On the plus side, because of the SciTE lexer, “contextversion’ and “\contextversion” are separate entries in the search index. And you can search for “\@@kldirection”.
> 


This is quite great!
I was often annoyed that I couldn’t really search for commands that are also regular words.

Thank you so much!

And thanks again to Garulfo for the whole effort of restructuring the wiki!


Hraban
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: ConTeXt source browser website update

[Leah Neukirchen]

Taco Hoekwater <taco@elvenkind.com> writes:

> Hi all,
>
> I just replaced the software of https://source.contextgarden.net/ by a
> Lua implementation that uses Hans Hagen’s SciTE lexers for syntax
> highlighting and a Lua-generated search index.

> Suggestions for improvements are welcome.

The download feature for binary files is broken, at least a leading
newline is prefixed to the download.  But even then
https://source.contextgarden.net/source/luametatex/luametatex-source.tar.xz
is corrupted.

Cheers,
-- 
Leah Neukirchen  <leah@vuxu.org>  https://leahneukirchen.org/

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: ConTeXt source browser website update

[Hans Hagen]

On 7/5/2020 9:11 PM, Leah Neukirchen wrote:
> Taco Hoekwater <taco@elvenkind.com> writes:
> 
>> Hi all,
>>
>> I just replaced the software of https://source.contextgarden.net/ by a
>> Lua implementation that uses Hans Hagen’s SciTE lexers for syntax
>> highlighting and a Lua-generated search index.
> 
>> Suggestions for improvements are welcome.
> 
> The download feature for binary files is broken, at least a leading
> newline is prefixed to the download.  But even then
> https://source.contextgarden.net/source/luametatex/luametatex-source.tar.xz
> is corrupted.
just ignore it ... it's just a left over crap file from some test

Hans

-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
        tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: ConTeXt source browser website update

[Taco Hoekwater]

Hi,

> On 5 Jul 2020, at 21:41, Hans Hagen <j.hagen@xs4all.nl> wrote:
> 
> On 7/5/2020 9:11 PM, Leah Neukirchen wrote:
>> Taco Hoekwater <taco@elvenkind.com> writes:
>>> Hi all,
>>> 
>>> I just replaced the software of https://source.contextgarden.net/ by a
>>> Lua implementation that uses Hans Hagen’s SciTE lexers for syntax
>>> highlighting and a Lua-generated search index.
>>> Suggestions for improvements are welcome.
>> The download feature for binary files is broken, at least a leading
>> newline is prefixed to the download. 
Binary downloads fixed.

Thanks for the report,
Taco
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki     : http://contextgarden.net
___________________________________________________________________________________