ntg-context - mailing list for ConTeXt users
 help / color / mirror / Atom feed
From: Taco Hoekwater <taco@elvenkind.com>
To: mailing list for ConTeXt users <ntg-context@ntg.nl>
Subject: ConTeXt source browser website update
Date: Sun, 7 Jun 2020 16:25:13 +0200	[thread overview]
Message-ID: <80B404F6-8812-42C1-8522-F17B21D4FC52@elvenkind.com> (raw)

Hi all,

I just replaced the software of https://source.contextgarden.net/ by a Lua implementation that uses Hans Hagen’s SciTE lexers for syntax highlighting and a Lua-generated search index.

There is no official documentation yet (because I may want to change some stuff still) but here is some relevant information:

* The search field uses a Lua string pattern to match a single word (no multi-word search as of yet). 

  See https://www.lua.org/manual/5.3/manual.html#6.4.1 for the syntax of lua string patterns.

* The search index only contains 7-bit ASCII ‘words’ that contain at least one of 'a-zA-Z’ (writing an LPEG for unicode matching is a tricky and probably would be quite slow even if I could make it work). 

  Still, this should be ok-ish because most of the ConTeXt source is ASCII, except for the interface files.

* The files context-en.xml, mtxrun{.lua}, and all of the files in the tex/context/patterns subtree are skipped in the search index. Also skipped are words shorter than 3 bytes, lua keywords, and words having more than (approx) 1500 hits

* On the plus side, because of the SciTE lexer, “contextversion’ and “\contextversion” are separate entries in the search index. And you can search for “\@@kldirection”.

* For text file types that have no SciTE lexer there is no syntax highlighting, and binary files are ignored altogether.

The rest of the interface should be self-explanatory.

Suggestions for improvements are welcome.


Index statistics:

4085 files processed, 178 skipped, 3744 with matches
120733 words found
119154 words exported to word.idx
37928 comment words found
36880 words exported to comment.idx
26756 documentation words found
26018 words exported to docs.idx

The word.idx is roughly 11 megabytes.

Have fun,

Taco




___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki     : http://contextgarden.net
___________________________________________________________________________________

             reply	other threads:[~2020-06-07 14:25 UTC|newest]

Thread overview: 10+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-06-07 14:25 Taco Hoekwater [this message]
2020-06-07 22:05 ` wiki update - status Garulfo
2020-06-08  7:50   ` Hans Hagen
2020-06-08 14:35   ` Marcus Vinicius Mesquita
2020-06-08 15:34   ` Aditya Mahajan
2020-06-10 12:25 ` ConTeXt source browser website update Willi Egger
2020-06-15 17:45 ` Henning Hraban Ramm
2020-07-05 19:11 ` Leah Neukirchen
2020-07-05 19:41   ` Hans Hagen
2020-07-05 21:27     ` Taco Hoekwater

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=80B404F6-8812-42C1-8522-F17B21D4FC52@elvenkind.com \
    --to=taco@elvenkind.com \
    --cc=ntg-context@ntg.nl \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).