From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.io/gmane.comp.tex.context/94632 Path: news.gmane.org!not-for-mail From: Philipp Gesang Newsgroups: gmane.comp.tex.context Subject: [lhf@tecgraf.puc-rio.br: [ANN] ltokenp, a token processor for Lua] Date: Wed, 4 May 2016 08:28:58 +0200 Message-ID: <20160504062858.GB23376@phlegethon> Reply-To: mailing list for ConTeXt users NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============8978511296164630770==" X-Trace: ger.gmane.org 1462343366 415 80.91.229.3 (4 May 2016 06:29:26 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Wed, 4 May 2016 06:29:26 +0000 (UTC) To: ConTeXt ML Original-X-From: ntg-context-bounces@ntg.nl Wed May 04 08:29:12 2016 Return-path: Envelope-to: gctc-ntg-context-518@m.gmane.org Original-Received: from zapf.boekplan.nl ([5.39.185.232] helo=zapf.ntg.nl) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1axqJ4-0000V0-OI for gctc-ntg-context-518@m.gmane.org; Wed, 04 May 2016 08:29:10 +0200 Original-Received: from localhost (localhost [127.0.0.1]) by zapf.ntg.nl (Postfix) with ESMTP id C2B60B9E6; Wed, 4 May 2016 08:28:44 +0200 (CEST) X-Virus-Scanned: Debian amavisd-new at zapf.boekplan.nl Original-Received: from zapf.ntg.nl ([127.0.0.1]) by localhost (zapf.ntg.nl [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id xp2RCOGxJmZd; Wed, 4 May 2016 08:28:40 +0200 (CEST) Original-Received: from zapf.ntg.nl (localhost [IPv6:::1]) by zapf.ntg.nl (Postfix) with ESMTP id 5D15CB9ED; Wed, 4 May 2016 08:28:40 +0200 (CEST) Original-Received: from localhost (localhost [127.0.0.1]) by zapf.ntg.nl (Postfix) with ESMTP id 06FBEB9E8 for ; Wed, 4 May 2016 08:28:40 +0200 (CEST) X-Virus-Scanned: Debian amavisd-new at zapf.boekplan.nl Original-Received: from zapf.ntg.nl ([127.0.0.1]) by localhost (zapf.ntg.nl [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id JPCSm4JXywjV for ; Wed, 4 May 2016 08:28:35 +0200 (CEST) Original-Received: from relay2.uni-heidelberg.de (relay2.uni-heidelberg.de [129.206.119.212]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by zapf.ntg.nl (Postfix) with ESMTPS id D2A1FB9E6 for ; Wed, 4 May 2016 08:28:25 +0200 (CEST) Original-Received: from ix.urz.uni-heidelberg.de (cyrus-portal01.urz.uni-heidelberg.de [129.206.100.176]) by relay2.uni-heidelberg.de (8.13.8/8.13.8) with ESMTP id u446SO93021937; Wed, 4 May 2016 08:28:24 +0200 Original-Received: from extmail.urz.uni-heidelberg.de (extmail.urz.uni-heidelberg.de [129.206.100.140]) by ix.urz.uni-heidelberg.de (Postfix) with ESMTPS id 4067A21DF34D; Wed, 4 May 2016 08:28:24 +0200 (CEST) Original-Received: from localhost (p5DED4FCD.dip0.t-ipconnect.de [93.237.79.205]) (authenticated bits=0) by extmail.urz.uni-heidelberg.de (8.13.4/8.13.1) with ESMTP id u446SNLJ010905 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Wed, 4 May 2016 08:28:23 +0200 Mail-Followup-To: ConTeXt ML X-RFC-6648: Please ignore, this header is deprecated. User-Agent: Mutt/1.5.23 (2014-03-12) X-BeenThere: ntg-context@ntg.nl X-Mailman-Version: 2.1.16 Precedence: list List-Id: mailing list for ConTeXt users List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ntg-context-bounces@ntg.nl Original-Sender: "ntg-context" Xref: news.gmane.org gmane.comp.tex.context:94632 Archived-At: --===============8978511296164630770== Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="neYutvxvOLaeuPCA" Content-Disposition: inline --neYutvxvOLaeuPCA Content-Type: multipart/mixed; boundary="x+6KMIRAuhnl3hBn" Content-Disposition: inline --x+6KMIRAuhnl3hBn Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hi Hans, all, this was just announced on lua-l. Might be relevant to Context since the advertised functionality intersects with mtx-package / util-mrg.lua: Potential uses include: - Compressing Lua programs by removing comments and whitespace (strip.l= ua) - Removing assertions (assert.lua) - Adding new syntax sugar (self.lua) - Experimenting with new syntax without hacking the Lua source (reserve= d.lua) I especially like the idea of having some kind of a preprocessor. The original announcement can be found at Gmane in case the attachment is eaten by the list: http://thread.gmane.org/gmane.comp.lang.lua.general/121329 Have fun, Philipp ----- Forwarded message from Luiz Henrique de Figueiredo ----- Date: Wed, 4 May 2016 02:17:22 -0300 =46rom: Luiz Henrique de Figueiredo To: lua-l@lists.lua.org Subject: [ANN] ltokenp, a token processor for Lua User-Agent: Mutt/1.5.20 (2009-12-10) ----- End forwarded message ----- --x+6KMIRAuhnl3hBn Content-Type: text/plain; charset=us-ascii Content-Disposition: inline ltokenp is now available at http://www.tecgraf.puc-rio.br/~lhf/ftp/lua/5.3/ltokenp.tar.gz It should work for Lua 5.1, 5.2, and 5.3. The README is attached. Enjoy. All feedback welcome. --lhf ltokenp is a token processor for Lua: it allows you to process the stream of tokens coming from the Lua lexer. Potential uses include: - Compressing Lua programs by removing comments and whitespace (strip.lua) - Removing assertions (assert.lua) - Adding new syntax sugar (self.lua) - Experimenting with new syntax without hacking the Lua source (reserved.lua) ltokenp accepts Lua scripts to run and files to process. The scripts and files are executed and processed as seen. Each script appears as a separate argument after '-s', one '-s' per script. Scripts typically output the contents of the files with some modifications. Unfortunately, all comments and whitespace are eaten by the lexer and never reach the token stream. A typical usage is ltokenp -s script.lua [file.lua ...] but you can also do ltokenp -s s1.lua f1.lua -s s2.lua f2.lua A global function named FILTER will be called once for each token seen in the files. So, scripts typically define FILTER. If no scripts are given, ltokenp just dumps the token stream with this: function FILTER(line,token,text,value) print(line,token,text,value) end As can been above, the FILTER function receives 4 arguments from the lexer: - the line number where the token appears - the token as a number - the token as text - the value of names, numbers, and strings; for other tokens, the value is the same as the text. To try ltokenp, just edit Makefile to reflect your installation of Lua and then run make. This will build ltokenp and run a simple test. Note that ltokenp needs inside information from the Lua private headers and so it needs to be built using a Lua build directory, not just the usual installation directories. This code is hereby placed in the public domain. Please send comments, suggestions, and bug reports to lhf@tecgraf.puc-rio.br . --x+6KMIRAuhnl3hBn-- --neYutvxvOLaeuPCA Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBCgAGBQJXKZapAAoJELdcG6d+hz92p2gP/RNyz2Pa7N6skX3ICmCliSZg ns6RRU5RR+8IHPy86mKvc4cfQo54Smhg71rQy6jCFaLEHpOPVDi12E1td8EpUHv+ FqRchDVmel7Wlw1ESQw1dBzC5VgyKthSuHGK+oOUXvOr+M9DNOGaac7803PfPmhg 6toHKo1jgpGmHu/m+/TjTP+U6VsjPbQpcQRAGpH9abYy02CM7C4vB/SHuP986L70 nW6UrZ0cVSMJ1dfNu12wiodesCeMamk9QLV6yS2ewsyAPKvLKAE5LzST8gmcOXvu v50L3/xTIeM17bRVjwBmFktC0hZhytvJb+pTsKR8KKPgXyRl5q1ndGIBGp5pY+HP LDiFjvB0ysza3nN/66GEPg3IrdIB/gNXZdGH1uwZwrFbqZLjK6uxdHjhh9PVdOMr g09b+Qpen5vXKyHdkzjoOBEwyNlxmLlugLAV0gCJpvUzWmJio4qtETTqoNZT9/0d Xkk+zSnsQgmMZ2RVBXmtmB3q10lCciw6Wprt31Jw6RAxW6/dq6eAv3p1Sz4r1DH3 nBqsnGiXvsvufp3ZJyhYvbqN2f4CDNsKARGzOrSEWNLfvAZWABdNLMAsd1O7nMEG ji1kna/rXDP07qnvmDtvH7XxfIkE9XRvh0NCB8vOS7p89Ehf8yBHbHkMFWVp6y6C MjHOsQ6R4eky3itUsTQD =fW9Z -----END PGP SIGNATURE----- --neYutvxvOLaeuPCA-- --===============8978511296164630770== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: inline X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX18KSWYgeW91ciBxdWVzdGlvbiBpcyBvZiBpbnRlcmVz dCB0byBvdGhlcnMgYXMgd2VsbCwgcGxlYXNlIGFkZCBhbiBlbnRyeSB0byB0aGUgV2lraSEKCm1h aWxsaXN0IDogbnRnLWNvbnRleHRAbnRnLm5sIC8gaHR0cDovL3d3dy5udGcubmwvbWFpbG1hbi9s aXN0aW5mby9udGctY29udGV4dAp3ZWJwYWdlICA6IGh0dHA6Ly93d3cucHJhZ21hLWFkZS5ubCAv IGh0dHA6Ly90ZXguYWFuaGV0Lm5ldAphcmNoaXZlICA6IGh0dHA6Ly9mb3VuZHJ5LnN1cGVsZWMu ZnIvcHJvamVjdHMvY29udGV4dHJldi8Kd2lraSAgICAgOiBodHRwOi8vY29udGV4dGdhcmRlbi5u ZXQKX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX18= --===============8978511296164630770==--