Re: Can ZWC be optimized, for lesser depths of recursive exec* calls?

[Sebastian Gniazdowski <psprint@zdharma.org>]

On 20.07.2017 at 19:40:33, Bart Schaefer (schaefer@brasslantern.com) wrote:
> To some extent you're having a deeper variation of the same confusion
> that Ray Andrews sometimes has, namely, distinguishing characteristics
> of an interpreted language from a compiled one. With a true compiled
> language, the syntax and semantics can both be analyzed at compile
> time, but in an interpreted language the semantics aren't fully
> expressed until run time. In this case the chain of exec.c function
> calls traces the semantics of the language -- reconstructing the parse
> tree, traversing it, and executing it, all at the same time.

Today instead of looking at large OS X Instruments call tree, I've added function-name-printing to each *exec* (plus addvars) function in exec.c. This way execution looks much more normal and exec.c code seems quite compact now, not so many such functions there. Basically following repeats for any execution-code item (not e.g. function-definition item):
  execlist -> execpline -> execpline2 -> execcmd_exec.

Function execsimple() can do assignment and function-definition (there's also some general-purpose else {} but it's rather not used, as control should go general-purpose earlier, before execsimple, if needed). Didn't track WC_SIMPLE yet.

I have a hard time establishing where CPU-time dissipates. It's a gradual process, and exec.c functions happen to be long. However I see that my requests to help in optimizing zwc have been rather asking for doing something with the 4 above typical calls, and yeah, they're typical, rather nothing to do with them.

> This is crucial for accurately reproducing the original script flow in
> xtrace output. Simplifying the ZWC would amount to rewriting the
> script -- looking at xtrace output would be like stepping through an
> optimized C program with a debugger: the program counter would no
> longer correspond to the source.

I have this automated. (z) flag and state tracking, and voila, automatic function extraction:

% zplg lexicon convert zdharma/fast-syntax-highlighting
Extracted  47-line function `_zsh_highlight'...
Extracted  21-line function `_zsh_highlight_apply_zle_highlight'...
Extracted  34-line function `_zsh_highlight_bind_widgets'...
Extracted   2-line function `_zsh_highlight_buffer_modified'...
Extracted   2-line function `_zsh_highlight_call_widget'...
Extracted   2-line function `_zsh_highlight_cursor_moved'...
Extracted   3-line function `_zsh_highlight_preexec_hook'...
Generated preamble ( 21 lines) # this is .plugin.zsh file stripped of funs

% pwd
/Users/sgniazdowski/.zplugin/lexicon

% ls
_zsh_highlight                     _zsh_highlight_call_widget
_zsh_highlight_apply_zle_highlight _zsh_highlight_cursor_moved
_zsh_highlight_bind_widgets        _zsh_highlight_preexec_hook
_zsh_highlight_buffer_modified

% file ../lexicon.zwc
../lexicon.zwc: data

So if user wants xtrace, he can revert to-lexicon-conversion and run plugin normally. The problem is that lexicon seems to be 2-3 ms slower. Not much, can treat this as the-same-speed, however, it should be faster to justify doing this. Or at least have some not-speed-related features. Seeing plugin's functions as files is cool, but I need something more.

--  
Sebastian Gniazdowski
psprint /at/ zdharma.org