[RFC] zsh/zpython module

[RFC] zsh/zpython module

[zyx]

Wondering if I write zsh/zpython module will it be included?

This is the module that embeds python interpreter into zsh. Proposed features:

Minimal:
- zpython command similar to :python in vim.
- zsh python module capable of setting the environment variables.
Intended:
- zpython command
- zsh python module capable of
  - getting and setting local, global and exported variables, including special ones
  - defining zsh commands
  - defining “magic variables” (those that are running python code when their value is requested)
  - defining zle widgets

Reasoning:
1. there are some jobs that when implemented using shell scripting or external commands are very time-consuming. I know two examples of this: zsh-syntax-highlighting and new powerline prompt (it has very noticeable delay in its current status, but even just `PS1='$(python -c "print 2")'` has noticeable delay).
2. I know no other good ways of doing this: if I put powerline into background all IPC I can imagine is a crap:
  - pipes are not available;
  - zsh/zpty is an overkill;
  - coproc can run only one process in background;
  - UNIX sockets/FIFO files require some place you can write to on filesystem, require some effort to generate unique names, require garbage-collecting when shell quits or stops using prompt, require collecting stale files after a crash.
3. Any method is going to lose performance for IPC: fist you need to tell that new prompt is needed and update changed environment variables (like `$?`), then wait until kernel wakes the background process up and it will generate the prompt, then wait until kernel wakes your process up to get and display new value. Zsh/zpython is running python in the same process, no need in IPC. And Vim has already shown that powerline performs pretty well in this case.

I will welcome any comments about where I should look first when writing this module, what else I could include and what I could not because it is not easily doable.


------------
http://titov.by - Путь выхода из кризиса!

Re: [RFC] zsh/zpython module

[Peter Stephenson]

On Wed, 23 Jan 2013 21:20:46 +0300 (FET)
zyx@np.by wrote:
> Wondering if I write zsh/zpython module will it be included?

If it's compact enough, we could include it, probably turned off by
default to avoid inflicting configuration problems on people who don't
need it.

I think the key things it would need would be a way of accessing shell
variables, including special variables, and a way of calling shell
functions and builtins.  None of those should be particularly difficult.

-- 
Peter Stephenson <p.w.stephenson@ntlworld.com>
Web page now at http://homepage.ntlworld.com/p.w.stephenson/

Re: [RFC] zsh/zpython module

[zyx]

  > On Wed, 23 Jan 2013 21:20:46 +0300 (FET)
> zyx@np.by wrote:
>> Wondering if I write zsh/zpython module will it be included?
>
> If it's compact enough, we could include it, probably turned off by
> default to avoid inflicting configuration problems on people who don't
> need it.
>
> I think the key things it would need would be a way of accessing shell
> variables, including special variables, and a way of calling shell
> functions and builtins.  None of those should be particularly difficult.

MInimal working version is here: https://bitbucket.org/ZyX_I/zsh/src/zpython (branch zpython) (supports only zpython built-in and zsh.eval() in zsh module). Wondering whether there exists a list of API functions I can use in my module. I failed to figure out how to get variable values alongside with their type. Did not try functions and builtins.


------------
http://titov.by - Путь выхода из кризиса!

Re: [RFC] zsh/zpython module

[zyx]

 >   > On Wed, 23 Jan 2013 21:20:46 +0300 (FET)
>> zyx@np.by wrote:
>>> Wondering if I write zsh/zpython module will it be included?
>>
>> If it's compact enough, we could include it, probably turned off by
>> default to avoid inflicting configuration problems on people who don't
>> need it.
>>
>> I think the key things it would need would be a way of accessing shell
>> variables, including special variables, and a way of calling shell
>> functions and builtins.  None of those should be particularly difficult.
>
> MInimal working version is here:
> https://bitbucket.org/ZyX_I/zsh/src/zpython (branch zpython) (supports
> only zpython built-in and zsh.eval() in zsh module). Wondering whether
> there exists a list of API functions I can use in my module. I failed to
> figure out how to get variable values alongside with their type. Did not
> try functions and builtins.

I guess I need getvalue[1] and the same switch as here[2] to get parameter with its type. Am I right that I have to use only functions that are prefixed with “mod_export”?

[1]: https://bitbucket.org/ZyX_I/zsh.git/src/8c95eaf152e222beb5538d77fd9b12b445956b7a/Src/params.c#cl-1816
[2]: https://bitbucket.org/ZyX_I/zsh.git/src/8c95eaf152e222beb5538d77fd9b12b445956b7a/Src/params.c#cl-1956

Re: [RFC] zsh/zpython module

[Peter Stephenson]

On Mon, 28 Jan 2013 07:44:35 +0300 (FET)
zyx@np.by wrote:
> I guess I need getvalue[1] and the same switch as here[2] to get
> parameter with its type.

getvalue's the right way in if you're supporting accesses to partial
arrays, for example, although there ought to be simpler ways of
e.g. simply retrieving a pamraeter expression as a string, as is done in
other modules -- although most other modules make more limited use of
parameters, and the one that doesnt, zsh/parameter, is mostly there to
define new parameters so isn't much use as a reference.  What
zsh/parameter might help with is showing you how to access functions
etc. since some of the parameters give you back doors for manipluating
them.

>  Am I right that I have to use only functions that are prefixed with
> “mod_export”?

Yes, however there's no major penalty in exporting other functions.  We
never sat down and defined a complete API, so if you need to mark
another function as mod_export that's OK.

In practice, mod_export may be largely redundant: of the systems I've
encountered, only AIX actually needs the functions marking specially for
the linker, which is what mod_export is for.

pws

Re: [RFC] zsh/zpython module

[zyx]

Wondering whether there is a way to call some function automatically when zsh forks. I think can cope without this, but code like

    if (zsh_subshell > zpython_subshell) {
        zpython_subshell = zsh_subshell;
        PyOS_AfterFork();
    }
that needs to be added to every topmost function that may invoke python interpreter (now there is only one, but if I figure out how to define magic variables, other builtins and whatever may be defined to conveniently interact with python in C without hardcoding them as features there will be much more) does not seem like the best way to cope with this.


------------
http://titov.by - Путь выхода из кризиса!

Re: [RFC] zsh/zpython module

[Peter Stephenson]

On Mon, 28 Jan 2013 19:57:49 +0300 (FET)
zyx@np.by wrote:
> Wondering whether there is a way to call some function automatically
> when zsh forks.

I think you'd probably need a new hook.  There's a mechanism for this,
widely used by completion, and it's probably not that heavyweight, but
I've never dealt directly with it.

pws