How does zsh internally store and process variables?

How does zsh internally store and process variables?

[vlkr1]

[-- Attachment #1: Type: text/plain, Size: 1697 bytes --]

Hey there

I am helping revive with a rust crate/library (https://github.com/Diegovsky/zsh-module-rs) that allows rust programmers to make zsh modules, you know, the ones you load like zmodload zsh/modulename​.

One of the things that we are having trouble with are internal variables. We can use std::env::var("VARIABLE")​ to get environment variables, but only if they are exported by the shell. For example, you would have to run export ZSH_VERSION​ before it would show up, since rust's env::var()​ function was only designed to handle exported scalar environment variables.

We do have a way to make the shell do this, basically calling the zsh builtin eval​ to force it to export either these variables or dummy ones, and then running eval​ again to unset them, but this is a pretty janky hack -- what if the user had important data stored in the internal variable $YG76987HBHLUBHJ​, or what if they didn't really want to export PS1​? Then you end up looping eval​s to try and find empty vars to throw your 5KB associative array in, or just writing shell in a multiline string to run callbacks and the task is needlessly complex.

Anyways, we wanted to implement internal variable inspection, but was very confused trying to understand the code. We would like to know the following:

- how zsh handles its variables
- how its variables are exported (setenv?)
- If the keys and values are copied or expected to be leaked/alloccd with malloc by the caller

Could you help us out here?

"Don't believe everything you read on the internet just because there's a picture with a fancy quote next to it."- Abraham Lincoln

Sent with [Proton Mail](https://proton.me/) secure email.

[-- Attachment #2: Type: text/html, Size: 3727 bytes --]

Re: How does zsh internally store and process variables?

[Bart Schaefer]

On Sat, Jan 27, 2024 at 2:00 PM vlkr1 <vlkr1@proton.me> wrote:
>
> One of the things that we are having trouble with are internal variables.

Have you looked at Etc/zsh-development-guide in the zsh distribution?

There's a section on "Parameters" and how modules should make use of
them.  It's mostly focused on how modules can add their own parameters
rather than on how to access existing parameters, but the basics of
the structures and accessors are all explained (albeit with pointers
to zsh.h for structure definitions).

As a side-effect of evolutionary development, zsh uses a single global
pointer "paramtab" for most of the get/set functions, so it's not
threading-safe code and signal handling has to be carefully managed.
A second global pointer "realparamtab" points to the actual table for
all the variables, so you'll see tests for "paramtab == realparamtab"
in various places.  Internally, associative array key/value pairs are
implemented as parameter tables, so paramtab may point to the
internals of an associative array in some cases, where the associative
array itself is contained in realparamtab (we don't presently allow
them to nest).

In the current development git head, the files
Src/Modules/param_private.c and Src/Modules/ksh93.c may provide some
useful coding hints, although neither is exactly what you're after.

> how its variables are exported (setenv?)

There's a preprocessor definitions for HAVE_PUTENV and
USE_SET_UNSET_ENV.  Primarily these apply in the zgetenv(), zputenv(),
and delenv() functions defined in Src/params.c which are what you
should call, but there are lower-level functions if you really need
them or you can use #ifdef with those macros to choose the system
library routines.

> If the keys and values are copied or expected to be leaked/alloccd with malloc by the caller

Keys are managed by the functions in Src/params.c, so you can pass
constant strings e.g. as the first argument of setsparam() or the
other parameter functions.  String values (second parameter of
setsparam()) should be allocated by the caller but are freed by
Src/params.c when the value changes or is unset.  Values represented
by numeric types (int, float) are not memory-managed.

String values also need to be "metafied".  If you are copying them
from another parameter, they are likely already metafied.  If coming
from user input or the like, use for example ztrdup_metafy() to
allocate a metafied copy for setsparam().