zsh-users
 help / color / mirror / code / Atom feed
* Portability of zwc files between zsh versions
@ 2022-08-14 18:19 Glenn Morris
  2022-08-14 19:38 ` Bart Schaefer
  0 siblings, 1 reply; 4+ messages in thread
From: Glenn Morris @ 2022-08-14 18:19 UTC (permalink / raw)
  To: zsh-users


Hi,

If I use zcompile to create a digest with zsh 5.5, I find I cannot load
the resulting file with zsh 5.8 (or vice versa).

'zcompile -t' prints: "zcompile: zwc file has wrong version"

It seems from the source:
https://github.com/zsh-users/zsh/blob/master/Src/parse.c#L3250
that in order to use a zwc file, the version of zsh it was created with
must match exactly the version of zsh trying to load it?

I was expecting zwc files to be more portable than this, given the
comments in the manual:

     The created file always contains two versions of the compiled
     format, one for big-endian machines and one for small-endian
     machines. The upshot of this is that the compiled file is machine
     independent [...]

The comments in Src/parse.c make me think that the intent was that zwc
files should be loadable so long as FD_MAGIC|FD_OMAGIC haven't changed -
they last changed in 52b830353 (2001).
Ref also https://www.zsh.org/mla/workers/2000/msg01327.html

It seems that the ZSH_VERSION check was added back in d296535d3 (2007),
"23313: fix test for incorrect version in ZWC file".

I noticed this issue after zcompiling on a RHEL9 system (zsh 5.8) where
HOME is also accessible to RHEL8 (zsh 5.5) hosts. It's a bit of a
nuisance to have to maintain a separate zwc file for each zsh version.

Is it intentional that zwc files aren't portable between zsh versions?
If so, it would seem like the endian duplication wouldn't be that useful
in practice?

Thanks.


^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: Portability of zwc files between zsh versions
  2022-08-14 18:19 Portability of zwc files between zsh versions Glenn Morris
@ 2022-08-14 19:38 ` Bart Schaefer
  2022-08-14 20:16   ` Peter Stephenson
  0 siblings, 1 reply; 4+ messages in thread
From: Bart Schaefer @ 2022-08-14 19:38 UTC (permalink / raw)
  To: Glenn Morris; +Cc: zsh-users

On Sun, Aug 14, 2022 at 11:19 AM Glenn Morris <rgm@stanford.edu> wrote:
>
> The comments in Src/parse.c make me think that the intent was that zwc
> files should be loadable so long as FD_MAGIC|FD_OMAGIC haven't changed

In fact the files are not loadable in the event that any of the
wordcode definitions in parse.c have changed, such as the WC_FUNCDEF
edit in March 2020, among other possible conditions.

Possibly the right thing to do would be to remove the version check
and instead be more diligent about updating the *MAGIC values, I
suspect that the habit of doing the latter was forgotten when the
version check subsumed it.

> It's a bit of a
> nuisance to have to maintain a separate zwc file for each zsh version.

compinit already maintains separate compdumps of autoloads for each
version, so the expectation that the function bodies might need to be
re-zcompiled isn't especially outlandish?

> Is it intentional that zwc files aren't portable between zsh versions?
> If so, it would seem like the endian duplication wouldn't be that useful
> in practice?

The endian duplication is in case you have equivalent versions of zsh
on different machine architectures.  That might not be as common as it
once was.

There have also been some significant improvements in file parsing
speed, so zcompile might not be as advantageous as in the past.


^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: Portability of zwc files between zsh versions
  2022-08-14 19:38 ` Bart Schaefer
@ 2022-08-14 20:16   ` Peter Stephenson
  2022-08-14 21:05     ` Bart Schaefer
  0 siblings, 1 reply; 4+ messages in thread
From: Peter Stephenson @ 2022-08-14 20:16 UTC (permalink / raw)
  To: zsh-users

On Sun, 2022-08-14 at 12:38 -0700, Bart Schaefer wrote:
> Possibly the right thing to do would be to remove the version check
> and instead be more diligent about updating the *MAGIC values, I
> suspect that the habit of doing the latter was forgotten when the
> version check subsumed it.

Indeed, I think the shell version check was inspired by the knowledge
that we were already being less than diligent in updating the dumped
wordcode version number.

Maybe a bit of creative thinking about how we detect relevant changes is
needed.  The only real requirement is picking up incompatibilities when
we make a release.

pws




^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: Portability of zwc files between zsh versions
  2022-08-14 20:16   ` Peter Stephenson
@ 2022-08-14 21:05     ` Bart Schaefer
  0 siblings, 0 replies; 4+ messages in thread
From: Bart Schaefer @ 2022-08-14 21:05 UTC (permalink / raw)
  To: Peter Stephenson; +Cc: zsh-users

On Sun, Aug 14, 2022 at 1:16 PM Peter Stephenson
<p.w.stephenson@ntlworld.com> wrote:
>
> Maybe a bit of creative thinking about how we detect relevant changes is
> needed.  The only real requirement is picking up incompatibilities when
> we make a release.

We could drop a zcompile'd file in the Test directory and add a check
that it loads+runs correctly.  It might be a bit tricky to construct
an example that uses all possible syntax tokens, and there's a
question of how to rebuild it such that we know it wasn't just
forcibly recompiled using the current build without updating the magic
numbers.


^ permalink raw reply	[flat|nested] 4+ messages in thread

end of thread, other threads:[~2022-08-14 21:06 UTC | newest]

Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-08-14 18:19 Portability of zwc files between zsh versions Glenn Morris
2022-08-14 19:38 ` Bart Schaefer
2022-08-14 20:16   ` Peter Stephenson
2022-08-14 21:05     ` Bart Schaefer

Code repositories for project(s) associated with this public inbox

	https://git.vuxu.org/mirror/zsh/

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).