A way to find a bug (or explain a missuse of) realpath expansion?

A way to find a bug (or explain a missuse of) realpath expansion?

[Marc Chantreux]

hello,

The /usr shade of path will disapear from debian distribution so
right now we have:

	realpath /usr/bin /usr/bin/ /bin /bin/| sort -u
	# /usr/bin

Let's forget extra / for the moment: I can get the same result with:

	foo=( {/usr,}/bin )
	foo=( ${foo%/} )
	foo=( $foo(:A) )
	print -l ${(u)foo}
	# /usr/bin

but I don't know how to oneline it:

	foo=( {/usr,}/bin )
	print -l ${(u):-${foo%/}(:A)}
	# /usr/bin
	# /usr/bin

neither I can deal with backslashed twins.

	foo=( {/usr,}/bin{,/} ) ; print -l ">>>> setting foo" $foo
	foo=( ${foo%/} )        ; print -l ">>>> remove tailing /" $foo
	foo=( $foo(:A) )        ; print -l ">>>> realpath" $foo
	foo=( ${(u)foo} )       ; print -l ">>>> uniq" $foo

	>>>> setting foo
	/usr/bin
	/usr/bin/
	/bin
	/bin/
	>>>> remove tailing /
	/usr/bin
	/usr/bin
	/bin
	/bin
	>>>> realpath
	/usr/bin
	/usr/bin
	/bin
	/usr/bin
	>>>> uniq
	/usr/bin
	/bin

I also realize I post those kind of questions for 20 years now just
because the zsh community is friendly enough to reply again and again:
I still don't know how to give a closer look to the way zsh work to fix
it by myself. To me the next step seems to be:

* compile my own version of zsh with symbols (from apt source and
  dpkg-buildpackage?)
* run a gdb session and start to explore the code

Is it? is there still something easier to try at this point? I want to
learn fishery.

regards,
-- 
Marc Chantreux
Pôle CESAR (Calcul et services avancés à la recherche)
Université de Strasbourg
14 rue René Descartes,
BP 80010, 67084 STRASBOURG CEDEX
03.68.85.60.79

Re: A way to find a bug (or explain a missuse of) realpath expansion?

[Bart Schaefer]

On Tue, Apr 30, 2024 at 6:59 AM Marc Chantreux <mc@unistra.fr> wrote:
>
> The /usr shade of path will disapear from debian distribution

Can you provide a reference for this?  Web search is clogged with
reports of accidental deletions of the /usr tree, etc.

>         foo=( {/usr,}/bin )
>         print -l ${(u):-${foo%/}(:A)}

You're trying to use a glob qualifier form (:A) but you're not doing
globbing, you're doing parameter expansion.  Just use the
colon-modifier directly:

print -l ${(u):-${foo:A}}

> neither I can deal with backslashed twins.

The above handles that, too.  You don't need to explicitly remove the
trailing slashes yourself, :A will normalize them and then (u) will
collapse all four results to one.

> I still don't know how to give a closer look to the way zsh work to fix
> it by myself. To me the next step seems to be:
>
> * compile my own version of zsh with symbols (from apt source and
>   dpkg-buildpackage?)
> * run a gdb session and start to explore the code

I think that would only confuse you further.  That would be like
trying to learn programming by disassembling a compiler.

In the example here, I would say start by understanding the order of
operations from the "Expansion" chapter in the doc, and then look at
how you're transforming the steps to determine where you might be
changing to a different operation, and what that means for your
desired result.

Re: A way to find a bug (or explain a missuse of) realpath expansion?

[Marc Chantreux]

hello Bart!

On Tue, Apr 30, 2024 at 08:43:51PM -0700, Bart Schaefer wrote:
> > The /usr shade of path will disapear from debian distribution
> Can you provide a reference for this?  Web search is clogged with
> reports of accidental deletions of the /usr tree, etc.

	https://lists.debian.org/debian-devel-announce/2023/10/msg00003.html
	https://wiki.debian.org/UsrMerge

> You're trying to use a glob qualifier form (:A) but you're not doing
> globbing, you're doing parameter expansion.

I was globing to remove non-existing directories.

But let's say it wasn't intentional: don't you think the result is
flawed anyway? I mean: I don't understand how the same expansion on
2 identical elements of the list can lead to 2 different results.

> >   dpkg-buildpackage?)
> > * run a gdb session and start to explore the code
>
> I think that would only confuse you further. That would be like
> trying to learn programming by disassembling a compiler.

I had a lot of epiphanies in the past just by dumping the call stack at
the good moment. The possibility of this way of learning really depends
the way functions are writen (atomicity, name, …) so the underlining
question of my precedent mail was: "is the code of zsh easy to dive in"?

> In the example here, I would say start by understanding the order of
> operations from the "Expansion" chapter in the doc

zshexpn is by far the manual i spent most of my time!

-- 
Marc Chantreux
Pôle CESAR (Calcul et services avancés à la recherche)
Université de Strasbourg
14 rue René Descartes,
BP 80010, 67084 STRASBOURG CEDEX
03.68.85.60.79

Re: A way to find a bug (or explain a missuse of) realpath expansion?

[Bart Schaefer]

On Thu, May 2, 2024 at 1:06 AM Marc Chantreux <mc@unistra.fr> wrote:
>
> On Tue, Apr 30, 2024 at 08:43:51PM -0700, Bart Schaefer wrote:
> > > The /usr shade of path will disapear from debian distribution
> > Can you provide a reference for this?  Web search is clogged with
> > reports of accidental deletions of the /usr tree, etc.
>
>         https://lists.debian.org/debian-devel-announce/2023/10/msg00003.html
>         https://wiki.debian.org/UsrMerge

Ah, I must have misunderstood "shade of path will disappear".

> > You're trying to use a glob qualifier form (:A) but you're not doing
> > globbing, you're doing parameter expansion.
>
> I was globing to remove non-existing directories.

What I meant was, syntactically, globbing does not occur in the order
that you think it did here:
  ${(u):-${foo%/}(:A)}
That is, the stuff to the right of :- in that expression is not
globbed until after the entire parameter expansion is resolved, so the
(u) flag does not "see" the result of globbing, it just sees
  /usr/bin /bin(:A)
and does nothing useful with it.

Note also that (:A) has been appended to the entire $foo expansion,
not to each element of the array $foo, so even once you get to the
globbing step only /bin has realpath applied to it -- the result would
be different for the reversed order
  foo=( {,/usr}/bin )

There's no way to embed filename generation (globbing) inside a
${param} expansion without using an intermediate command substitution.
The three $-prefixed forms of expansion -- parameter ${var}, command
$(cmd), and arithmetic $((expr)) -- are processed "together"
left-to-right and inside-out, but the other five -- history, process,
brace, tilde, and globbing -- are strictly independent.  (The
Expansion introduction could be more explicit about this.)

> But let's say it wasn't intentional: don't you think the result is
> flawed anyway? I mean: I don't understand how the same expansion on
> 2 identical elements of the list can lead to 2 different results.

In your attempt to work this out step-by-step you have
  foo=( $foo(:A) )        ; print -l ">>>> realpath" $foo
but there again $foo(:A) is
  /usr/bin /usr/bin /bin /bin(:A)
What you wanted was
  foo=( $^foo(:A) )
to get
  /usr/bin(:A) /usr/bin(:A) /bin(:A) /bin(:A)

Conversely, ${foo:A} is one step (parameter expansion) rather than two
(parameter, then glob) so realpath is "automatically" applied to every
element of the array.

Does that explain it?

You could have removed all the non-existing directories earlier in the
process, like this:
  foo=( {/usr,}/bin{,/}(/N) ) ; print -l ">>>> setting foo" $foo
That attaches the (/N) -- match directories, discard non-matches -- to
the brace expansion, yielding (before globbing)
  /usr/bin(/N) /usr/bin(/N) /bin(/N) /bin(/N)

In fact you could do the entire process right there:
  typeset -aU foo=( {/usr,}/bin{,/}(/N:A) )
  print -l $foo
"Create an array foo of unique elements and assign the realpaths of
the existing directories in the set /usr/bin /bin /usr/bin/ /bin/"

> [...] so the underlining
> question of my precedent mail was: "is the code of zsh easy to dive in"?

I would have to say it's not, at least for what you're trying to
accomplish here.