trapped in bash

trapped in bash

[Ray Andrews]

So I'm setting up a new machine with Debian 64 (using Debian 32 still on 
old machine) and I'm thinking to just copy over my whole zsh install 
from here to there and ... nope ... I 'cd' to where the binary is and 
type  'zsh' and bash reports that it can't find the file.  Copy it to 
'/bin' and it still can't find it.  Even the desperate '. ./zsh' reports 
'cannot execute binary file'.  I hope to gawd that an apt-get install of 
zsh would work, but I'd like to use my existing setup exactly as is. Do 
I have to rebuild? I'm new with 64 bit issues, whatever they may happen 
to be, but I thought a 64 bit environment could always run 32 bit apps.  
Or have I forgotten something simple? Meanwhile  I'm trapped in the 
pre-fire age of bash out of the box. Even 'ls -l' doesn't work, 
filenames only.  They say that suffering purifies the soul ;-)

BTW:

     _array=( ${(f)_variable} )
     _count=$#_array   # Count the number of lines.

Can that be simplified? I don't need the _array, just the _count. Of 
course there are any number of utilities to count lines, but I mean the 
simplest zsh way, although the above is nicely self-explanatory.

Re: trapped in bash

[Aaron Schrab]

At 07:33 -0800 16 Dec 2016, Ray Andrews <rayandrews@eastlink.ca> wrote:
>So I'm setting up a new machine with Debian 64 (using Debian 32 still 
>on old machine) and I'm thinking to just copy over my whole zsh 
>install from here to there and ... nope ... I 'cd' to where the binary 
>is and type  'zsh' and bash reports that it can't find the file.  Copy 
>it to '/bin' and it still can't find it.  Even the desperate '. ./zsh' 
>reports 'cannot execute binary file'.  I hope to gawd that an apt-get 
>install of zsh would work, but I'd like to use my existing setup 
>exactly as is. Do I have to rebuild? I'm new with 64 bit issues, 
>whatever they may happen to be, but I thought a 64 bit environment 
>could always run 32 bit apps.  Or have I forgotten something simple? 

The hardware and kernel are able to run 32bit binaries, but you can't 
use 64bit shared libraries with 32bit executables (or vice-versa).  You 
also need the 32bit version of the run-time dynamic linker; lack of that 
is why you're being told that a file isn't found.

Unless there was something really special about that build I'd either 
just go with installing via apt or rebuilding.

If you *really* want to use the same build you can likely do that with a 
multi-arch setup. To start with that would involve:

  dpkg --add-architecture i386
  apt update
  apt install libc6:i386

Once that is done the system should at least recognize that the old zsh 
build is executable. But you'll likely need to install several other 
i386 libraries; depending on the age of the system where that was built 
it's possible that the required versions won't be found for the current 
version of Debian.

The number of i386 library packages needed for this shouldn't be large, 
but a multi-arch system does mean that every `apt update` will fetch the 
package list for both of the architectures.

>Meanwhile  I'm trapped in the pre-fire age of bash out of the box.  
>Even 'ls -l' doesn't work, filenames only.

That shouldn't have anything to do with bash vs. zsh.

Re: trapped in bash

[Ray Andrews]

On 12/16/2016 01:51 PM, Aaron Schrab wrote:
> The hardware and kernel are able to run 32bit binaries, but you can't 
> use 64bit shared libraries with 32bit executables (or vice-versa).  
> You also need the 32bit version of the run-time dynamic linker; lack 
> of that is why you're being told that a file isn't found.
>
> Unless there was something really special about that build I'd either 
> just go with installing via apt or rebuilding.
>

Thanks Aaron.  No, I've no reason not to rebuild I just wondered about 
the 'not found' thing, it seems like an unhelpful message. What you say 
doesn't surprise me as to the libraries.  It is almost comical how 
helpless I am with out zsh now -- I can't install it for lack of having 
it.  I'll use apt-get and then go from there.

Re: trapped in bash

[Bart Schaefer]

On Dec 16,  2:39pm, Ray Andrews wrote:
}
} I've no reason not to rebuild I just wondered about 
} the 'not found' thing, it seems like an unhelpful message.

Agreed, it's very unhelpful/baffling, because what's not found is
not the binary itself, but one of the libraries needed to load it.
Sadly, the calling shell can't tell the difference, because all it
gets back is a numeric error code from the execve() system call.

Re: trapped in bash

[Ray Andrews]

On 12/16/2016 03:05 PM, Bart Schaefer wrote:
> On Dec 16,  2:39pm, Ray Andrews wrote:
> }
> } I've no reason not to rebuild I just wondered about
> } the 'not found' thing, it seems like an unhelpful message.
Agreed, it's very unhelpful/baffling, because what's not found is not 
the binary itself, but one of the libraries needed to load it. Sadly, 
the calling shell can't tell the difference, because all it gets back is 
a numeric error code from the execve() system call.
Seems  I have to rebuild all my own C projects as they have the same 
'not found' characteristic.

One little anomaly , the zsh right now is via the package manager in 
Debian: "zsh 5.2 (x86_64-debian-linux-gnu)" and I source my usual .zshrc 
and so far everything is where I put it, except that command completion, 
if there is no match, brings up the last command, whereas my setup 
should do nothing (leave the prompt blank).  Where might that change be 
effected from?

Re: trapped in bash

[Bart Schaefer]

On Dec 17, 12:27pm, Ray Andrews wrote:
}
} Seems  I have to rebuild all my own C projects as they have the same 
} 'not found' characteristic.

Or you need to follow Aaron's instructions for installing the 32-bit
runtime on your 64-bit system.  Although that's not *guaranteed* to
make everything work without rebuilding.

} and so far everything is where I put it, except that command completion, 
} if there is no match, brings up the last command, whereas my setup 
} should do nothing (leave the prompt blank).  Where might that change be 
} effected from?

That doesn't sound familiar at all.  Bringing up the last command seems
to imply that one of the history completers is in the zstyle.

Your best bet may be:

1. type ctrl-x h and see what completions are active in that context;

2. if that doesn't give you an obvious hint, use ctrl-x ? for a debug
trace and look through that.

Re: trapped in bash

[Ray Andrews]

On 12/17/2016 12:43 PM, Bart Schaefer wrote:
> On Dec 17, 12:27pm, Ray Andrews wrote:
> }
> } Seems  I have to rebuild all my own C projects as they have the same
> } 'not found' characteristic.
>
> Or you need to follow Aaron's instructions for installing the 32-bit
> runtime on your 64-bit system.  Although that's not *guaranteed* to
> make everything work without rebuilding.
No need to risk doing something that might be flaky, I'm just rebuilding 
as needed. So far there hasn't been a hint of a problem beyond needing 
to retrieve some 64 bit libraries.  So I'm pumped that zsh will rebuild 
with similar results.
>
> } and so far everything is where I put it, except that command completion,
> } if there is no match, brings up the last command, whereas my setup
> } should do nothing (leave the prompt blank).  Where might that change be
> } effected from?
>
> That doesn't sound familiar at all.  Bringing up the last command seems
> to imply that one of the history completers is in the zstyle.
>
> Your best bet may be:
>
> 1. type ctrl-x h and see what completions are active in that context;
>
> 2. if that doesn't give you an obvious hint, use ctrl-x ? for a debug
> trace and look through that.
>
>
Ah ... it's the stock files in /etc/zsh, I renamed the directory and 
everything is as I expect. It's one of my perpetual mini-rants  to hate 
the way most linux programs hide bits of themselves all over the place.  
Actually I deleted  all those  on my main  machine for fear of them 
doing funny things on Tuesday, but it's time I took a look at them again 
and put them work, maybe, now that I have some clue what I'm doing.  
Thanks Bart.

Re: trapped in bash

[Jim]

[-- Attachment #1: Type: text/plain, Size: 592 bytes --]

On Fri, Dec 16, 2016 at 9:33 AM, Ray Andrews <rayandrews@eastlink.ca>
wrote:
>
> BTW:
>
>     _array=( ${(f)_variable} )
>     _count=$#_array   # Count the number of lines.
>
> Can that be simplified? I don't need the _array, just the _count. Of
> course there are any number of utilities to count lines, but I mean the
> simplest zsh way, although the above is nicely self-explanatory


_count=${#${(f)_variable}}

This will remove the need for the variable _array. One caution, if your
not already aware, (f) removes blank lines, so they won't show up in
the count.

Hope this helps,

Jim

Re: trapped in bash

[Ray Andrews]

On 12/17/2016 03:16 PM, Jim wrote:
>
>
> _count=${#${(f)_variable}}
>
> This will remove the need for the variable _array. One caution, if your
> not already aware, (f) removes blank lines, so they won't show up in
> the count.
>
> Hope this helps,
>
> Jim
>
Thanks Jim, my experiments failed 'cuz I insisted on including the 
parenthesis, but since no array is created, no parenthesis are needed.  
Clear in hindsight.

Re: trapped in bash

[Daniel Shahaf]

Ray Andrews wrote on Sat, Dec 17, 2016 at 14:40:50 -0800:
> Ah ... it's the stock files in /etc/zsh, I renamed the directory and
> everything is as I expect.

If you just rename the directory, it'll be recreated the next time you
upgrade your zsh package (which is probably very soon).  You could
instead 'setopt NO_GLOBAL_RCS' in .zshenv.

dpkg-divert(1) may work as well (I'm not sure how it handles directory
diversions).

Cheers,

Daniel

Re: trapped in bash

[Ray Andrews]

On 12/18/2016 07:06 AM, Daniel Shahaf wrote:
> Ray Andrews wrote on Sat, Dec 17, 2016 at 14:40:50 -0800:
>> Ah ... it's the stock files in /etc/zsh, I renamed the directory and
>> everything is as I expect.
> If you just rename the directory, it'll be recreated the next time you
> upgrade your zsh package (which is probably very soon).

Sure, that's to be expected.
>    You could
> instead 'setopt NO_GLOBAL_RCS' in .zshenv.

I'll look into that, thanks.  Meanwhile just shooting the directory was 
easy enough to understand.  But as I said I want to take a look at those 
half a dozen various ifs ands buts and maybes as to the startup files, 
right now one plain old .zshrc seems to do everything nicely however.

BTW, as to the 'file not found' thing last time, not that  this is worth 
any bother, but surely the shell knows that it has found its target file 
vs. not being able to run it for some secondary reason? zsh and bash 
seem to  have the same message there, as I found trying to start zsh 
from bash, or any of my 32 bit binaries from zsh  under 64 bit Debian.  
Would it be doable to distinguish between *really* 'not found' and 
'can't run this binary'?  If nothing else it would have stopped me from 
having to ask that question :-/

Re: trapped in bash

[Daniel Shahaf]

Ray Andrews wrote on Sun, Dec 18, 2016 at 07:54:27 -0800:
> Would it be doable to distinguish between *really* 'not found' and
> 'can't run this binary'?  If nothing else it would have stopped me
> from having to ask that question :-/

As far as zsh is concerned, a binary file is simply something that can
be passed to execve().  zsh does not attempt to parse or load the file;
the C library and OS do.  So it is they who are in a position to issue
an error message explaining _why_ the file can't be executed.

I suppose you could write a command_not_found_handler() function that
tried to reverse engineer _why_ the command wasn't found — with the
obvious limitations, e.g., race conditions between the library's
execution and the handler's.

Cheers,

Daniel

Re: trapped in bash

[Bart Schaefer]

On Dec 19, 12:32am, Daniel Shahaf wrote:
}
} I suppose you could write a command_not_found_handler() function that
} tried to reverse engineer _why_ the command wasn't found - with the
} obvious limitations, e.g., race conditions between the library's
} execution and the handler's.

I don't think there can be a race here because the handler won't be
invoked if the execve() hasn't already failed.

Something like this:

command_not_found_handler () {
  local attempt="$1"
  if [[ "$1" != /* && ( -o pathdirs || "$1
" != */* ) ]]
  then
    emulate zsh -c 'attempt=$^path/$1(N[1])'
  fi
  if [[ -f $attempt ]]
  then
    local hashbang
    read -u 0 -k 2 hashbang < "$attempt"
    case "$hashbang" in
      ('#!') print -u2 -- $1\: bad '#!' interpreter ;;
      (*) print -u2 failed to load $1 ;;
    esac
  else
    print -u2 -- $1\: no such file in \$PATH
  fi
  return 127
}


-- 
Barton E. Schaefer

Re: trapped in bash

[Ray Andrews]

On 18/12/16 06:30 PM, Bart Schaefer wrote:
> On Dec 19, 12:32am, Daniel Shahaf wrote:
> }
> } I suppose you could write a command_not_found_handler() function that
> } tried to reverse engineer _why_ the command wasn't found - with the
> } obvious limitations, e.g., race conditions between the library's
> } execution and the handler's.
>
> I don't think there can be a race here because the handler won't be
> invoked if the execve() hasn't already failed.
>
> Something like this:
>
> command_not_found_handler () {
>    local attempt="$1"
>    if [[ "$1" != /* && ( -o pathdirs || "$1
> " != */* ) ]]
>    then
>      emulate zsh -c 'attempt=$^path/$1(N[1])'
>    fi
>    if [[ -f $attempt ]]
>    then
>      local hashbang
>      read -u 0 -k 2 hashbang < "$attempt"
>      case "$hashbang" in
>        ('#!') print -u2 -- $1\: bad '#!' interpreter ;;
>        (*) print -u2 failed to load $1 ;;
>      esac
>    else
>      print -u2 -- $1\: no such file in \$PATH
>    fi
>    return 127
> }
>
>
Not for me to say, but it seems to me that the simple fact that a file 
exists or does not exist is simple enough to determine in the tradition 
of 'whence' or something like that.  *Then* we run into issues about 
executing it *after* the thing has (obviously) been found.  Sometimes a 
command can't be found because it can't be found, that is, the file 
simply does not exist.  Or is that too simple?  Anyway I sure like the idea.

Re: trapped in bash

[Bart Schaefer]

On Dec 18,  8:29pm, Ray Andrews wrote:
}
} Not for me to say, but it seems to me that the simple fact that a file 
} exists or does not exist is simple enough to determine

Well, yes.

The shell finds the file, because it's responsible for path search.

It then forks, and the child process passes the full path to execve(),
which replaces the child shell with the kernel code that is supposed
to start up the new executable.

That kernel code then fails for whatever reason -- in this case, it's
because the 32-bit library is not found -- and returns a nonzero
numeric exit code to the parent shell.

The parent shell prints the message associated with that error code.
It isn't responsible for guessing what caused the error; there could
for example have been additional rounds of finding and exec'ing other
paths of which the parent has no knowledge, before a final exit.

Re: trapped in bash

[Ray Andrews]

On 18/12/16 09:48 PM, Bart Schaefer wrote:
> On Dec 18,  8:29pm, Ray Andrews wrote:
> }
> } Not for me to say, but it seems to me that the simple fact that a file
> } exists or does not exist is simple enough to determine
>
> Well, yes.
>
> The shell finds the file, because it's responsible for path search.
Exactly, I'd expect zsh to handle that with an appropriately helpful 
message, but past that point I quite understand that it's not the 
shell's affair.  Mind, even subsequent error messages might be preceded 
by 'execve()' reports: ...' or something like that, it would just make 
further diagnosis a bit faster.  In my situation I'd be angling for 
something like:  "Error: binary file '/aa/bb/xxx' is not executable.  
execve() reports error #54321: "file not found".
>
> It then forks, and the child process passes the full path to execve(),
> which replaces the child shell with the kernel code that is supposed
> to start up the new executable.
>
> That kernel code then fails for whatever reason -- in this case, it's
> because the 32-bit library is not found -- and returns a nonzero
> numeric exit code to the parent shell.
>
> The parent shell prints the message associated with that error code.
> It isn't responsible for guessing what caused the error; there could
> for example have been additional rounds of finding and exec'ing other
> paths of which the parent has no knowledge, before a final exit.
>

Re: trapped in bash

[Daniel Shahaf]

Ray Andrews wrote on Mon, Dec 19, 2016 at 08:26:43 -0800:
> On 18/12/16 09:48 PM, Bart Schaefer wrote:
> >The shell finds the file, because it's responsible for path search.
> Exactly, I'd expect zsh to handle that with an appropriately helpful
> message, but past that point I quite understand that it's not the shell's
> affair.  Mind, even subsequent error messages might be preceded by
> 'execve()' reports: ...' or something like that, it would just make further
> diagnosis a bit faster.

Yes, error messages should generally be signed.

Perhaps something along these lines? —

diff --git a/Src/exec.c b/Src/exec.c
index 7e29e9c..37d4690 100644
--- a/Src/exec.c
+++ b/Src/exec.c
@@ -699,7 +699,7 @@ execute(LinkList args, int flags, int defpath)
 	    if (arg0 == s || unset(PATHDIRS) ||
 		(arg0[0] == '.' && (arg0 + 1 == s ||
 				    (arg0[1] == '.' && arg0 + 2 == s)))) {
-		zerr("%e: %s", lerrno, arg0);
+		zerr("execve: %e: %s", lerrno, arg0);
 		_exit((lerrno == EACCES || lerrno == ENOEXEC) ? 126 : 127);
 	    }
 	    break;
@@ -774,7 +774,7 @@ execute(LinkList args, int flags, int defpath)
     }
 
     if (eno)
-	zerr("%e: %s", eno, arg0);
+	zerr("execve: %e: %s", eno, arg0);
     else if (commandnotfound(arg0, args) == 0)
 	_exit(lastval);
     else
diff --git a/Test/C05debug.ztst b/Test/C05debug.ztst
index 9a8df1d..ee2c1b8 100644
--- a/Test/C05debug.ztst
+++ b/Test/C05debug.ztst
@@ -52,15 +52,9 @@
 	# Failure
 	exit 10
 	HERE
-  $ZTST_testdir/../Src/zsh -f ./zsh-trapreturn-bug2 2>erroutput.dif
-  mystat=$?
-  (
-   setopt extendedglob
-   print ${"$(< erroutput.dif)"%%:[^:]#: ./fdasfsdafd}
-  )
-  (( mystat == 0 ))
+  $ZTST_testdir/../Src/zsh -f ./zsh-trapreturn-bug2
 0: trapreturn handling bug is properly fixed
->./zsh-trapreturn-bug2:5
+*?./zsh-trapreturn-bug2:5: execve:*./fdasfsdafd
 
   fn() {
     setopt localtraps localoptions debugbeforecmd
diff --git a/Test/E01options.ztst b/Test/E01options.ztst
index 45df9f5..d40dc2e 100644
--- a/Test/E01options.ztst
+++ b/Test/E01options.ztst
@@ -782,7 +782,7 @@
 >File in upper dir
 >File in lower dir
 >unsetting option...
-?(eval):14: no such file or directory: pathtestdir/findme
+?(eval):14: execve: no such file or directory: pathtestdir/findme
 
   (setopt pathdirs; path+=( /usr/bin ); type ./env)
 1:whence honours PATH_DIRS option