zsh-workers
 help / color / mirror / code / Atom feed
* first adventures
@ 2014-10-26 17:20 Ray Andrews
  2014-10-26 17:50 ` Bart Schaefer
  2014-10-26 17:52 ` Peter Stephenson
  0 siblings, 2 replies; 18+ messages in thread
From: Ray Andrews @ 2014-10-26 17:20 UTC (permalink / raw)
  To: zsh-workers

All,

These are my very first pokings at the code. So far, my feelings are 
about the same as
expressed by this comment in 'hist.c':

     * However, I'm so confused it could simply be baking Bakewell tarts.

I'm trying to figure out where/how command lines are grabbed (I think 
this is done by zle?).
I'd like to be able to grab commands before any processing whatsoever is 
done.  If I type:

     $ some-command $1 *.* !{1, 1 & @ >>! $2 "no parsing please!" .. () 
'<<' `# ^ -n(*&<)` {}  [enter]

... I'm hoping there will be some_string somewhere such that:

    puts(some_string)

... will print exactly:

    some-command $1 *.* !{1, 1 & @ >>! $2 "no parsing please!" .. () 
'<<' `# ^ -n(*&<)` {}


Surely there is an input buffer somewhere? That saves the
command line before it gets parsed?

One other thing: 'make' produces so many dull msg. about moving
into and out of directories and such.  Can these be turned off? I'd
like to see just warnings and errors and 'important' msg.

Thanks


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: first adventures
  2014-10-26 17:20 first adventures Ray Andrews
@ 2014-10-26 17:50 ` Bart Schaefer
  2014-10-26 18:04   ` Bart Schaefer
  2014-10-26 19:57   ` Ray Andrews
  2014-10-26 17:52 ` Peter Stephenson
  1 sibling, 2 replies; 18+ messages in thread
From: Bart Schaefer @ 2014-10-26 17:50 UTC (permalink / raw)
  To: zsh-workers

On Oct 26, 10:20am, Ray Andrews wrote:
}
} I'm trying to figure out where/how command lines are grabbed (I think 
} this is done by zle?).

Well, it's done by ZLE when ZLE is active, which it isn't always.

However, I think what you're looking for is Src/Zle/zle_main.c in the
function zle_main_entry() under "case ZLE_CMD_READ" where the call to
zleread() is made.  Or perhaps where that's called, which is usually
from Src/input.c:inputline() via zleentry(ZLE_CMD_READ, ...).

} ... I'm hoping there will be some_string somewhere such that:
} 
}     puts(some_string)
} 
} ... will print exactly:
} 
}     some-command $1 *.* !{1, 1 & @ >>! $2 "no parsing please!" .. () 
} '<<' `# ^ -n(*&<)` {}

No, that there isn't, though if you look around line 296 of input.c where
the VERBOSE option is handled, you'll get close.  The input line is read
into what's called "metafied" form by zleread() and has to be unmetafied
[by the aptly named function unmetafy(), Src/utils.c] to return it to
the raw input form.

} One other thing: 'make' produces so many dull msg. about moving
} into and out of directories and such.  Can these be turned off? I'd
} like to see just warnings and errors and 'important' msg.

Run it as "make -k" and it will shut up about that.  This is a GNU make
thing, not a zsh build configuration thing.


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: first adventures
  2014-10-26 17:20 first adventures Ray Andrews
  2014-10-26 17:50 ` Bart Schaefer
@ 2014-10-26 17:52 ` Peter Stephenson
  2014-10-28 17:48   ` Ray Andrews
  1 sibling, 1 reply; 18+ messages in thread
From: Peter Stephenson @ 2014-10-26 17:52 UTC (permalink / raw)
  To: zsh-workers

On Sun, 26 Oct 2014 10:20:47 -0700
Ray Andrews <rayandrews@eastlink.ca> wrote:
> I'm trying to figure out where/how command lines are grabbed (I think 
> this is done by zle?).

That's how it works for interactive shells.  The lowest level of input
in the main shell is within input.c.  Because zle is a separate library
that may be loaded at run time the interaction actually goes through the
function zleentry() in init.c, which ensures that either zle is loaded
or if it can't be there's some fallback to get the input another way.
ZLE_CMD_READ is the command to zleentry() that causes a line to be read.

shingetline() is the fallback for non-interactive shells or if zle can't
be loaded.

> One other thing: 'make' produces so many dull msg. about moving
> into and out of directories and such.  Can these be turned off? I'd
> like to see just warnings and errors and 'important' msg.

make -s

pws


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: first adventures
  2014-10-26 17:50 ` Bart Schaefer
@ 2014-10-26 18:04   ` Bart Schaefer
  2014-10-26 18:41     ` Ray Andrews
  2014-10-26 19:57   ` Ray Andrews
  1 sibling, 1 reply; 18+ messages in thread
From: Bart Schaefer @ 2014-10-26 18:04 UTC (permalink / raw)
  To: zsh-workers

On Oct 26, 10:50am, Bart Schaefer wrote:
}
} } One other thing: 'make' produces so many dull msg. about moving
} 
} Run it as "make -k" and it will shut up about that.

Oops, as PWS points out, that should say "make -s".


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: first adventures
  2014-10-26 18:04   ` Bart Schaefer
@ 2014-10-26 18:41     ` Ray Andrews
  0 siblings, 0 replies; 18+ messages in thread
From: Ray Andrews @ 2014-10-26 18:41 UTC (permalink / raw)
  To: zsh-workers

On 10/26/2014 11:04 AM, Bart Schaefer wrote:
> On Oct 26, 10:50am, Bart Schaefer wrote:
> }
> } } One other thing: 'make' produces so many dull msg. about moving
> }
> } Run it as "make -k" and it will shut up about that.
>
> Oops, as PWS points out, that should say "make -s".
>
Thank you gentlemen. I was trying to find something in the makefile 
itself :(


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: first adventures
  2014-10-26 17:50 ` Bart Schaefer
  2014-10-26 18:04   ` Bart Schaefer
@ 2014-10-26 19:57   ` Ray Andrews
  2014-10-26 21:04     ` Bart Schaefer
  1 sibling, 1 reply; 18+ messages in thread
From: Ray Andrews @ 2014-10-26 19:57 UTC (permalink / raw)
  To: zsh-workers

On 10/26/2014 10:50 AM, Bart Schaefer wrote:
> On Oct 26, 10:20am, Ray Andrews wrote:
> }
> } I'm trying to figure out where/how command lines are grabbed (I think
> } this is done by zle?).
>
> Well, it's done by ZLE when ZLE is active, which it isn't always.
>
> However, I think what you're looking for is Src/Zle/zle_main.c in the
Nuts! I never even thought to look in a subdir.  No wonder all trails 
went cold.
> function zle_main_entry() under "case ZLE_CMD_READ" where the call to
> zleread() is made.  Or perhaps where that's called, which is usually
> from Src/input.c:inputline() via zleentry(ZLE_CMD_READ, ...).
>
> } ... I'm hoping there will be some_string somewhere such that:
> }
> }     puts(some_string)
> }
> } ... will print exactly:
> }
> }     some-command $1 *.* !{1, 1 & @ >>! $2 "no parsing please!" .. ()
> } '<<' `# ^ -n(*&<)` {}
>
> No, that there isn't, though if you look around line 296 of input.c where
> the VERBOSE option is handled, you'll get close.  The input line is read
> into what's called "metafied" form by zleread() and has to be unmetafied
> [by the aptly named function unmetafy(), Src/utils.c] to return it to
> the raw input form.
But.  But ... we can't get the string before it's metafied in the first 
place? How is it possible
for it to be metafied before 'it' exists? If I can have a meta-fried 
egg, surely at some point there
was a raw egg? I was looking in hist.c, because the one place where 
things seem to be
stored 'literally' is in .histfile.

The Tao of zsh is a deep thing.

But somehow I just can't get anything to print if interactive.  I've got 
lots of feedback
if adding " puts("Hi. This is line xxx of file xxx")  " ... sort of 
things in the code
when  running a script, but if interactive nothing ever shows:

//    if (isset(VERBOSE)) {
     /* Output the whole line read so far. */
     zputs(ingetcline, stderr);
     fflush(stderr);
  //   }

... nothing doing, no output. I tried it from console too. Nothing. Is 
the xterm somehow
interfering?



^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: first adventures
  2014-10-26 19:57   ` Ray Andrews
@ 2014-10-26 21:04     ` Bart Schaefer
  2014-10-27  3:48       ` Ray Andrews
  0 siblings, 1 reply; 18+ messages in thread
From: Bart Schaefer @ 2014-10-26 21:04 UTC (permalink / raw)
  To: zsh-workers

On Oct 26, 12:57pm, Ray Andrews wrote:
}
} > No, that there isn't, though if you look around line 296 of input.c where
} > the VERBOSE option is handled, you'll get close.  The input line is read
} > into what's called "metafied" form by zleread() and has to be unmetafied
} But.  But ... we can't get the string before it's metafied in the first 
} place?

The following is a lot oversimplified, but:

It's read one character at a time and each character is then metafied
before being aggregated into the string buffer.

} How is it possible for it to be metafied before 'it' exists? If I can
} have a meta-fried egg, surely at some point there was a raw egg?

It's more like having a bag of popcorn; they don't all pop at the same
time, but ideally you don't open the bag until all of them have.

} I was looking in hist.c, because the one place where 
} things seem to be stored 'literally' is in .histfile.

History isn't stored until after bangchar references have been expanded, 
so although it's closer to literal it's still not the actual input line.

} The Tao of zsh is a deep thing.

Partly this is from separating ZLE into a module years after the shell
was first written.  It's designed so that somebody else could write and
plug in a different line editor module, but no one got that ambitious.

} But somehow I just can't get anything to print if interactive.

It works for me:

torch% echo hello
echo hello
hello
torch% 

Are you sure that you're actually executing the correct binary when you
are running interactively?  Maybe a startup file is "exec"-ing another
path?  What happens if you start zsh with the -v option?


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: first adventures
  2014-10-26 21:04     ` Bart Schaefer
@ 2014-10-27  3:48       ` Ray Andrews
  2014-10-27  6:08         ` Bart Schaefer
  0 siblings, 1 reply; 18+ messages in thread
From: Ray Andrews @ 2014-10-27  3:48 UTC (permalink / raw)
  To: zsh-workers

On 10/26/2014 02:04 PM, Bart Schaefer wrote:
> On Oct 26, 12:57pm, Ray Andrews wrote:
> }
> It's read one character at a time and each character is then metafied
> before being aggregated into the string buffer.
I see, so the raw string never exists.  I'd never contemplate doing it 
that way, but
I suppose there's a reason for it.
> Partly this is from separating ZLE into a module years after the shell
> was first written.  It's designed so that somebody else could write and
> plug in a different line editor module, but no one got that ambitious.
I remember reading about that. It would seem to me that ZLE is 
profoundly part of the
shell, so that's a bit wierd, but ...
> Are you sure that you're actually executing the correct binary when you
> are running interactively?  Maybe a startup file is "exec"-ing another
> path?  What happens if you start zsh with the -v option?
Right again Bart.  My last Debian upgrade seems to have reset my 
/etc/alternative/zsh*
links back to /bin/zsh5, which is the normal thing with Debian.  So I've 
been flogging the
wrong horse ;-)

So now I see the command there. What is this 'bangchar' you mention?


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: first adventures
  2014-10-27  3:48       ` Ray Andrews
@ 2014-10-27  6:08         ` Bart Schaefer
  0 siblings, 0 replies; 18+ messages in thread
From: Bart Schaefer @ 2014-10-27  6:08 UTC (permalink / raw)
  To: zsh-workers

On Oct 26,  8:48pm, Ray Andrews wrote:
}
} I see, so the raw string never exists.  I'd never contemplate doing it 
} that way, but I suppose there's a reason for it.

It probably makes more sense if you consider that every character typed
has to be analyzed to invoke an editor function (or be cached as part of
a multi-character sequence that will eventually invoke an editor function).
In fact the "self-insert" widget is what adds a character to the input
buffer, so really there are no characters that aren't editor functions.

} So now I see the command there. What is this 'bangchar' you mention?

The first character of $histchars, normally '!', which resolves what
follows it to a history substitution, is called bangchar in the C code.


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: first adventures
  2014-10-26 17:52 ` Peter Stephenson
@ 2014-10-28 17:48   ` Ray Andrews
  2014-10-29  4:05     ` Bart Schaefer
  0 siblings, 1 reply; 18+ messages in thread
From: Ray Andrews @ 2014-10-28 17:48 UTC (permalink / raw)
  To: zsh-workers

On 10/26/2014 10:52 AM, Peter Stephenson wrote:


Peter, Bart:

First hints of traction:

> That's how it works for interactive shells. The lowest level of input 
> in the main shell is within input.c. Because zle is a separate library 
> that may be loaded at run time the interaction actually goes through 
> the function zleentry() in init.c, which ensures that either zle is 
> loaded or if it can't be there's some fallback to get the input 
> another way. ZLE_CMD_READ is the command to zleentry() that causes a 
> line to be read. shingetline() is the fallback for non-interactive 
> shells or if zle can't be loaded. va_end(ap);

At the bottom of zleentry():

     ///
     printf("FILE: %s LINE: %d\n", __FILE__, __LINE__);
     puts(ret);
         return ret;

   }

I get:

     $ !4562; ls zsh.*
     ls e*; ls zsh.*                        << exactly as typed, but 
with bangchar expanded.
     FILE: init.c LINE: 1535
     !4562; ls zsh.*                      << exactly exactly as typed :)

     (output)

... so, I finally have my gloriously, deliciously unexpanded, plain 
vanilla command, exactly as I
typed it. (BTW, 'cmd = -1' and this doesn't seem to be handled by 
'ZLE_CMD_READ' but nevermind.)
However, as Bart has explained, I really do want to be able to capture 
the bangchar/history thing expanded
line as well, precisely as shown by the shell anyway:

     ls e*; ls zsh.*                        << exactly as typed, but 
with bangchar expanded.

Where can I capture that output? AFAICT, that's exactly as it gets 
written to .histfile, so it should
be easy enough to intercept.

Finally, if we have several commands on one line, at some point the line 
must be chopped up
at the semicolons for execution, since each individual command only 
'gets' that part of the
line that applies to it.  How can that be done?  (I was speculating that 
if each command
could somehow know that it was (say) command #3 on that line, then one 
might very inelegantly
parse the .histfile line, counting semicolons, and chop it up that way, 
but how ugly is that.)


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: first adventures
  2014-10-28 17:48   ` Ray Andrews
@ 2014-10-29  4:05     ` Bart Schaefer
  2014-10-29 15:41       ` Ray Andrews
  0 siblings, 1 reply; 18+ messages in thread
From: Bart Schaefer @ 2014-10-29  4:05 UTC (permalink / raw)
  To: zsh-workers

On Oct 28, 10:48am, Ray Andrews wrote:
}
} At the bottom of zleentry():
} 
}      ///
}      printf("FILE: %s LINE: %d\n", __FILE__, __LINE__);
}      puts(ret);
}          return ret;
} 
}    }
} 
} I get:
} 
}      FILE: init.c LINE: 1535
}      !4562; ls zsh.*                      << exactly exactly as typed :)
}      (output)
} 
} ... so, I finally have my gloriously, deliciously unexpanded, plain 
} vanilla command, exactly as I

Well, you think so, but really it just happens that nothing you typed
needed to be metafied.  If you were to step through with a debugger,
you'd find that what's ultimately been called is zleread(), which is
prefixed with this comment:

/* Read a line.  It is returned metafied. */

} typed it. (BTW, 'cmd = -1' and this doesn't seem to be handled by 
} 'ZLE_CMD_READ' but nevermind.)

There are two switch statements in zleentry().  The first one calls
zle_entry_ptr(cmd) and then assigns cmd = -1 to cause the second
switch to be skipped.

} However, as Bart has explained, I really do want to be able to capture 
} the bangchar/history thing expanded
} 
} Where can I capture that output?

That's a complicated question.  Input is actually read by the lexer,
which is called repeatedly by the parser until an entire runnable
shell command has been consumed.  (This is referred to as an "event"
in much of the internals, because that's how it's recorded in the
history when interactive, but the same applies to a script file.)

So if you actually want to try to look at it line by line, your best
bet is to look at the str argument passed to input.c:inputsetline(),
though that has the same caveats about metafication as above.  This
is still before history expansion, though.

History is expanded by the lexer as each word is consumed, because as
I mentioned in private email, the expansion has to be re-analyzed to
correctly tokenize it.  So the previously-discussed point in input.c
where VERBOSE is handled, is the line with history expanded.

The parser itself is driven by Src/init.c:loop().  At around line 160
through 180 is where the preexec function is called after a complete
input event has been read (which may be many lines).  The argument
list of that shell function has the strings that you're looking for:

preexec
     Executed just after a command has been read and is about to be
     executed.  If the history mechanism is active (and the line was not
     discarded from the history buffer), the string that the user typed
     is passed as the first argument, otherwise it is an empty string.
     The actual command that will be executed (including expanded
     aliases) is passed in two different forms: the second argument is
     a single-line, size-limited version of the command (with things
     like function bodies elided); the third argument contains the full
     text that is being executed.

(The doc is actually somewhat inaccurate there -- the first argument is
the string after history expansion, not exactly as typed.)

} Finally, if we have several commands on one line, at some point the line 
} must be chopped up
} at the semicolons for execution, since each individual command only 
} 'gets' that part of the
} line that applies to it.  How can that be done?

It's done by the parser.  In loop() you'll find the line:

        if (!(prog = parse_event())) {  /* if we couldn't parse a list */

When prog is non-NULL (the else branch of that if), it contains the
compiled form of the input code, which is already broken up into the
set of commands that will be executed.  There is no human-readable
decomposed form of the individual commands that make up a finished
input event; getpermtext(prog) returns the readable form of the entire
thing, which is the closest you get.

} (I was speculating that 
} if each command
} could somehow know that it was (say) command #3 on that line, then one 
} might very inelegantly
} parse the .histfile line, counting semicolons, and chop it up that way, 
} but how ugly is that.)

If you're examining the history file from the shell, you can use the (z)
and (Z:...:) parameter expansion flags to simulate parsing.


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: first adventures
  2014-10-29  4:05     ` Bart Schaefer
@ 2014-10-29 15:41       ` Ray Andrews
  2014-10-29 20:46         ` Bart Schaefer
  0 siblings, 1 reply; 18+ messages in thread
From: Ray Andrews @ 2014-10-29 15:41 UTC (permalink / raw)
  To: zsh-workers

On 10/28/2014 09:05 PM, Bart Schaefer wrote:
> } ... so, I finally have my gloriously, deliciously unexpanded, plain
> } vanilla command, exactly as I
>
> Well, you think so, but really it just happens that nothing you typed
> needed to be metafied.
Deo volente! For now I don't even want to think about metafying, my head 
would explode.

> History is expanded by the lexer as each word is consumed, because as
> I mentioned in private email, the expansion has to be re-analyzed to
> correctly tokenize it.  So the previously-discussed point in input.c
> where VERBOSE is handled, is the line with history expanded.
Yeah, that's the right form, but at that point, it's so generous, it 
prints everything the
shell does, like preexec(), and the whole init of the shell.
>
> When prog is non-NULL (the else branch of that if), it contains the
> compiled form of the input code, which is already broken up into the
> set of commands that will be executed.  There is no human-readable
> decomposed form of the individual commands that make up a finished
> input event; getpermtext(prog) returns the readable form of the entire
> thing, which is the closest you get.
Right, it looks good, and each command is on it's own line;

     $ ls init.c*; ls $PWD/main.* > /tmp/junk
     FILE: init.c LINE: 185
     ls init.c*
     ls $PWD/main.* > /tmp/junk

     (output)

It's proof against expansion of glob, variables and redirection ... 
pretty much all I'm thinking of.
bangchar/history is expanded, and so are aliases, but that's good.
So, if there was just some way of matching each line of that output from 
getpermtext() with the
appropriate command, then I'd be close to the finish line. if, in:

    $ ls main.c; ls init.c; ls version.h

... 'ls init.c' could 'know' that it is  command #2 of 3, then I could 
grab line #2 from getpermtext(), and
if I could then assign that to a shell variable, I'd have exactly what I 
want: each command would have
access to the verbatim command line 'segment' that called it (that is, 
between the semicolons).



^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: first adventures
  2014-10-29 15:41       ` Ray Andrews
@ 2014-10-29 20:46         ` Bart Schaefer
  2014-10-29 23:31           ` Ray Andrews
  0 siblings, 1 reply; 18+ messages in thread
From: Bart Schaefer @ 2014-10-29 20:46 UTC (permalink / raw)
  To: zsh-workers

On Oct 29,  8:41am, Ray Andrews wrote:
}
} So, if there was just some way of matching each line of that output
} from getpermtext() with the appropriate command, then I'd be close to
} the finish line.

Maybe if you told us WHY you're trying to extract this particular bit
of information, in more detail than just --

} if I could then assign that to a shell variable, I'd have exactly what
} I want: each command would have access to the verbatim command line
} 'segment' that called it

-- it might be possible to make better suggestions.  I've been answering
on the assumption that you were just trying to learn how the internals
work, but it seems there's a larger motive.


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: first adventures
  2014-10-29 20:46         ` Bart Schaefer
@ 2014-10-29 23:31           ` Ray Andrews
  2014-10-30  4:07             ` Bart Schaefer
  0 siblings, 1 reply; 18+ messages in thread
From: Ray Andrews @ 2014-10-29 23:31 UTC (permalink / raw)
  To: zsh-workers

[-- Attachment #1: Type: text/plain, Size: 5325 bytes --]

On 10/29/2014 01:46 PM, Bart Schaefer wrote:
> On Oct 29,  8:41am, Ray Andrews wrote:
> }
> } So, if there was just some way of matching each line of that output
> } from getpermtext() with the appropriate command, then I'd be close to
> } the finish line.
>
> Maybe if you told us WHY you're trying to extract this particular bit
> of information, in more detail than just --
>
> } if I could then assign that to a shell variable, I'd have exactly what
> } I want: each command would have access to the verbatim command line
> } 'segment' that called it
>
> -- it might be possible to make better suggestions.  I've been answering
> on the assumption that you were just trying to learn how the internals
> work, but it seems there's a larger motive.
>
Well, both.  I have a real issue, and chewing on it seems like a good 
first motivation to dive
into the internals.  But I think my questions have been focused enough 
that, were it possible to
do as I'm seeking: break up the command line (in the .histfile format as 
we've discussed) into
separate individual commands (by semicolons and/or however else that is 
done), and be able to
pass that 'literal' command string to the command to which it applies:

    whats-my-line()
    {
        echo "My line was: $some-imaginary-variable"
    }

    $ whats-my-line $PWD * < > /temp/junk ; what's-my-line '$foo' bar;
    whats-my-line "I think that I shall never see"

    My line was: $PWD * < > /temp/junk
    My line was: '$foo' bar
    My line was:  "I think that I shall never see"

... then I'd be able to handle the rest.

Really, I just want to grab command arguments 'raw', I look at .histfile 
because that's where they
*are* stored raw (except for bangchar expansion, which is fine).

But since you ask about the larger motive (but I waste enough of your 
time already):

I like wrappers:


ell is the wrapper around 'ls'. The ,H switch says, 'show me the 
'expanded' (or 'real')
command you are about to execute, and write it to history
(for recall with possible modifications, as desired):

Comments are interspersed with screen capture:

     $ l ,H main.c*

    ... it produces a tarted up output:

    Sorting All file types Backwards by Modification Time (main.c*):

    ... and it shows you the 'real' command, and saves it to history:

    EXECUTING: ls --time-style=+%F//%T -AGFhrgdt
    --group-directories-first --color=always main.c* | egrep '' | egrep
    -v "^total"

    -rw-r--r-- 1 3.7K 2014-10-28//15:45:56 main.c.unc-backup~
    -rw-r--r-- 1 3.7K 2014-10-28//16:21:23 main.c
    -rw-r--r-- 1   41 2014-10-28//16:21:23 main.c.unc-backup.md5~

    ... The last two lines in history:

    14561* l ,H main.c*
    14562  ls --time-style=+%F//%T -AGFhrgdt --group-directories-first
    --color=always main.c* | egrep '' | egrep -v "^total"

    ... now, I recall and execute the last command in history:

    pts/7 HP-y5--5-Debian1 root /aMisc/Z/zsh-5.0.7/Src $ ls
    --time-style=+%F//%T -AGFhrgdt --group-directories-first
    --color=always main.c* | egrep '' | egrep -v "^total"
    -rw-r--r-- 1 3.7K 2014-10-28//15:45:56 main.c.unc-backup~
    -rw-r--r-- 1 3.7K 2014-10-28//16:21:23 main.c
    -rw-r--r-- 1   41 2014-10-28//16:21:23 main.c.unc-backup.md5~

      ... and the ouptut it exactly the same because it is exactly the 
same 'real' command.

It already works very much as I want, but the scripting is laborious 
because the shell can't be
restrained from expanding things.  Arguments to the function want to 
expand, but that makes
it difficult to show the 'EXECUTING: ...' line because I DONT want any 
expansions at all,
just the various arguments from the literal string that I typed. (If 
globs expanded you can
imagine the result.)

Oversimplified, I do it like this:

     alias l='noglob _l'
     function _l ()
     {
         ...
         # And once I've cobbled together the 'real' 'ls' command, I use 
'eval' to  expand any globs
         # and it's fine. (Tho I'd like to restrain variable expansion 
too, but it's not vital).
         echo "$the-big-long-ls-command" >> $HOME/.histfile
         fc -R
         echo "EXECUTING: $the-big-long-ls-command"
         eval "$the-big-long-ls-command"
     }

It doesn't matter what I type on CLI, whatever I type goes to history 
just the way
I want it. How to get it back? Easy if there was only one command on the 
line, but if several:

     $ l main.c*; l version.h; l * >! $PWD/file

     ... .histfile:

     14564  l main.c*; l version.h; l * >! $PWD/file

Now, If I could just 'match' each command at execution with the literal 
string from whence it
was called,  I'd have precisely what I'm looking for: The third call to 
ell above would know at
execution that it was called via the literal string: " l * >! $PWD/file 
". Knowing the literal string,
I could put together the 'EXECUTING' string with " $PWD " NOT whatever $PWD
evaluates to.

So ... if I can't grab the arguments to a command 'raw', then I'm hoping 
that somewhere in the code it might
be possible to tease apart the line in .histfile and send each fragment 
to the appropriate command via
a variable.  But I think that on principal a command should be able to 
grab it's arguments 'raw' if
it wants to. " $@ " = expanded argument list vs:  (made up:) " $!@ " = 
raw argument list.

Or something ....




^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: first adventures
  2014-10-29 23:31           ` Ray Andrews
@ 2014-10-30  4:07             ` Bart Schaefer
  2014-10-30 17:22               ` Ray Andrews
  2014-10-31  1:59               ` Ray Andrews
  0 siblings, 2 replies; 18+ messages in thread
From: Bart Schaefer @ 2014-10-30  4:07 UTC (permalink / raw)
  To: zsh-workers

On Oct 29,  4:31pm, Ray Andrews wrote:
>
> I have a real issue, and chewing on it seems like a good first
> motivation to dive into the internals. But I think my questions have
> been focused enough that, were it possible to do as I'm seeking: break
> up the command line (in the .histfile format as we've discussed) into
> separate individual commands (by semicolons and/or however else that
> is done), and be able to pass that 'literal' command string to the
> command to which it applies:

The problem with asking focused questions is that they feed on one's
preconceived notions of what needs to be done, rather than revealing
what has already BEEN done ...

torch% setopt DEBUG_BEFORE_CMD

(That's the default, but just in case.)

torch% TRAPDEBUG() { print -u2 "XX $ZSH_DEBUG_CMD XX" }
torch% print $path * > /dev/null
XX print $path * > /dev/null XX
torch% 

This only gets you as far as what the grammar calls a "sublist", so a
pipeline will get put into $ZSH_DEBUG_CMD in its entirety, but a new
trap is run after each semicolon:

torch% print $path * > /dev/null ; print foo | grep bar
XX print $path * > /dev/null XX
XX print foo | grep bar XX
torch% print $(echo foo | grep bar)
XX print $(echo foo | grep bar) XX
XX echo foo | grep bar XX

torch% 

Also for structured commands (loops, if-then) the DEBUG trap is called
once for the entire structure and then again for each sublist inside:

torch% repeat 3
repeat> do print foo
repeat> done
XX repeat 3
do
        print foo
done XX
XX print foo XX
foo
XX print foo XX
foo
XX print foo XX
foo
torch% 

This means you can work around the pipeline thing by using braces:

torch% { print $path * > /dev/null } && { print foo } | { grep bar }
XX {
        print $path * > /dev/null
} && {
        print foo
} | {
        grep bar
} XX
XX print $path * > /dev/null XX
XX print foo XX
XX grep bar XX
torch% 

The commands that appear inside the debug trap are not themselves
debugged, so you can examine other state there to decide for example
whether you want to copy $ZSH_DEBUG_CMD into another variable that
your wrapper can then examine.  (( $#functrace == 1 )) is probably
a good test for whether you are at the shell prompt, e.g.:

typeset -g THIS_TOPLEVEL_CMD
TRAPDEBUG() {
  (( $#functrace == 1 )) && THIS_TOPLEVEL_CMD="$ZSH_DEBUG_CMD"
}

Then your ell function can look at $THIS_TOPLEVEL_CMD like you want.

One final word:

>          echo "$the-big-long-ls-command" >> $HOME/.histfile
>          fc -R

Ouch; please don't do that.

	print -S "$THIS_TOPLEVEL_CMD"


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: first adventures
  2014-10-30  4:07             ` Bart Schaefer
@ 2014-10-30 17:22               ` Ray Andrews
  2014-10-31  1:59               ` Ray Andrews
  1 sibling, 0 replies; 18+ messages in thread
From: Ray Andrews @ 2014-10-30 17:22 UTC (permalink / raw)
  To: zsh-workers

On 10/29/2014 09:07 PM, Bart Schaefer wrote:
> On Oct 29,  4:31pm, Ray Andrews wrote:
>>
>> The problem with asking focused questions is that they feed on one's
>> preconceived notions of what needs to be done, rather than revealing
>> what has already BEEN done ...
Indeed sir.  I am very well aware of my limitations.  I work within the 
paradigm that I'm
used to, and expanding that will take time. To misquote Rumsfeld, there 
are things I know I don't know
and things I don't know I don't know and things I know that are wrong ;-)
>> torch% setopt DEBUG_BEFORE_CMD
Later ...
>
> Ouch; please don't do that.
>
> 	print -S "$THIS_TOPLEVEL_CMD"
>
Seems to work, and probably a whole lot less less trouble for the shell, 
I'm guessing. That's
a good example of something I wouldn't have even imagined could be done 
so simply.

Arigatou gozaimasu Sensei


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: first adventures
  2014-10-30  4:07             ` Bart Schaefer
  2014-10-30 17:22               ` Ray Andrews
@ 2014-10-31  1:59               ` Ray Andrews
  2014-10-31  2:59                 ` Bart Schaefer
  1 sibling, 1 reply; 18+ messages in thread
From: Ray Andrews @ 2014-10-31  1:59 UTC (permalink / raw)
  To: zsh-workers

[-- Attachment #1: Type: text/plain, Size: 2271 bytes --]

On 10/29/2014 09:07 PM, Bart Schaefer wrote:
> On Oct 29,  4:31pm, Ray Andrews wrote:
I believe you have done it, sir:

    #!/usr/bin/zsh


    typeset -g TLC
    TRAPDEBUG()
    {

       # This works perfectly to print directly to screen:
       #(( $#functrace == 1 )) && print -u2 "$ZSH_DEBUG_CMD"

       # Top Level Command capture:
       (( $#functrace == 1 )) && TLC="${ZSH_DEBUG_CMD}"
    }

    test()
    {
       # We only want the arguments so chop off the command itself:
       chop_cmd=${TLC/$0/}
       echo
       print -ru2 "My unexpanded arguments were: $chop_cmd"
       print -u2 "\n...and my arguments broken to an array:\n"

       array=("${=chop_cmd}")

       print -ru2 "two:   $array[2]"
       print -ru2 "three: $array[3]"
       print -ru2 "four:  $array[4]"
       print -ru2 "five:  $array[5]"
       print -ru2 "six:   $array[6]"
    }

It is proof against glob or variable expansion, it swallows redirection, 
and it doesn't mind compound command lines, however it does expand 
bangchar, as wanted:

    $ test ls $PWD; test ls g* > /dev/null; test l ,H $PWD/test; test 'I
    wandered \n lonely as a cloud'; test "that \n floats on high"

    My unexpanded arguments were:  ls $PWD

    ...and my arguments broken to an array:

    two:   ls
    three: $PWD
    four:
    five:
    six:
    My unexpanded arguments were:  ls g* > /dev/null

    ...and my arguments broken to an array:

    two:   ls
    three: g*
    four:  >
    five:  /dev/null
    six:

    My unexpanded arguments were:  l ,H $PWD/test

    ...and my arguments broken to an array:

    two:   l
    three: ,H
    four:  $PWD/test
    five:
    six:

    My unexpanded arguments were:  'I wandered \n lonely as a cloud'

    ...and my arguments broken to an array:

    two:   'I
    three: wandered
    four:  \n
    five:  lonely
    six:   as

    My unexpanded arguments were:  "that \n floats on high"

    ...and my arguments broken to an array:

    two:   "that
    three: \n
    four:  floats
    five:  on
    six:   high"


... I think I can't even  quibble about the quoted strings not being 
treated as single arguments
because I can't have it both ways--the whole exercise is to have zero 
processing.

And I don't even need to rape the src.






^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: first adventures
  2014-10-31  1:59               ` Ray Andrews
@ 2014-10-31  2:59                 ` Bart Schaefer
  0 siblings, 0 replies; 18+ messages in thread
From: Bart Schaefer @ 2014-10-31  2:59 UTC (permalink / raw)
  To: zsh-workers

On Oct 30,  6:59pm, Ray Andrews wrote:
>
>        # We only want the arguments so chop off the command itself:
>        chop_cmd=${TLC/$0/}
>        array=("${=chop_cmd}")

If you do this ...

	array=( ${(z)TLC} )
	chop_cmd = $array[1]
	shift array

... then this ...

> ... I think I can't even quibble about the quoted strings not being 
> treated as single arguments

... won't be a quibble any more.  Or you could even start out with TLC
as an array by doing TLC=( ${(z)ZSH_DEBUG_CMD} ) in TRAPDEBUG.

(We've drifted back to stuff that would be fine on zsh-users now ...)


^ permalink raw reply	[flat|nested] 18+ messages in thread

end of thread, other threads:[~2014-10-31  2:59 UTC | newest]

Thread overview: 18+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2014-10-26 17:20 first adventures Ray Andrews
2014-10-26 17:50 ` Bart Schaefer
2014-10-26 18:04   ` Bart Schaefer
2014-10-26 18:41     ` Ray Andrews
2014-10-26 19:57   ` Ray Andrews
2014-10-26 21:04     ` Bart Schaefer
2014-10-27  3:48       ` Ray Andrews
2014-10-27  6:08         ` Bart Schaefer
2014-10-26 17:52 ` Peter Stephenson
2014-10-28 17:48   ` Ray Andrews
2014-10-29  4:05     ` Bart Schaefer
2014-10-29 15:41       ` Ray Andrews
2014-10-29 20:46         ` Bart Schaefer
2014-10-29 23:31           ` Ray Andrews
2014-10-30  4:07             ` Bart Schaefer
2014-10-30 17:22               ` Ray Andrews
2014-10-31  1:59               ` Ray Andrews
2014-10-31  2:59                 ` Bart Schaefer

Code repositories for project(s) associated with this public inbox

	https://git.vuxu.org/mirror/zsh/

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).