timing builtins etc.

timing builtins etc.

[Peter Stephenson]

(Thought I'd sent this:  it looks like I exited Emacs before MH
finished sending it.)

I have been working on the bug that builtins and shell functions can't
be timed.  I found a neat way of doing it which simply involved adding
the current process to the job list.  This even supplied a more
natural way of fixing the previous bug that a builtin run at the end
of the pipeline lost its status to something before.  It was all very
swish, and it works fine for builtins.

Unfortunately it doesn't work for shell functions.  The culprit is the
list_pipe code in exec.c, which randomly deletes the job from the
table and it gets used for other purposes later (you can't simply
hijack the new current job).  So the code worked where the shell
function was inside a pipe, such as `echo foo | fn', but not on its
own.

Unfortunately, too, the list_pipe code is horrendously complicated and
completely uncommented and I've never understood any of it.  Can
anybody suggest anything?  Does anybody know why it randomly deletes
jobs?  Isn't there a neater way of fixing the original problem?

I'm tempted to pretend that the code is already in a pipeline when
`time' is used.  I'm not quite sure yet what effect this will have.

-- 
Peter Stephenson <pws@ifh.de>       Tel: +49 33762 77366
WWW:  http://www.ifh.de/~pws/       Fax: +49 33762 77330
Deutches Electronen-Synchrotron --- Institut fuer Hochenergiephysik Zeuthen
DESY-IfH, 15735 Zeuthen, Germany.

Re: timing builtins etc.

[Richard Coleman]

> (Thought I'd sent this:  it looks like I exited Emacs before MH
> finished sending it.)

Do you use mh-e?  It appears there are a lot of mh/exmh/mh-e
users on here (I use both mh and exmh).

> I have been working on the bug that builtins and shell functions can't
> be timed.  I found a neat way of doing it which simply involved adding
> the current process to the job list.  This even supplied a more
> natural way of fixing the previous bug that a builtin run at the end
> of the pipeline lost its status to something before.  It was all very
> swish, and it works fine for builtins.
> 
> Unfortunately it doesn't work for shell functions.  The culprit is the
> list_pipe code in exec.c, which randomly deletes the job from the
> table and it gets used for other purposes later (you can't simply
> hijack the new current job).  So the code worked where the shell
> function was inside a pipe, such as `echo foo | fn', but not on its
> own.
> 
> Unfortunately, too, the list_pipe code is horrendously complicated and
> completely uncommented and I've never understood any of it.  Can
> anybody suggest anything?  Does anybody know why it randomly deletes
> jobs?  Isn't there a neater way of fixing the original problem?
> 
> I'm tempted to pretend that the code is already in a pipeline when
> `time' is used.  I'm not quite sure yet what effect this will have.

Peter, could you send me a copy of what you do so far?  That way I can
keep that in mind when I work on this.

I'm also unhappy with the code in execpline and execpline2.  I've been
planning on cleaning that up, but I wanted to wait until we were at
a semi-stable positition (since I'm sure I'll break something).  Since
beta12 appears fairly stable, I'll try to work on that some.

There are a couple of things that I'll been thinking about for the
pipeline code.  The first is that it is completely undocumented.  I'll
try to fix that one.  The second is how it handles current shell
jobs.  In execpline2, there is the comment

/* if we are doing "foo | bar" where foo is a current *
 * shell command, do foo in a subshell and do the     *
 * rest of the pipeline in the current shell.         */

Does anyone know why this special case was added?  Because of this,
if you try to do something like

jobs | wc -l

it doesn't do what people expect.  Since `jobs' runs in a subshell,
there are no jobs to report, so the above line always returns 0.
I checked bash and it returns the correct answer.

rc

Re: timing builtins etc.

[Barton E. Schaefer]

On Nov 23, 11:16am, Richard Coleman wrote:
} Subject: Re: timing builtins etc.
}
} In execpline2, there is the comment
} 
} /* if we are doing "foo | bar" where foo is a current *
}  * shell command, do foo in a subshell and do the     *
}  * rest of the pipeline in the current shell.         */
} 
} Does anyone know why this special case was added?

It's not really a special case for pipelines; pipelines always fork
off the left side and then process the rest of the pipeline in the
current shell.  It's a special case for builtins and shell functions
only because they are not normally forked off at all.

You have to fork one side of the pipeline or the other in order to
set up the pipe between the processes.  You could do the builtin in
the current shell and fork a subshell for the right side, but it's
more logical to fork the left side first because it's the producer
on which the right side consumer depends.  Besides, a builtin
or shell function might appear somewhere in the right side, too.

} Because of this, if you try to do something like
} 
} jobs | wc -l
} 
} it doesn't do what people expect.  Since `jobs' runs in a subshell,
} there are no jobs to report, so the above line always returns 0.

Csh has this restriction, too.

You're surely aware that the reason that there are no jobs to report
has nothing to do with the actual process tree.  Any subshell could
print out the jobs list as easily as the parent could.  It's just
that the jobs list gets thrown away when a subshell is entered, so
there's nothing to print.

Zsh does the left side in a subshell because executing in a subshell
sets up all the correct entry and exit conditions (true fork rather
than vfork, etc.), so that e.g. the child won't mess up the "real"
shell's tty and so forth.  Ideally, the notion of being a forked copy
of the current shell would be separate from the notion of being a
lexical subshell, so that entry and exit conditions would be separate
from clearing the job table and other subshell requirements.  This
must be what bash is doing.

} I checked bash and it returns the correct answer.

What does bash do if you put "fg" on the left side of a pipe?  Note
that "jobs" and "fg" have traditionally been the same builtin at the
C code level in zsh, and for "fg" the process tree really does make
a difference.  And what does bash do with

    ( jobs ) | wc -l

??

-- 
Bart Schaefer                     Vice President, Technology, Z-Code Software
schaefer@z-code.com                  Division of NCD Software Corporation
http://www.well.com/www/barts

Re: timing builtins etc.

[wischnow]

Peter Stephenson wrote:

> 
> (Thought I'd sent this:  it looks like I exited Emacs before MH
> finished sending it.)
> 
> I have been working on the bug that builtins and shell functions can't
> be timed.  I found a neat way of doing it which simply involved adding
> the current process to the job list.  This even supplied a more
> natural way of fixing the previous bug that a builtin run at the end
> of the pipeline lost its status to something before.  It was all very
> swish, and it works fine for builtins.
> 
> Unfortunately it doesn't work for shell functions.  The culprit is the
> list_pipe code in exec.c, which randomly deletes the job from the
> table and it gets used for other purposes later (you can't simply
> hijack the new current job).  So the code worked where the shell
> function was inside a pipe, such as `echo foo | fn', but not on its
> own.
> 
> Unfortunately, too, the list_pipe code is horrendously complicated and
> completely uncommented and I've never understood any of it.  Can
> anybody suggest anything?  Does anybody know why it randomly deletes
> jobs?  Isn't there a neater way of fixing the original problem?
> 

Well, I should be able to say something about, but it's a while back
since I wrote it (one thing drives another thing out, you know...).

I only remember that after most of the stuff I wrote there worked
someone came up with an example of some compicated pipe that suddenly
stopped working. That made me add the deletepipejob() thing and the
first deletejob() in execpline(), but I don't remember the exact
problem.

I think it is clear that the whole exec.c/jobs.c stuff should be
rewritten (not doing so was the real mistake I made), but who will do
that (certainly not me ;-).

> I'm tempted to pretend that the code is already in a pipeline when
> `time' is used.  I'm not quite sure yet what effect this will have.
> 



Bye
 Sven


--
Sven Wischnowsky                         wischnow@informatik.hu-berlin.de