Re: wishlist

Re: wishlist

[Tom Culliton x2278]

Alan finds a reason why he doesn't need read:

> So, there I was, walking home after work, muttering under my breath,
> and mulling over the analogy between rc having to exec to read a line
> and C having to perform a system call to read a char.  I though I had
> the clincher in this argument.  Then it dawned on me -- we can use
> buffered IO in rc (read many lines with one exec) exactly as we do in C
> (read many chars with one system call).

Strangely enough it was thoughts along this line that that convinced me
that read might be a good idea!  My first thought was that if execing
for each line was too slow why not just do something like this:

	buf = ``($nl) { cat }

And then just step through the list in any of the usual ways.  Of course
the problem is that even rc may not be able to stuff the whole file into
the environment, other programs could choke big time, and it would slow
down forking.  The next step is to get clever and use sed to grab
chunks:

	eof=()
	buf = ``($nl) { sed 20q }
	if (~ $#buf 0) eof = 1;

Of course this is also going to bloat the environment, albeit to a
lesser extent, with the buffer, code to fetch from the buffer and
transparently refill it when it empties, not to mention code to reset
the buffer, etc.  (I may write this up and post it later.)

In shell scripts read is used for two different purposes, getting one
line from a user, or chewing through a theoretically infinite input
stream one line at a time.  In the first case efficiency is usually not
a concern, but in the second case it obviously is.  The fact that the
same generalized facility would serve both cases well, and that these
are fairly common things to do, seem like a good argument for adding
read. (or something like it)

Tom

Re: wishlist

[Tom Culliton x2278]

> You can't use sed, as you don't know how much it buffers stuff up internally.
> Same with awk.
> 
> You cannot use ifs = $nl { foo = `{ cat } }, as you lose blank lines
> and cannot tell if the final line ends in a $nl or not.  I had enough
> of that kind of stuff 

You're absolutely right.  My first draft actually started out saying
that sed wouldn't work for pipes because you couldn't rescan the input
and provided an external command called "lines" (a generalization of
"line").  Unfortunately when I came back from lunch the reason for this
had slipped my mind, so it got cut.  (How embarassing!)  All the more
reason to add read.

Tom

Re: wishlist

[Alan Watson]

I didn't intend to cc that last message to the list.  The C code is
awfully embarrassing.

Re: wishlist

[Alan Watson]

You can't use sed, as you don't know how much it buffers stuff up internally.
Same with awk.

You cannot use ifs = $nl { foo = `{ cat } }, as you lose blank lines
and cannot tell if the final line ends in a $nl or not.  I had enough
of that kind of stuff 

I coded this up in about 10 minutes last night -- it works as a
technology demonstrator, but (a) the C code needs re-writing to read N
chars first and then one char at a time until the end of the line, and
(b) it all needs some better error checking.

bufread reads a line at a time into the variable, including any
trailing newline, and sets the variable to () at EOF.

Hope this helps you get a start,

Alan.

fn copy {
   @ {
      exec <$1 >$2
      while ( bufread line ibuf && ! ~ $#line 0 ) 
	 bufwrite line obuf
      bufflush obuf
   }
}

fn bufread {
   _var = $1 _buf = $2 {
      if ( ~ $#$_buf 0 ) {
	 ifs = () { eval '*' '=' `{ ./buffill } }
      } else {
	 * = $$_buf
      }
      $_var = $1
      if ( ! ~ $#* 0 )
	 shift
      $_buf = $*
   }
}

fn bufwrite {
   _var = $1 _buf = $2 {
      if ( ~ $#$_buf 4 ) {
	 bufflush $_buf
	 $_buf = ()
      }
      $_buf = ( $$_buf $$_var )
   }
}

fn bufflush {
   _buf = $1 _text = () {
      * = $$_buf
      while ( ! ~ $#* 0 ) {
	 _text = $text^$1
	 shift
      }
      echo -n $text
   }
}

buffill.c:


#include <unistd.h>

#define N 256

#define writestr(fd,s) (write ((fd),(s), sizeof (s) - 1))

int main ()
{
   char c;
   char lastc;
   int n;

   writestr (1, "( ");

   n = 0;
   while (n < N || c != '\n') {
      if (read (0, &c, 1) != 1)
	 break;
      if (n == 0 || lastc == '\n')
	 writestr (1, "'");
      if (c == '\'')
	 writestr (1, "''");
      else if (c == '\n')
	 writestr (1, "\n' ");
      else
	 write (1, &c, 1);
      lastc = c;
      n++;
   }

   if (n != 0 && c != '\n')
      writestr (1, "' ");

   writestr (1, ")");

   return 0;
}

Re: wishlist

[Bengt KLEBERG]

> From rc-owner@hawkwind.utcs.toronto.edu Wed May 26 05:07:39 1993
> Date: 	Tue, 25 May 1993 23:02:18 -0400
> From: netapp!byron@netcom.com (Byron Rakitzis)
> To: culliton@srg.af.mil, rc@hawkwind.utcs.toronto.edu
> Subject: Re: wishlist
> Content-Length: 329
> 
> I think it is a little too late to change $^ to $", but maybe that's
> just me. I know it was a drag for me when I switched from if not to
> else, and that was when there were just a few rc users. Now I put the
> install base at 3 figures, possibly 4. I would have to have an extremely
> good reason to break existing scripts right now.
> 
The question is perhaps more rightly put as: How many uses $^ ( and
would need to change to $" ). I must admit that I don't and therefore
I am in favor for getting $" in. It might make me lookup what it is :-)

I also think 'if not' was cute and would happily fix the (very) few
'else' that I have if I got it back.

Bengt

Re: wishlist

[Chris Siebenmann]

 Let's not break things without an excellent reason.
 At a minimum, let's support both $^ and $" for one release, so people
*don't* have to have a flag day where everything breaks until they fix
it.

	- cks

Re: wishlist

[mycroft]

> The question is perhaps more rightly put as: How many uses $^ ( and
> would need to change to $" ).

I think nearly every rc script I've ever written has a $^ in it
somewhere.

Re: wishlist

[Alan Watson]

So, there I was, walking home after work, muttering under my breath,
and mulling over the analogy between rc having to exec to read a line
and C having to perform a system call to read a char.  I though I had
the clincher in this argument.  Then it dawned on me -- we can use
buffered IO in rc (read many lines with one exec) exactly as we do in C
(read many chars with one system call).

The interface might be a little tricky, and one would want to be
careful of having so much stuff in the environment, but it may well be
possible.  

Therefore, I withdraw my request for a built-in read, withdraw my
sugestion that echo should be a de jure built-in, and consequently
withdraw my suggestion that herestrings be removed.  To celebrate I may
even go and compile up rc without echo.

Sorry to have wasted your time with this one,

Alan.

Re: wishlist

[Byron Rakitzis]

I think it is a little too late to change $^ to $", but maybe that's
just me. I know it was a drag for me when I switched from if not to
else, and that was when there were just a few rc users. Now I put the
install base at 3 figures, possibly 4. I would have to have an extremely
good reason to break existing scripts right now.

Re: wishlist

[Scott Schwartz]

You could add $" without removing $^.

Re: wishlist

[Tom Culliton x2278]

(Damn!  How do I manage to get such vigorous "discussions" going?)

Re: what to take out (Alan Watson)

This was not my intent but I do have a favorite candidate, which never
made sense to me: `` which is just syntactic sugar for ifs=(...) {...}
Everything else seems like it really belongs.

Re: empowering features/simplifications (Erik Quanstrom)

I think rc has already done pretty well here, and can't think of any
proposed addition that really seems like a big win to me.  Possibly read
so you can chew up text files (awk is probably a better choice), or flag
so you can manipulate the behaviour.  Nothing else seems like it gives
you the kind of big win that lack of rescanning, variables as lists,
clean quoting, or even builtin ~ does.

Re: Conformance with Plan 9 rc (John Mackin)

John correctly points out that you'll probably never be able to port a
Unix rc script to Plan 9 or vice-versa just because the environments are
so different, and that Byron has made some major improvements and fixed
a number of the bugs listed in the Plan 9 man page.  I still think it
would be nice to keep the two as close as possible, especially for
arbitrary differences like $" vs. $^.  This makes the various Plan 9
docs and papers more useful and otherwise broadens our experience base.

Tom

Re: wishlist

[Alan Watson]

John writes:

> Why are people looking for features to throw out of rc, anyway?

I'm English, so I like whining; an Australian should understand that.

> There are still some of us -- probably everyone on this list -- who believe
> that software bloat is bad, and that smaller, faster code is better.

Yes, but man page bloat is worse.

So, let me defend some of my statements:

(a) Here strings.

If echo is a built-in, here strings are redundant with echo and pipes
(more on why I think echo should be a built-in later).  Will anyone
argue with me that a redundant feature WHICH USERS NEVER USE DIRECTLY
should leave the language?

The only caveat is if the code to re-write functions in terms of echo
and pipes is more likely to be a problem to maintain than the current
code, and if Byron feels he has better things to do with his time (and
he almost certainly does).

(b) ``

> Its convenience certainly justifies its redundancy many times
> over.  It must stay.

I will accept an argument of convenience if you calculate the factional
occurance of `` in your history file, and if that faction is larger
than, say, 0.5%.  Typing convenience just doesn't wash in scripts.

Two ways to do one thing is bad, bad, bad.  Really sinful.  We should
either junk `` or ifs, and since the later comes handed down from the
heavens, the former must go.

(c) newpgrp

Perhaps I'm speaking from ignorance here, but what about a stand-alone
binary along the lines of nice, nohup, et al?  If people really want it
built-in, then they can hack addon.h.

(d) $^foo

Oops, Malte shows me up.  Same arguments as (b).

As I've mentioned in private email to Byron, I'm not going to stop
using rc just because these feature remain.  But my honest opinion is
that if the time Byron had to devote to rc was limitless, they should
be purged.  I'm sure he has better things to do, though.

Now, the big one, why should echo (and read) be built-in?

If all a shell does is fork and exec commands, they need not be there.
However, I would like to be able to use rc in the same manner as `awk'
-- read some data, match on it using ~, mess with it with some external
programs, and spit it out the other end.  awk is great for some stuff,
but it has it's limitations.  For example, I want be be able to parse
mail folders in rc.  To do this efficiently in rc, I need echo and read
to be built-in.  (This was never done very much in sh because quoting
becomes a nightmare.)

Perhaps this is misguided, but it seems a modest investment (two
primitives, both with historical precedent) for a vast increase in rc's
potential.

If echo becomes a de jure built-in, rather than remaining de facto, you
can still use "fn echo { /bin/echo $* }".

Re: wishlist

[malte]

In response to  Alan Watson <alan@oldp.astro.wisc.edu>:

	(a) Here strings: use "echo 'foo' | bar" instead of "bar <<< 'foo'"
		For more complex cases, like "bar <<<[0]'foo0' <<<[1]'foo1'" use "echo
		'foo0' |[1=0] { echo 'foo1' |[1=1] bar }"

Well, I made this mistake too:
	echo 'foo' != 'foo'
but
	echo -n 'foo' == 'foo'

I think it's too much typing ( see answer to backqoutes ). But I wouldn't
bother if here string were removed because I don't use them anyway.


	(b) The flattening operator: use a function instead

rc lists are always flat, so reading your function definition, I assume
you mean joining a lists elements into a single element. If this is the case
use
	fn flatten {$1=``(){echo -n $$1}}

because your flattening function uses the flattening operator it tries to
replace.

	(c) `` backquote substitution: use

Yes, this will surely simplify the parser, but in my experience simplicity
is not the ultimate goal. In a real worlds shell you have to watch for
conveniencies, too. Whenever I try to persuade people to use rc, after
a while they'll complain about too much typing because of oversimplified
quoting rules. That's why I asked for backslash escapes. I like the "``"
mechanism very much. And besides, what should be The Right Thing to do

	; ifs = () { echo $#ifs $ifs }
	0
or
	1
I mean, should ifs be unset or set to an empty list ? What does it mean if
ifs not set ? If set to an empty list, this would be different from every
other variable.

	(d) newpgrp: use a wrapper around rc

You'll loose the history list when using readline or editline.

And another thing that hasn't been discussed here: What about compatibility
with plan 9 rc ?

Malte

Re: wishlist

[John Mackin]

I'm quoting Malte.

    [...] I wouldn't bother if here string were removed because I don't
    use them anyway.

[He basically supports here strings, though.]  The point all the here-string
bashers are missing is that here strings are ESSENTIAL.  Quite apart from
arguments from NCARGS, which will doubtless be shot down on the grounds
that ``echo is a builtin'' (not in _my_ rc it isn't!!), the point is that
here _documents_ are convenient (I don't hear anyone arguing that here
documents should be ditched), and rc needs here strings in order to
export functions containing here documents!  Yes?  Try exporting a
function using a here document sometime using a Pike sh.  Make
sure you have something to vomit into handy.  And for the curious,
the very last sentence of the Plan 9 rc manual entry is:

	Functions that use here documents don't work.

Is that what you'd rather have?  Here strings stay.

    [Referring to a proposal to ditch ``:]
    Yes, this will surely simplify the parser, but in my experience simplicity
    is not the ultimate goal. In a real worlds shell you have to watch for
    conveniencies, too.

This is precisely, 100%, absolutely on-target!  We don't want a shell we
can prove theorems about.  We want a shell that is _nice to use._  Everything
should be made as simple as possible, _and no simpler._  Oversimplifying
leads to, as Byron once put it very well, a ``Bell Labs pissing contest.''
We don't pursue minimalism for its own sake.  We pursue it for _our_ sakes.
There are still some of us -- probably everyone on this list -- who believe
that software bloat is bad, and that smaller, faster code is better.  That
doesn't mean that we want to edit text files using "cat >" (I'm typing
this in sam :).

    Whenever I try to persuade people to use rc, after a while they'll
    complain about too much typing because of oversimplified quoting
    rules.

I don't know who these people are, or what kind of people they are,
but if that's how they think I am inclined to think that their opinions
are valueless, since they have no taste in software.  rc's quoting rules
are just about the best thing about the whole damn design, along with
the `this is not a macro language, there is no rescanning' paradigm.
I have heard many complaints directed at rc since mid-1991 when I
first started proselytising about it, but this one (that the quoting
rules are too simple) has _never_ been among them.  I think you have
just shown the shell to some very, very weird people.  Let me guess --
they edit with emacs, yes?

    That's why I asked for backslash escapes.

Happily, you didn't and won't get them.  With luck, Paul will throw them
out of es in the current pruning drive (plug plug :).

    I like the "``" mechanism very much.

So do I.  Its convenience certainly justifies its redundancy many times
over.  It must stay.  Why are people looking for features to throw out
of rc, anyway?  It isn't too big.  This whole discussion started with
Tom pointing out features that _should not be added_, changes that
_shouldn't_ be made.  IMNSHO, design-wise, Byron's rc is very, very
close to perfect right now.  There's no need to look for things to
take out.

    	(d) newpgrp: use a wrapper around rc

    You'll loose the history list when using readline or editline.

newpgrp is just an essential builtin in a non-job-control shell that
lives in a job-control world.  Really.

    And another thing that hasn't been discussed here: What about compatibility
    with plan 9 rc ?

This is a big, big issue (in my mind, anyway, I don't know how others feel).
The way I see it, Byron has improved on Duff's design very substantially.
I can't speak to quality of implementation, since I have never read
Duff's sources (although I now have legal access to them), but Byron's
sources are roughly twice as many characters as Duff's.  I don't think
that is in any way indicative that Byron's version is twice as complicated,
since Byron's rc is a rock-solid, production-quality _portable Unix
program_.  Duff's rc is a Plan 9 program.  Like most Plan 9 native
programs, you might even be lucky if you can compile it with a
non-Plan 9 compiler, and when it comes to true portability, forget it.
Plan 9 programs are not designed or implemented to port to other
than Plan 9.  Yes, I know the title of Duff's paper.  I still claim
his rc will definitely NOT compile and work in anything like the
variety of environments that Byron's will.

Gratuitous back-porting of Plan 9 features into Unix programs _must
be avoided_.  I don't have time to speak to this in detail now,
but the point is, just because something is The Right Thing to
do in the Plan 9 environment does NOT make it The Right Thing
to do in a Unix/X environment.  Plan 9 is nice, but it's Plan 9.
Unix is Unix.  rc scripts for Unix won't port to Plan 9 anyway,
unless they're trivial -- and do you REALLY want to see "if not"
instead of "else"?  Come off it.  Byron has _done the job_.  Let
him fix these last few tiny rough spots and then _that's it_, OK?

Really.  Byron's rc is better than Duff's and certainly isn't
broken.  And we _all_ know what not being broken means when
it comes time to think about fixing...

OK,
John.

Re: wishlist

[Alan Watson]

So what should go?

(a) Here strings: use "echo 'foo' | bar" instead of "bar <<< 'foo'".
For more complex cases, like "bar <<<[0]'foo0' <<<[1]'foo1'" use "echo
'foo0' |[1=0] { echo 'foo1' |[1=1] bar }"

(b) The flattening operator: use a function instead

   fn flatten {
      if ( ~ $#* 0 ) {
	 echo >[2=1] 'usage: flatten list ...'
	 return 1
      }
      list = $1 {
	 shift
	 $list = $1
	 while ( ! ~ $#* 0 1 ) {
	    shift
	    $list = $list^' '^$1
	 }
      }
   }

(c) `` backquote substitution: use

   ifs = ( foo ) { bar = `{ baz } }

(d) newpgrp: use a wrapper around rc.

(e) cdpath: do it with a function.

Re: wishlist

[Erik Quanstrom]

Suppose we considered changes to rc which might promote small & 
simple.

Re: wishlist

[Tom Culliton x2278]

WHOA!  My earlier message seems to have been misinterpreted.

> From: Scott Schwartz <schwartz@groucho.cs.psu.edu>
----------------------
> My biggest wish:
> 
> I'd like 
> 	fn / { echo do stuff }
> to work.

> From: Emin Gun Sirer <egsirer@phoenix.princeton.edu>
----------------------
> P.S. Someone forgot to add the controversy about hash not introducing a
> comment if it is not preceded by white space (e.g. lpr -#2), my personal
> pet peeve, to the wish list.

My intent WAS NOT to publish a wish list of things which should be
changed in or added to rc.  Instead it was to show some of the things
that people had asked for and hopefully point out that if we did all
these things we would end up with a ugly bloated mess.  I was urging
restraint rather than soliciting more crazed featurism.  In case I'm
still not making myself clear, some reasonable ideas have been discussed
along with some REALLY bad ones, I don't think most of them belong in
rc.  It is supposed to be small and simple, IT IS small and simple, LETS
KEEP IT THAT WAY!

Tom

wishlist

[Scott Schwartz]

My biggest wish:

I'd like 
	fn / { echo do stuff }
to work.