[9fans] rc here docs

[9fans] rc here docs

[boyd]

this is a a horrible bug:

brahma% fn foo {
	cat <<!
	echo foo
	!
	}
	!
brahma% whatis foo
fn foo {<<'/tmp/here1aa7.0000' cat;echo foo;! }
brahma% ls -l /tmp
--rw------- M 5073 boyd boyd 0 Aug 31 19:53 /tmp/A6818.boydsam
--rw-r--r-- M 5073 boyd boyd 0 Aug 31 19:27 /tmp/here18cc.0000
brahma% foo
/tmp/here1aa7.0000: rc: can't open: '/tmp/here1aa7.0000' directory entry not found
brahma% date
Sun Aug 31 20:56:57 EDT 2003

Re: [9fans] rc here docs

[David Presotto]

[-- Attachment #1: Type: text/plain, Size: 286 bytes --]

So what's the fix?  For that matter, what should the semantics be?
Should it be correct that while scanning a function body, the
action of <<token should be in abeiance?  That seems reasonable
to me, but I don't write very complicated shell scripts.  Is there
anyone that disagrees?

[-- Attachment #2: Type: message/rfc822, Size: 2065 bytes --]

From: boyd@sdgm.net
To: 9fans@cse.psu.edu
Subject: [9fans] rc here docs
Date: Sun, 31 Aug 2003 20:58:31 -0400
Message-ID: <57dbcd0684da6830d1654755467e74ad@sdgm.net>

this is a a horrible bug:

brahma% fn foo {
	cat <<!
	echo foo
	!
	}
	!
brahma% whatis foo
fn foo {<<'/tmp/here1aa7.0000' cat;echo foo;! }
brahma% ls -l /tmp
--rw------- M 5073 boyd boyd 0 Aug 31 19:53 /tmp/A6818.boydsam
--rw-r--r-- M 5073 boyd boyd 0 Aug 31 19:27 /tmp/here18cc.0000
brahma% foo
/tmp/here1aa7.0000: rc: can't open: '/tmp/here1aa7.0000' directory entry not found
brahma% date
Sun Aug 31 20:56:57 EDT 2003

Re: [9fans] rc here docs

[Scott Schwartz]

This works correctly (or at least as I expected) in Byron's rc.

; whatis foo
fn foo {cat <<<'        echo foo
'}

The `<<<' is a builtin operator, of which the manpage says:

     Additionally, rc supports ``here strings'', which  are  like
     here  documents,  except that input is taken directly from a
     string on the command line.  Their use is illustrated here:

          cat <<< 'this is a here string' | wc

    (This feature enables rc  to  export  functions  using  here
     documents  into  the environment; the author does not expect
     users to find this feature useful.)

Re: [9fans] rc here docs

[Geoff Collyer]

Getting here documents right inside functions and loops and loops
inside functions and in the background is tricky.  Even Bourne didn't
get it right (see BUGS in V7's sh(1)).  I think the SVR2 shell (thus
the V8 shell) got it mostly right, but I didn't push it hard.  There
are two main issues to sort out: when to start reading a given here
document (and collecting it into a temp file or perhaps into memory
nowadays) and when to remove the temp file.  This needs to work in the
presence of notes too.  So consider scripts like this one:

fn sigint sigexit {
	{
		for (i in 1 2 3) {
			cat <<!
going $i times...
!
			sleep 1
		}
	} &
	sleep 4
}
{
	for (f) {
		cat <<! $f <<! | lp
header
!
trailer
!
	}
} &
sleep 3
exec date

Note the extra complications: variable substitution in a here document
in a loop, and multiple here documents for a single command.  (Do all
the here-document temp files get removed if you type an interrupt?)
Getting all of this to work together is tricky, so I can't fault
Bourne or td.  As I recall, the SVR2 sh added a separate stack for
here documents, but even then I'm not sure it got all the cases right
(for example, does an "exec" command remove any outstanding
here-document temp files?).  If /tmp were always a ramfs, we could
probably ignore the hard cases of removing here-document temp files.
This still leaves cases like variable substitution in a here document
in a loop, for which you need to generate a different temporary file
on each iteration.  According to rc(1), we don't have to deal with one
of the messy cases that sh had to:

cat <<!
stuff
`cat <<!!
inner stuff for $USER
!!
`
more stuff
!

It's evident from the code (V7 sh) that here documents were a late
addition to the shell, hacked in rather hurriedly, perhaps to
eliminate the need for the PWB/UNIX pump command, which had been used
with the PWB shell to give a similar effect.

So: do we need here documents at all?  Is

cat <<! | somecommand
stuff
$user
more stuff
!

any more useful or noticeably more convenient than

echo 'stuff
'$user'
more stuff' | somecommand

on Plan 9, where we have more-or-less unrestricted command-line
arguments (unlike V7 Unix, which had a limit of 5120 bytes, including
environment if I can trust my memory).

How are others using here documents?

Re: [9fans] rc here docs

[boyd, rounin]

guess i use here docs 'cos i've written a lotta complex shell scripts.

i guess, back with V7, a lotta echo's wouldn't be cheap.

Re: [9fans] rc here docs

[boyd, rounin]

> This works correctly (or at least as I expected) in Byron's rc.

i'm drawterm'd to ny and it's plan 9 4th ed.

i'm using echo.

Re: [9fans] rc here docs

[Scott Schwartz]

| > This works correctly (or at least as I expected) in Byron's rc.
|
| i'm drawterm'd to ny and it's plan 9 4th ed.

I heard questions posed about what the right behaviour was, and how
to implement it, so I provided an example of what I think is the right
answer.

| i'm using echo.

You might be able to port Byron's shell. :)
Did anyone ever get es running under Plan 9?

Re: [9fans] rc here docs

[boyd, rounin]

> You might be able to port Byron's shell. :)

hasn't vita already done that with the plethora of shells they have?

Re: [9fans] rc here docs

[Geoff Collyer]

I'm not suggesting

	{ echo foo; echo bar; echo baz; echo quux } | cmd

but rather

	echo 'foo
bar
baz
quux' | cmd

Given one echo command to replace one here document, I don't think the
here documents are substantially cheaper than using echo.  With echo,
you have to load echo from disk (modulo caching) and it writes your
message down a pipe.  With a here document, the shell writes the here
document into a temporary file, then attaches that temporary file as
the command's standard input, so the command can read what the shell
just wrote, then eventually the shell removes the temporary file;
again this is all nominally disk i/o.  (To make matters slightly
worse, sh did unaligned writes of lengths not a multiple of 512 to the
temporary files.)  So here documents are not cheap and they do seem to
clutter shells with considerable added complexity.  I could probably
live without them except that bundle uses them, sometimes to transmit
large files that might even exceed Plan 9's ability to pass them as
command-line arguments.

Re: [9fans] rc here docs

[boyd, rounin]

> I could probably live without them except that bundle uses them, ...

oh i had forgotten about that.  bundle is a cool thing.

i hate all that tar.gz nonsense; if you have to compress it, well it's too big.

Re: [9fans] rc here docs

[boyd, rounin]

> { echo foo; echo bar; echo baz; echo quux } | cmd

now it's affordable and it's because more readable i'm using it:

{
    echo Either you''''re a spammer or I don''''t like you or I don''''t know
you.
    echo
    echo Email me this rejected message with:
    echo
    echo '     ' $code
    echo
    echo in the Subject: line and I''''ll decide if I''''ll add you to my list
    echo of people I''''ll accept mail from.
} | ...

Re: [9fans] rc here docs

[andrey mirtchovski]

have you looked at this:

	http://tmda.sourceforge.net/

it has the same feel to it as yours does (judging from the little snippet
below), even though it's considerably larger and more complex (but can be
fooled by the latest '6 degrees of freedom' viruses, just like your filter
can)...

an added bonus -- it's being used by the FSF for @gnu.org addresses :)
and another bonus -- it's written in python, so it could just work in
Plan 9.

andrey

On Mon, 1 Sep 2003, boyd, rounin wrote:

> > { echo foo; echo bar; echo baz; echo quux } | cmd
>
> now it's affordable and it's because more readable i'm using it:
>
> {
>     echo Either you''''re a spammer or I don''''t like you or I don''''t know
> you.
>     echo
>     echo Email me this rejected message with:
>     echo
>     echo '     ' $code
>     echo
>     echo in the Subject: line and I''''ll decide if I''''ll add you to my list
>     echo of people I''''ll accept mail from.
> } | ...
>

Re: [9fans] rc here docs

[boyd, rounin]

> have you looked at this:
>
> http://tmda.sourceforge.net/

thanks, but no.  i can't stand sourcefarce (sic) and the lunix zealots.

Re: [9fans] rc here docs

[Charles Forsyth]

[-- Attachment #1: Type: text/plain, Size: 146 bytes --]

we kept mash in case people were using it, but we do everything (or at
least roger does) with sh.  arguably they are all still too complicated.

[-- Attachment #2: Type: message/rfc822, Size: 2609 bytes --]

From: "boyd, rounin" <boyd@insultant.net>
To: <9fans@cse.psu.edu>
Subject: Re: [9fans] rc here docs
Date: Mon, 1 Sep 2003 05:14:16 +0200
Message-ID: <020901c37037$2081de40$b9844051@insultant.net>

> You might be able to port Byron's shell. :)

hasn't vita already done that with the plethora of shells they have?

Re: [9fans] rc here docs

[Douglas A. Gwyn]

boyd, rounin wrote:
> i hate all that tar.gz nonsense; if you have to compress it, well it's too big.

A lot of software systems worth going to the trouble of porting
are inherently pretty large, and the compression helps a lot for
folks who are stuck with dial-up modem access.

Re: [9fans] rc here docs

[rog]

the main advantage of here documents as far as i can see is a
syntactic one; it's nicer to see:

cat << EOF
don't care about ' marks.
$home is where the heart is.
EOF

than its near-equivalent:

echo 'don't care about '' marks.
'^$"home^' is where the heart is.' | cat

particularly when you've got lots of text as a template.  balancing
single quotes is very easy to get wrong.

i do sometime regret the lack of here documents in the inferno shell
for that reason (although actually putting them into /tmp seems a
little archaic)

i wonder if there might be room for a "here word" i.e.  something that
is equivalent to a word, but looks syntactically something like a here
document.  dunno what the syntax would look like though.