[9fans] rc: token buffer too short

[9fans] rc: token buffer too short

[Pietro Gagliardi]

Run /n/sources/contrib/pietro/eg (a work-in-progress troff  
preprocessor for graphs of equations). You get the error described  
above on a line that contains simply "else". What happened?

Re: [9fans] rc: token buffer too short

[Martin Neubauer]

Apparrently in quoted strings each character (rune, actually) constitutes
one token. The lexical scanner only holds NTOK (==8192) tokens at a time and
sam and wc convince me you have a 8555 byte string you pass to awk. You
should probably put the awk stuff into a separate file eg.awk and just do
`awk -f eg.awk $*' in eg (cf. chem(1) ). You might have to fiddle with where
you put things, but that seems to be the easiest way out.

	Martin

* Pietro Gagliardi (pietro10@mac.com) wrote:
> Run /n/sources/contrib/pietro/eg (a work-in-progress troff  
> preprocessor for graphs of equations). You get the error described  
> above on a line that contains simply "else". What happened?

Re: [9fans] rc: token buffer too short

[roger peppe]

a reason to use here files? :-)

On Dec 1, 2007 7:09 PM, Martin Neubauer <m.ne@gmx.net> wrote:
> Apparrently in quoted strings each character (rune, actually) constitutes
> one token. The lexical scanner only holds NTOK (==8192) tokens at a time and
> sam and wc convince me you have a 8555 byte string you pass to awk. You
> should probably put the awk stuff into a separate file eg.awk and just do
> `awk -f eg.awk $*' in eg (cf. chem(1) ). You might have to fiddle with where
> you put things, but that seems to be the easiest way out.
>
>         Martin
>
>
> * Pietro Gagliardi (pietro10@mac.com) wrote:
> > Run /n/sources/contrib/pietro/eg (a work-in-progress troff
> > preprocessor for graphs of equations). You get the error described
> > above on a line that contains simply "else". What happened?
>

Re: [9fans] rc: token buffer too short

[Pietro Gagliardi]

Not in this case, no. The format of awk is

	awk 'program' files
	awk -f prgm files

What I could do instead is

	u=/tmp/$0$pid$apid$0
	cat > $u <<\END
	program
	END
	awk -f $u $*
	rm $u

but I'd rather not go that way to avoid possible collisions.

Here's another problem. The error check function cats to [1=2].  
However, instead of going to standard output, it makes a file [1=2]  
and writes the message there. How do I fix this?

On Dec 11, 2007, at 8:32 AM, roger peppe wrote:

> a reason to use here files? :-)
>
> On Dec 1, 2007 7:09 PM, Martin Neubauer <m.ne@gmx.net> wrote:
>> Apparrently in quoted strings each character (rune, actually)  
>> constitutes
>> one token. The lexical scanner only holds NTOK (==8192) tokens at  
>> a time and
>> sam and wc convince me you have a 8555 byte string you pass to  
>> awk. You
>> should probably put the awk stuff into a separate file eg.awk and  
>> just do
>> `awk -f eg.awk $*' in eg (cf. chem(1) ). You might have to fiddle  
>> with where
>> you put things, but that seems to be the easiest way out.
>>
>>         Martin
>>
>>
>> * Pietro Gagliardi (pietro10@mac.com) wrote:
>>> Run /n/sources/contrib/pietro/eg (a work-in-progress troff
>>> preprocessor for graphs of equations). You get the error described
>>> above on a line that contains simply "else". What happened?
>>

Re: [9fans] rc: token buffer too short

[Douglas A. Gwyn]

Pietro Gagliardi wrote:
> What I could do instead is
>         u=/tmp/$0$pid$apid$0
>         ...
> but I'd rather not go that way to avoid possible collisions.

Seems that there could be a filesystem that hands out unique names.

Another similar approach would be for the open of a new special
file to create a unique temp file, which would vanish upon final
close, and have the app share/clone the file descriptor, which
could be dup2()ed (or whatever Plan9 equivalent is) to stdout,
stdin, etc., for the various processes in your script.

Re: [9fans] rc: token buffer too short

[roger peppe]

On Dec 11, 2007 7:46 PM, Pietro Gagliardi <pietro10@mac.com> wrote:
> Not in this case, no. The format of awk is
>
>         awk 'program' files
>         awk -f prgm files
>
> What I could do instead is
>
>         u=/tmp/$0$pid$apid$0
>         cat > $u <<\END
>         program
>         END
>         awk -f $u $*
>         rm $u
>
> but I'd rather not go that way to avoid possible collisions.

actually, you could use a here file and a named pipe:
awk -f <{cat << 'EOF'} $*
program
EOF

i do think that a quoted argument should be able
to take an arbitrary amount of text though.

> Here's another problem. The error check function cats to [1=2].
> However, instead of going to standard output, it makes a file [1=2]
> and writes the message there. How do I fix this?

awk uses ape/sh to run its commands, so you can fix it
by using >&2 as under unix.

On Dec 12, 2007 10:14 AM, Douglas A. Gwyn <DAGwyn@null.net> wrote:
> Another similar approach would be for the open of a new special
> file to create a unique temp file, which would vanish upon final
> close, and have the app share/clone the file descriptor, which
> could be dup2()ed (or whatever Plan9 equivalent is) to stdout,
> stdin, etc., for the various processes in your script.

that's similar to the named-pipe approach, except that you can't
seek on named pipes. i often find myself mounting
a new instance of ramfs, thus getting rid of the problem in most
cases (the exception being if you don't want to fork the namespace).