Re: [9fans] multiple inputs

Re: [9fans] multiple inputs

[npe]

How would one go about merging several files at once?

I'm working on a crappy little barebones AIM toc client running in plan 9. It tentatively implements itself as a file server like this

	/mnt/aim/buddies/bob/send # messages you send
	/mnt/aim/buddies/bob/recv # messages you get
	/mnt/aim/buddies/bob/status # away, idle etc...

messages are sent by
	echo hello>/mnt/aim/buddies/bob/send

and read by
	tail -f /mnt/aim/buddies/bob/recv

bots attach by:
	tail -f /mnt/aim/buddies/bob/recv | eliza >/mnt/aim/buddies/bob/send

I want to get functionality something like a traditional aim client by combining all three files, so new data in any of the three files come out at the same place(like doing three tail -f's in the same acme errors window).

What's the best way in Plan 9?

Also, is the interface reasonable(thanks to Andrey for the initial help)? Any other ideas?

Noah

Re: [9fans] multiple inputs

[matt]

will this do ?

tail -f file1 &
tail -f file2 &
tail -f file3 &

m

Re: [9fans] multiple inputs

[glenda]

[-- Attachment #1: Type: text/plain, Size: 314 bytes --]

That's pretty close thanks :)

I had forgotten the way rc's brackets work.

This is what I wanted

fn bt {
		eval '{' `{for (i in $*)  echo 'tail -f '$i '&'} '}'
	}

Gives me the behavior I wanted for toc and let's me put my log files together for debugging(i.e. smtp and smtp.fail }

Thanks,

Noah

[-- Attachment #2: Type: message/rfc822, Size: 2622 bytes --]

From: matt@proweb.co.uk
To: 9fans@cse.psu.edu
Subject: Re: [9fans] multiple inputs
Date: Tue, 30 Mar 2004 09:22:39 +0100
Message-ID: <dbc73bf3ff6fea1664236932103db873@juice.thebigchoice.com>

will this do ?

tail -f file1 &
tail -f file2 &
tail -f file3 &

m

Re: [9fans] multiple inputs

[matt]

untested but surely :

fn bt {
	for (i in $*)   { {	tail -f $i} & }
	}

or even

fn tail_it {
	tail -f $1 &
}

fn bt {
	for (i in $*)  tail_it $i 
}

m

Re: [9fans] multiple inputs

[glenda]

[-- Attachment #1: Type: text/plain, Size: 439 bytes --]

Sorry I wasn't clear enough in my explaination.

You need to put the brackets on the outside to make the combination of tail -f's act like a single 'tail -f'. That was my real conceptual misunderstanding the first time around.

If you just do all the 'tail &' without bracketing them you get dumped back into the shell immediately which isn't the sort of behavior I was thinking about, although perfectly valid in many cases.

Noah

[-- Attachment #2: Type: message/rfc822, Size: 2717 bytes --]

From: matt@proweb.co.uk
To: 9fans@cse.psu.edu
Subject: Re: [9fans] multiple inputs
Date: Tue, 30 Mar 2004 15:03:10 +0100
Message-ID: <22c65f87f1174822b66d9c35b6ed9c14@juice.thebigchoice.com>

untested but surely :

fn bt {
	for (i in $*)   { {	tail -f $i} & }
	}

or even

fn tail_it {
	tail -f $1 &
}

fn bt {
	for (i in $*)  tail_it $i 
}

m

Re: [9fans] multiple inputs

[rog]

surely rather than using the rather slow polling behaviour of tail -f
it would be better to have the files block on read until data is
available.

then you could just do:

	cat send & cat recv & cat status

no long latency on messages arriving, and you get the same
functionality.

also, if you've got a lot of traffic, you get correct interleaving of
messages, as the server controls the size of the data packets that cat
reads, which isn't the case for tail -f.

slightly harder on the server side, but not difficult, and a good
excuse to explore ways of structuring that kind of thing.

Re: [9fans] multiple inputs

[Noah Evans]

[-- Attachment #1: Type: text/plain, Size: 30 bytes --]

Good advice, thanks.

Noah

[-- Attachment #2: Type: message/rfc822, Size: 3110 bytes --]

From: rog@vitanuova.com
To: 9fans@cse.psu.edu
Subject: Re: [9fans] multiple inputs
Date: Tue, 30 Mar 2004 17:07:24 +0100
Message-ID: <0bfa30b4e4570a291f4ea406d1a699db@vitanuova.com>

surely rather than using the rather slow polling behaviour of tail -f
it would be better to have the files block on read until data is
available.

then you could just do:

	cat send & cat recv & cat status

no long latency on messages arriving, and you get the same
functionality.

also, if you've got a lot of traffic, you get correct interleaving of
messages, as the server controls the size of the data packets that cat
reads, which isn't the case for tail -f.

slightly harder on the server side, but not difficult, and a good
excuse to explore ways of structuring that kind of thing.

Re: [9fans] multiple inputs

[rog]

> You need to put the brackets on the outside to make the combination of
> tail -f's act like a single 'tail -f'.  That was my real conceptual
> misunderstanding the first time around.

erm, i don't think so.

{a & b &}

should be exactly the same as

a & b &

you almost never need eval in rc, unless you genuinely want
to evaluate an rc expression that's unknown beforehand
(which isn't true in your case).

you could have got the behaviour i think you were after
with something like:

fn bt {
	x=$1
	shift
	for(i) {tail -f $i &}
	tail -f $x
}

Re: [9fans] multiple inputs

[Noah Evans]

[-- Attachment #1: Type: text/plain, Size: 665 bytes --]

I'll agree with you there. The eval is unnecessary.

But 

fn bt {
	eval '{'for {tail -f $i &} '}'
}

mimics how I would write it by hand.

{tail -f file1 &; tail -f file2 &; tail -f file3 &}>file

Which allows me to consider the command one block, which I can pipe to and from etc.

The idiom that you and matt give doesn't work for pipes

tail -f file1 &; tail -f file2 &; tail -f file3 >file

only puts the text from file3 in the pipe. 

The for loop you gave obviates that problem, since it counts as a single block, but it's considerably different than my interactive shell usage and is more verbose(though much more readable :)).

Noah

[-- Attachment #2: Type: message/rfc822, Size: 3075 bytes --]

From: rog@vitanuova.com
To: 9fans@cse.psu.edu
Subject: Re: [9fans] multiple inputs
Date: Tue, 30 Mar 2004 17:25:46 +0100
Message-ID: <19236d27e19c4ebbe4ba92e507e29149@vitanuova.com>

> You need to put the brackets on the outside to make the combination of
> tail -f's act like a single 'tail -f'.  That was my real conceptual
> misunderstanding the first time around.

erm, i don't think so.

{a & b &}

should be exactly the same as

a & b &

you almost never need eval in rc, unless you genuinely want
to evaluate an rc expression that's unknown beforehand
(which isn't true in your case).

you could have got the behaviour i think you were after
with something like:

fn bt {
	x=$1
	shift
	for(i) {tail -f $i &}
	tail -f $x
}

Re: [9fans] multiple inputs

[rog]

[-- Attachment #1: Type: text/plain, Size: 447 bytes --]

> fn bt {
> 	eval '{'for {tail -f $i &} '}'
> }
> 
> mimics how I would write it by hand.

if that's all you need, then

fn bt {
	for(i){tail -f $i&}
}

should be just fine.
then

	bt > file

might work (if file is append-only, otherwise you'd need a process
(e.g.  cat) in the middle to avoid overlapping writes)

using eval is disastrous...  consider what happens if you've got a
filename with a shell metacharacter in it.

[-- Attachment #2: Type: message/rfc822, Size: 5882 bytes --]

[-- Attachment #2.1.1: Type: text/plain, Size: 665 bytes --]

I'll agree with you there. The eval is unnecessary.

But 

fn bt {
	eval '{'for {tail -f $i &} '}'
}

mimics how I would write it by hand.

{tail -f file1 &; tail -f file2 &; tail -f file3 &}>file

Which allows me to consider the command one block, which I can pipe to and from etc.

The idiom that you and matt give doesn't work for pipes

tail -f file1 &; tail -f file2 &; tail -f file3 >file

only puts the text from file3 in the pipe. 

The for loop you gave obviates that problem, since it counts as a single block, but it's considerably different than my interactive shell usage and is more verbose(though much more readable :)).

Noah

[-- Attachment #2.1.2: Type: message/rfc822, Size: 3075 bytes --]

From: rog@vitanuova.com
To: 9fans@cse.psu.edu
Subject: Re: [9fans] multiple inputs
Date: Tue, 30 Mar 2004 17:25:46 +0100
Message-ID: <19236d27e19c4ebbe4ba92e507e29149@vitanuova.com>

> You need to put the brackets on the outside to make the combination of
> tail -f's act like a single 'tail -f'.  That was my real conceptual
> misunderstanding the first time around.

erm, i don't think so.

{a & b &}

should be exactly the same as

a & b &

you almost never need eval in rc, unless you genuinely want
to evaluate an rc expression that's unknown beforehand
(which isn't true in your case).

you could have got the behaviour i think you were after
with something like:

fn bt {
	x=$1
	shift
	for(i) {tail -f $i &}
	tail -f $x
}

Re: [9fans] multiple inputs

[Noah Evans]

[-- Attachment #1: Type: text/plain, Size: 89 bytes --]

True and much more elegant.

:)

Good point about the security hole too :( 

Noah

[-- Attachment #2: Type: message/rfc822, Size: 9159 bytes --]

[-- Attachment #2.1.1: Type: text/plain, Size: 447 bytes --]

> fn bt {
> 	eval '{'for {tail -f $i &} '}'
> }
> 
> mimics how I would write it by hand.

if that's all you need, then

fn bt {
	for(i){tail -f $i&}
}

should be just fine.
then

	bt > file

might work (if file is append-only, otherwise you'd need a process
(e.g.  cat) in the middle to avoid overlapping writes)

using eval is disastrous...  consider what happens if you've got a
filename with a shell metacharacter in it.

[-- Attachment #2.1.2: Type: message/rfc822, Size: 5884 bytes --]

[-- Attachment #2.1.2.1.1: Type: text/plain, Size: 665 bytes --]

I'll agree with you there. The eval is unnecessary.

But 

fn bt {
	eval '{'for {tail -f $i &} '}'
}

mimics how I would write it by hand.

{tail -f file1 &; tail -f file2 &; tail -f file3 &}>file

Which allows me to consider the command one block, which I can pipe to and from etc.

The idiom that you and matt give doesn't work for pipes

tail -f file1 &; tail -f file2 &; tail -f file3 >file

only puts the text from file3 in the pipe. 

The for loop you gave obviates that problem, since it counts as a single block, but it's considerably different than my interactive shell usage and is more verbose(though much more readable :)).

Noah

[-- Attachment #2.1.2.1.2: Type: message/rfc822, Size: 3075 bytes --]

From: rog@vitanuova.com
To: 9fans@cse.psu.edu
Subject: Re: [9fans] multiple inputs
Date: Tue, 30 Mar 2004 17:25:46 +0100
Message-ID: <19236d27e19c4ebbe4ba92e507e29149@vitanuova.com>

> You need to put the brackets on the outside to make the combination of
> tail -f's act like a single 'tail -f'.  That was my real conceptual
> misunderstanding the first time around.

erm, i don't think so.

{a & b &}

should be exactly the same as

a & b &

you almost never need eval in rc, unless you genuinely want
to evaluate an rc expression that's unknown beforehand
(which isn't true in your case).

you could have got the behaviour i think you were after
with something like:

fn bt {
	x=$1
	shift
	for(i) {tail -f $i &}
	tail -f $x
}

Re: [9fans] multiple inputs

[boyd, rounin]

> I'll agree with you there. The eval is unnecessary.

'eval' -- therein lies dragons.

Re: [9fans] multiple inputs

[Noah Evans]

[-- Attachment #1: Type: text/plain, Size: 80 bytes --]

Yup, consider me chagrined. What do the 9fans think of hygenic macros?

Noah

[-- Attachment #2: Type: message/rfc822, Size: 2913 bytes --]

From: "boyd, rounin" <boyd@insultant.net>
To: 9fans@cse.psu.edu
Subject: Re: [9fans] multiple inputs
Date: Wed, 31 Mar 2004 22:22:51 +0200
Message-ID: <000a01c417f0$56f4f320$34fea8c0@SOMA>

> I'll agree with you there. The eval is unnecessary.

'eval' -- therein lies dragons.

Re: [9fans] multiple inputs

[rog]

> What do the 9fans think of hygenic macros?

dunno what others think but i reckon macros (hygienic or not) make it
much harder to maintain a program.  you have to get through the custom
syntax before you can start seeing what's really going on.

if you need that kind of expressivity, consider writing an
interpreter, a translator or a preprocessor.

limbo doesn't have macros, and although i've occasionally thought "a
macro would be nice here" the benefits of having all source code in
standard form far outweigh the costs.

Re: [9fans] multiple inputs

[boyd, rounin]

> limbo doesn't have macros, and although i've occasionally thought "a
> macro would be nice here" the benefits of having all source code in
> standard form far outweigh the costs.

yup

Re: [9fans] multiple inputs

[Russ Cox]

rog@vitanuova.com wrote:

>>What do the 9fans think of hygenic macros?
>>    
>>
>
>dunno what others think but i reckon macros (hygienic or not) make it
>much harder to maintain a program.  you have to get through the custom
>syntax before you can start seeing what's really going on.
>
>if you need that kind of expressivity, consider writing an
>interpreter, a translator or a preprocessor.
>  
>

I think it would be great if the C compiler had some sort
of nice extension facility (call it hygenic macros with syntax) so that
we could write

alt {
<- done =>
    print("done\n");
c <-= val =>
    print("sent val\n");
}

But you're right, that would make it much harder to maintain
the program.  Instead we should content ourselves with writing

Alt alts[3];
alts[0].c = done;
alts[0].v = 0;
alts[0].op = CHANRCV;
alts[1].c = c;
alts[1].v = val;
alts[1].op = CHANSND;
alts[2].op = CHANEND;
switch(alt(alts)){
case 0:
    print("done\n");
    break;
case 1:
    print("sent val\n");
    break;
}

which is much clearer and doesn't require us to wade through
custom macro syntax.

Russ

Re: [9fans] multiple inputs

[rog]

> which is much clearer and doesn't require us to wade through
> custom macro syntax.

i sympathise completely.

problem is the moment you have something that's flexible enough
to allow alt, you'll get someone that wants to do all kinds of other
things with it. remember:

LOCAL TREPTR	list(flg)
{
	REG TREPTR	r;
	REG INT		b;

	r = term(flg);
	WHILE r ANDF ((b=(wdval==ANDFSYM)) ORF wdval==ORFSYM)
	DO	r = makelist((b ? TAND : TORF), r, term(NLFLG));
	OD
	return(r);
}

what you're after, i think, is a different language, one with one or
two extra features, not really a way to customise C in an arbitrary
way.  a pre-processor could do the job, but you'd pay the price.

if many people defined syntax like the alt stuff, you'd really struggle
to work out what was going on. IMHO a language requires a small,
coherently designed set of features, not a way to add every feature
under the sun.

Re: [9fans] multiple inputs

[Noah Evans]

[-- Attachment #1: Type: text/plain, Size: 744 bytes --]

I think the idea of the hygenic macro is to implement the macro facility in to the compiler as a change to the parse tree rather than a change to the program text with a preprocessor(sorry if I'm being painfully obvious here).

This means that your macros don't have the normal problems associated with macros(precedence problems, multiple evaluation of arguments etc.). I haven't put very much thought into this, but from I'm guessing an implementation would act like a simplified yacc that worked with the compiler directly.

Again I'm not sure how practical this would be. The only implementation I know of, in scheme, is aided by scheme's use of prefix notation. Figuring out the precedence order would be a pain.

Thoughts?

Noah

[-- Attachment #2: Type: message/rfc822, Size: 3411 bytes --]

From: rog@vitanuova.com
To: 9fans@cse.psu.edu
Subject: Re: [9fans] multiple inputs
Date: Thu, 01 Apr 2004 18:36:36 +0100
Message-ID: <8cdad86de9428072d82f8764a903f798@vitanuova.com>

> which is much clearer and doesn't require us to wade through
> custom macro syntax.

i sympathise completely.

problem is the moment you have something that's flexible enough
to allow alt, you'll get someone that wants to do all kinds of other
things with it. remember:

LOCAL TREPTR	list(flg)
{
	REG TREPTR	r;
	REG INT		b;

	r = term(flg);
	WHILE r ANDF ((b=(wdval==ANDFSYM)) ORF wdval==ORFSYM)
	DO	r = makelist((b ? TAND : TORF), r, term(NLFLG));
	OD
	return(r);
}

what you're after, i think, is a different language, one with one or
two extra features, not really a way to customise C in an arbitrary
way.  a pre-processor could do the job, but you'd pay the price.

if many people defined syntax like the alt stuff, you'd really struggle
to work out what was going on. IMHO a language requires a small,
coherently designed set of features, not a way to add every feature
under the sun.

Re: [9fans] multiple inputs

[rog]

> I think the idea of the hygenic macro [...]

one way or another you're talking about syntax transformation, so the
syntax of the program you read is not that which is documented in the
language reference manual.

i think that this can easily lead to unmaintainable software.  others
disagree.

off topic, anyway...

Re: [9fans] multiple inputs

[Noah Evans]

[-- Attachment #1: Type: text/plain, Size: 615 bytes --]

Right. But that's the point.

What hygenic macros are(to me) a way of creating domain specific languages based on C. 

This can, when used badly, lead to unmaintainable software. You're exactly right. 

At the same time as Russ said it can abstract a complex and opaque computation and clarify its intention.

To bring the subject of macros back to the subject of Plan 9, I made the initial comment thinking about a way to make the shell less dangerous when it expands metacharacters(in other words, protect me from myself). It's only an intuition, and I wanted to see what everyone thought of it.

Noah

[-- Attachment #2: Type: message/rfc822, Size: 2831 bytes --]

From: rog@vitanuova.com
To: 9fans@cse.psu.edu
Subject: Re: [9fans] multiple inputs
Date: Thu, 01 Apr 2004 19:42:27 +0100
Message-ID: <810c47b392ed21dbd01b75286fa1ec73@vitanuova.com>

> I think the idea of the hygenic macro [...]

one way or another you're talking about syntax transformation, so the
syntax of the program you read is not that which is documented in the
language reference manual.

i think that this can easily lead to unmaintainable software.  others
disagree.

off topic, anyway...

Re: [9fans] multiple inputs

[rog]

> a way to make the shell less dangerous when it expands metacharacters

it's easy...  don't use eval!  (i've written loads of rc scripts and i
don't think i've ever used eval).

if you really wanted to use eval in a safe way, you'd need some
support for quoting; for instance:

% quote a 'b c' d
a 'b c' d
% 

but it'd be awkward to use this in rc due to the `{} $ifs splitting.

i made this kind of thing easier to do in the inferno shell, but i'm
still not that convinced - it's very easy to build up inpenetrable
layers.

Re: [9fans] multiple inputs

[Noah Evans]

[-- Attachment #1: Type: text/plain, Size: 180 bytes --]

:) 

Exactly, But I love code that writes itself so I have a hard time giving it up ;)  hence the interest in playing with the parse tree. 

Thanks for your insights,

Noah

[-- Attachment #2: Type: message/rfc822, Size: 3046 bytes --]

From: rog@vitanuova.com
To: 9fans@cse.psu.edu
Subject: Re: [9fans] multiple inputs
Date: Thu, 01 Apr 2004 20:47:56 +0100
Message-ID: <94ec8e21ec12b4dc3ae41b46df9420b8@vitanuova.com>

> a way to make the shell less dangerous when it expands metacharacters

it's easy...  don't use eval!  (i've written loads of rc scripts and i
don't think i've ever used eval).

if you really wanted to use eval in a safe way, you'd need some
support for quoting; for instance:

% quote a 'b c' d
a 'b c' d
% 

but it'd be awkward to use this in rc due to the `{} $ifs splitting.

i made this kind of thing easier to do in the inferno shell, but i'm
still not that convinced - it's very easy to build up inpenetrable
layers.

Re: [9fans] multiple inputs

[Geoff Collyer]

If you find any of that code that writes itself, send some my way.
Clearly AI has achieved greater things than I'd realised.  ☺

Re: [9fans] multiple inputs

[David Tolpin]

> Right. But that's the point.
>
> What hygenic macros are(to me) a way of creating domain specific languages
> based on C. 

hugs is ported to Plan 9. Use haskell for domain-specific languages,
it is great for the purpose. C is different -- it encourages to
write thin, transparent programs; when it is used appropriately
it is great.

Re: [9fans] multiple inputs

[Geoff Collyer]

Actually, writing out alts initialisation by hand has always seemed
clumsy to me.

void
newalt(Alt *ap, ChanOp op, Channel *c, void *v)
{
	ap->op = op;
	ap->c = c;
	ap->v = v;
}

	Alts alts[3], *ap = alts;

	newalt(ap++, CHANRCV, done, nil);
	newalt(ap++, CHANSND, c, val);
	newalt(ap,   CHANEND, nil, nil);

seems a bit more obvious and only requires one additional line per
channel.

Re: [9fans] multiple inputs

[Noah Evans]

:) 

I picked up my bad habits when I figured out that I could make scheme eat syntax trees and spit out recognizers for particular patterns.

It's been downhill ever since. ;)

Noah

----- Original Message -----
From: Geoff Collyer <geoff@collyer.net>
Date: Thursday, April 1, 2004 8:48 pm
Subject: Re: [9fans] multiple inputs

> If you find any of that code that writes itself, send some my way.
> Clearly AI has achieved greater things than I'd realised.  ?
> 
>