Re: [9fans] Distributed Pipelines

[Akshat Kumar <akumar@mail.nanosouffle.net>]

Hi Erik,

Thanks for figuring that bit out!
Indeed, it seems TCP is the
problem, and IL seems to work
fine for me for the moment:

echo '1 2 3' | rx il!$cpu!17009 awk -f $home/comp.awk | gview

works perfectly!

I'll try to dig deeper into the TCP case.


Best,
ak


On 4/26/10, erik quanstrom <quanstro@labs.coraid.com> wrote:
>>
>> "...
>>           eqn paper | rx kremvax troff -ms | rx deepthought lp
>>                Parallel processing: do each stage of a pipeline on a
>>                different machine.
>> "
>>
>> however, it seems not to work this way.
>> My basic test has been something like:
>>
>> echo '1 2 3' | rx $cpu awk -f $home/comp.awk | gview
>
> cool that you tracked this down.
>
> this just doesn't work.  the problem seems to be that tcp
> is eating the eof.  awk doesn't know to exit.
>
> if you are using il, this does work.
>
> 	; echo 1 2 3| 8.rx bureau sed s/1/x/ | sed s/2/y/
> 	x y 3
>
> but if you are using tcp, it hangs.  sed never sees the
> eof and doesn't generate output (because it's buffered).
>
> the answer isn't obvious to me with tcp as i don't
> know of a way to half-close a tcp connection —
> from userspace anyway.
>
> - erik
>
>