From mboxrd@z Thu Jan 1 00:00:00 1970 Date: Wed, 27 Jan 2010 21:44:50 +1100 From: Sam Watkins To: Fans of the OS Plan 9 from Bell Labs <9fans@9fans.net> Message-ID: <20100127104450.GC13116@nipl.net> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.13 (2006-08-11) Subject: Re: [9fans] dataflow programming from shell interpreter Topicbox-Message-UUID: c8064054-ead5-11e9-9d60-3106f5b1d025 On Mon, Jan 18, 2010 at 10:58:30AM +0000, Tim Climber wrote: > Is this possible for UNIX philosophy to develop further? Let's say, > XML-coded trees or graphs instead of one-line strings in stdin/ > stdout.Or LISP S-expressions. New set of utilities for filtering such > streams, grep for XML trees, etc. Building environment for dataflow > programming from shell interpreter. > Any interesting papers exist on this topic? This is interesting. I'm not keen on XML (except for actual markup of documents) but dataflow programming is interesting and worthwhile. I like graphical programming (with nodes and arcs) for this, but programs can also be represented as plain text. I haven't implemented much to do with this yet, but I have thought about it at length, determined what basic operators and forms can be used for lambda, math, sets, lists etc. I'm interested in relational / logic / dataflow programming where for example: a + b c expresses a relationship. If b and c are known, a can be calculated: a = b + c If a and b are known, c can be calculated: c = b - a This can lead to much shorter code. Immediately you can see there is no need for a separate subtraction operator, it is just a different mode of addition. Likewise / is only a mode of *. Raising to the power, taking a root, exponentiation and logarithms are different modes of a single operator. + - * / ^ log becomes + * ^ One basic operator is needed for sets: the discrete union, which is like a sum: A B # C means A OR B = C, A AND B = empty Various list relationships such as reversal, membership, head, tail, sublist, etc. can be represented simply with a single list symbol. Operators can be defined using lambda closure (a box). An operator over paramters A, B, C, D is effectively (from some point of view) a set of all tuples satisfying the contained relationships { (A,B,C,D) } So the "lambda" box thing creates a set of possible values from a free variable. It can be used in reverse to produce a free variable that is a member of a set. (also, an instance of an operator). Lambda, application, set definition, and set membership are the same thing (expressed by a box). A single processor might perform both splitting and joining of text, depending which direction you feed the data to it, etc. Or parsing and formatting, maybe encoding and decoding video, etc. There will be some components of the program used only for encoding, some only for decoding, but much can be shared between them. I am not fully sure how to integrate the dataflow system with the relational system, but it seems like these models would go very well together. Perhaps an abstract model of a stream (like in Haskell) would be the way, perhaps a lazy list achieves this already. I need to think more about this. Sam