* [9fans] typed sh (was: what features would you like in a shell?)
@ 2009-04-02 18:48 fgergo
2009-04-02 19:28 ` roger peppe
0 siblings, 1 reply; 13+ messages in thread
From: fgergo @ 2009-04-02 18:48 UTC (permalink / raw)
To: Fans of the OS Plan 9 from Bell Labs
On Thu, Apr 2, 2009 at 8:41 PM, John Stalker <stalker@maths.tcd.ie> wrote:
> What I most often miss in shell programming is a proper type system.
You should have a look at alphabet. It is cool.
http://www.vitanuova.com/inferno/man/1/sh-alphabet.html
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [9fans] typed sh (was: what features would you like in a shell?) 2009-04-02 18:48 [9fans] typed sh (was: what features would you like in a shell?) fgergo @ 2009-04-02 19:28 ` roger peppe 2009-04-02 20:27 ` tlaronde 2009-04-06 3:55 ` Bakul Shah 0 siblings, 2 replies; 13+ messages in thread From: roger peppe @ 2009-04-02 19:28 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs 2009/4/2 <fgergo@gmail.com>: > On Thu, Apr 2, 2009 at 8:41 PM, John Stalker <stalker@maths.tcd.ie> wrote: >> What I most often miss in shell programming is a proper type system. > You should have a look at alphabet. It is cool. > http://www.vitanuova.com/inferno/man/1/sh-alphabet.html i certainly enjoyed creating it. unfortunately it's unfinished - i ran into unwarranted-complexity problems trying to simulate polymorphism with processes and channels... limbo's not a great language for writing interpreters in when you don't know all your types in advance. for those thinking of ideas for new shells, i still think there's mileage in some of alphebet's ideas, principally the way it lives in a half-way house between almost no types (sh) and unrestricted types (most other languages). that idea came from the thought that part of what makes sh so powerful is the fact that so many commands use the same types (string, stream of bytes + exit status). i wanted to go a little beyond sh while stopping short of the type profligacy of most other languages, hoping to create a situation where many commands used exactly the same types, and hence were viable to pipeline together. a pipeline is an amazingly powerful thing considering that it's not a turing-complete abstraction. alphabet was actually a generalisation of the fs command (see http://www.vitanuova.com/inferno/man/1/fs.html), that i'd found really useful (and more powerful than other filesystem traversal tools i've seen, for minimal code). some time i'll finish it! ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [9fans] typed sh (was: what features would you like in a shell?) 2009-04-02 19:28 ` roger peppe @ 2009-04-02 20:27 ` tlaronde 2009-04-06 3:55 ` Bakul Shah 1 sibling, 0 replies; 13+ messages in thread From: tlaronde @ 2009-04-02 20:27 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs I don't know if others have already hit this kind of problematic, but I was dealing with a fair amount of C code, usable both as a library and accessible by a shell. Plus debugging needs. So I was, again and again, writing a wrapper to access a C function from the shell. So I ended concluding that I needed a kind of C interpreter as a shell. (I have an implementation, but it is not pure C---sentential calculus is distinct; it's a 4 values logic (NONSENSE, TRUE, FALSE, UNDECIDABLE) that has already real application in geometrical calculus; and integer and real calculus is added too for mathematical tasks---but it is not ready for prime time and I have still unanswered questions for special things I want to be present.) typedef, i.e. the ability to define other types above primary ones is perhaps what you are looking for? ISTR that on the early PCees, there was a basic interpreter in BIOS to let you play with the almost bare machine. I'd like to have a shell that lets me play with the bare OS. -- Thierry Laronde (Alceste) <tlaronde +AT+ polynum +dot+ com> http://www.kergis.com/ Key fingerprint = 0FF7 E906 FBAF FE95 FD89 250D 52B1 AE95 6006 F40C ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [9fans] typed sh (was: what features would you like in a shell?) 2009-04-02 19:28 ` roger peppe 2009-04-02 20:27 ` tlaronde @ 2009-04-06 3:55 ` Bakul Shah 2009-04-06 11:09 ` erik quanstrom 2009-04-16 17:24 ` roger peppe 1 sibling, 2 replies; 13+ messages in thread From: Bakul Shah @ 2009-04-06 3:55 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs On Thu, 02 Apr 2009 20:28:57 BST roger peppe <rogpeppe@gmail.com> wrote: > 2009/4/2 <fgergo@gmail.com>: > i wanted to go a little beyond sh while stopping > short of the type profligacy of most other languages, > hoping to create a situation where many commands > used exactly the same types, and hence were > viable to pipeline together. Nitpick: the output type of one command and the input type of the next command in the pipeline has to match, not every command. > a pipeline is an amazingly powerful thing considering > that it's not a turing-complete abstraction. "f | g" is basically function composition, where f and g are stream functions. Of course, this simple analogy breaks down the moment we add more input/output channels -- may be that is why anything beyond a simple pipeline seems to get people in trouble (see the rc output redirection thread). To go beyond simple char streams, one can for example build a s-expr pipeline: a stream of self identifying objects of a few types (chars, numbers, symbols, lists, vectors). In Q (from kx.com) over an IPC connection you can send strings, vectors, dictionaries, tables, or arbitray Q expressions. But there the model is more of a client/server. ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [9fans] typed sh (was: what features would you like in a shell?) 2009-04-06 3:55 ` Bakul Shah @ 2009-04-06 11:09 ` erik quanstrom 2009-04-06 15:40 ` Bakul Shah 2009-04-16 17:24 ` roger peppe 1 sibling, 1 reply; 13+ messages in thread From: erik quanstrom @ 2009-04-06 11:09 UTC (permalink / raw) To: 9fans > Nitpick: the output type of one command and the input type of > the next command in the pipeline has to match, not every > command. i think this is wrong. there's no requirement that the programs participating in a pipeline are compatable at all; that's the beauty of pipes. you can do things that were not envisioned at the time the programs were written. > To go beyond simple char streams, one can for example build a > s-expr pipeline: a stream of self identifying objects of a > few types (chars, numbers, symbols, lists, vectors). In Q > (from kx.com) over an IPC connection you can send strings, > vectors, dictionaries, tables, or arbitray Q expressions. But > there the model is more of a client/server. or ntfs where files are databases. not sure if streams can look the same way. - erik ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [9fans] typed sh (was: what features would you like in a shell?) 2009-04-06 11:09 ` erik quanstrom @ 2009-04-06 15:40 ` Bakul Shah 2009-04-06 16:02 ` erik quanstrom 0 siblings, 1 reply; 13+ messages in thread From: Bakul Shah @ 2009-04-06 15:40 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs On Mon, 06 Apr 2009 07:09:47 EDT erik quanstrom <quanstro@quanstro.net> wrote: > > Nitpick: the output type of one command and the input type of > > the next command in the pipeline has to match, not every > > command. > > i think this is wrong. there's no requirement > that the programs participating in a pipeline are compatable > at all; that's the beauty of pipes. If program A outputs numbers in big-endian order and B expects input in little-endian order, A|B won't do the "right thing". Even for programs like wc have a concept of a 'character' and if the prev prog. produces something else you will be counting something meaningless. Perhaps it is impossible to capture such type compatibility in anything but runtime IO routines but the concept exists. > you can do things > that were not envisioned at the time the programs were > written. That comes from composability. > > To go beyond simple char streams, one can for example build a > > s-expr pipeline: a stream of self identifying objects of a > > few types (chars, numbers, symbols, lists, vectors). In Q > > (from kx.com) over an IPC connection you can send strings, > > vectors, dictionaries, tables, or arbitray Q expressions. But > > there the model is more of a client/server. > > or ntfs where files are databases. not sure if streams > can look the same way. May be not. What I was getting at was that one can do a lot with a small number of IO types -- no need for "type profligacy"! [Unless your definition of profligacy is anything more than one.] The nice thing about s-expr is that you have a syntax for structured IO and its printer, parser are already written for you. Anyway, a typed sh would be an interesting experiment. ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [9fans] typed sh (was: what features would you like in a shell?) 2009-04-06 15:40 ` Bakul Shah @ 2009-04-06 16:02 ` erik quanstrom 2009-04-06 16:22 ` [9fans] typed sh maht ` (2 more replies) 0 siblings, 3 replies; 13+ messages in thread From: erik quanstrom @ 2009-04-06 16:02 UTC (permalink / raw) To: 9fans > If program A outputs numbers in big-endian order and B > expects input in little-endian order, A|B won't do the "right > thing". non-marshaled data considered harmful. film at 11. ☺ what i said was not that A|B "makes sense" for all A and B and for any data but rather that using text streams makes A|B possible for any A and any B and any input. the output might not be useful, but that is a problem on a completely different semantic level, one that computers are usually no good at. alsi, i don't think that "type compatability" is sufficient to insure that the output "makes sense". what if A produces big-endian times in ms while B expects big-endian times in µs. > Even for programs like wc have a concept of a > 'character' and if the prev prog. produces something else you > will be counting something meaningless. that's why plan 9 uses a single character set. but forcing compability seems worse. where are these decisions centralized? how do you change decisions? can you override these decisions (cast)? how does the output of, say, awk get typed? - erik ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [9fans] typed sh 2009-04-06 16:02 ` erik quanstrom @ 2009-04-06 16:22 ` maht 2009-04-06 16:56 ` erik quanstrom 2009-04-06 16:50 ` [9fans] typed sh (was: what features would you like in a shell?) Bakul Shah 2009-04-06 21:18 ` John Stalker 2 siblings, 1 reply; 13+ messages in thread From: maht @ 2009-04-06 16:22 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs > but rather that using text streams makes > A|B possible for any A and any B and any input. What is this "text" of which you speak ? ASCII EBCDIC UTF-16 UTF-8 ISO8859 etc. etc. etc. ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [9fans] typed sh 2009-04-06 16:22 ` [9fans] typed sh maht @ 2009-04-06 16:56 ` erik quanstrom 0 siblings, 0 replies; 13+ messages in thread From: erik quanstrom @ 2009-04-06 16:56 UTC (permalink / raw) To: 9fans > > but rather that using text streams makes > > A|B possible for any A and any B and any input. > What is this "text" of which you speak ? ASCII EBCDIC UTF-16 UTF-8 > ISO8859 etc. etc. etc. there's got to be a latin term parallel to reducto ad absurdum that means the opposite. make the problem gratituiously harder until no sensible statements can be made at all. or maybe it's the opposite of divide and conquer. accrete and be subjugated? ☺. by the way, tcs works fine for me. - erik ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [9fans] typed sh (was: what features would you like in a shell?) 2009-04-06 16:02 ` erik quanstrom 2009-04-06 16:22 ` [9fans] typed sh maht @ 2009-04-06 16:50 ` Bakul Shah 2009-04-06 21:18 ` John Stalker 2 siblings, 0 replies; 13+ messages in thread From: Bakul Shah @ 2009-04-06 16:50 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #1: Type: text/plain, Size: 2104 bytes --] On Mon, 06 Apr 2009 12:02:21 EDT erik quanstrom <quanstro@quanstro.net> wrote: > > If program A outputs numbers in big-endian order and B > > expects input in little-endian order, A|B won't do the "right > > thing". > > non-marshaled data considered harmful. film at 11. ☺ In effect you are imposing a constraint (a type discipline). Even if the programs themselves check such constraints, the compatibility idea exists. > what i said was not that A|B "makes sense" for all A and B > and for any data but rather that using text streams makes > A|B possible for any A and any B and any input. the output > might not be useful, but that is a problem on a completely > different semantic level, one that computers are usually no good at. > alsi, i don't think that "type compatability" is sufficient > to insure that the output "makes sense". what if A produces > big-endian times in ms while B expects big-endian times in µs. In effect you are saying that text streams allow nonsensical pipelines as well as sensible ones and anything other than text streams would imply giving up freedom to create sensible pipelines as yet unthought of. No disagreement there but see below. > > Even for programs like wc have a concept of a > > 'character' and if the prev prog. produces something else you > > will be counting something meaningless. > > that's why plan 9 uses a single character set. > > but forcing compability seems worse. where are these decisions > centralized? how do you change decisions? can you override > these decisions (cast)? how does the output of, say, awk get > typed? I am not suggesting forcing anything; I am suggesting experimenting with s-expr streams (in the context of "typed sh" idea). I don't know if that buys you anything more or if you give up any essential freedom. My guess is you'd build something more scalable, more composable but I wouldn't really know until it is tried. I imagine s-expr-{grep,awk} would look quite different from {grep,awk}. May be you'd end up with something like a Lisp machine. ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [9fans] typed sh (was: what features would you like in a shell?) 2009-04-06 16:02 ` erik quanstrom 2009-04-06 16:22 ` [9fans] typed sh maht 2009-04-06 16:50 ` [9fans] typed sh (was: what features would you like in a shell?) Bakul Shah @ 2009-04-06 21:18 ` John Stalker 2 siblings, 0 replies; 13+ messages in thread From: John Stalker @ 2009-04-06 21:18 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs > but forcing compability seems worse. where are these decisions > centralized? how do you change decisions? can you override > these decisions (cast)? how does the output of, say, awk get > typed? The output of awk is a byte stream, same as its input. The same holds for any program. If you want to give it some other type then you need at least a casting mechanism. You probably also want a splitting mechanism, with a regular expression for the field separator. -- John Stalker School of Mathematics Trinity College Dublin tel +353 1 896 1983 fax +353 1 896 2282 ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [9fans] typed sh (was: what features would you like in a shell?) 2009-04-06 3:55 ` Bakul Shah 2009-04-06 11:09 ` erik quanstrom @ 2009-04-16 17:24 ` roger peppe 2009-04-16 18:52 ` Bakul Shah 1 sibling, 1 reply; 13+ messages in thread From: roger peppe @ 2009-04-16 17:24 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs 2009/4/6 Bakul Shah <bakul+plan9@bitblocks.com>: > On Thu, 02 Apr 2009 20:28:57 BST roger peppe <rogpeppe@gmail.com> wrote: >> a pipeline is an amazingly powerful thing considering >> that it's not a turing-complete abstraction. > > "f | g" is basically function composition, where f and g are > stream functions. Of course, this simple analogy breaks down > the moment we add more input/output channels -- may be that > is why anything beyond a simple pipeline seems to get people > in trouble (see the rc output redirection thread). actually, the analogy works fine if we add more input channels - it's multiple output channels that make things hard, as they mean that you have an arbitrary directed graph rather than a tree, which doesn't have such a convenient textual representation and is harder to comprehend to boot. in alphabet, i had a diagnostic channel, which was strictly textual and not easily accessible from the language, which was arguably not the best solution, but i didn't want things to get too complex. > To go beyond simple char streams, one can for example build a > s-expr pipeline: a stream of self identifying objects of a > few types (chars, numbers, symbols, lists, vectors). the difficulty with s-exprs (and most nested structures, e.g. XML) from a pipeline point of view is that their nested nature means that any branch might contain unlimited quantities of stuff, so you can't always process in O(1) space, which is one of the things i really like about pipeline processing. i found a nice counter-example in the fs stuff - the fundamental type was based around a "conditional-push" protocol for sending trees of files - the sender sends some information on a file/directory and the receiver replies whether to descend into that file or not. the tree had a canonical order (alphabetical on name), so tree merging could be done straightforwardly in O(1) space. this kind of streaming "feels" like a regular pipeline, but you can't do this with a regular pipeline. for instance, a later element in the pipeline can prevent an earlier from descending into a part of the file system that might block indefinitely. every language has a trade-off between typed and untyped representations; with alphabet i was trying to create something where it was *possible* to create new kinds of types where necessary (as in the fs example), but where it wouldn't be customary or necessary to do so in the vast majority of cases. perhaps it was folly, but i still think it was an interesting experiment, and i don't know of anything similar. ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [9fans] typed sh (was: what features would you like in a shell?) 2009-04-16 17:24 ` roger peppe @ 2009-04-16 18:52 ` Bakul Shah 0 siblings, 0 replies; 13+ messages in thread From: Bakul Shah @ 2009-04-16 18:52 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs On Thu, 16 Apr 2009 18:24:36 BST roger peppe <rogpeppe@gmail.com> wrote: > 2009/4/6 Bakul Shah <bakul+plan9@bitblocks.com>: > > On Thu, 02 Apr 2009 20:28:57 BST roger peppe <rogpeppe@gmail.com> =C2=A0w= > rote: > >> a pipeline is an amazingly powerful thing considering > >> that it's not a turing-complete abstraction. > > > > "f | g" is basically function composition, where f and g are > > stream functions. Of course, this simple analogy breaks down > > the moment we add more input/output channels -- may be that > > is why anything beyond a simple pipeline seems to get people > > in trouble (see the rc output redirection thread). > > actually, the analogy works fine if we add more > input channels - it's multiple output channels > that make things hard, as they mean that you have > an arbitrary directed graph rather than a tree, which doesn't > have such a convenient textual representation > and is harder to comprehend to boot. True in general but certain graphs are relatively easy to comprehend depending on what you are doing (trees, hub & spokes, rings). Shells don't provide you a convenient mechanism for constructing these graphs (I'd use macros in Scheme/Lisp, or a graphics editor). For DAGs you can use something like the example below but it doesn't have the nice aesthetics of a pipeline! let s0,s1 = function-with-two-output-streams function-with-two-input-streams(f0(s0), f1(s1), ...) > > To go beyond simple char streams, one can for example build a > > s-expr pipeline: a stream of self identifying objects of a > > few types (chars, numbers, symbols, lists, vectors). > > the difficulty with s-exprs (and most nested structures, e.g. XML) > from a pipeline point of view is > that their nested nature means that any branch might contain unlimited > quantities > of stuff, so you can't always process in O(1) space, which is one of the > things i really like about pipeline processing. You can have arbitrarily long lines in a text file so if you operate on lines, you need arbitrary buffer space. It is the same problem. Also note that I was talking about a stream of s-exprs, not one s-expr as a stream (which makes no sense). For example, (attach ...) (walk ...) (open ...) (read ...) (clunk ...) > i found a nice counter-example in the fs stuff - the fundamental type > was based around a "conditional-push" protocol for sending trees > of files - the sender sends some information on a file/directory > and the receiver replies whether to descend into that file or > not. the tree had a canonical order (alphabetical on name), so > tree merging could be done straightforwardly in O(1) space. > > this kind of streaming "feels" like a regular pipeline, but you can't > do this with a regular pipeline. for instance, a later element in the > pipeline can prevent an earlier from descending into a part > of the file system that might block indefinitely. > > every language has a trade-off between typed and untyped representations; > with alphabet i was trying to create something where it was *possible* > to create new kinds of types where necessary (as in the fs example), > but where it wouldn't be customary or necessary to do so in the > vast majority of cases. > > perhaps it was folly, but i still think it was an interesting experiment, > and i don't know of anything similar. ^ permalink raw reply [flat|nested] 13+ messages in thread
end of thread, other threads:[~2009-04-16 18:52 UTC | newest] Thread overview: 13+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2009-04-02 18:48 [9fans] typed sh (was: what features would you like in a shell?) fgergo 2009-04-02 19:28 ` roger peppe 2009-04-02 20:27 ` tlaronde 2009-04-06 3:55 ` Bakul Shah 2009-04-06 11:09 ` erik quanstrom 2009-04-06 15:40 ` Bakul Shah 2009-04-06 16:02 ` erik quanstrom 2009-04-06 16:22 ` [9fans] typed sh maht 2009-04-06 16:56 ` erik quanstrom 2009-04-06 16:50 ` [9fans] typed sh (was: what features would you like in a shell?) Bakul Shah 2009-04-06 21:18 ` John Stalker 2009-04-16 17:24 ` roger peppe 2009-04-16 18:52 ` Bakul Shah
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).