From mboxrd@z Thu Jan 1 00:00:00 1970 From: bogus@does.not.exist.com () Date: Mon, 4 May 2009 19:42:15 +0000 Subject: No subject Message-ID: Topicbox-Message-UUID: 00560166-ead5-11e9-9d60-3106f5b1d025 OS, but I am not able to do so because it lacks the data analysis tools available in some other systems, like linux. Because my work involves dealing with data coming from experiments in astro-particle physics, I am more or less tied to data analysis software like the R programming language, Python's Numpy, Cern's ROOT and even gnuplot. While using them, I realized that most of the time I deal with text files that go here and there as input or output of small specific programs that perform a given task (I don't know if this is the result of my Unix/Plan 9 background or just a coincidence). Say I have a command 'clean' that removes undesired points from a body of data, and another command 'four' that performs the FFT; so they are used together as clean data.txt | four > results.txt so it occurred to me that one can create single commands to interact among them to perform some analysis on data, just like in the original Unix style. Awk can be used as glue among them, with some other small glue utilities. Plotting data is another thing that I would like to integrate into this, since plots are quite frequent while analysing data, but I am not sure how. Also, something similar to GSL (http://www.gnu.org/software/gsl/) would be invaluable or maybe even indispensable. Maybe some day I'll start to write some commands for plan 9 to begin working on it, but I want to convince myself that this is worth the time spent. What do you think of this? my main concern is that perhaps the "do one thing well" design falls short for data analysis. I've never seen people work like this on data analysis before (but I do not think I am the first to do it) because in general, they tend to use large data analysis frameworks. I'd really appreciate some feedback on this from people working on data analysis and also from the plan 9 community (otherwise I wouldn't be writing here :-) Saludos -- Hugo