Re: [Caml-list] looping recursion

[Brian Hurt <bhurt@spnz.org>]

On Tue, 27 Jul 2004 briand@aracnet.com wrote:

> it really is.
> although I'd actually like it to be an array.
> it's the classic situation, read whole file - iterate on elements.
> easy to do in a list since I don't need to know the size.
> 
> some minor tweaks and I can include the size in a data file and just
> fill an array directly.
> 
> however, my larger point is the fact that the "standard" map blows up
> like that.  as a long time scheme user I just find that plain weird.
> Now Everybody else seems to think I'm weird because I think that's
> weird ;-)
> 
>   Brian> Very long lists are a sign that you're using the wrong data
>   Brian> structure.
> 
> I'm not sure I understand that.  I have 10^6 data points to work on.
> I'm thinking a list or an array is the right data structure.  Although
> an array is more right.  I mean if you are not worried about
> efficiency then incremental transformation by map'ing is a very
> elegant and clean way to go about doing that sort of work.
> 
> Part of the problem is that powerful computers make for lazy
> programmers :-) It's just so easy to read the 10^6 elements into a
> list and then just keep map'ing them to the final value when it only
> takes seconds to do :-) If it took 2 minutes I'd be more inclined to
> think about the problem.
> 

You want an enumeration.  You're basically taking a file, reading it into 
a list, mapping the list multiple different times, and then writting out 
the result, right?  Well, enumerations work the same way.  You take the 
file, and create an enumeration of the lines of the file (there's a 
function that does that).  You then map the enumeration multiple times, 
same map functions.  Then you write out the result.  As a programmer, it 
doesn't change the problem all that much.

But it allows the library to optimize things.  Maps of enumerations are 
applied lazily- that means they aren't calculated until the result is 
needed.  By creating the enumeration of the file doesn't read the file at 
that point.  When you go to write the result out, the output function 
grabs the first line from the enumeration.  This causes the enumeration to 
actually go read the first line of the input file, and apply all the 
appropriate maps to it, before writting it to the output file.  And so on.
You write the code as if the entire data structure were present right now, 
but it works like a stream processor.  With no extra work on the part of 
the programmer.


-- 
"Usenet is like a herd of performing elephants with diarrhea -- massive,
difficult to redirect, awe-inspiring, entertaining, and a source of
mind-boggling amounts of excrement when you least expect it."
                                - Gene Spafford 
Brian


-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners