From mboxrd@z Thu Jan 1 00:00:00 1970 Received: (from majordomo@localhost) by pauillac.inria.fr (8.7.6/8.7.3) id TAA05878; Tue, 15 Oct 2002 19:11:16 +0200 (MET DST) X-Authentication-Warning: pauillac.inria.fr: majordomo set sender to owner-caml-list@pauillac.inria.fr using -f Received: (from weis@localhost) by pauillac.inria.fr (8.7.6/8.7.3) id TAA05961 for caml-list@pauillac.inria.fr; Tue, 15 Oct 2002 19:11:16 +0200 (MET DST) Received: from concorde.inria.fr (concorde.inria.fr [192.93.2.39]) by pauillac.inria.fr (8.7.6/8.7.3) with ESMTP id RAA01032 for ; Tue, 15 Oct 2002 17:12:11 +0200 (MET DST) Received: from mail.interaktif.net.tr ([62.248.102.120]) by concorde.inria.fr (8.11.1/8.11.1) with ESMTP id g9FFC9514822 for ; Tue, 15 Oct 2002 17:12:09 +0200 (MET DST) Received: from abn166-176.ank-avrupa-ports.kablonet.net.tr (unknown [195.174.166.176]) by mail.interaktif.net.tr (Postfix) with ESMTP id DFBC61E6D7; Tue, 15 Oct 2002 18:09:02 +0300 (EEST) From: Eray Ozkural Organization: Bilkent University CS Dept. To: Oleg Subject: Re: [Caml-list] productivity improvement Date: Tue, 15 Oct 2002 18:08:17 +0300 User-Agent: KMail/1.4.5 Cc: caml-list References: <20020716172916.4903.qmail@web10702.mail.yahoo.com> <200210151231.18731.erayo@cs.bilkent.edu.tr> <200210151233.IAA08545@apakabar.cc.columbia.edu> In-Reply-To: <200210151233.IAA08545@apakabar.cc.columbia.edu> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 8bit Content-Disposition: inline Message-Id: <200210151808.17342.erayo@cs.bilkent.edu.tr> Sender: owner-caml-list@pauillac.inria.fr Precedence: bulk On Tuesday 15 October 2002 15:34, Oleg wrote: > On Tuesday 15 October 2002 05:31 am, Eray Ozkural wrote: > > I have a feeling I can beat any ANN implementation written in C++ for > > that matter ;) I was writing a generic ANN library in C++ and found it to > > be quite difficult to put together different kinds of networks and > > algorithms in the same basket. It would really benefit from a well > > designed generic graph library which I can imagine would be possible only > > in a functional language. > > "Beat" in what sense? Should you decide to write an O'Caml ANN library that > learns better or faster than PDP++, SNNS and Torch, I can't imagine anyone > trying to keep you from doing it. > I think better in the sense of extensibility, it could be made to allow more sophisticated learning algorithms or ANN models. It could be made just as efficient as any C code, or even faster who knows. Actually, what I have in mind is a general purpose machine learning library which has all the standard networks under the "ANN" module: single layer, multi layer feed forward (together with BP), hopfield and kohonen nets... I wrote all that in C++ for a grad course but I think it has its shortcomings, so I intend to rewrite it in ocaml so that I can have a convenient machine learning shell. Nevertheless, it's an awful lot of work if you want to have your interfaces tidy. > BTW BOOST has some sort of template graph library by Jeremy Siek. Yes I know, but I prefer to use my own stuff. Even though that library is supposed to go into C++ standard some time in the future ;) I think C++ will be obsolete by then ;) Here is some C++ client code for character recognition to give you a feel of the approach I have in mind. I would like to have an ANN library that is more generic than the one I crafted in C++ ;) Any ideas welcome. typedef Neuron< Bipolar_Sigmoidal_Custom > Neuron; typedef Sqr_Matrix Matrix; typedef Matrix_Source< Matrix > Source; typedef pair > Training_Pair; // use a multi layered neural network Feed_Forward_Net ff_net; // add hidden neurons initialized at small random values. for (int i=0; i<11; i++) ff_net.add_hidden(Neuron(Rand::rand_double(-0.005, 0.005), 0)); // initialize with random values ff_net.init_random(); // a square 5x5 matrix Matrix Amtx; // we now configure our network for ebp Source source(Amtx); ff_net.connect_input(source); ff_net.connect_output(10); // our coding requires ten outputs // the net has been put to required topology // read the training sets into this training pairs list vector< Training_Pair > pairs; // this is all hardwired, not much config // is required. list< Matrix* > matrices; // list< auto_ptr > matrices; list< Source > sources; for (int character = 0; character < 10; character++) { ostrstream name_stream; name_stream << "data/char-" << character << ".txt" << ends; ifstream file_in( name_stream.str() ); // 4 patterns each for (int i = 0; i<4; i++) { Matrix *Amtx = new Matrix; matrices.push_back( Amtx ); sources.push_back( Source(*Amtx) ); vector desired = cons_max(10, character); pairs.push_back( Training_Pair(&sources.back(), desired) ); file_in >> *Amtx; } } // train the network with this data // the learning coefficient is 0.02 CPU_Time start_time; ff_net.train(pairs, 0.02, 0.00002, 100000); cout << "Trained in " << CPU_Time() - start_time << endl; // voila nlog << ff_net << endl; // now testing with the original training set list< Source >::iterator source_it = sources.begin(); int correct_results = 0; for (int character = 0; character < 10; character++) { cout << "testing character " << character << endl; for (int i = 0; i<4; i++) { ff_net.compute(*source_it++); vector result_vec(10); for (int res=0; res<10; res++) result_vec[res] = ff_net.output_layer[res].read(); double result =select_max(result_vec); cout << " pattern " << i << " : " << result << endl; if (result == character) correct_results++; } } double train_success = double(correct_results) / 40 * 100; cout << "Training success is " << train_success << "%" << endl; and so forth... -- Eray Ozkural (exa) Comp. Sci. Dept., Bilkent University, Ankara www: http://www.cs.bilkent.edu.tr/~erayo Malfunction: http://mp3.com/ariza GPG public key fingerprint: 360C 852F 88B0 A745 F31B EA0F 7C07 AE16 874D 539C ------------------- To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/ Beginner's list: http://groups.yahoo.com/group/ocaml_beginners