Re: [Caml-list] productivity improvement

[Eray Ozkural <erayo@cs.bilkent.edu.tr>]

On Tuesday 15 October 2002 15:34, Oleg wrote:
> On Tuesday 15 October 2002 05:31 am, Eray Ozkural wrote:
> > I have a feeling I can beat any ANN implementation written in C++ for
> > that matter ;) I was writing a generic ANN library in C++ and found it to
> > be quite difficult to put together different kinds of networks and
> > algorithms in the same basket. It would really benefit from a well
> > designed generic graph library which I can imagine would be possible only
> > in a functional language.
>
> "Beat" in what sense? Should you decide to write an O'Caml ANN library that
> learns better or faster than PDP++, SNNS and Torch, I can't imagine anyone
> trying to keep you from doing it.
>

I think better in the sense of extensibility, it could be made to allow more 
sophisticated learning algorithms or ANN models. It could be made just as 
efficient as any C code, or even faster who knows. Actually, what I have in 
mind is a general purpose machine learning library which has all the standard 
networks under the "ANN" module: single layer, multi layer feed forward 
(together with BP), hopfield and kohonen nets... I wrote all that in C++ for 
a grad course but I think it has its shortcomings, so I intend to rewrite it 
in ocaml so that I can have a convenient machine learning shell.

Nevertheless, it's an awful lot of work if you want to have your interfaces 
tidy.

> BTW BOOST has some sort of template graph library by Jeremy Siek.

Yes I know, but I prefer to use my own stuff. Even though that library is 
supposed to go into C++ standard some time in the future ;) I think C++ will 
be obsolete by then ;)


Here is some C++ client code for character recognition to give you a feel of 
the approach I have in mind. I would like to have an ANN library that is more 
generic than the one I crafted in C++ ;) Any ideas welcome.

  typedef Neuron< Bipolar_Sigmoidal_Custom > Neuron;
  typedef Sqr_Matrix<double,5> Matrix;
  typedef Matrix_Source< Matrix > Source;
  typedef pair<Source *, vector<double> > Training_Pair;

  // use a multi layered neural network
  Feed_Forward_Net<Neuron, Raw_Input_Neuron, Neuron> ff_net;

  // add hidden neurons initialized at small random values.
  for (int i=0; i<11; i++)
    ff_net.add_hidden(Neuron(Rand::rand_double(-0.005, 0.005), 0));
  // initialize with random values
  ff_net.init_random();

  // a square 5x5 matrix
  Matrix Amtx;

  // we now configure our network for ebp

  Source source(Amtx);
  ff_net.connect_input(source);
  ff_net.connect_output(10); // our coding requires ten outputs

  // the net has been put to required topology

  // read the training sets into this training pairs list
  vector< Training_Pair > pairs;

  // this is all hardwired, not much config
  // is required.

  list< Matrix* > matrices;
//   list< auto_ptr<Matrix> > matrices;
  list< Source > sources;
  for (int character = 0; character < 10; character++) {
    ostrstream name_stream;
    name_stream << "data/char-" << character  << ".txt" << ends;
    ifstream file_in( name_stream.str()  );
    // 4 patterns each
    for (int i = 0; i<4; i++) {
      Matrix *Amtx = new Matrix;
      matrices.push_back( Amtx );
      sources.push_back( Source(*Amtx) );
      vector<double> desired = cons_max(10, character);
      pairs.push_back( Training_Pair(&sources.back(), desired) );
      file_in >> *Amtx;
    }
  }

  // train the network with this data
  // the learning coefficient is 0.02
  CPU_Time start_time;
  ff_net.train(pairs, 0.02, 0.00002, 100000);
  cout << "Trained in " << CPU_Time() - start_time << endl;

  // voila
  nlog << ff_net << endl;

  // now testing with the original training set
  list< Source >::iterator source_it = sources.begin();
  int correct_results = 0;
  for (int character = 0; character < 10; character++) {
    cout << "testing character " << character << endl;
    for (int i = 0; i<4; i++) {
      ff_net.compute(*source_it++);
      vector<double> result_vec(10);
      for (int res=0; res<10; res++)
        result_vec[res] = ff_net.output_layer[res].read();
      double result =select_max(result_vec);
      cout << "  pattern " << i
           << " : " << result << endl;
      if (result == character)
        correct_results++;
    }
  }
  double train_success = double(correct_results) / 40 * 100;
  cout << "Training success is " << train_success << "%" << endl;

and so forth...

-- 
Eray Ozkural (exa) <erayo@cs.bilkent.edu.tr>
Comp. Sci. Dept., Bilkent University, Ankara
www: http://www.cs.bilkent.edu.tr/~erayo  Malfunction: http://mp3.com/ariza
GPG public key fingerprint: 360C 852F 88B0 A745 F31B  EA0F 7C07 AE16 874D 539C
-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners