CamlMPI: sending marshalled objects

[Thomas Fischbacher <tf@functionality.de>]

Hello everybody,

there is an old Caml/MPI interface, written by Xavier himself in 1998.
This provides a pair of C functions caml_mpi_send/caml_mpi_receive,
which may look a bit funny considering present day ML-C-interface coding
conventions, but GC-wise is actually valid code as far as I can judge.
For reference, I added them at the end of this email.

As far as I can see, output_value_to_malloc() actually should be able to
serialize virtually any ML object (which does not contain alloc_custom()
blocks and similar stuff) to a buffer. Sending this over the net also
should not be much of a problem, regardless of the data being sent. If,
on the other hand, I manually serialized to a string and tried to send
that, I would be bound by the 16 MB maximal string length limitation on
32-bit architectures.

So much the theory. But what I actually find in reality is that -- even
using caml_mpi_send/caml_mpi_receive rather than intermediate strings --
my code crashes when I try to pass around serialized ML data over MPI
which is larger than 16 MB. I am not yet 100% sure that this really is
the problem, but at present, I have every reason to believe so.

So, (1) what is going on here, and (2) how can I repair it?
Is there a quick fix?


value caml_mpi_send(value data, value flags,
                     value dest, value tag, value vcomm)
{
   MPI_Comm comm = Comm_val(vcomm);
   char * buffer;
   long len;

   Begin_root(vcomm)             /* prevent deallocation of communicator */
     output_value_to_malloc(data, flags, &buffer, &len);
     /* This also allocates the buffer */
     enter_blocking_section();
     MPI_Send(buffer, len, MPI_BYTE, Int_val(dest), Int_val(tag), comm);
     leave_blocking_section();
   End_roots();
   stat_free(buffer);
   return Int_val(len);
}


value caml_mpi_receive(value vlen, value source, value tag, value vcomm)
{
   MPI_Comm comm = Comm_val(vcomm);
   mlsize_t len = Long_val(vlen);
   char * buffer;
   MPI_Status status;
   value res;

   Begin_root(vcomm)             /* prevent deallocation of communicator */
     buffer = stat_alloc(len);
     enter_blocking_section();
     MPI_Recv(buffer, len, MPI_BYTE,
              Int_val(source), Int_val(tag), comm, &status);
     leave_blocking_section();
     res = input_value_from_malloc(buffer, 0);
     /* This also deallocates the buffer */
   End_roots();
   return res;
}

-- 
best regards,
Thomas Fischbacher
t.fischbacher@soton.ac.uk