Bigarray question

Bigarray question

[Thomas Fischbacher]

As far as I can see from the documentation, it is not explicitly permitted 
to change a once allocated bigarray data structure in such a way that one 
replaces the data pointer by some other data pointer (while retaining 
size and type). This would, however, be quite useful in some situations.

Could this be changed?

-- 
regards,               tf@cip.physik.uni-muenchen.de              (o_
 Thomas Fischbacher -  http://www.cip.physik.uni-muenchen.de/~tf  //\
(lambda (n) ((lambda (p q r) (p p q r)) (lambda (g x y)           V_/_
(if (= x 0) y (g g (- x 1) (* x y)))) n 1))                  (Debian GNU)

Re: [Caml-list] Bigarray question

[Dmitry Bely]

Thomas Fischbacher <Thomas.Fischbacher@Physik.Uni-Muenchen.DE> writes:

> As far as I can see from the documentation, it is not explicitly permitted 
> to change a once allocated bigarray data structure in such a way that one 
> replaces the data pointer by some other data pointer (while retaining 
> size and type). This would, however, be quite useful in some situations.

Is your bigarray a wrapper around some externally allocated data, created
via alloc_bigarray() C call? If yes, you can safely replace the data
pointer with another one via Caml-C interface. If not, why do you need that?

- Dmitry Bely

Re: [Caml-list] Bigarray question

[Thomas Fischbacher]

On Fri, 25 Nov 2005, Dmitry Bely wrote:

> Thomas Fischbacher <Thomas.Fischbacher@Physik.Uni-Muenchen.DE> writes:
> 
> > As far as I can see from the documentation, it is not explicitly permitted 
> > to change a once allocated bigarray data structure in such a way that one 
> > replaces the data pointer by some other data pointer (while retaining 
> > size and type). This would, however, be quite useful in some situations.
> 
> Is your bigarray a wrapper around some externally allocated data, created
> via alloc_bigarray() C call?

It's a somewhat special case of just that situation, yes.

I supposed I may just do so, however, the specification (documentation) 
does not give me any guarantee that this is valid, so, at least the 
documentation has to be changed before I can rely on this.

There is a subtle difference between "just accidentally works" 
and "works because it is guaranteed by specification".

-- 
regards,               tf@cip.physik.uni-muenchen.de              (o_
 Thomas Fischbacher -  http://www.cip.physik.uni-muenchen.de/~tf  //\
(lambda (n) ((lambda (p q r) (p p q r)) (lambda (g x y)           V_/_
(if (= x 0) y (g g (- x 1) (* x y)))) n 1))                  (Debian GNU)

Re: [Caml-list] Bigarray question

[Dmitry Bely]

Thomas Fischbacher <Thomas.Fischbacher@Physik.Uni-Muenchen.DE> writes:

>> > As far as I can see from the documentation, it is not explicitly permitted 
>> > to change a once allocated bigarray data structure in such a way that one 
>> > replaces the data pointer by some other data pointer (while retaining 
>> > size and type). This would, however, be quite useful in some situations.
>> 
>> Is your bigarray a wrapper around some externally allocated data, created
>> via alloc_bigarray() C call?
>
> It's a somewhat special case of just that situation, yes.
>
> I supposed I may just do so, however, the specification (documentation) 
> does not give me any guarantee that this is valid, so, at least the 
> documentation has to be changed before I can rely on this.

1. For the current implementation it's safe (use Data_bigarray_val(v) to
change the data pointer). I don't see any reason why it could become unsafe.
2. It is up to you whether to use this feature or not.

> There is a subtle difference between "just accidentally works"
> and "works because it is guaranteed by specification".

OCaml is a developing language, so IMHO nobody guarantees that nothing will
change in the future.

- Dmitry Bely

Re: [Caml-list] Bigarray question

[Thomas Fischbacher]

On Mon, 28 Nov 2005, Dmitry Bely wrote:

> 1. For the current implementation it's safe (use Data_bigarray_val(v) to
> change the data pointer). I don't see any reason why it could become unsafe.
> 2. It is up to you whether to use this feature or not.

I will neither use undocumented programming techniques, nor recommend 
doing so to other programmers, no matter what kind of language. That would 
- quite in general - be a very bad habit that eventually must lead to bugs 
that are hard to impossible to diagnose and debug.

Assigning to Data_bigarray_val is not explicitly permitted.

Nevertheless, being allowed to do so would be helpful. So, a simple change 
to the documentation would suffice to make me happy.

-- 
regards,               tf@cip.physik.uni-muenchen.de              (o_
 Thomas Fischbacher -  http://www.cip.physik.uni-muenchen.de/~tf  //\
(lambda (n) ((lambda (p q r) (p p q r)) (lambda (g x y)           V_/_
(if (= x 0) y (g g (- x 1) (* x y)))) n 1))                  (Debian GNU)

Re: [Caml-list] Bigarray question

[Thomas Fischbacher]

Xavier Leroy wrote:

(on whether it is permissible to dynamically change the dim and data
pointers in a ML bigarray):

>>as far as I can tell, it is easy (but not explicitly allowed in the
>>documentation) to allocate a Caml bigarray and change the data
>>pointer inside that bigarray afterwards.
>>There is no problem in implementing this, technically speaking.
>>But this would require a change to the ML documentation stating
>>that it is explicitly considered as permissible to change the
>>data pointer in a bigarray. Can I get that, please?
> 
> 
> I see no reason why this would cause problems, as long as the data
> pointer points to C data of the right shape and dimensions.  (Of
> course, you could update the dimensions of the bigarray as
> appropriate, if needed.)  I don't think I would document this, as I
> wouldn't quite know how to word it in the docs, but you have my
> encouragements to try and report problems if any.


As a matter of fact, considering the two C functions below which I use
in our "nsim" abstract field theory/multiphysics simulation engine to
speed up some internal computation, the first variant would be desirable
but crashes. The second works. What this actually is about is to
systematically go through the rows of a sparse matrix, handled by the
PETSc library. (So, basically, I am using a sparse matrix as sort-of a
hash of row index to entries here.)

CAMLprim value caml_petsc_matrix_call_on_rows_raw(value ml_mx,
						  value ml_fun,
						  value ml_start_row,
						  value ml_end_row
						  )
{
   CAMLparam4(ml_mx,ml_fun,ml_start_row,ml_end_row);
   Mat mx;
   int start_row,end_row,start_own,end_own;
   int i,ncols=0;
   const int *cols=0;
   const PetscScalar *vals=0;
   CAMLlocal2(ba_indices,ba_vals);

   petsc_checkinit();
   petsc_check_mat(ml_mx);

   mx=(Mat)Field(ml_mx,1);

   MatGetOwnershipRange(mx,&start_own,&end_own);

   start_row=Int_val(ml_start_row);
   end_row=Int_val(ml_end_row);

   if(start_row == -1)start_row=start_own;
   if(end_row == -1)end_row=end_own;

   /* We would like to do things this way, but while one may guess that 
it should work,
      this (1) is not covered by what is permitted according to the 
OCaml documentation,
      and (2) indeed produces crashes.
   */
   ba_indices=alloc_bigarray_dims(BIGARRAY_NATIVE_INT | BIGARRAY_C_LAYOUT,
				 1, cols, ncols);

   ba_vals=alloc_bigarray_dims(BIGARRAY_FLOAT64 | BIGARRAY_C_LAYOUT,
			      1, vals, ncols);


   for(i=start_row;i<end_row;i++)
     {
       MatGetRow(mx,i,&ncols,&cols,&vals);

       Bigarray_val(ba_indices)->dim[0]=ncols;
       Bigarray_val(ba_vals)->dim[0]=ncols;

       Bigarray_val(ba_indices)->data=(void*)cols;
       Bigarray_val(ba_vals)->data=(void*)vals;

       callback3(ml_fun,Val_int(i),ba_indices,ba_vals);

       MatRestoreRow(mx,i,&ncols,&cols,&vals);
     }

   CAMLreturn(Val_unit);
}



CAMLprim value caml_petsc_matrix_call_on_rows_raw_defensive(value ml_mx,
							    value ml_fun,
							    value ml_start_row,
							    value ml_end_row
							    )
{
   CAMLparam4(ml_mx,ml_fun,ml_start_row,ml_end_row);
   Mat mx;
   int start_row,end_row,start_own,end_own;
   int i,ncols=0;
   const int *cols=0;
   const PetscScalar *vals=0;
   CAMLlocal2(ba_indices,ba_vals);

   petsc_checkinit();
   petsc_check_mat(ml_mx);

   mx=(Mat)Field(ml_mx,1);

   MatGetOwnershipRange(mx,&start_own,&end_own);

   start_row=Int_val(ml_start_row);
   end_row=Int_val(ml_end_row);

   if(start_row == -1)start_row=start_own;
   if(end_row == -1)end_row=end_own;


   for(i=start_row;i<end_row;i++)
     {
       MatGetRow(mx,i,&ncols,&cols,&vals);

       ba_indices=alloc_bigarray_dims(BIGARRAY_NATIVE_INT | 
BIGARRAY_C_LAYOUT,
				     1, cols, ncols);

       ba_vals=alloc_bigarray_dims(BIGARRAY_FLOAT64 | BIGARRAY_C_LAYOUT,
				  1, vals, ncols);

       callback3(ml_fun,Val_int(i),ba_indices,ba_vals);

       MatRestoreRow(mx,i,&ncols,&cols,&vals);
     }

   CAMLreturn(Val_unit);
}


-- 
best regards,
Thomas Fischbacher
tf@functionality.de

Re: [Caml-list] Bigarray question

[Thomas Fischbacher]

Xavier,

>>as far as I can tell, it is easy (but not explicitly allowed in the
>>documentation) to allocate a Caml bigarray and change the data
>>pointer inside that bigarray afterwards.
>>There is no problem in implementing this, technically speaking.
>>But this would require a change to the ML documentation stating
>>that it is explicitly considered as permissible to change the
>>data pointer in a bigarray. Can I get that, please?
> 
> I see no reason why this would cause problems, as long as the data
> pointer points to C data of the right shape and dimensions.  (Of
> course, you could update the dimensions of the bigarray as
> appropriate, if needed.)  I don't think I would document this, as I
> wouldn't quite know how to word it in the docs, but you have my
> encouragements to try and report problems if any.

Just adding a line to the documentation which states that
"it is permitted to dynamically change the data and dims
fields of the bigarray data structure from within C" would be
sufficient.

-- 
best regards,
Thomas Fischbacher
t.fischbacher@soton.ac.uk

Re: [Caml-list] Bigarray question

[Xavier Leroy]

> as far as I can tell, it is easy (but not explicitly allowed in the
> documentation) to allocate a Caml bigarray and change the data
> pointer inside that bigarray afterwards.
> There is no problem in implementing this, technically speaking.
> But this would require a change to the ML documentation stating
> that it is explicitly considered as permissible to change the
> data pointer in a bigarray. Can I get that, please?

I see no reason why this would cause problems, as long as the data
pointer points to C data of the right shape and dimensions.  (Of
course, you could update the dimensions of the bigarray as
appropriate, if needed.)  I don't think I would document this, as I
wouldn't quite know how to word it in the docs, but you have my
encouragements to try and report problems if any.

- Xavier Leroy