Re: [Caml-list] Common code over different fields of a record, using macros?

[Anthony Tavener <anthony.tavener@gmail.com>]

That explains everything, thank-you Gabriel for providing a patch. I
will be modest in my expectations from the macro parser. It does help
to understand that it's more of an example or proof-of-concept.

I've been looking more seriously into camlp4 now and will continue.
Also, your cautionary points about too much syntactic alteration are
well noted. I'd rather keep things as stock as possible. Though I
always hope for a performant and expressive language which provides
good (compile-time) metaprogramming facilities. OCaml has 2.5 out of
3. ;) Enough that I'm happy. :)

A note on the hashtable approach... that is how most of my data is
stored, though in larger blocks. One table is these records of
characteristics. There's a certain ideal in being able to access a
variable by key, but I'm worried about getting the granularity too
fine and having problems with time/space costs in the long run. Your
suggestion is tempting though. I also am trying to keep more purity in
low-level functions, while the outer levels handle gathering from or
updating the database (hashtables).

My current solution, after a reply from G.Yziquel, is a set of
accessor functions which makes the high-level use rather succinct and
clear, but rather verbose in support functions...

(* in Stat module... this provides means for a read-compute-update *)
module Stat = struct
  let strength s = (s.strength,fun x -> {s with strength=x})
  (* and more... *)
end

(* this part is a little clunky due to all my tuple returns *)
let age access (state,age) =
  let s,sets = access state  (* 'access' gives us the current value
and a way to create an updated record *)
  and a,seta = access age in
  let s,a = age_stat s a in
  sets s,seta a

(* usage of 'age' with accessor function *)
  age Stat.strength input

A little messy, but this doesn't need any syntax change and redundancy
factored out. I will continue learning camlp4 for possible suitability
in this and other situations (keeping in mind that acceptable 'plain
OCaml' solutions would be preferable!).

And you make another great point about the 'ends' of an application
often dealing with 'untyped' or raw data. I might need conversions
anyway, and could possibly leverage them in other places if
appropriate.

Thank-you again for your very in-depth reply!


On Fri, Feb 18, 2011 at 2:11 AM, Gabriel Scherer
<gabriel.scherer@gmail.com> wrote:
> 1. This discontinuity (field names in access position get replaced, but not
> in a "with ...") is a bug of the current macro extension of Camlp4. I have
> just written a patch that fixes this precise issue.
> The good news is that you don't need to patch and recompile the whole OCaml
> distribution to benefit from the fix. The "macro parser" (the part of camlp4
> that supports DEFINE and so on) is a camlp4 extension like any other, so you
> just have to compile the modified code somewhere and pass the ".cmo" to the
> camlp4* preprocessor you run on your source code.
> The bad news is that while the proposed patch fixes your particular use
> case, I have found other similar inconsistencies -- eg. use in pattern
> position -- that are harder to fix and would require a deeper
> re-implementation of this syntax extension logic. I know how I should do
> that but, unless someone is specifically interested in fixing this, I'm not
> sure I will actually do it.
>
> My patch is distributed as an individual source file, which is a modified
> version of the original Camlp4Parsers/Camlp4MacroParser.ml file of the ocaml
> distribution :
>   http://bluestorm.info/camlp4/dev/define_and_fields_bug/macro.ml.html
> I use a small test file exhibiting your use case, an other use fixed by the
> change, an unrelated bug, and an issue which isn't fixed by the change.
>   http://bluestorm.info/camlp4/dev/define_and_fields_bug/test.ml.html
> All source files are available directly
>   http://bluestorm.info/camlp4/dev/define_and_fields_bug/list.php
>
> It's easy to compare test.ml against both the original and the modified
> macro parser. After compilation of macro.ml (ocamlfind-using compilation
> command given at the top of the file), use :
>   `camlp4of test.ml` for processing with the original macro parser
>   `camlp4o macro.cmo` for processing with the modifier macro parser
>
> Hope this helps.
>
> PS : I will eventually file a ticket on the ocaml bug tracker on this
> Camlp4MacroParser issue. Xavier Clerc is quite responsive on these kind of
> things, and if I (or someone else; don't hesitate!) come with a reasonable
> fix it may get into 3.13.
>
>
> 2. As a more general answer to your post, you should keep in mind that the
> macro facility, as implemented by Camlp4Parsers/Camlp4MacroParser.ml , is
> not intended to be an all-powerful syntax extension facility for OCaml. It
> is rather a simple example of the power of writing Camlp4 extensions, that
> provides simple substitute-and-replace facilities for expressions in
> patterns. The particular misbehavior you encountered should be considered a
> bug and it is legitimate to try to fix it, but it doesn't mean that this
> macro definition facility ever will suit all your potential needs of clever
> metaprogramming. Don't expect a much finer support for syntactic classes ("i
> want this parameter to accept all syntactically valid patterns, and nothing
> more") or hygienic macros any time soon.
> If you want a powerful syntactical metaprogramming tool, your best bet would
> be to use Camlp4 directly. While still limited, it is much more powerful
> that the simple Camlp4MacroParser facility. I'm not sure however that
> relying heavily on such syntactic facilities is a good idea in the long
> turn; at least, you shouldn't expect other developers to easily accept to
> use your non-standard extensions. However, for projects that you develop
> alone, or where you have strong control on the compilation chain, camlp4 (or
> camlp5) can be a nice tool.
>
>
> 3. In your specific example, a good compromise would be to use a hashtable
> indexed by an algebraic datatype.
>
>   type field = Strength | Agility
>
>   type stats_table = (field, int) Hashtbl.t
>
>   let age_state _ _ = failwith "not implemented"
>
>   let _AGE field = fun (state, age) ->
>     let s,a = age_state (Hashtbl.find state field) (Hashtbl.find age field)
> in
>     Hashtbl.add state field s;
>     Hashtbl.add age field a;
>     (state, age)
>
>   let age_character input = function
>     | n when n < 2 -> input
>     | 2 -> _AGE(Strength) input
>     | _ -> _AGE(Agility) input
>
> In my example, an impure data structure is used (Hashtbl) and updated
> imperatively; if you want to keep a pure update-by-copy operation like the
> {... with ..} facility of records, you should rather use Map instead of
> Hashtbl. As thread all state through a state monad, I considered this was
> not an issue.
>
> If you want a more concise syntax for hasthable access (and do not use the
> related syntax extension), you can define infix operators, following an idea
> from Paolo Donadeo:
>
>   let (-->) table key = Hashtbl.find table key
>   let (<--) table (key, value) = Hashtbl.add table key value
>
> The downside of associative data structure instead of records here is that
> force each field to have the same type. You couldn't have strength be an int
> and agility be a float. It is fine however in the common situation where you
> can have a uniform operation over field names because they are all used with
> the same type.
>
> Finally, it can be a good idea to determine a confined part of your program
> where you need that indexing flexibility, where you would use that
> "stats_table" representation, and convert in and out to a record "stats"
> type so that the rest of your program see the more standard record type.
> In my experience, the less typed but more flexible representations are often
> only useful in the "ends" of your application, typically the place where you
> do input-output; a common case is when constructing the data by parsing a
> configuration file.
>
>
> On Thu, Feb 17, 2011 at 12:53 AM, Anthony Tavener
> <anthony.tavener@gmail.com> wrote:
>>
>> I find that records often result in redundant code... the same operation
>> being specified for different fields (of the same type). I could use arrays
>> in these cases, but then lose meaningful (to programmers) field names.
>>
>> I tried using camlp4 (and camlp5) macros to get the effect of passing a
>> field "name" to a "function". I could get my example to work if I update
>> mutable fields (record.field <- x), but using the functional record-update
>> {record with field=x} doesn't work...
>>
>>
>> --- Simplified mock-up of my current situation (not enough to compile) ---
>>
>> (* a record with some fields of the same type... imagine there might be
>> many more fields *)
>> type stats = { strength: int; agility: int }
>>
>> (* macro which doesn't work... "field" which follows "with" doesn't get
>> replaced *)
>> DEFINE AGE(field) = fun (state,age) ->
>>   let s,a = age_stat state.field age.field in
>>   {state with field=s}, {age with field=a}
>>
>> (* val f : (stats * stats) -> int -> (stats * stats) *)
>> let age_character input = function
>>   | n when n < 2 -> input
>>   | 2 -> AGE(strength) input
>>   | 3 -> AGE(agility) input
>>   | _ -> AGE2(strength,agility) input
>>
>> (* a mock-up of usage... *)
>> let state = { strength=3; agility=1 } in
>> let age = { strength=0; agility=0 } in
>> let state',age' = age_character (state,age) (rand 8) in ...
>>
>> ---
>>
>> After processing by camlp4 with macros this is what the first case
>> becomes:
>>
>>   | 1 -> let s,a = age_stat state.strength age.strength in
>>          {state with field=s}, {age with field=a}
>>
>> "field" isn't replaced with "strength" in the record-update. I tried
>> looking at Camlp4MacroParser.ml... but it makes my head swim. It must be
>> doing something much more careful than literal text replacement, but perhaps
>> too careful... or incomplete? Does anyone know how these macros work? Is
>> this proper behavior for some reason, or an unhandled edge case?
>>
>> The problem is I don't want a block of code like this to be repeated 10
>> times with only a field-name change for each (and 4 field names each time!).
>> That's not well readable, prone to error, and harder to maintain properly.
>>
>> Sometimes I wish I could present an alternative view of the same data,
>> such as having an array "view" into part of a record... verified to be
>> typesafe by the compiler... and compiled into the same simple offsets in the
>> end. Maybe it's my asm/C origins which I never seem to escape. I mention
>> this in hope that someone says "Oh, that's exactly what you can do... like
>> this!" :)
>>
>> Thank-you for any help caml-list!
>>
>>  Tony
>>
>
>