[Caml-list] Functional GUI programming: looking for good practices

[Caml-list] Functional GUI programming: looking for good practices

[Philippe Veber]

[-- Attachment #1: Type: text/plain, Size: 924 bytes --]

Dear camlers,

I'm looking for advanced examples of GUI programming in functional style.
As I'm aware there is no definitive answer on this topic, I'll gladly read
about pragmatic approaches which may fail to be fully declarative, but do
work well in practice. Lately I've been trying to write a little GUI API,
replacing all mutable values by React signals (
http://erratique.ch/software/react), but it turns out to be more difficult
than expected, for example with layout management. In order to compute a
layout for the widgets, some information has to travel bottom up the widget
hierarchy, and some goes top down. While there is a well-founded order for
defining all signals, it's more difficult to group them in their respective
widget and still avoid mutually recursive definitions. More generally I'd
interested in good (and pragmatic !) techniques for GUI programming, with
some real life code.
Cheers,
  Philippe.

[-- Attachment #2: Type: text/html, Size: 1022 bytes --]

Re: [Caml-list] Functional GUI programming: looking for good practices

[Adrien]

On 13/02/2012, Philippe Veber <philippe.veber@gmail.com> wrote:
> Dear camlers,
>
> I'm looking for advanced examples of GUI programming in functional style.
> As I'm aware there is no definitive answer on this topic, I'll gladly read
> about pragmatic approaches which may fail to be fully declarative, but do
> work well in practice. Lately I've been trying to write a little GUI API,
> replacing all mutable values by React signals (
> http://erratique.ch/software/react), but it turns out to be more difficult
> than expected, for example with layout management. In order to compute a
> layout for the widgets, some information has to travel bottom up the widget
> hierarchy, and some goes top down. While there is a well-founded order for
> defining all signals, it's more difficult to group them in their respective
> widget and still avoid mutually recursive definitions. More generally I'd
> interested in good (and pragmatic !) techniques for GUI programming, with
> some real life code.
> Cheers,
>   Philippe.

Since FRP is quite "new" and not well understood, I'm going to try to
summarize how I understan it: it makes it possible to use functional code
for a task that has typically relied on mutability, with all the benefits
we're used to.


I've created a lablgtk branch named "adrien/react" to get react signals out
of gtk properties and react events out of gtk signals (they match quite
well). Support isn't perfect but enough to test and experiment.

The issue was to write the application itself: it was way too complicated
and it involved many many react values which had to be somehow kept alive.
It was also not very useful. The reason was that the very first thing I
would do with all the events was React.E.select: I would create distinct
signals only to merge them!

What I've started doing instead is to have one "object" with a corresponding
event: callbacks only send a message to that event and look like "send (`Foo
bar)" and from then I can use match over the message in a different location
in the code, typically in an FRP context.


(* Warning: this is about work in-progress; it might not always make sense,
might have some weird things and I might not be able to explain everything
properly. *)

My current application is a web browser which I want to make much more
intelligent than the browsers available today. For this reason, I store
web pages in a data structure which is of course purely functional. I can
have several layers of data structures containing objects in the same way.

My objects start with a default state and evolve (in a functional way)
through a fold according to the messages they receive. Each time the state
changes, two sets of callbacks are triggered: first, to change the UI;
second, to update the data structure containing the object which is needed
because of functional updates. I also use that last mechanism to propagate
messages from the inner objects to the outter ones.

One last characteristic is that I have a UI side besides the functional one.
It contains a handful of things which are needed to work with GTK.I also use
it to propagate messages from the outter objects to the inner ones.

This is work-in-progress and some details could be improved but I think that
the big picture is there. As far as I can tell, the UI and functional sides
are properly separated, constraints aren't heavy and I seem to be able to
get the usual qualities of ocaml in GUIs.


Generally speaking, FRP is not the silver bullet for GUIs. Maybe for Haskell
but definitely not for OCaml. The main reason is probably that most C
libraries have a specific API which is often very imperative. OCaml provides
references, mutability and objects. If you don't have a big beautiful data
structure and an actual model for the state of your program, you might want
to go the easy route and not use FRP but mutability everywhere.


PS: for layout management, I found that using the ~packing option when
creating the widgets is usually much better than #add'ing the afterwards.


Hope this helps,
Adrien Nader

Re: [Caml-list] Functional GUI programming: looking for good practices

[Philippe Veber]

[-- Attachment #1: Type: text/plain, Size: 5797 bytes --]

2012/2/13 Adrien <camaradetux@gmail.com>

> On 13/02/2012, Philippe Veber <philippe.veber@gmail.com> wrote:
> > Dear camlers,
> >
> > I'm looking for advanced examples of GUI programming in functional style.
> > As I'm aware there is no definitive answer on this topic, I'll gladly
> read
> > about pragmatic approaches which may fail to be fully declarative, but do
> > work well in practice. Lately I've been trying to write a little GUI API,
> > replacing all mutable values by React signals (
> > http://erratique.ch/software/react), but it turns out to be more
> difficult
> > than expected, for example with layout management. In order to compute a
> > layout for the widgets, some information has to travel bottom up the
> widget
> > hierarchy, and some goes top down. While there is a well-founded order
> for
> > defining all signals, it's more difficult to group them in their
> respective
> > widget and still avoid mutually recursive definitions. More generally I'd
> > interested in good (and pragmatic !) techniques for GUI programming, with
> > some real life code.
> > Cheers,
> >   Philippe.
>
> Since FRP is quite "new" and not well understood, I'm going to try to
> summarize how I understan it: it makes it possible to use functional code
> for a task that has typically relied on mutability, with all the benefits
> we're used to.
>
With, I think, an additional benefit: making things difficult to write when
we would have done something fishy with mutable references. It's just that
sometimes it's difficult to avoid fishy stufff :o).


>
>
> I've created a lablgtk branch named "adrien/react" to get react signals out
> of gtk properties and react events out of gtk signals (they match quite
> well). Support isn't perfect but enough to test and experiment.
>
> The issue was to write the application itself: it was way too complicated
> and it involved many many react values which had to be somehow kept alive.
> It was also not very useful. The reason was that the very first thing I
> would do with all the events was React.E.select: I would create distinct
> signals only to merge them!
>
> What I've started doing instead is to have one "object" with a
> corresponding
> event: callbacks only send a message to that event and look like "send
> (`Foo
> bar)" and from then I can use match over the message in a different
> location
> in the code, typically in an FRP context.
>
I did that too, and it does seem a good practice: a central bus for
information that is relevant to the whole application. I have a question on
this particular aspect: suppose this bus is updated through the primitive
(broadcast : message -> unit), and that I have a bool event ev that
(indirectly) depends on the bus. Am I allowed to define the following event:

let action = React.E.map (fun b -> if b then broadcast `action ; not b) ev

In other words, am I allowed to call a primitive in a lifted function?


>
>
> (* Warning: this is about work in-progress; it might not always make sense,
> might have some weird things and I might not be able to explain everything
> properly. *)
>
> My current application is a web browser which I want to make much more
> intelligent than the browsers available today.

Just curious: in what way?


> For this reason, I store
> web pages in a data structure which is of course purely functional. I can
> have several layers of data structures containing objects in the same way.
>
> My objects start with a default state and evolve (in a functional way)
> through a fold according to the messages they receive. Each time the state
> changes, two sets of callbacks are triggered: first, to change the UI;
> second, to update the data structure containing the object which is needed
> because of functional updates. I also use that last mechanism to propagate
> messages from the inner objects to the outter ones.
>
> One last characteristic is that I have a UI side besides the functional
> one.
> It contains a handful of things which are needed to work with GTK.I also
> use
> it to propagate messages from the outter objects to the inner ones.
>
> This is work-in-progress and some details could be improved but I think
> that
> the big picture is there. As far as I can tell, the UI and functional sides
> are properly separated, constraints aren't heavy and I seem to be able to
> get the usual qualities of ocaml in GUIs.
>
What module would you recommend reading to get a taste of it?



>
>
> Generally speaking, FRP is not the silver bullet for GUIs. Maybe for
> Haskell
> but definitely not for OCaml. The main reason is probably that most C
> libraries have a specific API which is often very imperative. OCaml
> provides
> references, mutability and objects. If you don't have a big beautiful data
> structure and an actual model for the state of your program, you might want
> to go the easy route and not use FRP but mutability everywhere.
>
Actually I don't use a GUI binding, but directly drawing primitives from
sdl, it looked too difficult for me to use FRP in conjunction with an
existing GUI library, for the very reason you invoke. However I shall have
a close look at the way you made gtk and react coexist. That looks
promising.


>
>
> PS: for layout management, I found that using the ~packing option when
> creating the widgets is usually much better than #add'ing the afterwards.
>

I was hoping to build APIs with this kind of formulation:

let window = panel (hpack [
  picture "picture.bmp" ;
  vpack [
    button "Ok" (fun () -> set_some_event true) ;
    button "Cancel" (fun () -> set_some_event false)
  ]
])

that is avoiding a "create and configure" style. If I can't get that, I'll
probably stick with a traditional, imperative approach.

Thanks for your feedback !
pH.



>
>
> Hope this helps,
> Adrien Nader
>

[-- Attachment #2: Type: text/html, Size: 7644 bytes --]

Re: [Caml-list] Functional GUI programming: looking for good practices

[Daniel Bünzli]

Le mardi, 14 février 2012 à 11:02, Philippe Veber a écrit :

> In other words, am I allowed to call a primitive in a lifted function?
No. This is documented here [1]. One way to side-step the issue is to put these updates in a queue and execute them after the update cycle ended.  

Best,

Daniel

[1] http://erratique.ch/software/react/doc/React#update

Re: [Caml-list] Functional GUI programming: looking for good practices

[Philippe Veber]

[-- Attachment #1: Type: text/plain, Size: 1087 bytes --]

Hi Daniel,

2012/2/14 Daniel Bünzli <daniel.buenzli@erratique.ch>

> Le mardi, 14 février 2012 à 11:02, Philippe Veber a écrit :
>
> > In other words, am I allowed to call a primitive in a lifted function?
> No. This is documented here [1].

Well I did read the paragraph, but thought the described limitation might
be about updating a signal from different threads. Of course it makes sense
that even in a single-threaded code, an update cycle should not be
interrupted by another if there are shared dependencies. May I nevertheless
humbly suggest to add the "not calling a primitive inside a lifted
function" rule more explicitely, as it is a tempting sin for a beginner?


> One way to side-step the issue is to put these updates in a queue and
> execute them after the update cycle ended.
>
 That seems a good work-around, to gather all side-effects in one place,
namely a (unit -> unit) Queue.t where to defer all calls to the primitives.
That do help, thanks !
ph.


> Best,
>
> Daniel
>
> [1] http://erratique.ch/software/react/doc/React#update
>

[-- Attachment #2: Type: text/html, Size: 1760 bytes --]

Re: [Caml-list] Functional GUI programming: looking for good practices

[Adrien]

On 14/02/2012, Philippe Veber <philippe.veber@gmail.com> wrote:
> Hi Daniel,
>
> 2012/2/14 Daniel Bünzli <daniel.buenzli@erratique.ch>
>
>> Le mardi, 14 février 2012 à 11:02, Philippe Veber a écrit :
>>
>> > In other words, am I allowed to call a primitive in a lifted function?
>> No. This is documented here [1].
>
> Well I did read the paragraph, but thought the described limitation might
> be about updating a signal from different threads. Of course it makes sense
> that even in a single-threaded code, an update cycle should not be
> interrupted by another if there are shared dependencies. May I nevertheless
> humbly suggest to add the "not calling a primitive inside a lifted
> function" rule more explicitely, as it is a tempting sin for a beginner?

Quite often, doing so does not make a lot of sense.
Suppose that you have a signal A. An external event triggers an update which
will create A1. But during that update, you trigger another update which
will create A2. Now, how does your signal change? Is A2 going to be the next
value of A or is will it be A1?

A simple (and probably not proper) implementation of FRP that I've done
simply used A2 and then A1: the additional work that had been done for A2
was being dropped.

Some implementations or approaches will probably solve that one way or
another but that's not my point: when you do that, the code becomes
non-obvious.

>> One way to side-step the issue is to put these updates in a queue and
>> execute them after the update cycle ended.
>>
>  That seems a good work-around, to gather all side-effects in one place,
> namely a (unit -> unit) Queue.t where to defer all calls to the primitives.
> That do help, thanks !
> ph.

I've had to use that until I changed the order of some operations in my
code. I might have managed to break a bigger cycle into two smaller and
independant cycles but I need to check, document, triple-check, ... what
I've done.

Anyway, I quickly benchmarked "GLib.Timeout.Add ~ms cb" from gtk/lablgtk2
which calls cb after at least ms milliseconds from the mainloop. Its latency
was around 122µs for ms = 0 in my case (obviously, the program was otherwise
idle).

I think you could also use lwt here but I haven't done anything with it.


Btw, I think it's worthwhile to mention some issues that can create such a
need. In my case of a web browser, web pages have javascript and the
javascript can create a new page and get a corresponding object.
  var popup = window.open (); // or something like that

In webkit-gtk (and probably all other webkit ports), this triggers a GTK
signal to which you can attach a callback which returns a WebView object.
However, my code does not allow a webpage to create a new page: all it can
do is request that one gets created by the Zipper which holds the pages.

So the page changes its state and stores a request for a pop-up from the
callback. The request is seen by the Zipper which creates a new page in the
UI side of my code. Once the page is ready, it gets added to the Zipper and
a message is sent to the parent page. All that has been done in the
callback. It now works with my "MiniFRP" implementation but it definitely
had the A/A1/A2 issue I mentionned above.

If you put the updates in a queue and delay them, you're probably not going
to solve this issue which turns out to be pretty simple. With the callback,
you've started an update cycle and you cannot finish that update cycle
before a new page, and therefore, another update cycle, has taken place.

The API of webkit-gtk could allow to get an ID for the popup request, exit
the callback with only the promise the answer will be given at some point,
and do the actual work outside of the callback. But it doesn't. And there
will always be APIs like that and we need to find how to handle this.


Regards,
Adrien Nader

Re: [Caml-list] Functional GUI programming: looking for good practices

[Daniel Bünzli]

Le mardi, 14 février 2012 à 12:52, Adrien a écrit :

> Quite often, doing so does not make a lot of sense.
> Suppose that you have a signal A. An external event triggers an update which
> will create A1. But during that update, you trigger another update which
> will create A2. Now, how does your signal change? Is A2 going to be the next
> value of A or is will it be A1?

Yes it doesn't make any sense at all.  

An update cycle is made under a synchrony hypothesis, everything therein happens simultaneously, at the same time t. The semantics of a signal (resp. event) [1] is a *function* from time to a value (resp. optional value). There's no way a signal (resp. event) can have two different values for the same time t, this should raise a red flag. Besides by definition two primitive events cannot (should not, since the system cannot enforce it) occur at the same time t [2].

When in doubt, it helps to get back to the semantics to see if your interfacing mechanic to react actually respects FRP's semantics. And if it doesn't and you can't fix it, I wouldn't try to use react for the problem.  

Best,

Daniel

P.S. to Philipe
I'm not exactly sure what you problem is. But if you are trying to define a signal that somehow depends on itself, make sure you did understand the {E,S}.fix combinators [3]. I'm sure I had to use it once or twice to define the (very primitive) UI of the breakout.ml example from the distribution.  

[1] http://erratique.ch/software/react/doc/React#sem
[2] http://erratique.ch/software/react/doc/React#simultaneity
[3] http://erratique.ch/software/react/doc/React#recursion

Re: [Caml-list] Functional GUI programming: looking for good practices

[Adrien]

On 14/02/2012, Philippe Veber <philippe.veber@gmail.com> wrote:
> 2012/2/13 Adrien <camaradetux@gmail.com>
>> I've created a lablgtk branch named "adrien/react" to get react signals
>> out
>> of gtk properties and react events out of gtk signals (they match quite
>> well). Support isn't perfect but enough to test and experiment.
>>
>> The issue was to write the application itself: it was way too complicated
>> and it involved many many react values which had to be somehow kept alive.
>> It was also not very useful. The reason was that the very first thing I
>> would do with all the events was React.E.select: I would create distinct
>> signals only to merge them!
>>
>> What I've started doing instead is to have one "object" with a
>> corresponding
>> event: callbacks only send a message to that event and look like "send
>> (`Foo
>> bar)" and from then I can use match over the message in a different
>> location
>> in the code, typically in an FRP context.
>>
> I did that too, and it does seem a good practice: a central bus for
> information that is relevant to the whole application. [snip]

There is one clear benefit: less links.

When you have n elements that can trigger y actions, you need to setup (n*y)
links. If you go through a "central bus", you will only need (n+y) links.

Moreover, the callback code is often going to be very simple and you will be
able to touch the UI code less.

>> (* Warning: this is about work in-progress; it might not always make
>> sense,
>> might have some weird things and I might not be able to explain everything
>> properly. *)
>>
>> My current application is a web browser which I want to make much more
>> intelligent than the browsers available today.
>
> Just curious: in what way?

I want to do everything but make a browser that is a mere collection of
tabs without any relation between them. I consider a web browser to be a
tool and I want to improve my own tools.

Currently, if you don't use a hundred firefox add-ons, you get:
- linear history (if you use vim, try ':h undo-tree)
- linear tabs instead of a tree on screen and no way or a poor way to group
  pages together (including in the bookmarks)
- stupid popup/ad blockers because a popup opened by a popup is treated like
  a page opened on purpose by the user
- no way to recreate your browsing session: that you moved from page A to
  page B to page C, went back to page B, moved to page D, back to B and then
  to C again; ever browsed through your history looking for one element, and
  finding one which you had visited right before but still couldn't get a
  hold on the one you wanted?

There's a lot of semantic going on in browsing these days and I want take
advantage of it to improve browsing, especially in these horrible days of UI
simplification, which I'm having troubles using and which I believe are
dumbing down everything, moving everything away from the client and only to
facebook and twitter while making all the other non-stupid uses harder.


Of course, there's more to it: not throwing everything in an sqlite
database, hoping it'll be good for all uses of everything, including
full-text search. Faster everything. Configuration settings (chrom* has
almost none and firefox has a lump of completely undocumented crap).


And I want OCaml scripting inside the browser. ;-)


>> For this reason, I store
>> web pages in a data structure which is of course purely functional. I can
>> have several layers of data structures containing objects in the same way.
>>
>> My objects start with a default state and evolve (in a functional way)
>> through a fold according to the messages they receive. Each time the state
>> changes, two sets of callbacks are triggered: first, to change the UI;
>> second, to update the data structure containing the object which is needed
>> because of functional updates. I also use that last mechanism to propagate
>> messages from the inner objects to the outter ones.
>>
>> One last characteristic is that I have a UI side besides the functional
>> one.
>> It contains a handful of things which are needed to work with GTK.I also
>> use
>> it to propagate messages from the outter objects to the inner ones.
>>
>> This is work-in-progress and some details could be improved but I think
>> that
>> the big picture is there. As far as I can tell, the UI and functional
>> sides
>> are properly separated, constraints aren't heavy and I seem to be able to
>> get the usual qualities of ocaml in GUIs.
>>
> What module would you recommend reading to get a taste of it?

I have the "lablgtk-react" project but it's not well documented (partly
because it has changed a lot and documentation hasn't been updated).

Actually, it has changed so much that the "lablgtk" part has disappeared,
and react, which was not needed since it's using a functor for any FRP
implementation, is now not working anymore (I'll be fixing it soonish).

The best example is maybe caravel itself (the browser I'm doing):
  http://git.ocamlcore.org/cgi-bin/gitweb.cgi?p=caravel/caravel.git;a=tree;f=src/browser

The model/ folder has, huh, models which are purely functionals, and the ui/
folder has the UI code. At the top of the ui/page.ml, you can see:

  module Page = LablgtkReact.Core (MiniFRP) (BrowserModel.Page)

and a bit below:

  let core = new Page.core ~state_callbacks:[propagator] $foo in

That's because it's the UI which is the source of the messages.

The UI send message to "core" which contains an FRP signal of a data
structure. This data structure then changes according to the signal and both
the UI side and the parent data structure get a message about that.

The UI receives a message in through "core#msg_callbacks" with self#sink
(there is usually little work to do there).

The parent data structure gets a message through "core#state_callbacks" with
the propagator argument in the code above. In my case, this is a `Set
message which is handled like all the messages which are sent to the model
(i.e. through the on_action_func function, here
model/Navigation.on_action_func).

I need to make 2 or 3 diagrams to show the messaging that takes place.

>> PS: for layout management, I found that using the ~packing option when
>> creating the widgets is usually much better than #add'ing the afterwards.
>>
>
> I was hoping to build APIs with this kind of formulation:
>
> let window = panel (hpack [
>   picture "picture.bmp" ;
>   vpack [
>     button "Ok" (fun () -> set_some_event true) ;
>     button "Cancel" (fun () -> set_some_event false)
>   ]
> ])
>
> that is avoiding a "create and configure" style. If I can't get that, I'll
> probably stick with a traditional, imperative approach.

What about this?

  let window = GWindow.window () in
  let vbox = GPack.vbox ~packing:window#add () in
  let picture = foo "picture.bmp" ~packing:vbox#pack () in
  let hbox = GPack.hbox ~packing:vbox#pack ()in
  let ok = gbutton.button ~pack:hbox#pack () in
  let cancel = gbutton.button ~pack:hbox#pack () in
  ignore (ok#connect#clicked (fun () -> set_some_event true));
  ignore (cancel#connect#clicked (fun () -> set_some_event false))

It's a bit longer but not more difficult and it avoids a number of issues
you have in your code:
- typing issues: you can't store a button and a box in the same list: you
  have to use variants which make handling a bit heavier
- no easy way to access widgets (no variable name) with their "full" types
- very simple API
- not much freedom

That can work quite well depending on what you do. If your UI only ever
generates messages but never has to receive any, it should be good. As far
as I remember, there is support for such code in Lablgtk-extras (on the
ocaml forge).

It's quite nice to have something purely declarative and for which the code
can be formatted to look like the UI but I don't see this anymore as the
biggest issue for GUI programming and have therefore not spent much on that
recently.


Regards,
Adrien Nader

Re: [Caml-list] Functional GUI programming: looking for good practices

[Raoul Duke]

On Mon, Feb 13, 2012 at 3:01 AM, Philippe Veber
<philippe.veber@gmail.com> wrote:
> than expected, for example with layout management. In order to compute a
> layout for the widgets, some information has to travel bottom up the widget
> hierarchy, and some goes top down. While there is a well-founded order for

academic thoughts from others:

http://lambda-the-ultimate.org/node/2913

Re: [Caml-list] Functional GUI programming: looking for good practices

[Anthony Tavener]

[-- Attachment #1: Type: text/plain, Size: 2993 bytes --]

I've been taking a stab at this too; for a video game. The approach I'm
taking is with delimited continuations (using Oleg's delimcc). This allows
me to run UI code as a coroutine. The UI code then looks like any other
code, where user inputs are results from a function. Really, the UI code
stops, waiting for input (while the mainline coroutine runs), then returns
the input value.

An example:

let act = user (menu_of_list
  [ ("Equip",`Equip);
    ("Spell",`Spell);
    ("End",`End) ] ) in

match act with
| Some `Spell ->
    try
      let spellList = (* build a list of casting options for this character
*) in
      let spell = ?.(user (menu_of_list spellList)) in
      let target = ?.(user (select (target_filter spell) scene)) in
      Cast.cast id spell target
    with OptionNone -> ()

This example creates a small menu of special options with the given string
labels "Equip", etc. The "user" function is a yield of the UI coroutine,
which means we await resuming with the input value. Interacting with GUI
elements is handled by querying the scene or OpenGL state based on SDL
input events. In this case clicking on the created "Spell" menu item will
return a value of Some `Spell.

Later in the code, more user interactions are handled for providing
specific spell options and then targets. (The "?." prefix operator is just
turning None values into exceptions, to implement user abort.)

I have a yield function, using Delimcc's shift and abort:

let yield level fn =
  shift level (fun k ->
    fn k;
    abort level () )

which is the basis of the 'user' function in the UI:

  let user fn = yield ui_process fn

So far, I'm not sure how well this works out for a complete project. I like
it so far, but I have complexity growing in some "update loop" stuff, which
are little closures I add to be run each frame-update for reacting to
mouse-over/hover.

I've tried dabbling with FRP, but it kept getting hairy too. I will
probably use it to replace my kludge of "update loop" closures. Someday,
someone will figure out something slick, whether it's a way to use these
tools, or something new. I'm at least happy not using "signals and slots"!

Good luck in the hunt for elegant UI (code)!

 Tony

Inside the UI code, the 'process' delimits the UI coroutine
On Mon, Feb 13, 2012 at 11:13 AM, Raoul Duke <raould@gmail.com> wrote:

> On Mon, Feb 13, 2012 at 3:01 AM, Philippe Veber
> <philippe.veber@gmail.com> wrote:
> > than expected, for example with layout management. In order to compute a
> > layout for the widgets, some information has to travel bottom up the
> widget
> > hierarchy, and some goes top down. While there is a well-founded order
> for
>
> academic thoughts from others:
>
> http://lambda-the-ultimate.org/node/2913
>
> --
> Caml-list mailing list.  Subscription management and archives:
> https://sympa-roc.inria.fr/wws/info/caml-list
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs
>
>

[-- Attachment #2: Type: text/html, Size: 4334 bytes --]

Re: [Caml-list] Functional GUI programming: looking for good practices

[Philippe Veber]

[-- Attachment #1: Type: text/plain, Size: 617 bytes --]

Hi Anthony,

This looks interesting, however as I'm not very familiar with delimcc
(that's a shame, I admit), I fail to understand the flow of the program.
Would you mind giving a snippet of the update loop you mentionned?


> So far, I'm not sure how well this works out for a complete project. I
> like it so far, but I have complexity growing in some "update loop" stuff,
> which are little closures I add to be run each frame-update for reacting to
> mouse-over/hover.
>
>

Good luck in the hunt for elegant UI (code)!
>

Let's hope I'm not just Dahu hunting!
(http://en.wikipedia.org/wiki/Dahu)



>
>  Tony
>
>

[-- Attachment #2: Type: text/html, Size: 1302 bytes --]

Re: [Caml-list] Functional GUI programming: looking for good practices

[Anthony Tavener]

[-- Attachment #1: Type: text/plain, Size: 5188 bytes --]

Apologies Philippe, this is a bit long...

The "update loop" I mentioned might be a bit of a red-herring, as I'm only
using that for continuously active UI elements: such as changing cursor to
represent the action which would be taken on click. It has nothing to do
with the basic UI flow.

I didn't understand delimcc right away, and I hardly understand it now! :)

I was looking to write UI code much as your example of packing buttons
together with directly bound activation functions.

Here's my "menu_of_list", which takes a list of string * 'a pairs, calling
GUI code to make a menu using the strings as labels, and using the 'a
values as return values...

let menu_of_list lst return =
  (* snipped: preamble which handles positioning, stacking order, getting
font, etc *)
  Gui.add_widget gui
    (new Gui.vbox pos stacking spacing
      (lst |> List.map
        (fun (label,value) ->
          Gui.wit_fn gui
            (new Gui.rectangle_text w h fnt label)
            (fun () -> return (Some value) ) )))

The important part here is the "return" function. This will resume the UI
coroutine. It is given to "menu_of_list" by the UI code, then when the GUI
has an activation of a menu button this "return" is called... resuming the
UI where it left off, and with the "Some value" which was associated to the
button.

The magic of how this works is in delimcc capturing portions of the run
stack. Here I've extracted the relevant bits of code:


(* this runs as a coroutine with the "mainline" *)
let ui ui_process () =

  (* in actual code, this menu comes after a "right click", for special
actions *)
  let act = yield ui_process (menu_of_list coord
    [ ("Equip",`Equip);
      ("Spell",`Spell);
      ("End",`End) ] ) in
  (* ... handle whatever action the user chose! *)


(* given SDL events, calls activation functions of 'hit' widgets *)
let gui_react gui events =
  let hits = gui_select gui events in
  match hits with
  | h::t -> begin
    match get_binding gui h with
    | Some f -> f ()  (* runs activation function of widget, such as
"return" to coroutine! *)
    | None -> ()
    end
  | [] -> ()

let _ =
  (* A prompt is delimcc's mechanism for marking the stack to -limit- the
   * continuation, rather than creating whole-program continuations. *)
  (* So here, the "ui" is initially run, and will *resume* this mainline
continuation
   * at the end of the "user" function call. *)
  let ui_prompt = new_prompt () in
  push_prompt ui_prompt (ui ui_prompt);

  ...

  (* in mainloop *)
    gui_react gui events;

----------------------

Now I'm going to restate the yield function here, and try to explain...

let yield level fn =
  shift level (fun k ->
    fn k;
    abort level () )

"shift" and "abort" are delimcc. This runs the given function "fn" with a
continuation 'k'... so this continuation is the "return" function passed to
menu_of_list, and therefore bound to each menu-button. The abort exits the
"ui coroutine", resuming the mainline, hence the name: yield.

The continuation 'k' comes from the shift... the continuation is the code
"outside" of the shift call... so when it's called (by the mainline's
gui_react), it resume at the end of 'yield' and keep on going... in this
example, handling the returned action!

I hope this conveys at least the gist of what's going on... I read a lot of
papers over-and-over, not understanding... although none were specifically
GUI. Delimited continuations have numerous applications and a surprising
number of configurations for just a pair of functions! (Abort is really a
special case of "reset"... so it's shift and reset, in ump-teen
configurations.)

I'll try to explain if you have any further questions! However, I'm still
trying to sort out how best to write my GUI code -- there is a lot of room
for improvement. :)

 -Tony

PS. I'd like to blame X. Leroy ;)... for a post some 10 years ago replying
to someone's attempt to do a GUI in OCaml... Xavier casually replied
something like "oh, you can use coroutines". That was a wild goose chase
(or Dahu?)! Delimcc didn't exist at the time, and it's native-code
implementation came about just a few years ago (thank-you Oleg!). Even
then, I had no idea how I was supposed to use a coroutine to write GUI
code! Oh well, it was an adventure, and I still don't know if this is a
good thing, but I like it a lot more than "signals and slots" -- the usual
scattered GUI code which is connected by messages/events.


On Tue, Feb 14, 2012 at 3:17 AM, Philippe Veber <philippe.veber@gmail.com>wrote:

> Hi Anthony,
>
> This looks interesting, however as I'm not very familiar with delimcc
> (that's a shame, I admit), I fail to understand the flow of the program.
> Would you mind giving a snippet of the update loop you mentionned?
>
>
>> So far, I'm not sure how well this works out for a complete project. I
>> like it so far, but I have complexity growing in some "update loop" stuff,
>> which are little closures I add to be run each frame-update for reacting to
>> mouse-over/hover.
>>
>>
>
> Good luck in the hunt for elegant UI (code)!
>>
>
> Let's hope I'm not just Dahu hunting!
> (http://en.wikipedia.org/wiki/Dahu)
>
>
>
>>
>>  Tony
>>
>>
>

[-- Attachment #2: Type: text/html, Size: 7460 bytes --]

Re: [Caml-list] Functional GUI programming: looking for good practices

[Anthony Tavener]

[-- Attachment #1: Type: text/plain, Size: 6509 bytes --]

Hrm... when I re-read my prior post before sending, it made sense. Several
hours later it seems inadequate, and I've thought of something to say more
clearly...

The execution of code bounces between the UI and the mainline. When the
mainline processes a "gui hit" it will resume the UI code *right where it
left off (yielded)*... then the UI will do something and get to another
point where it yields, awaiting input and thereby resuming the mainline
where it was (back to processing gui hits).

Why would I want this? So that I don't have stateful UI code which has to
trickle down to the right place all the time. The UI code is clearer
because it doesn't need to be re-entered from the top each frame. This
suits the declarative style of GUI specification, like the example you
gave, since we don't need special state or messages to communicate what the
GUI is doing. GUI state becomes a property of the... well, the stack. It's
the current scope of the code.

Hope this helps!

 -Tony


On Tue, Feb 14, 2012 at 11:02 AM, Anthony Tavener <anthony.tavener@gmail.com
> wrote:

> Apologies Philippe, this is a bit long...
>
> The "update loop" I mentioned might be a bit of a red-herring, as I'm only
> using that for continuously active UI elements: such as changing cursor to
> represent the action which would be taken on click. It has nothing to do
> with the basic UI flow.
>
> I didn't understand delimcc right away, and I hardly understand it now! :)
>
> I was looking to write UI code much as your example of packing buttons
> together with directly bound activation functions.
>
> Here's my "menu_of_list", which takes a list of string * 'a pairs, calling
> GUI code to make a menu using the strings as labels, and using the 'a
> values as return values...
>
> let menu_of_list lst return =
>   (* snipped: preamble which handles positioning, stacking order, getting
> font, etc *)
>   Gui.add_widget gui
>     (new Gui.vbox pos stacking spacing
>       (lst |> List.map
>         (fun (label,value) ->
>           Gui.wit_fn gui
>             (new Gui.rectangle_text w h fnt label)
>             (fun () -> return (Some value) ) )))
>
> The important part here is the "return" function. This will resume the UI
> coroutine. It is given to "menu_of_list" by the UI code, then when the GUI
> has an activation of a menu button this "return" is called... resuming the
> UI where it left off, and with the "Some value" which was associated to the
> button.
>
> The magic of how this works is in delimcc capturing portions of the run
> stack. Here I've extracted the relevant bits of code:
>
>
> (* this runs as a coroutine with the "mainline" *)
> let ui ui_process () =
>
>   (* in actual code, this menu comes after a "right click", for special
> actions *)
>   let act = yield ui_process (menu_of_list coord
>     [ ("Equip",`Equip);
>       ("Spell",`Spell);
>       ("End",`End) ] ) in
>   (* ... handle whatever action the user chose! *)
>
>
> (* given SDL events, calls activation functions of 'hit' widgets *)
> let gui_react gui events =
>   let hits = gui_select gui events in
>   match hits with
>   | h::t -> begin
>     match get_binding gui h with
>     | Some f -> f ()  (* runs activation function of widget, such as
> "return" to coroutine! *)
>     | None -> ()
>     end
>   | [] -> ()
>
> let _ =
>   (* A prompt is delimcc's mechanism for marking the stack to -limit- the
>    * continuation, rather than creating whole-program continuations. *)
>   (* So here, the "ui" is initially run, and will *resume* this mainline
> continuation
>    * at the end of the "user" function call. *)
>   let ui_prompt = new_prompt () in
>   push_prompt ui_prompt (ui ui_prompt);
>
>   ...
>
>   (* in mainloop *)
>     gui_react gui events;
>
> ----------------------
>
> Now I'm going to restate the yield function here, and try to explain...
>
> let yield level fn =
>   shift level (fun k ->
>     fn k;
>     abort level () )
>
> "shift" and "abort" are delimcc. This runs the given function "fn" with a
> continuation 'k'... so this continuation is the "return" function passed to
> menu_of_list, and therefore bound to each menu-button. The abort exits the
> "ui coroutine", resuming the mainline, hence the name: yield.
>
> The continuation 'k' comes from the shift... the continuation is the code
> "outside" of the shift call... so when it's called (by the mainline's
> gui_react), it resume at the end of 'yield' and keep on going... in this
> example, handling the returned action!
>
> I hope this conveys at least the gist of what's going on... I read a lot
> of papers over-and-over, not understanding... although none were
> specifically GUI. Delimited continuations have numerous applications and a
> surprising number of configurations for just a pair of functions! (Abort is
> really a special case of "reset"... so it's shift and reset, in ump-teen
> configurations.)
>
> I'll try to explain if you have any further questions! However, I'm still
> trying to sort out how best to write my GUI code -- there is a lot of room
> for improvement. :)
>
>  -Tony
>
> PS. I'd like to blame X. Leroy ;)... for a post some 10 years ago replying
> to someone's attempt to do a GUI in OCaml... Xavier casually replied
> something like "oh, you can use coroutines". That was a wild goose chase
> (or Dahu?)! Delimcc didn't exist at the time, and it's native-code
> implementation came about just a few years ago (thank-you Oleg!). Even
> then, I had no idea how I was supposed to use a coroutine to write GUI
> code! Oh well, it was an adventure, and I still don't know if this is a
> good thing, but I like it a lot more than "signals and slots" -- the usual
> scattered GUI code which is connected by messages/events.
>
>
> On Tue, Feb 14, 2012 at 3:17 AM, Philippe Veber <philippe.veber@gmail.com>wrote:
>
>> Hi Anthony,
>>
>> This looks interesting, however as I'm not very familiar with delimcc
>> (that's a shame, I admit), I fail to understand the flow of the program.
>> Would you mind giving a snippet of the update loop you mentionned?
>>
>>
>>> So far, I'm not sure how well this works out for a complete project. I
>>> like it so far, but I have complexity growing in some "update loop" stuff,
>>> which are little closures I add to be run each frame-update for reacting to
>>> mouse-over/hover.
>>>
>>>
>>
>> Good luck in the hunt for elegant UI (code)!
>>>
>>
>> Let's hope I'm not just Dahu hunting!
>> (http://en.wikipedia.org/wiki/Dahu)
>>
>>
>>
>>>
>>>  Tony
>>>
>>>
>>
>

[-- Attachment #2: Type: text/html, Size: 9057 bytes --]

Re: [Caml-list] Functional GUI programming: looking for good practices

[Philippe Veber]

[-- Attachment #1: Type: text/plain, Size: 6891 bytes --]

Thank you very much Anthony and Adrien, your comments helped me a lot, I
now have plenty of material to think on!
Best,
  ph.

2012/2/15 Anthony Tavener <anthony.tavener@gmail.com>

> Hrm... when I re-read my prior post before sending, it made sense. Several
> hours later it seems inadequate, and I've thought of something to say more
> clearly...
>
> The execution of code bounces between the UI and the mainline. When the
> mainline processes a "gui hit" it will resume the UI code *right where it
> left off (yielded)*... then the UI will do something and get to another
> point where it yields, awaiting input and thereby resuming the mainline
> where it was (back to processing gui hits).
>
> Why would I want this? So that I don't have stateful UI code which has to
> trickle down to the right place all the time. The UI code is clearer
> because it doesn't need to be re-entered from the top each frame. This
> suits the declarative style of GUI specification, like the example you
> gave, since we don't need special state or messages to communicate what the
> GUI is doing. GUI state becomes a property of the... well, the stack. It's
> the current scope of the code.
>
> Hope this helps!
>
>  -Tony
>
>
> On Tue, Feb 14, 2012 at 11:02 AM, Anthony Tavener <
> anthony.tavener@gmail.com> wrote:
>
>> Apologies Philippe, this is a bit long...
>>
>> The "update loop" I mentioned might be a bit of a red-herring, as I'm
>> only using that for continuously active UI elements: such as changing
>> cursor to represent the action which would be taken on click. It has
>> nothing to do with the basic UI flow.
>>
>> I didn't understand delimcc right away, and I hardly understand it now! :)
>>
>> I was looking to write UI code much as your example of packing buttons
>> together with directly bound activation functions.
>>
>> Here's my "menu_of_list", which takes a list of string * 'a pairs,
>> calling GUI code to make a menu using the strings as labels, and using the
>> 'a values as return values...
>>
>> let menu_of_list lst return =
>>   (* snipped: preamble which handles positioning, stacking order, getting
>> font, etc *)
>>   Gui.add_widget gui
>>     (new Gui.vbox pos stacking spacing
>>       (lst |> List.map
>>         (fun (label,value) ->
>>           Gui.wit_fn gui
>>             (new Gui.rectangle_text w h fnt label)
>>             (fun () -> return (Some value) ) )))
>>
>> The important part here is the "return" function. This will resume the UI
>> coroutine. It is given to "menu_of_list" by the UI code, then when the GUI
>> has an activation of a menu button this "return" is called... resuming the
>> UI where it left off, and with the "Some value" which was associated to the
>> button.
>>
>> The magic of how this works is in delimcc capturing portions of the run
>> stack. Here I've extracted the relevant bits of code:
>>
>>
>> (* this runs as a coroutine with the "mainline" *)
>> let ui ui_process () =
>>
>>   (* in actual code, this menu comes after a "right click", for special
>> actions *)
>>   let act = yield ui_process (menu_of_list coord
>>     [ ("Equip",`Equip);
>>       ("Spell",`Spell);
>>       ("End",`End) ] ) in
>>   (* ... handle whatever action the user chose! *)
>>
>>
>> (* given SDL events, calls activation functions of 'hit' widgets *)
>> let gui_react gui events =
>>   let hits = gui_select gui events in
>>   match hits with
>>   | h::t -> begin
>>     match get_binding gui h with
>>     | Some f -> f ()  (* runs activation function of widget, such as
>> "return" to coroutine! *)
>>     | None -> ()
>>     end
>>   | [] -> ()
>>
>> let _ =
>>   (* A prompt is delimcc's mechanism for marking the stack to -limit- the
>>    * continuation, rather than creating whole-program continuations. *)
>>   (* So here, the "ui" is initially run, and will *resume* this mainline
>> continuation
>>    * at the end of the "user" function call. *)
>>   let ui_prompt = new_prompt () in
>>   push_prompt ui_prompt (ui ui_prompt);
>>
>>   ...
>>
>>   (* in mainloop *)
>>     gui_react gui events;
>>
>> ----------------------
>>
>> Now I'm going to restate the yield function here, and try to explain...
>>
>> let yield level fn =
>>   shift level (fun k ->
>>     fn k;
>>     abort level () )
>>
>> "shift" and "abort" are delimcc. This runs the given function "fn" with a
>> continuation 'k'... so this continuation is the "return" function passed to
>> menu_of_list, and therefore bound to each menu-button. The abort exits the
>> "ui coroutine", resuming the mainline, hence the name: yield.
>>
>> The continuation 'k' comes from the shift... the continuation is the code
>> "outside" of the shift call... so when it's called (by the mainline's
>> gui_react), it resume at the end of 'yield' and keep on going... in this
>> example, handling the returned action!
>>
>> I hope this conveys at least the gist of what's going on... I read a lot
>> of papers over-and-over, not understanding... although none were
>> specifically GUI. Delimited continuations have numerous applications and a
>> surprising number of configurations for just a pair of functions! (Abort is
>> really a special case of "reset"... so it's shift and reset, in ump-teen
>> configurations.)
>>
>> I'll try to explain if you have any further questions! However, I'm still
>> trying to sort out how best to write my GUI code -- there is a lot of room
>> for improvement. :)
>>
>>  -Tony
>>
>> PS. I'd like to blame X. Leroy ;)... for a post some 10 years ago
>> replying to someone's attempt to do a GUI in OCaml... Xavier casually
>> replied something like "oh, you can use coroutines". That was a wild goose
>> chase (or Dahu?)! Delimcc didn't exist at the time, and it's native-code
>> implementation came about just a few years ago (thank-you Oleg!). Even
>> then, I had no idea how I was supposed to use a coroutine to write GUI
>> code! Oh well, it was an adventure, and I still don't know if this is a
>> good thing, but I like it a lot more than "signals and slots" -- the usual
>> scattered GUI code which is connected by messages/events.
>>
>>
>> On Tue, Feb 14, 2012 at 3:17 AM, Philippe Veber <philippe.veber@gmail.com
>> > wrote:
>>
>>> Hi Anthony,
>>>
>>> This looks interesting, however as I'm not very familiar with delimcc
>>> (that's a shame, I admit), I fail to understand the flow of the program.
>>> Would you mind giving a snippet of the update loop you mentionned?
>>>
>>>
>>>> So far, I'm not sure how well this works out for a complete project. I
>>>> like it so far, but I have complexity growing in some "update loop" stuff,
>>>> which are little closures I add to be run each frame-update for reacting to
>>>> mouse-over/hover.
>>>>
>>>>
>>>
>>> Good luck in the hunt for elegant UI (code)!
>>>>
>>>
>>> Let's hope I'm not just Dahu hunting!
>>> (http://en.wikipedia.org/wiki/Dahu)
>>>
>>>
>>>
>>>>
>>>>  Tony
>>>>
>>>>
>>>
>>
>

[-- Attachment #2: Type: text/html, Size: 9624 bytes --]

Re: [Caml-list] Functional GUI programming: looking for good practices

[Adrien]

Hi,

I've made a kind of "type diagram" for caravel in an attempt to document
it and explain the approach (see the post-scriptum on the "how"). The
_main_ reason was that I was often lost in some code/behaviour paths.

  http://notk.org/~adrien/ocaml/caravel/t13.png

  (NB: graphviz wouldn't let me write "Page.t" so I used commas instead
  of dots in the labels; similarly, I used brackets instead of parens)

This graph is far from perfect and I have other documentation efforts to
do.

It shows two modules in caravel: Page and Navigation. Page is a browser
tab: it mostly includes the webpage and the address bar but also has to
track the history for instance. Navigation is a set of Page elements.

As I mentionned earlier, the components are exchanging messages
(directly or through callbacks on each state change) and this graph
shows the polymorphic variants which are used for messages, not objects.

The diagrams are not always perfectly readable but only have a few
rules: the first line of blocks is the type name and after that, the
fields are listed in declaration order, possibly with "sub" polymorphic
variants. For instance, for Page.action_functional, the type declaration
is:

  type action_functional = [
    | action_common
    | `Request of [ request | Download of string ]
  ]

Caravel's global architecture is close to the following diagram (I'm
using module packs named "UI" and "Model"):

(* you _really_ want a monospace font to display this *)

< USER > |  < UI (object) >  |  < MODEL (functional/object) >

                ----------------------------
                |                          |
                v (A)                      ^ (9)            ^
           UI.Navigation  >---   Model.Navigation       Navigation
                             |             ^ (8)            v
                             ------------->|
                                           |
                                           |
                                      (5)  ^ (7)
                --------------------------<|
                |                          |
                v (6)                      ^ (4)            ^
User ---> (1) UI.Page (2) >---       Model.Page           Page
                             |             ^ (3)            v
                             ------------->|

Steps are numbered from 1 to A (hex: it saved space on the graph).

(1) The user does an action which is caught with a GTK callback in an
    ocaml object which holds the various UI elements for a page.

(2) The callback send()'s a message to its model. The messages are of
    type "Model.Page.action_functional".
(3) The model has an "on_action_functional" function through which all
    the messages go. This function is called by an FRP fold function
    with two arguments: the message and a Model.Page.t which is model
    for the page (with React, it'd give Model.Page.t React.signal).
(4) The "on_action_functional" returns two values: a new value for the
    model and a message of type "Model.Page.action_ui".

(5) The message of type action_ui is first sent to the UI object.
(6) The UI object handles the messages with a "sink" function and does
    whatever the model told it to do (the model decides, the ui obeys).

(7) The messages are also sent to the "parent" data structure (no magic
    and no implicit: it's a programmer-specified callback). The main
    reason for this is that the elements in the set are all functional
    and we have to tell the set that a new version of one element is
    available (this is done with Navigation.action_functional's `Set).
(8,9,A) The same thing happens as for Page.


I'm probably wrong on some points and I also lied a bit (notably, steps
7 to A happens before steps 5 and 6 because of the issue that Daniel
commented on: it's been working for me but I don't know whether the
approach is always valid).


My current big issue is that I'm getting lost in the flow of types. For
instance, when the webpage wants to download a file, it sends a message
"`Request (`Download uri)" to the model (step (2) in the diagram above).
Then the model returns a message "`Request (`Download (uri, referer))"
(step (4)). This message is seen by Navigation (step (8)) and forwarded
up again with the same process (which is basically the action of setting
a flag).

More generally, because the model will often add information to the
messages it receives, I have different versions of the same type:
`Download of string, and `Download of (string * string); `Close of id,
and `Close of (id * int option), ...

I'm wondering how I could have fewer types. I could remove the type with
the least number of arguments: have only `Download of (string * string
option) and always use None for the string option type in the UI and
always Some in the model. That wouldn't work in all cases and typing
would be a (tiny) bit weaker however.

Something else I'd like to have is stricter types: currently the flow of
types between components is `Download of x -> `Download of y ->
`Download of z, but nothing in the types prevents me from writing
`Download x -> `CloseEverything -> `StartNuclearWar. That's not the
biggest concern today but it's something that should eventually be
improved.


PS: I've generated the graph with camllexer, a hand-written parser, and
some logic that took advantage of caravel's organization (module packs
and predefined type names because of functors). I had tried Maxence
Guesdon's Oug which did a nice first work but it doesn't currently
(fully?) handle polymorphic variants and it obviously wasn't specialized
for caravel.


-- 
Adrien Nader