Why 'Graphics.wait_next_event' doesn't reply anymore ?

Why 'Graphics.wait_next_event' doesn't reply anymore ?

[Fabrice Marchant]

  Hi !

  This issue with 'Graphics.wait_next_event' seems only to happen in some cases with threads.
Please consider the code snippet below :
---------------------------------------------------
let id = ref None

let run () =
  while !id <> None do
    step ();  (* Changes drawing state *)
    plot () (* ;
    Thread.delay 0.01 *)
  done

let rec test () =
  print "Graphics.wait_next_event, next statement...\n"; (* A flushed printf *)
  let status = wait_next_event [ Key_pressed ] in
  print "Graphics.wait_next_event returned.\n";
  match status.key with
    'r' ->
      (match !id with
        None -> id := Some (Thread.create run ())
      | Some _ -> ());
      test ()
  | 's' ->
      id := None;
      test ()
  | _ ->
      ()
-----------------------------------------------
  First time you press 'r', a thread is created. ( There is no delay problem in changing thread 'id' after creation and process test '!id <> None' is soon true at first call. )
  So the thread does its animation job. Here is the full short - stuck - test code :
http://fabrice.marchant.free.fr/code/wait_next_event/
After the thread is launched, we can check the console to state that the "main" thread is waiting for a key press.

The issue arises now.
 You can hit any key : 'Graphics.wait_next_event' couldn't care less.

Please why ?

  Another thing I do not understand :
if you insert a delay inside the 'run' thread code loop ( please see uncommentable Thread.delay above ), the program does not stick anymore, 'Graphics.wait_next_event' normally listening to the keyboard.
( Nice, but my aim was to run this thread at full speed... )

Thanks,

Fabrice

Re: [Caml-list] Why 'Graphics.wait_next_event' doesn't reply anymore ?

[Fabrice Marchant]

  Replying to myself :

  Found this 2002 - unanswered ... thread - : "Graphics.wait_next_event () blocks other threads"
http://caml.inria.fr/pub/ml-archives/caml-list/2002/09/d29ca3379fe68e3cdbd36323f5f409c0.en.html

  Please help.

Regards,

Fabrice
-------
 . o .
 . . o
 o o o

Re: [Caml-list] Why 'Graphics.wait_next_event' doesn't reply anymore ?

[Oliver Bandel]

Hi,


Zitat von Fabrice Marchant <fabrice.marchant@orange.fr>:

>   Replying to myself :
>
>   Found this 2002 - unanswered ... thread - : "Graphics.wait_next_event ()
> blocks other threads"
>
http://caml.inria.fr/pub/ml-archives/caml-list/2002/09/d29ca3379fe68e3cdbd36323f5f409c0.en.html
>
[...]

This code looks ugly.

Why?

I don't mean the indentation and not the use of names...

... I mean the way, how it handles the threading.


It forces calls to the scheduler all too often.
And it does it at three places. And in the print-loop
it clals it every time!
This is the way of wasting ressources.


I have rewritten it, but not the time to explain it now,
in some minutes I have guests, so no time for long emails.

Here is the code (I have tested it, and it works):

============================================
open Thread
open Graphics


  let code1 () =
        let x = ref 0 in
        while true do
          incr x;
          Printf.printf "alpha %d\n" !x;
          flush stdout
        done

  let code2 () =
        open_graph " 300x300";
        while true do
          let st = wait_next_event [Key_pressed] in
          Printf.printf "key %c pressed...\n" st.key;
          flush stdout
        done

let _ =
  ignore(Thread.create code1());
  code2()

============================================


Ciao,
   Oliver

Re: [Caml-list] Why 'Graphics.wait_next_event' doesn't reply anymore ?

[Julien Moutinho]

On Sun, Nov 11, 2007 at 05:48:34PM +0100, Oliver Bandel wrote:
> [...]
> This code looks ugly.
I've just noticed the id := Some (Thread.create run ())
coupled with the [while !id <> None do] in run:
it seems possible that the while loop never starts.
And the useless call to Thread.exit.
But I think these are unrelated stuffs to the real matter:
the thread running wait_next_event seems to never get
the attention of the thread scheduler.

> Why?
> [...]
> It forces calls to the scheduler all too often.
> And it does it at three places. And in the print-loop
> it clals it every time!
Are you sure?

> This is the way of wasting ressources.
> 
> Here is the code (I have tested it, and it works):
Unfortunately, if I remove the calls to Printf/flush in code1,
then the thread running wait_next_event still hangs up,
I mean has no time to run as in Fabrice's code (without the call
to Thread.delay).

I tried to replace Printf/flush calls by a simple call to:
CAMLprim value
release_master_lock (value unit)
{
    enter_blocking_section();
    leave_blocking_section();
    return Val_unit;
}
and I observed that the thread running wait_next_event
does not hang up, as with the Printf/flush calls.

Hence, I think it's only because Printf/flush release the master
lock when they try to lock the mutex of stdout, that your proposal
works better.

Fabrice: I would expect Thread.yield to help in such situation,
but I am unable to get something out of it.

Just my humble remarks,
  Julien.

Re: [Caml-list] Why 'Graphics.wait_next_event' doesn't reply anymore ?

[Fabrice Marchant]

  Hi,

 Thanks for the help !

On Sun, 11 Nov 2007 19:35:30 +0100
Julien Moutinho <julien.moutinho@gmail.com> wrote:

> On Sun, Nov 11, 2007 at 05:48:34PM +0100, Oliver Bandel wrote:
> > [...]
> > This code looks ugly.
> I've just noticed the id := Some (Thread.create run ())
> coupled with the [while !id <> None do] in run:
> it seems possible that the while loop never starts.
   Oliver didn't spoke about my code...
Anyway these are not real codes but rather snippets intended to track the issues.

  However what you say is right. I mentionned it in original post :
>>> There is no delay problem in changing thread 'id' after creation and process test '!id <> None' is soon true at first call.

> And the useless call to Thread.exit.
  Indeed, I removed it in original post :
>>> let run () =
>>>   while !id <> None do
>>>     step ();  (* Changes drawing state *)
>>>     plot () (* ;
>>>     Thread.delay 0.01 *)
>>>   done
>>> 
>>> let rec test () =
Should have updated blockIssue.ml source...
Thread.exit () (* <- Is it really useful, at end of thread code here ? *)

> But I think these are unrelated stuffs to the real matter:
> the thread running wait_next_event seems to never get
> the attention of the thread scheduler.
  I agree.

> > Here is the code (I have tested it, and it works):
> Unfortunately, if I remove the calls to Printf/flush in code1,
> then the thread running wait_next_event still hangs up,
> I mean has no time to run as in Fabrice's code (without the call
> to Thread.delay).
  I thought it was a time question too. But I performed some trials and
now doubt of this.
> 
> I tried to replace Printf/flush calls by a simple call to:
> CAMLprim value
> release_master_lock (value unit)
> {
>     enter_blocking_section();
>     leave_blocking_section();
>     return Val_unit;
> }
> and I observed that the thread running wait_next_event
> does not hang up, as with the Printf/flush calls.
> 
> Hence, I think it's only because Printf/flush release the master
> lock when they try to lock the mutex of stdout, that your proposal
> works better.

> Fabrice: I would expect Thread.yield to help in such situation,
> but I am unable to get something out of it.
  I tried to put some 'yields' at different locations of the code,
just like Berke Durak did. Unfortunately they didn't change the
program behaviour.

I'm grateful for the interesting results of the trials you exposed
here.

Regards,

Fabrice

Re: [Caml-list] Why 'Graphics.wait_next_event' doesn't reply anymore ?

[Oliver Bandel]

Zitat von Julien Moutinho <julien.moutinho@gmail.com>:

> On Sun, Nov 11, 2007 at 05:48:34PM +0100, Oliver Bandel wrote:
> > [...]
> > This code looks ugly.
> I've just noticed the id := Some (Thread.create run ())
> coupled with the [while !id <> None do] in run:
[...]

I didn't saw this code in the program that I'm talking about :-)


[...]
>
> > Why?
> > [...]
> > It forces calls to the scheduler all too often.
> > And it does it at three places. And in the print-loop
> > it clals it every time!
> Are you sure?

Yes!

And the loop in the main-thread, that does nothing else but
calling Thread.yield is even worse!

Two busy threads running wild against one
most-of-the-time sleeping thread...
...and always these Thread.yield-calls,
that only add unnecessary business...

Ciao,
   Oliver

Re: [Caml-list] Why 'Graphics.wait_next_event' doesn't reply anymore ?

[Fabrice Marchant]

  Hi Oliver !

 Thanks to have considered my problem.

On Sun, 11 Nov 2007 17:48:34 +0100
Oliver Bandel <oliver@first.in-berlin.de> wrote:

> http://caml.inria.fr/pub/ml-archives/caml-list/2002/09/d29ca3379fe68e3cdbd36323f5f409c0.en.html

> This code looks ugly.
...
> It forces calls to the scheduler all too often.
> And it does it at three places. And in the print-loop
> it clals it every time!
> This is the way of wasting ressources.
  I think these 'yields' are only desperate attempts to hint the scheduler to do its job - that it doesn't as we could expected -.

> open Thread
> open Graphics
> 
> 
>   let code1 () =
>         let x = ref 0 in
>         while true do
>           incr x;
>           Printf.printf "alpha %d\n" !x;
>           flush stdout
(*                      ;
            display_mode false;
            fill_circle 100 100 20; 
            synchronize ()*)
>         done
> 
>   let code2 () =
>         open_graph " 300x300";
>         while true do
>           let st = wait_next_event [Key_pressed] in
>           Printf.printf "key %c pressed...\n" st.key;
>           flush stdout
>         done
> 
> let _ =
>   ignore(Thread.create code1());
>   code2()
  Yes, it perfectly works.

So I compared it with my hanging example to try to understood what was responsible for blocking.

  If you uncomment the lines I inserted in code1, things begins to go more or less wrong.
On my computer, from time to time, when you alternatively focus console output and graphical window, some characters can be suddenly spat. Or keyboard input is completely ignored...

   Thanks to have make things progress ! I think we aren't far to understand.

Regards,

  Fabrice

Re: [Caml-list] Why 'Graphics.wait_next_event' doesn't reply anymore ?

[Oliver Bandel]

Hi Fabrice,

sorry, have not much time to answer, I think it's time to sleep ;-)

But a short one...


Zitat von Fabrice Marchant <fabrice.marchant@orange.fr>:

>   Hi Oliver !
>
>  Thanks to have considered my problem.
>
> On Sun, 11 Nov 2007 17:48:34 +0100
> Oliver Bandel <oliver@first.in-berlin.de> wrote:
>

[...]
> >   let code1 () =
> >         let x = ref 0 in
> >         while true do
> >           incr x;
> >           Printf.printf "alpha %d\n" !x;
> >           flush stdout
> (*                      ;
>             display_mode false;
>             fill_circle 100 100 20;
>             synchronize ()*)
> >         done
[...]

Why do you want to call synchronize that often?

I didn't tested your code, so I don't know what's going wrong.
But calling synchronize that often IMHO makes no sense.
It's similar to calling Thread.yield to often.

For an average eye/viewer, it would be enough to call
synchronize all 1/25 seconds.
So, a thread that makes the timing and then calls
synchronize would be the right way, IMHO.
Or if you need higher frame rates, call it more often.
But I doubt in today machines, putting it inside a loop as you did,
would make sense.
And it also will waste ressources. It's not needed that fast,
because the eye is not fast enough.
So, there is no reason to call that function so often.

BTW: What's about your code, you sent?
Did you throw it away, or do you want some help there?
It looked to big for spare time, but maybe the next days
I can look on it. But possibly, if the toto.ml
(the code on which I already answered) is OK for you,
I would ignore your first code.


Ciao,
   Oliver

P.S.: Good night ;-)

Re: [Caml-list] Why 'Graphics.wait_next_event' doesn't reply anymore ?

[Fabrice Marchant]

> > >   let code1 () =
> > >         let x = ref 0 in
> > >         while true do
> > >           incr x;
> > >           Printf.printf "alpha %d\n" !x;
> > >           flush stdout
> > (*                      ;
> >             display_mode false;
> >             fill_circle 100 100 20;
> >             synchronize ()*)
> > >         done
> [...]
> 
> Why do you want to call synchronize that often?
> 
> I didn't tested your code, so I don't know what's going wrong.
> But calling synchronize that often IMHO makes no sense.
> It's similar to calling Thread.yield to often.
> 
> For an average eye/viewer, it would be enough to call
> synchronize all 1/25 seconds.
> So, a thread that makes the timing and then calls
> synchronize would be the right way, IMHO.
> Or if you need higher frame rates, call it more often.
> But I doubt in today machines, putting it inside a loop as you did,
> would make sense.
> And it also will waste ressources. It's not needed that fast,
> because the eye is not fast enough.
> So, there is no reason to call that function so often.
 Hi Oliver !

  I agree doing this call to 'synchronize' at this rate has absolutely
no meaning.
  The reason it appears here : because I removed the statements of
my hanging code one by one until it remains enough to stick
'wait_next_event'.

> BTW: What's about your code, you sent?
> Did you throw it away, or do you want some help there?
> It looked to big for spare time, but maybe the next days
> I can look on it.
  The first link was to another dummy code only intended to demonstrate
the encountered issue with 'wait_next_event' + threads.

  In fact the true application where the problem originally raised is :
http://fabrice.marchant.free.fr/funLife/
Y. A. Game of Life.
 ..*
(*** is a good test pattern.)
 .*.

  The thread code locates in ten lines autoplay.ml.
Suppressing the loop delay will cause 'wait_next_event' (and me)
to be stuck : the aim was to run the game at full speed.

  I didn't found anything about a bug in 'wait_next_event' so I ask :
who could say what's wrong in a thread loop without this f*ckin delay ?

 Regards,

Fabrice

Re: [Caml-list] Why 'Graphics.wait_next_event' doesn't reply anymore ?

[Oliver Bandel]

Hi Fabrice,

Zitat von Fabrice Marchant <fabrice.marchant@orange.fr>:
[...]
>   In fact the true application where the problem originally raised is :
> http://fabrice.marchant.free.fr/funLife/
[...]

I only have looked some minutes into the code.
fast browsed ;-)

Both threads use the Graphics-module (via View-module).
Possibly the problem is, that the access
is not protected: use Mutexes, when calling the View.<function>,
so that each Thread only has access to the View-functions
(and the Graphics module's functions), when the other thread
is ready with it's work. Otherwise possibly things become confused.

So, wrap each of the View.<function> calls with
calls to Mutex.lock and Mutex.unlock.

Possibly this is your main problem!


Ciao,
   Oliver


P.S.: The Mutex must be in the environment of both threads,
      so must be defined at the toplevel of the autoPlay.ml,
      which means unit-global, as "running" also is.

Re: [Caml-list] Why 'Graphics.wait_next_event' doesn't reply anymore ?

[Fabrice Marchant]

> Both threads use the Graphics-module (via View-module).
> Possibly the problem is, that the access
> is not protected: use Mutexes, when calling the View.<function>,
> so that each Thread only has access to the View-functions
> (and the Graphics module's functions), when the other thread
> is ready with it's work. Otherwise possibly things become confused.
> 
> So, wrap each of the View.<function> calls with
> calls to Mutex.lock and Mutex.unlock.
> 
> Possibly this is your main problem!

  Great ! Oliver, it was this. All perfectly rocks at full speed now.
I'm very happy my Game of Life finally works.
However, I've not understood why this Mutex was needed : what could be
the data shared with 'wait_next_event' ?
  Some internal of Graphics module ?

 Regards,

Fabrice

Re: [Caml-list] Why 'Graphics.wait_next_event' doesn't reply anymore ?

[Oliver Bandel]

Hi,


Zitat von Fabrice Marchant <fabrice.marchant@orange.fr>:

> > Both threads use the Graphics-module (via View-module).
> > Possibly the problem is, that the access
> > is not protected: use Mutexes, when calling the View.<function>,
> > so that each Thread only has access to the View-functions
> > (and the Graphics module's functions), when the other thread
> > is ready with it's work. Otherwise possibly things become confused.
> >
> > So, wrap each of the View.<function> calls with
> > calls to Mutex.lock and Mutex.unlock.
> >
> > Possibly this is your main problem!
>
>   Great ! Oliver, it was this. All perfectly rocks at full speed now.

Heheh, fine.
So, it was not OCaml nor the Graphics module that is buggy ;-)


> I'm very happy my Game of Life finally works.

Are the new sources online?
How to play it?
Now that it works, I may play around with it for a while.... :)




> However, I've not understood why this Mutex was needed : what could be
> the data shared with 'wait_next_event' ?
>   Some internal of Graphics module ?

Yes, Graphics modules internals.
Both threads might share a lot of data.
They are drawing at the same screen,
so if their work might be interrupted by the
scheduler before the data was written completely,
something might go wrong.

If you can't be sure that the data is isolated,
use Mutexes. And in the case of the Graphics-module you
don't get back a value that holds something,
e.g. when using the initialization functions.
So seemingly they all write on the same data.
(And to have a copy of the whole screen always,
 would be wasting memory, so I think it's ok this way.)

One could write a wrapper-lib for it, so that
the Mutex-stuff is already inside then.
But that is something that should be up to the
users of the Graphics-module (and not the OCaml-team),
because needs and design might be differing a lot.
If people for example would not want to have Threads inside,
with such a wrapper-lib they would be inside nevertheless.
And that's, how bloat-projects start to evolve. ;-)


Ciao,
   Oliver

Re: [Caml-list] Why 'Graphics.wait_next_event' doesn't reply anymore ?

[Fabrice Marchant]

On Tue, 13 Nov 2007 11:14:32 +0100
Oliver Bandel <oliver@first.in-berlin.de> wrote:

> So, it was not OCaml nor the Graphics module that is buggy ;-)
  But I wonder why I didn't see any word about these necessary
"precaution" in doc. (Maybe have missed something ?)

> > I'm very happy my Game of Life finally works.
> 
> Are the new sources online?
The code is'nt yet very clean but I seize the opportunity to show
it you offer to me :
http://fabrice.marchant.free.fr/fl/funLife/
> How to play it?
Press '?' or 'H' for help page ( directed to console ).

> Now that it works, I may play around with it for a while.... :)
  It features an "infinite" board, a zoom, mouse dragging of the
view, history and customizable colours. Many things can be changed
"on the fly" - when automaton is running - but maybe it remains
some bugs about this.
  ( Nothing to do with the powerful top speed "xlife". )
However, 'll try to improve and the TODO list is long...

> > However, I've not understood why this Mutex was needed : what could be
> > the data shared with 'wait_next_event' ?
> >   Some internal of Graphics module ?
> 
> Yes, Graphics modules internals.
> Both threads might share a lot of data.
> They are drawing at the same screen,
> so if their work might be interrupted by the
> scheduler before the data was written completely,
> something might go wrong.
> 
> If you can't be sure that the data is isolated,
> use Mutexes. And in the case of the Graphics-module you
> don't get back a value that holds something,
> e.g. when using the initialization functions.
> So seemingly they all write on the same data.
> (And to have a copy of the whole screen always,
>  would be wasting memory, so I think it's ok this way.)
  I tried to use Mutex again this way for changing autoplay speed.
Seems to work.

Regards,

  Fabrice

Re: [Caml-list] Why 'Graphics.wait_next_event' doesn't reply anymore ?

[Oliver Bandel]

Zitat von Fabrice Marchant <fabrice.marchant@orange.fr>:

> On Tue, 13 Nov 2007 11:14:32 +0100
> Oliver Bandel <oliver@first.in-berlin.de> wrote:
>
> > So, it was not OCaml nor the Graphics module that is buggy ;-)
>   But I wonder why I didn't see any word about these necessary
> "precaution" in doc. (Maybe have missed something ?)
[...]

There are so many C-libs that were (or even now are)
not threadsafe. And this is (was) seldom mentioned
in the documentation...

Who uses threads must be aware of a lot of things.
Again I can recommend to read the recommended books.


If you want to have thunghs mentioned in the OCaml-documentation,
you can write a feature wish for the Ocaml docs at the
appropriate tool:

  http://caml.inria.fr/bin/caml-bugs

It's used for bugs and feature wishes,
for OCaml sourcecode as well as the documentation.

So, if you want to have it inside, please use this.
I think it's easier for the core developpers to look
there, instead of scanning all the discussions here
on the list for something that they might possibly
should take care of.


Ciao,
    Oliver

Re: [Caml-list] Why 'Graphics.wait_next_event' doesn't reply anymore ?

[Oliver Bandel]

Zitat von Fabrice Marchant <fabrice.marchant@orange.fr>:

[...]
> >
> >   let code1 () =
> >         let x = ref 0 in
> >         while true do
> >           incr x;
> >           Printf.printf "alpha %d\n" !x;
> >           flush stdout
> (*                      ;
>             display_mode false;
>             fill_circle 100 100 20;
>             synchronize ()*)
> >         done
[...]


Oh, I see: display-mode also has to go out of the loop.

Things that must be called only once to work,
should only called once.

So, it has to go out of the loop.

And I has to go to bed now ;-)

Ciao,
   Oliver

Re: [Caml-list] Why 'Graphics.wait_next_event' doesn't reply anymore ?

[Oliver Bandel]

Zitat von Fabrice Marchant <fabrice.marchant@orange.fr>:

>   Hi Oliver !
>
>  Thanks to have considered my problem.
>
> On Sun, 11 Nov 2007 17:48:34 +0100
> Oliver Bandel <oliver@first.in-berlin.de> wrote:
>
> >
>
http://caml.inria.fr/pub/ml-archives/caml-list/2002/09/d29ca3379fe68e3cdbd36323f5f409c0.en.html
>
> > This code looks ugly.
> ...
> > It forces calls to the scheduler all too often.
> > And it does it at three places. And in the print-loop
> > it clals it every time!
> > This is the way of wasting ressources.
>   I think these 'yields' are only desperate attempts to hint the scheduler to
> do its job - that it doesn't as we could expected -.
[...]

Yes, that also was my view on using this function to often.

It's like using OCaml: ehy to use the GC-Module?
I have not used it, because I didn't needed.
I first look, if my code could possibly done better,
instead of thinking the gaerbageCollector might do something wrong.

The Gc-module is there to have a possibility to use some
screws for manual fine-tuning. But you should not start to
use it until your application seems to be ready.

It's like in LaTeX, where you should not too often
look at the results of line braking. You can do fine tuning
at the end.

And Gc or in Threading: explicit hints to the scheduler
should be used, if necessary, and not as early as possible.

But these are very general hints: optimization of details
comes at the end.

Best way to avoid "problems that have to be fixed by hand later"
(workarounds) is, to have some consideration on the design when
you start (and during the whole development phase of course).

(In LaTeX: use the pramble for selecting the right class and packages
and do not insert paragraph spacing throughout the whole document, if you
can change \parskip and \parindent ;-))


Ciao,
   Oliver

P.S.: And looking for Bugfixes in the libraries (as the one with the
      example code of "toto.ml") is the thing that comes in the very end,
      when the code is really good. If someone writes bad code and then
      looking for bugfixes of the langaueg he/she uses, that's a littlebid
      strange, IMHO.
      For concurrent programming you need to rethink many things, compared to
      non-concurrent programming. It's a littlebid like switching to FP,
      coming from non-FP-languages.