Re: [Caml-list] Severe loss of performance due to new signal handling

[Oliver Bandel <oliver@first.in-berlin.de>]

On Mon, Mar 20, 2006 at 01:37:39PM +0100, Gerd Stolpmann wrote:
> Am Montag, den 20.03.2006, 11:39 +0100 schrieb Oliver Bandel:
> > On Mon, Mar 20, 2006 at 10:29:49AM +0100, Xavier Leroy wrote:
> > > > It seems that changes to signal handling between OCaml 3.08.4 and 3.09.1
> > > > can lead to a very significant loss of performance (up to several orders
> > > > of magnitude!) in code that uses threads and performs I/O (tested on 
> > > Linux).
> > > > [...]
> > > > Maybe some assembler guru can repeat this result and explain to us
> > > > what's going on...
> > > 
> > > Short explanation: atomic instructions are dog slow.
> > > 
> > > Longer explanation:
> > > 
> > > OCaml 3.09 fixed a number of long-standing bugs in signal handling
> > > that could cause signals to be "lost" (not acted upon).  The fixes,
> > 
> > I'm not clear about what your proble is with lost signals,
> > but when using signals on Unix/Linux-systems, you can use
> > UNIX-API, with sigaction/sigprocmask etc. you can do things well,
> > and with the signal-function which C provides things are bad/worse.
> > The C-API signal-function signal(3) clears out the signal handler
> > after a call to it. In the sigaction/sigprocmask/... functions
> > the handler remains installed.
> 
> The problem is the following: The O'Caml runtime cannot handle signals
> immediately because this would break memory management (e.g. imagine a
> signal happens when memory has just been allocated but not initialized).
> To get around this the signal handler sets just a flag, and the compiler
> emits instructions that regularly check this flag at safe points of
> execution (i.e. memory is known to be initialised). These instructions
> are now atomic in 3.09. In 3.08, you have basically
> 
> if "flag is set" then (
>   (*)
>   "clear flag";
>   "call the signal handler function"
> )
> 
> If another signal happens at (*) it will be lost.

Well, I'm not an OCaml-internals specialist, so I can't say
if this would be necessary...

On the first look it looks like the problem one has when using
signal(3) instead of sigprocmask(), sigaction() and Co. 



> 
> As you mention sigprocmask: Of course, you can block signals before
> checking the flag and allow them again after clearing it, but this would
> be even _much_ slower than the solution in 3.09, because sigprocmask
> needs a context switch to do its work (it is a kernel function).

Why to call such functions often?

You can use sigaction() to handle signals when you want it;
even if signals are blocked, their occurence will be saved.
When you want to handle them, then you can do it.

It's too long ago to say details here, but if wanted,
I can look for details (not today, but tomorrow I will have some time
to do it).

(The only thing you can't find out with this mechanism is,
 which of the signals came first and which later.)



> 
> I don't know what Xavier has in mind to solve the problem, but I would
> think about reducing the frequency of the atomic check.
> This could work as follows:
> 
> - Revert the check to the 3.08 solution
> - Use the alarm clock timer to regularly call a signal_manager
>   function at a certain frequency (i.e. the signal flag is set
>   at a certain frequency)

Using alarm() is not reliable.


[...]
> > BTW: I saw that in the Unix-module the unix-signalling functions are
> >      now included... (the ywere not on older versions of Ocaml).
> 
> They have been included for a long time. New is Thread.sigmask.

Depends on the definition of "long time" ;-)
As I had first conact with OCaml, which really is some years ago,
it was not included (I think 3.04?).
I didn't looked for these functions, and just saw them, while
looking for other things at about 3.08 (?).
So then I was astouned. This makes OCaml better suited for
applications in the real world, because C's signal(3) is unreliable.
(When catching the signal, the handler will be deactivated, until it is
 re-established again - that's the same problem as you has mentioned above.
 So if a signal comes twice, you lost one. But on the other hand,
 this provides the system for recursive loops which could make it
 unreliable too. But only with sigprocmask()/sigaction() and so on
 you can do it reliable and clean and clear.)

Ciao,
   Oliver