[Caml-list] PortAudio on ocaml

[Caml-list] PortAudio on ocaml

[Likai Liu]

Hello,

I'm trying to evaluate on porting PortAudio API to O'Caml, which would
enable an entire genre of sound applications being written in this nice
language. What concerns me is that PortAudio uses asynchronous callbacks,
much like signals, to tell the application that some audio sample has
arrived, needs to be played, or both. I looked at the signal.c of asmrun
and byterun and found out that signals are normally deferred if it
interrupts a running O'Caml program until a garbage collection cycle
occurs. It doesn't seem fesible to have these real-time audio callbacks
implemented this way; threads seems to be the only way.

If I were to enforce using threads semantics, then the synchronization
issue might have some light. However, the original PortAudio API is meant
for a program to continue its execution when callback keeps coming.
Depending on the implementation on a certain platform, it might interrupt
the program completely while it processes the callback, or the program
might run concurrently with the callback on different threads. Since only
one O'Caml section can hold the global mutex lock at any given time, it's
likely that the callback will block trying to acquire the lock in either
cases. It seems necessary to change the PortAudio interface on O'Caml to
block the running program while the audio transaction is carried out.

Before I make such a radical decision, I'm wondering if there is a better
solution. A possibility after soul searching the source code of threads is
that, in the case of bytecode, when a thread does Unix.select, the thread
scheduler would let go the lock momentarily so the callback doesn't block
forever; in the case of native thread, when a thread calls Threads.yield,
then the lock is released momentarily. I'm not sure if it's a good idea to
rely on these deep internals of O'Caml, or maybe it's better to stay on
the light side, that is, restrict the API for workaround, instead. What do
the O'Caml developers say about this?

There is a simplified interface of PortAudio called Pablio that doesn't
use the callback, but a conventional blocking I/O style read/write
functions. Pablio interface is straightforward to implement. So at least
the current plan is to get pablio working first, and leave the actual
PortAudio API unimplemented with some provided interface stub for
reference purpose.

liulk

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] PortAudio on ocaml

[SooHyoung Oh]

I didn't read your message seriously,
but in my first glance, what you want to seems to be asychroneous threads.
Why don't you use Event module of threads library?
It's a Concurrent ML.

---
SooHyoung Oh
----- Original Message -----
From: "Likai Liu" <liulk@bu.edu>
To: <caml-list@inria.fr>
Cc: "Likai Liu" <liulk@acs.bu.edu>
Sent: Tuesday, July 15, 2003 5:55 AM
Subject: [Caml-list] PortAudio on ocaml


> Hello,
>
> I'm trying to evaluate on porting PortAudio API to O'Caml, which would
> enable an entire genre of sound applications being written in this nice
> language. What concerns me is that PortAudio uses asynchronous callbacks,
> much like signals, to tell the application that some audio sample has
> arrived, needs to be played, or both. I looked at the signal.c of asmrun
> and byterun and found out that signals are normally deferred if it
> interrupts a running O'Caml program until a garbage collection cycle
> occurs. It doesn't seem fesible to have these real-time audio callbacks
> implemented this way; threads seems to be the only way.
>
> If I were to enforce using threads semantics, then the synchronization
> issue might have some light. However, the original PortAudio API is meant
> for a program to continue its execution when callback keeps coming.
> Depending on the implementation on a certain platform, it might interrupt
> the program completely while it processes the callback, or the program
> might run concurrently with the callback on different threads. Since only
> one O'Caml section can hold the global mutex lock at any given time, it's
> likely that the callback will block trying to acquire the lock in either
> cases. It seems necessary to change the PortAudio interface on O'Caml to
> block the running program while the audio transaction is carried out.
>
> Before I make such a radical decision, I'm wondering if there is a better
> solution. A possibility after soul searching the source code of threads is
> that, in the case of bytecode, when a thread does Unix.select, the thread
> scheduler would let go the lock momentarily so the callback doesn't block
> forever; in the case of native thread, when a thread calls Threads.yield,
> then the lock is released momentarily. I'm not sure if it's a good idea to
> rely on these deep internals of O'Caml, or maybe it's better to stay on
> the light side, that is, restrict the API for workaround, instead. What do
> the O'Caml developers say about this?
>
> There is a simplified interface of PortAudio called Pablio that doesn't
> use the callback, but a conventional blocking I/O style read/write
> functions. Pablio interface is straightforward to implement. So at least
> the current plan is to get pablio working first, and leave the actual
> PortAudio API unimplemented with some provided interface stub for
> reference purpose.
>
> liulk
>
> -------------------
> To unsubscribe, mail caml-list-request@inria.fr Archives:
http://caml.inria.fr
> Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ:
http://caml.inria.fr/FAQ/
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
>

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] PortAudio on ocaml

[Likai Liu]

On Monday, July 14, 2003, at 08:37  PM, SooHyoung Oh wrote:

>
> I didn't read your message seriously,
> but in my first glance, what you want to seems to be asychroneous 
> threads.
> Why don't you use Event module of threads library?
> It's a Concurrent ML.

The implementation is up to the actual PortAudio implementors 
themselves, which is different on all the platforms that PortAudio runs 
on. They use a callback based API as the common interface that each 
PortAudio implementation would follow. I have no say over whether they 
should use Event module or otherwise. Event is synchronous. The problem 
of wrapping PortAudio callbacks using Event is that callbacks have a 
realtime requirement, that is, they're required to finish within a 
certain amount of time, so they must be processed right away, unlike 
Event that counts on being received at some convenient time. Callbacks 
need to return directly to PortAudio for further processing.

Hope this clarifies.

liulk

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] PortAudio on ocaml

[Xavier Leroy]

> I'm trying to evaluate on porting PortAudio API to O'Caml, which would
> enable an entire genre of sound applications being written in this nice
> language. What concerns me is that PortAudio uses asynchronous callbacks,
> much like signals, to tell the application that some audio sample has
> arrived, needs to be played, or both.

That's the crux of the problem, indeed.  I looked at the PortAudio
tutorial and found this little gem:

  Your callback function is often called by an interrupt, or low level
  process so you should not do any complex system activities like
  allocating memory, or reading or writing files, or printf(). Just
  crunch numbers and generate audio signals.

So, calling Caml code from the callback function is definitely out of
the question.

> [Snipped investigation of internals of OCaml runtime system]
> What do the O'Caml developers say about this?

That no matter how much you peek under the hood, you cannot
meet the requirements of the PortAudio "interrupt handler" model.

My advice is that there are essentially only two programming models
for C libraries that can reasonably be interfaced with OCaml:

- Synchronous calls (block the calling thread till results are available,
  then return into Caml).

- Event loops (register Caml callbacks, then enter a library-provider event
  handler that will block and invoke the callbacks one at a time,
  but not from a signal or interrupt context).

In both cases, you can make the design thread-compatible by using the
Caml runtime system functions enter_blocking_section and
leave_blocking_section, see
        http://caml.inria.fr/archives/200106/msg00199.html

- Xavier Leroy

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] PortAudio on ocaml

[Likai Liu]

On Wednesday, July 16, 2003, at 05:55  AM, Xavier Leroy wrote:

> That's the crux of the problem, indeed.  I looked at the PortAudio
> tutorial and found this little gem:
>
>   Your callback function is often called by an interrupt, or low level
>   process so you should not do any complex system activities like
>   allocating memory, or reading or writing files, or printf(). Just
>   crunch numbers and generate audio signals.

I'm giving a bit leeway on the activities that allocate memory, since 
most of the time it's just manipulation of some data structure in 
memory. Granted, it might cause hard-disk activity every once in a 
while for thrashing, but even simple memory access could cause this 
too. Also, note that the restriction is tight on some platforms but not 
so tight on others. I was hoping at least this would work on these not 
so stringent systems. However, I think invoking system calls that block 
the callback for possibly unspecified amount of time is the forbidden 
game.

I think garbage collection inside the callback is probably okay, if it 
doesn't do other fancy things, such as dispatching signals or influence 
thread scheduling. What does O'Caml runtime do during garbage 
collection?

> - Event loops (register Caml callbacks, then enter a library-provider 
> event
>   handler that will block and invoke the callbacks one at a time,
>   but not from a signal or interrupt context).

That's also a possibility; does it play well with O'Caml Threads?

Thanks for the response.

liulk

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] PortAudio on ocaml

[Damien Doligez]

> I think garbage collection inside the callback is probably okay, if it 
> doesn't do other fancy things, such as dispatching signals or 
> influence thread scheduling. What does O'Caml runtime do during 
> garbage collection?

Garbage collection inside the callback is probably OK, if you can 
guarantee
that the callback does not occur during garbage collection, or any other
operation that assumes that memory doesn't move under its feet.

-- Damien

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] PortAudio on ocaml

[Likai Liu]

is a "leave_blocking_section()" call enough to guarentee that?

liulk

On Wednesday, July 16, 2003, at 12:57  PM, Damien Doligez wrote:

>> I think garbage collection inside the callback is probably okay, if 
>> it doesn't do other fancy things, such as dispatching signals or 
>> influence thread scheduling. What does O'Caml runtime do during 
>> garbage collection?
>
> Garbage collection inside the callback is probably OK, if you can 
> guarantee
> that the callback does not occur during garbage collection, or any 
> other
> operation that assumes that memory doesn't move under its feet.
>
> -- Damien
>

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] PortAudio on ocaml

[Damien Doligez]

On Wednesday, July 16, 2003, at 07:46 PM, Likai Liu wrote:

> is a "leave_blocking_section()" call enough to guarentee that?
>
> liulk
>
> On Wednesday, July 16, 2003, at 12:57  PM, Damien Doligez wrote:
>
>>> I think garbage collection inside the callback is probably okay, if 
>>> it doesn't do other fancy things, such as dispatching signals or 
>>> influence thread scheduling. What does O'Caml runtime do during 
>>> garbage collection?
>>
>> Garbage collection inside the callback is probably OK, if you can 
>> guarantee
>> that the callback does not occur during garbage collection, or any 
>> other
>> operation that assumes that memory doesn't move under its feet.
>>

If the callback is invoked in a thread, yes.  But then you are back
to square 1: leave_blocking_section will wait for the master lock,
and your program is not real-time.

-- Damien

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] PortAudio on ocaml

[Likai Liu]

[-- Attachment #1: Type: text/plain, Size: 1010 bytes --]

> > is a "leave_blocking_section()" call enough to guarentee that?
>
> If the callback is invoked in a thread, yes.  But then you are back
> to square 1: leave_blocking_section will wait for the master lock,
> and your program is not real-time.

So what about the alternative option, as Xavier suggested earlier, to 
enter an event loop that that waits in an "enter_blocking_section" call 
with threads but no other running threads, and then callbacks are made 
during the loop one at a time?

I think at the end, more speculation probably won't help much. I think 
I should just try different ways and do ad-hoc experiments to see which 
ones actually work acceptably, in the presence of garbage collection, 
interrupting signals, other threads, etc. If none of them work, at 
least there is a blocking I/O interface left for O'Caml.

Speak of which, if I write a C function that blocks, does the bytecode 
thread scheduler know how to schedule other threads during the mean 
time? And native threads?

liulk

[-- Attachment #2: Type: text/enriched, Size: 1067 bytes --]

<excerpt><fontfamily><param>Courier</param>> is a
"leave_blocking_section()" call enough to guarentee that?


If the callback is invoked in a thread, yes.  But then you are back

to square 1: leave_blocking_section will wait for the master lock,

and your program is not real-time.</fontfamily>

</excerpt>

So what about the alternative option, as Xavier suggested earlier, to
enter an event loop that that waits in an "enter_blocking_section"
call with threads but no other running threads, and then callbacks are
made during the loop one at a time?


I think at the end, more speculation probably won't help much. I think
I should just try different ways and do ad-hoc experiments to see
which ones actually work acceptably, in the presence of garbage
collection, interrupting signals, other threads, etc. If none of them
work, at least there is a blocking I/O interface left for O'Caml.


Speak of which, if I write a C function that blocks, does the bytecode
thread scheduler know how to schedule other threads during the mean
time? And native threads?


liulk

Re: [Caml-list] PortAudio on ocaml

[Damien Doligez]

On Saturday, July 19, 2003, at 03:37 PM, Likai Liu wrote:

> So what about the alternative option, as Xavier suggested earlier, to 
> enter an event loop that that waits in an "enter_blocking_section" 
> call with threads but no other running threads, and then callbacks are 
> made during the loop one at a time?

You would have a Caml thread that does "enter_blocking_section" to
release the master lock, then wait for callbacks.  When a callback
occurs, you need to "leave_blocking_section", do the work, then
"enter_blocking_section" again before returning from the callback.


> Speak of which, if I write a C function that blocks, does the bytecode 
> thread scheduler know how to schedule other threads during the mean 
> time? And native threads?

This is the whole purpose of the "enter_blocking_section" and
"leave_blocking_section" calls: to tell the scheduler that it can
schedule another thread.

-- Damien

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners