[9fans] audio standards -- too many to choose from

[9fans] audio standards -- too many to choose from

[Tim Newsham]

I'm looking over the audio device formats because I would
like to make a service that is interoperable with the existing
services and much to my dismay there are several different
formats:

    - plan9 audio(3) with audio,volume,audiostat
    - plan9 usb(4) with audio,volume,audioctl,audioin
    - inferno audio(3) with audio,audioctl
      man page hints at a port driver that supports an
      older format

The original audio(3) seems fairly simplistic in that
it only supports one format of audio data.  The inferno
format for selecting audio format seems somewhat nice
except that it breaks out pcm variants across several
settings (encoding and bits, but probably to that should
be added "endian", too).

I would love to make a system that interops with the existing
"standard" but I don't know which standard to choose, and
it seems that at the least I will have to embrace and extend
one of them...

- Is there any interest in unifying the existing audio formats?
- If so, is anyone interested in bouncing around ideas of what
   this format should look like?
- What software exists for each of these formats?
- Which format is the most "popular"?

Tim Newsham
http://www.thenewsh.com/~newsham/

Re: [9fans] audio standards -- too many to choose from

[erik quanstrom]

> - Is there any interest in unifying the existing audio formats?
> - If so, is anyone interested in bouncing around ideas of what
>    this format should look like?

this is probablly super obvious.  but as one as you
pick a cannonical format as is the plan 9 custom
(like for character sets, images, etc.) you should do
well.

there are a couple of models for dealing with lots
and lots of standards.  there's the tcs/jpg translator
model.  but i think the upas/fs model might work better
for audio.  you can present a standard menu of resources
and translate on the fly with a fs.  might be kind of cool.

but i'm sure you'd figured this all out by now.  i'll shut up.

- erik

Re: [9fans] audio standards -- too many to choose from

[Tim Newsham]

> there are a couple of models for dealing with lots
> and lots of standards.  there's the tcs/jpg translator
> model.  but i think the upas/fs model might work better
> for audio.  you can present a standard menu of resources
> and translate on the fly with a fs.  might be kind of cool.

Instead of writing translators I'd rather pick a single convention
that seems the most suitable and fixup the other implementations
and clients to fall in line with those conventions.  My biggest
question is "is it worth my time?"  If I spend time unifying
the various implementations, will they be accepted back into
the mainline or will it be viewed as unnecessary churn...
How much do people even care about the audio drivers and they're
interfaces at the present time?

> - erik

Tim Newsham
http://www.thenewsh.com/~newsham/

Re: [9fans] audio standards -- too many to choose from

[erik quanstrom]

> Instead of writing translators I'd rather pick a single convention
> that seems the most suitable and fixup the other implementations
> and clients to fall in line with those conventions.  My biggest
> question is "is it worth my time?"  If I spend time unifying
> the various implementations, will they be accepted back into
> the mainline or will it be viewed as unnecessary churn...
> How much do people even care about the audio drivers and they're
> interfaces at the present time?

i didn't mean translating from one /dev/audio to the next.
i ment dealing with azalia audio vs. ac97 vs. soundblaster.
and ogg/vorbis vs. mp3 vs pem vs. *law.

i might be out-of-step.  but i care about audio drivers.
there hasn't been the time to tackle that problem, though.
i would be interested in a solid implementation.  i haven't
yet spent any serious time looking at the various ac'97 drivers
but they didn't seem to me to go quite far enough in defining
an audio universe.  that's one thing that draw(2) does do
that i would hope audio would do as well.

since neither of the ac'97 drivers are in the distribution,
i don't think you can cause more churn than there is already.
basically, the way to get plan 9 audio right now is to use
(a) ancient hardware (b) a nonstandard kernel (c) drawterm.
i don't want to short change 9vx, but i don't know about it
off the top of my head.

sounds like a cool project.

- erik

Re: [9fans] audio standards -- too many to choose from

[Tim Newsham]

> i didn't mean translating from one /dev/audio to the next.
> i ment dealing with azalia audio vs. ac97 vs. soundblaster.
> and ogg/vorbis vs. mp3 vs pem vs. *law.

I agree here.  I envision a separate codec server that
sits on top of an audio server and encapsulates a bunch
of this stuff.  It would be nice if it was practical
to  "cat foo.mp3 >/dev/codec/mp3" or something like that..
I haven't really thought this through much, just daydreaming
this feature...

> i would be interested in a solid implementation.  i haven't
> yet spent any serious time looking at the various ac'97 drivers
> but they didn't seem to me to go quite far enough in defining
> an audio universe.  that's one thing that draw(2) does do
> that i would hope audio would do as well.

I'm not familiar with what ac97 provides.  Can you describe
a little what the features are and what you think an interface
should be like?

Here's a brief rundown of what I was thinking about, somewhat
based on the inferno driver:

    $ cat /dev/audioctl
    enc pcm8 pcm16le pcmu16be ulaw
    rate 44100 8000 11025 22050
    chans 2 1

    meaning that the device is currently in pcm8 at 44.1khz
    and 2 channels (stereo).  (Inferno's driver would have
    said enc "pcm" with bits "8" instead of saying "pcm8",
    but I feel that separating out bits and possibly endian
    for "pcm" adds complication since the encoding "pcm-16-unsigned"
    isnt really a separable thing).

    $ echo "rate 11025" >/dev/audioctl
    $ echo "chans 1" >/dev/audioctl
    $ cat /dev/audioctl
    enc pcm8 pcm16le pcmu16be ulaw
    rate 11025 44100 8000 22050 44100
    chans 1 2
    $ dd if=/dev/audio of=sound bs=4096

    now we have 4k samples of 11khz mono audio in "sound"

additionally it might be useful to support some of the
stat features from usbaudio or plan9 audio(3), such
as the ability to figure out the dma size, how much data
is buffered, and the timing associated with samples being
read.

> - erik

Tim Newsham
http://www.thenewsh.com/~newsham/

Re: [9fans] audio standards -- too many to choose from

[ron minnich]

It would be nice to do plan 9 audio if only to show people how it can
be done. Anyone who deals with audio on linux knows how not to do it;
but it's probably very hard to get it right. I know I could do no
better.

It would be nice, I think, to do it out of the kernel ... still better
to do it in a way that makes it easy to adopt new audio formats
without having to rip out the guts each time and start over -- which
seems to be the linux problem.

The ideas here certainly sound like potentially good ones.

ron

Re: [9fans] audio standards -- too many to choose from

[erik quanstrom]

> It would be nice, I think, to do it out of the kernel ... still better
> to do it in a way that makes it easy to adopt new audio formats
> without having to rip out the guts each time and start over -- which
> seems to be the linux problem.

i'm just glad they don't do this with disks.  we have scsi and ata.
and that's enough for me.

- erik

Re: [9fans] audio standards -- too many to choose from

[Charles Forsyth]

>we have scsi and ata.
>and that's enough for me.

that's more than enough for me.

Re: [9fans] audio standards -- too many to choose from

[Russ Cox]

It's hard to do the low-level hardware stuff outside
the kernel.  It's possible, but it's a lot easier inside.
Just keep the inside simple.

I've done audio on a handful of operating systems
and all I ever want to do with the card is set it up
to play X kHz 16-bit little-endian PCM stereo and
then control the volume.  The rest can be done from
user space.  This is exactly what Plan 9's audio
driver already does, and I wish the others were so
simple.

The original post commented that only supporting
PCM was too simplistic, but I argue it's exactly right.
PCM is the native hardware sample format and is
basically the "uncompress bitmap" of the audio world.
Defining a fixed byte order avoids one level of
misconfiguration and complexity; it doesn't matter which
is chosen except that the existing driver is little endian.
Being able to change the sample rate is necessary for quality;
it is too hard to resample other speeds nicely on
the fly, and the hardware will take care of it for you.
The fixed 16-bit and stereo parts are perhaps debatable,
but basically no one has 8-bit audio they care about,
it is trival to convert 8-bit to 16-bit, and it is also trivial
to turn mono into stereo.

If you want a different device file into which you
can write mp3 and other sound file formats, great:
do it in user space, translating into the native hardware
format.

By analogy, Plan 9's kernel gives user space access to the
raw disk bytes.  It doesn't implement the interpretation
of those bytes as 9660, fat, ext2, kfs, venti, or other disk
"file formats".  That can be done better and more easily
in user space.  The same is true of audio.  There's already
a good native hardware format and a well-established and
capable (if simple) kernel interface.  Leave those alone
and put the extra complexity in user space for those who
want to use it.

Russ


P.S. My guess is that Inferno's implementation of mp3 in the
kernel device file was a quick way to make an mp3 decoder
available without writing one in Limbo.  It may have had
something to do with making a snazzy low-bandwidth
import-across-the-network demo too.  But neither of those
applies to Plan 9, since the same can be accomplished
with a C program running in user space.

Re: [9fans] audio standards -- too many to choose from

[Charles Forsyth]

> ... Inferno's implementation of mp3 in the kernel device file ...

it does?

Re: [9fans] audio standards -- too many to choose from

[Russ Cox]

On Tue, Aug 11, 2009 at 9:35 PM, Charles Forsyth<forsyth@terzarima.net> wrote:
>> ... Inferno's implementation of mp3 in the kernel device file ...
>
> it does?

sorry, i got my wires crossed.
it was plan b that did that.

russ

Re: [9fans] audio standards -- too many to choose from

[Roman Shaposhnik]

On Aug 11, 2009, at 9:24 PM, Russ Cox wrote:
> It's hard to do the low-level hardware stuff outside
> the kernel.  It's possible, but it's a lot easier inside.
> Just keep the inside simple.
>
> I've done audio on a handful of operating systems
> and all I ever want to do with the card is set it up
> to play X kHz 16-bit little-endian PCM stereo and
> then control the volume.  The rest can be done from
> user space.  This is exactly what Plan 9's audio
> driver already does, and I wish the others were so
> simple.

This sounds like exactly the kind of thing one wants
from an audio driver for playback. For recording things
get slightly more complicated.

Even for playback if you want to do passthrough (via
SPDIF or some such) things get slightly more complicated.
Of course, one can disregard passthrough as not
being an audio at all, but rather a datalink issue.

> It may have had something to do with making a snazzy low-bandwidth
> import-across-the-network demo too.

Personally, I'd place my bet on this explanation. ;-)

Thanks,
Roman.

Re: [9fans] audio standards -- too many to choose from

[hiro]

> This sounds like exactly the kind of thing one wants
> from an audio driver for playback. For recording things
> get slightly more complicated.

What exactly do you mean?

> Even for playback if you want to do passthrough (via
> SPDIF or some such) things get slightly more complicated.
> Of course, one can disregard passthrough as not
> being an audio at all, but rather a datalink issue.

What's so special about things like SPDIF? Why would it be not
suitible for the kernel audio driver? For me it's clearly an audio and
not a datalink issue, but perhaps I have only limited use cases.

Re: [9fans] audio standards -- too many to choose from

[Roman V. Shaposhnik]

On Wed, 2009-08-12 at 17:36 +0200, hiro wrote:
> > This sounds like exactly the kind of thing one wants
> > from an audio driver for playback. For recording things
> > get slightly more complicated.
>
> What exactly do you mean?

Now that I understand what Tim is trying to do my original
concern makes no sense. Sorry for this little bit of noise.

>
> > Even for playback if you want to do passthrough (via
> > SPDIF or some such) things get slightly more complicated.
> > Of course, one can disregard passthrough as not
> > being an audio at all, but rather a datalink issue.
>
> What's so special about things like SPDIF?

Two things: the data traveling over the link can be any data, not
just some form of PCM (it can be AC-3, it can be DTS, it can even
be MP3). In that respect the audio card acts as a pass-through
device. It is not really supposed to do anything with the
data. And that brings us to the second point -- the pass-through
APIs can be as ugly as hell, depending on the type of the device
(follow the link to the MPlayer docs I provided in this thread).

> Why would it be not suitible for the kernel audio driver?

Because it has very little to do with the PCM audio, and that's
what the driver discussed in this thread was supposed to be all about.
Now, I'm not saying that there's shouldn't be a special driver just for
pass-through type of functionality, all I'm saying is that
lumping this functionality together with the PCM one would
confuse things.

> For me it's clearly an audio and
> not a datalink issue, but perhaps I have only limited use cases.

Perhaps, everywhere in this thread I should've prefixed the word
audio with "some form of PCM".

Thanks,
Roman.

Re: [9fans] audio standards -- too many to choose from

[hiro]

I think I can consider myself lucky, that my equipment doesn't know
how to do AC-3 or DTS :)
Although my soundcard does have some other interfaces i.e. for a
S/PDIF clock source.

On Fri, Aug 14, 2009 at 4:45 AM, Roman V. Shaposhnik<rvs@sun.com> wrote:
> On Wed, 2009-08-12 at 17:36 +0200, hiro wrote:
>> > This sounds like exactly the kind of thing one wants
>> > from an audio driver for playback. For recording things
>> > get slightly more complicated.
>>
>> What exactly do you mean?
>
> Now that I understand what Tim is trying to do my original
> concern makes no sense. Sorry for this little bit of noise.
>
>>
>> > Even for playback if you want to do passthrough (via
>> > SPDIF or some such) things get slightly more complicated.
>> > Of course, one can disregard passthrough as not
>> > being an audio at all, but rather a datalink issue.
>>
>> What's so special about things like SPDIF?
>
> Two things: the data traveling over the link can be any data, not
> just some form of PCM (it can be AC-3, it can be DTS, it can even
> be MP3). In that respect the audio card acts as a pass-through
> device. It is not really supposed to do anything with the
> data. And that brings us to the second point -- the pass-through
> APIs can be as ugly as hell, depending on the type of the device
> (follow the link to the MPlayer docs I provided in this thread).
>
>> Why would it be not suitible for the kernel audio driver?
>
> Because it has very little to do with the PCM audio, and that's
> what the driver discussed in this thread was supposed to be all about.
> Now, I'm not saying that there's shouldn't be a special driver just for
> pass-through type of functionality, all I'm saying is that
> lumping this functionality together with the PCM one would
> confuse things.
>
>> For me it's clearly an audio and
>> not a datalink issue, but perhaps I have only limited use cases.
>
> Perhaps, everywhere in this thread I should've prefixed the word
> audio with "some form of PCM".
>
> Thanks,
> Roman.
>
>
>

Re: [9fans] audio standards -- too many to choose from

[Tim Newsham]

> I've done audio on a handful of operating systems
> and all I ever want to do with the card is set it up
> to play X kHz 16-bit little-endian PCM stereo and
> then control the volume.  The rest can be done from
> user space.  This is exactly what Plan 9's audio
> driver already does, and I wish the others were so
> simple.
>
> The original post commented that only supporting
> PCM was too simplistic, but I argue it's exactly right.

The simplicity is definitely attractive in its own right,
and I'll consider it.  However, the devices do provide hardware
support for other formats which do require some work to convert.
mu-law and a-law come to mind..  the conversion isnt very
computationally expensive, but the hardware will do it free..
Yah, this format doesnt come up that often.. perhaps its not
worth the effort, but then again the ability to switch a device's
encoding isnt very much work either...  About as hard as
changing the sampling rate or turning stereo on and off...

> If you want a different device file into which you
> can write mp3 and other sound file formats, great:
> do it in user space, translating into the native hardware
> format.

I agree wrt. "mp3".  I'm considering the possibility of supporting
alaw, ulaw, pcm8, pcm16 in big/little and signed/unsigned formats,
and adpcm, using the hardware features...

> Russ

Tim Newsham
http://www.thenewsh.com/~newsham/

Re: [9fans] audio standards -- too many to choose from

[Roman Shaposhnik]

On Aug 11, 2009, at 10:15 PM, Tim Newsham wrote:
> The simplicity is definitely attractive in its own right,
> and I'll consider it.  However, the devices do provide hardware
> support for other formats which do require some work to convert.
> mu-law and a-law come to mind..

In all my life doing multimedia I'm yet to come across real use
of mu-law and a-law (except in legacy telephony which is really not
multimedia, if you ask me).

Russ is absolutely correct -- PCM is all there is for audio.

Everything else is really more of a datalink issue.

>> If you want a different device file into which you
>> can write mp3 and other sound file formats, great:
>> do it in user space, translating into the native hardware
>> format.
>
> I agree wrt. "mp3".  I'm considering the possibility of supporting
> alaw, ulaw, pcm8, pcm16 in big/little and signed/unsigned formats,
> and adpcm, using the hardware features...


Here's a complete list of audio formats that one can make hardware
either generate or accept. Where do you draw the line?
    $ ffmpeg -formats | grep DEA | egrep -v "wmav|vorbis|flac"
  DEA    adpcm_4xm
  DEA    adpcm_adx
  DEA    adpcm_ct
  DEA    adpcm_ea
  DEA    adpcm_ima_dk3
  DEA    adpcm_ima_dk4
  DEA    adpcm_ima_qt
  DEA    adpcm_ima_smjpeg
  DEA    adpcm_ima_wav
  DEA    adpcm_ima_ws
  DEA    adpcm_ms
  DEA    adpcm_sbpro_2
  DEA    adpcm_sbpro_3
  DEA    adpcm_sbpro_4
  DEA    adpcm_swf
  DEA    adpcm_xa
  DEA    adpcm_yamaha
  DEA    g726
  DEA    gsm
  DEA    mp2
  DEA    pcm_alaw
  DEA    pcm_mulaw
  DEA    pcm_s16be
  DEA    pcm_s16le
  DEA    pcm_s24be
  DEA    pcm_s24daud
  DEA    pcm_s24le
  DEA    pcm_s32be
  DEA    pcm_s32le
  DEA    pcm_s8
  DEA    pcm_u16be
  DEA    pcm_u16le
  DEA    pcm_u24be
  DEA    pcm_u24le
  DEA    pcm_u32be
  DEA    pcm_u32le
  DEA    pcm_u8
  DEA    sonic
  DEA    vorbis

Thanks,
Roman.

Re: [9fans] audio standards -- too many to choose from

[Tim Newsham]

>> I agree wrt. "mp3".  I'm considering the possibility of supporting
>> alaw, ulaw, pcm8, pcm16 in big/little and signed/unsigned formats,
>> and adpcm, using the hardware features...
>
>
> Here's a complete list of audio formats that one can make hardware
> either generate or accept. Where do you draw the line?
>   $ ffmpeg -formats | grep DEA | egrep -v "wmav|vorbis|flac"
> DEA    adpcm_4xm
> DEA    adpcm_adx
> DEA    adpcm_ct
[...]

Draw the line at what the hardware can be told to decode
with a flip of a register?  The driver interface can easily
accomodate arbitrary encoding names (see inferno's driver
for an example).  The rest is just mapping strings to register
values.  With the appropriate API, supporting 30 audio formats
is hardly harder than supporting 3....

One thing that strikes me is that if the device interface is
designed in a flexible way that allows for arbitrary codings, then
it will be easy to layer a userland driver on top of it that
extends the API with more encodings that arent supported in hardware.
If the interface isnt flexible in this way, then a separate
interface will have to be made for such a userland driver...
I'm not sure what the value of unifying those two interface is...

> Roman.

Tim Newsham
http://www.thenewsh.com/~newsham/

Re: [9fans] audio standards -- too many to choose from

[Steve Simon]

Here is how I think it would work - please correct me if
I am wrong.

the status file gives a list for the supported
features and the current state of each. If particular
hardware does not support mu-law then no state is displayed
for it and the application layer can decide to emulate a
mu-law table or report the problem to the user and give up.

I do think that thought needs to be given to how a combined
audio and video player would be inplemented - ensuring correct
lipsink at a minimum.

also, might I suggest that the application level tools live in
aud/xxx or audio/xxx - I think it is a shame that the image
manipulation tools are not in img/xxx

-Steve

Re: [9fans] audio standards -- too many to choose from

[Roman Shaposhnik]

On Aug 12, 2009, at 1:28 AM, Tim Newsham wrote:
>>> I agree wrt. "mp3".  I'm considering the possibility of supporting
>>> alaw, ulaw, pcm8, pcm16 in big/little and signed/unsigned formats,
>>> and adpcm, using the hardware features...
>>
>>
>> Here's a complete list of audio formats that one can make hardware
>> either generate or accept. Where do you draw the line?
>>  $ ffmpeg -formats | grep DEA | egrep -v "wmav|vorbis|flac"
>> DEA    adpcm_4xm
>> DEA    adpcm_adx
>> DEA    adpcm_ct
> [...]
>
> Draw the line at what the hardware can be told to decode
> with a flip of a register?

Pretty much all of the above.

> The driver interface can easily
> accomodate arbitrary encoding names (see inferno's driver
> for an example).  The rest is just mapping strings to register
> values.  With the appropriate API, supporting 30 audio formats
> is hardly harder than supporting 3....

My list was only there to try and prove the point that Russ has
made -- pick a most common format and stick with it. Convert
everything else into it. Forget about the fact that hardware
can/could do gazillion variations. At least forget about it for
the time being while you're getting basics right.

> One thing that strikes me is that if the device interface is
> designed in a flexible way that allows for arbitrary codings, then
> it will be easy to layer a userland driver on top of it that
> extends the API with more encodings that arent supported in hardware.
> If the interface isnt flexible in this way, then a separate
> interface will have to be made for such a userland driver...
> I'm not sure what the value of unifying those two interface is...

I don't think I buy this point of view. Gratuitous flexibility is not
something Plan 9 is known for, nor should it. IMHO.

Thanks,
Roman.

Re: [9fans] audio standards -- too many to choose from

[erik quanstrom]

> I don't think I buy this point of view. Gratuitous flexibility is not
> something Plan 9 is known for, nor should it. IMHO.

those with such talents don't generally enjoy a good reputation.
but i hear they make great money on the internet.

- erik

Re: [9fans] audio standards -- too many to choose from

[James Tomaschke]

Roman Shaposhnik wrote:
> On Aug 12, 2009, at 1:28 AM, Tim Newsham wrote:
>>> Here's a complete list of audio formats that one can make hardware
>>> either generate or accept. Where do you draw the line?
This is incorrect, you don't "make hardware" do anything, the software
layer that sits on top of the hardware does this.  The SB16 hardware
does not support ADPCM and it never will.

> My list was only there to try and prove the point that Russ has
> made -- pick a most common format and stick with it. Convert
> everything else into it.
By this logic, I need to have my application to convert CDROM-XA ADPCM
audio from a device into PCM just to talk to an interface, which in turn
must convert it back into ADPCM to play it back because the DMA
transfers to the audio hardware buffer require ADPCM.

>> One thing that strikes me is that if the device interface is
>> designed in a flexible way that allows for arbitrary codings, then
>> it will be easy to layer a userland driver on top of it that
>> extends the API with more encodings that arent supported in hardware.
>> If the interface isnt flexible in this way, then a separate
>> interface will have to be made for such a userland driver...
>> I'm not sure what the value of unifying those two interface is...
>
> I don't think I buy this point of view. Gratuitous flexibility is not
> something Plan 9 is known for, nor should it. IMHO.
Why does this idea deserves a "gratuitous" label?  Preventing an
application from communicating to the hardware in a native format is
complete lack of flexibility.

If you do not care about native hardware formats, why choose PCM over
ADPCM as a transport?  ADPCM is ~3/4 the size of PCM in terms of
bandwidth.  MPEG audio and probably most telephony codecs use ADPCM, why
consider the 1980 RedBook PCM as the holy audio format?

Re: [9fans] audio standards -- too many to choose from

[erik quanstrom]

> > My list was only there to try and prove the point that Russ has
> > made -- pick a most common format and stick with it. Convert
> > everything else into it.
> By this logic, I need to have my application to convert CDROM-XA ADPCM
> audio from a device into PCM just to talk to an interface, which in turn
> must convert it back into ADPCM to play it back because the DMA
> transfers to the audio hardware buffer require ADPCM.

the problem with supporting everything the hardware will
do is that it's quite expensive in terms of development time.
and that is the scarce resource that needs to be optimized.

it also will make the interfaces much bulkier because you have
to make accomidations for the quirks of n formats.

you're right, there is a cost.  simple is expensive.

- erik

Re: [9fans] audio standards -- too many to choose from

[Christopher Nielsen]

On Thu, Aug 13, 2009 at 15:52, erik quanstrom<quanstro@quanstro.net> wrote:
>> > My list was only there to try and prove the point that Russ has
>> > made -- pick a most common format and stick with it. Convert
>> > everything else into it.
>> By this logic, I need to have my application to convert CDROM-XA ADPCM
>> audio from a device into PCM just to talk to an interface, which in turn
>> must convert it back into ADPCM to play it back because the DMA
>> transfers to the audio hardware buffer require ADPCM.
>
> the problem with supporting everything the hardware will
> do is that it's quite expensive in terms of development time.
> and that is the scarce resource that needs to be optimized.
>
> it also will make the interfaces much bulkier because you have
> to make accomidations for the quirks of n formats.
>
> you're right, there is a cost.  simple is expensive.

So spend time figuring out the best interface, write what you need,
add as you find more you need, refactor if it starts to get crufty.
Sometimes you don't get the interface right the first time. That's
fine! You can refactor later. It's better to write code and play with
it than to get stuck in analysis paralysis.

Maybe we're thinking the same thing, and I misunderstand what you're saying.

-- 
Christopher Nielsen
"They who can give up essential liberty for temporary
safety, deserve neither liberty nor safety." --Benjamin Franklin

Re: [9fans] audio standards -- too many to choose from

[James Tomaschke]

erik quanstrom wrote:
>>> My list was only there to try and prove the point that Russ has
>>> made -- pick a most common format and stick with it. Convert
>>> everything else into it.
>> By this logic, I need to have my application to convert CDROM-XA ADPCM
>> audio from a device into PCM just to talk to an interface, which in turn
>> must convert it back into ADPCM to play it back because the DMA
>> transfers to the audio hardware buffer require ADPCM.
>
> the problem with supporting everything the hardware will
> do is that it's quite expensive in terms of development time.
> and that is the scarce resource that needs to be optimized.
Creative Labs made an EMU10K1x chipset, the "x" means that it came with
no hardware mixer.  The software driver had to perform the mixing before
sending it to the hardware.  A device driver that only exposes what the
hardware has is much more simple than one that patches missing
functionality in the software driver.

However if you are instead suggesting that it will take time to support
other formats other than signed 16-bit little-endian samples.  I have no
problem with a driver developer initially starting there leaving it
incomplete.  At least someone has the potential to add such support.

> it also will make the interfaces much bulkier because you have
> to make accomidations for the quirks of n formats.
>
> you're right, there is a cost.  simple is expensive.
>
> - erik
I don't see the complexity, the interface merely needs to allow device
drivers to provide the information such as "I support x y z", the
application can query a "features" file, select a format and write back
through the interface configuring the hardware.  The interface need not
have any predetermined knowledge of available formats.  The only issue
would be for each device driver to agree to call the same format by the
same name, "s8 s16le s24be".

To support multiple mix buffers or audio channels, the interface only
needs to iterate information provided by the driver.  Again, there is no
requirement that initial device driver development need support all
possible features.

Re: [9fans] audio standards -- too many to choose from

[erik quanstrom]

> However if you are instead suggesting that it will take time to support
> other formats other than signed 16-bit little-endian samples.  I have no
> problem with a driver developer initially starting there leaving it
> incomplete.  At least someone has the potential to add such support.

i don't believe i've expressed any preference for one
audio format over another.  all i've said, is that one format
is better than many.

>
> > it also will make the interfaces much bulkier because you have
> > to make accomidations for the quirks of n formats.
> >
> > you're right, there is a cost.  simple is expensive.
> >
> > - erik
> I don't see the complexity, the interface merely needs to allow device
> drivers to provide the information such as "I support x y z", the
> application can query a "features" file, select a format and write back
> through the interface configuring the hardware.  The interface need not
> have any predetermined knowledge of available formats.  The only issue
> would be for each device driver to agree to call the same format by the
> same name, "s8 s16le s24be".

i think you're ignoring the complexity.  suppose we wish to
support 10 formats.  suppose that i have devices that support
all 10 formats.  then there are 90 different conversions to do
to fill in the gaps.  now suppose that we pick a format.  then
we need to write 10 conversions.  also each driver potentially
needs 1 conversion.  if we want to write things like audio
mixers or whatnot in software, we only need to support 1 input
and 1 output format.

the expense is that this is going to cost cycles.  it may be
a little impractical, but computers are pretty fast these days.
i think it might just work.

this is exactly the same problem that unix finds itself in supporting
n character sets rather than translating on the way in.  it's really a
great example of optimizing your problem.  unix has unwitting made
an small problem unsolvable (at least well).  (cf. gnu grep.)

- erik

Re: [9fans] audio standards -- too many to choose from

[James Tomaschke]

erik quanstrom wrote:

>> I don't see the complexity, the interface merely needs to allow device
>> drivers to provide the information such as "I support x y z", the
>> application can query a "features" file, select a format and write back
>> through the interface configuring the hardware.  The interface need not
>> have any predetermined knowledge of available formats.  The only issue
>> would be for each device driver to agree to call the same format by the
>> same name, "s8 s16le s24be".
>
> i think you're ignoring the complexity.  suppose we wish to
> support 10 formats.  suppose that i have devices that support
> all 10 formats.  then there are 90 different conversions to do
> to fill in the gaps.  now suppose that we pick a format.  then
> we need to write 10 conversions.  also each driver potentially
> needs 1 conversion.  if we want to write things like audio
> mixers or whatnot in software, we only need to support 1 input
> and 1 output format.
It's a matter of kernelspace vs userspace and freedom.

Nothing I have proposed prevents you from using a single format.
Nothing I have proposed requires you to implement N formats or
conversions.  This is because I allow for your freedom to do so in
userspace.

Rather, your suggestion of forcing a single format, prevents my
applications from using other formats, and it requires I implement
conversions.  This is because you limit freedom by placing a simple
interface into kernelspace.

If you desire a simple audio server to glue simple audio applications it
can be done in userspace, a |filter program can be used by an audio
server on top of the device driver.  The scope of the complexity of
which is completely up to you.

Re: [9fans] audio standards -- too many to choose from

[Anthony Sorace]

James Tomaschke<james@orcasystems.com> wrote:

// ...you limit freedom by placing a simple interface into kernelspace.

are you serious?

Re: [9fans] audio standards -- too many to choose from

[James Tomaschke]

Anthony Sorace wrote:
> James Tomaschke<james@orcasystems.com> wrote:
>
> // ...you limit freedom by placing a simple interface into kernelspace.
>
> are you serious?

Show me how forcing a single sample format on the user increases their
freedom in developing audio applications.

Re: [9fans] audio standards -- too many to choose from

[Devon H. O'Dell]

2009/8/13 James Tomaschke <james@orcasystems.com>:
> erik quanstrom wrote:
>
>>> I don't see the complexity, the interface merely needs to allow device
>>> drivers to provide the information such as "I support x y z", the
>>> application can query a "features" file, select a format and write back
>>> through the interface configuring the hardware.  The interface need not
>>> have any predetermined knowledge of available formats.  The only issue
>>> would be for each device driver to agree to call the same format by the
>>> same name, "s8 s16le s24be".
>>
>> i think you're ignoring the complexity.  suppose we wish to
>> support 10 formats.  suppose that i have devices that support
>> all 10 formats.  then there are 90 different conversions to do
>> to fill in the gaps.  now suppose that we pick a format.  then
>> we need to write 10 conversions.  also each driver potentially
>> needs 1 conversion.  if we want to write things like audio
>> mixers or whatnot in software, we only need to support 1 input
>> and 1 output format.
> It's a matter of kernelspace vs userspace and freedom.
>
> Nothing I have proposed prevents you from using a single format.
> Nothing I have proposed requires you to implement N formats or
> conversions.  This is because I allow for your freedom to do so in
> userspace.
>
> Rather, your suggestion of forcing a single format, prevents my
> applications from using other formats, and it requires I implement
> conversions.  This is because you limit freedom by placing a simple
> interface into kernelspace.

This is the silliest thing I've seen posted in this thread. No offense
intended, but if you choose the highest available sample size and bit
rate available to the card, you are not limiting anybody: the
limitation becomes the hardware. If that's an issue, get really
ridiculously high quality analog devices, and stop being so anal about
your perfect ears.

--dho

Re: [9fans] audio standards -- too many to choose from

[Devon H. O'Dell]

2009/8/13 Devon H. O'Dell <devon.odell@gmail.com>:
> 2009/8/13 James Tomaschke <james@orcasystems.com>:
>> Rather, your suggestion of forcing a single format, prevents my
>> applications from using other formats, and it requires I implement
>> conversions.  This is because you limit freedom by placing a simple
>> interface into kernelspace.
>
> This is the silliest thing I've seen posted in this thread. No offense
> intended, but if you choose the highest available sample size and bit
> rate available to the card, you are not limiting anybody: the
> limitation becomes the hardware. If that's an issue, get really
> ridiculously high quality analog devices, and stop being so anal about
> your perfect ears.

And if this is a problem for the processor, resample to the highest
input or output format provided. If that's an issue, don't mix so much
at once, or get newer hardware or buy a mixer :)

--dho

Re: [9fans] audio standards -- too many to choose from

[James Tomaschke]

Devon H. O'Dell wrote:
> 2009/8/13 James Tomaschke <james@orcasystems.com>:
>> Rather, your suggestion of forcing a single format, prevents my
>> applications from using other formats, and it requires I implement
>> conversions.  This is because you limit freedom by placing a simple
>> interface into kernelspace.
>
> This is the silliest thing I've seen posted in this thread. No offense
> intended, but if you choose the highest available sample size and bit
> rate available to the card, you are not limiting anybody: the
> limitation becomes the hardware. If that's an issue, get really
> ridiculously high quality analog devices, and stop being so anal about
> your perfect ears.

How can an application select a higher sample size if the interface to
the driver only supports signed 16-bit samples?

"the limitation becomes the hardware"
Music App 24bit@192kHz -> #audio 16bit -> Hardware 24bit@192kHz.

Re: [9fans] audio standards -- too many to choose from

[Charles Forsyth]

>Hardware 24bit@192kHz.

the human ear can't hear as high as that
still, it ought to please any passing bat!
Hi-fi, hi-fi, ...

Re: [9fans] audio standards -- too many to choose from

[James Tomaschke]

Charles Forsyth wrote:
>> Hardware 24bit@192kHz.
>
> the human ear can't hear as high as that
> still, it ought to please any passing bat!
> Hi-fi, hi-fi, ...

So if i instead said 24bit@44.1kHz would it make any difference on my
argument? please.

You are right, however 192kHz means the DAC bandwidth could support 4x
48kHz buffers, which is what it's used for.

Re: [9fans] audio standards -- too many to choose from

[Charles Forsyth]

[-- Attachment #1: Type: text/plain, Size: 230 bytes --]

sorry, i did realise. i'm afraid i just couldn't resist slightly
misquoting Flanders and Swann's `Song of Reproduction' (High Fidelity).
http://www.amazon.com/Complete-Flanders-Swann-Georges-Brassens/dp/B000006T4S/ref=pd_sim_b_1

[-- Attachment #2: Type: message/rfc822, Size: 2734 bytes --]

From: James Tomaschke <james@orcasystems.com>
To: Fans of the OS Plan 9 from Bell Labs <9fans@9fans.net>
Subject: Re: [9fans] audio standards -- too many to choose from
Date: Thu, 13 Aug 2009 20:42:43 -0700
Message-ID: <4A84DD33.6060307@orcasystems.com>

Charles Forsyth wrote:
>> Hardware 24bit@192kHz.
>
> the human ear can't hear as high as that
> still, it ought to please any passing bat!
> Hi-fi, hi-fi, ...

So if i instead said 24bit@44.1kHz would it make any difference on my
argument? please.

You are right, however 192kHz means the DAC bandwidth could support 4x
48kHz buffers, which is what it's used for.

Re: [9fans] audio standards -- too many to choose from

[James Tomaschke]

Charles Forsyth wrote:
> sorry, i did realise. i'm afraid i just couldn't resist slightly
> misquoting Flanders and Swann's `Song of Reproduction' (High Fidelity).
> http://www.amazon.com/Complete-Flanders-Swann-Georges-Brassens/dp/B000006T4S/ref=pd_sim_b_1

Yes, marketing in general feeds on "more is better".

I don't have a complex audio chain, but there are some with multiple
devices mixing audio and digital.  I believe their argument for higher
sampling rates is because the analog filters have to cut frequencies
higher than Nyquist, so the higher the sampling frequency the less of of
a "slope" the filter has to work on which reduces the noise from the
analog side.  Or this can be interpreted as being able to use cheaper
analog filters for the same noise requirement.  However, this field is
beyond my competence.

>From a hardware perspective, I could see how a single 192kHz DAC is
cheaper to use than four 48kHz ones.  Smaller chip die area or less
components to attach.

Either way I believe it has to do more with manufacturing costs.

Re: [9fans] audio standards -- too many to choose from

[Steve Simon]

Ok, My memory from about 1982...

first there was phillips who used 2 high speed, high linearity 14bit DACs
in their CD players using 4 times oversampling - as they had no apropriate
16bit converters at the time; which gives near 16bit resolution.

sony however developed a laser trimmed 16bit converter but due to the high price
they time ivison multiplexed a single converter between left and right channels.
This required very steep, very high Q reconstruction filters which had
significantly non-linear phase shifts in the audio band.

people where surprised how much nicer the phillips sounded nicer than the
sony solution. The nasties caused by filters in the sony design have resulted
in much of the that "digital sounds nasty" folklaw.

as time went past people realised that riunning at 256 times oversampling
means you can have a 1 bit converter (i.e. PWM) and get near perfect linearity
without the need for laser trimming.

I beleive the push for very high sample rates at 16 or 24 bits is because it
further simplifies the reconstruction filters and they can be made cleaner
and even less likely to introduce audiable artifacts.

just my opinion and not that of my employer.

-Steve

Re: [9fans] audio standards -- too many to choose from

[matt]

Charles Forsyth wrote:

>>Hardware 24bit@192kHz.
>>
>>
>
>the human ear can't hear as high as that
>still, it ought to please any passing bat!
>Hi-fi, hi-fi, ...
>
>
>
Personally I can't hear over 9119 hz (audio), but I might want to record
1s of 192Khz (samples I presume) and stretch them by 100x to 9600 hz
(audio) and have a (possibly) interesting time listening to the results
without interpolating.

Re: [9fans] audio standards -- too many to choose from

[Devon H. O'Dell]

2009/8/13 James Tomaschke <james@orcasystems.com>:
> Devon H. O'Dell wrote:
>> 2009/8/13 James Tomaschke <james@orcasystems.com>:
>>> Rather, your suggestion of forcing a single format, prevents my
>>> applications from using other formats, and it requires I implement
>>> conversions.  This is because you limit freedom by placing a simple
>>> interface into kernelspace.
>>
>> This is the silliest thing I've seen posted in this thread. No offense
>> intended, but if you choose the highest available sample size and bit
>> rate available to the card, you are not limiting anybody: the
>> limitation becomes the hardware. If that's an issue, get really
>> ridiculously high quality analog devices, and stop being so anal about
>> your perfect ears.
>
> How can an application select a higher sample size if the interface to
> the driver only supports signed 16-bit samples?

You're distorting what I'm saying. You choose the highest sample size
local to the card. If you have a 24b@96kHz file, and your card only
supports playback at 16@44.1, you resample to 16@44.1. If your card
supports 24@96, and you're playing 16@44.1, you don't resample at all,
because it makes no sense. I don't see what's so difficult about this.

> "the limitation becomes the hardware"
> Music App 24bit@192kHz -> #audio 16bit -> Hardware 24bit@192kHz.

I think you misunderstand.

#A bit/sample limited by hardware

If hardware is 24@192, #A is 24@192, unless what you're playing is
lower than that, in which case upconverting makes no sense.

I really don't understand why this isn't obvious. We're all smart
here, why would you think we would suggest something that dumb?

--dho

Re: [9fans] audio standards -- too many to choose from

[Devon H. O'Dell]

This is starting to remind me of two things:

1) The case where this guy did a review of two different audio
processors, and labeled the DAC of one as inferior to the other. He
posted audio files of the resulting output from one and the other.
Except he posted the exact same link for both of them. People
commented about how right he was, after allegedly listening to both
files. They were the same file. I'm an audiophile, but I know when to
quit.

2) A la Tim Newsham and Federico Benavento, less talk more code. I
have a mixerfs implementation from several years ago that Kris and I
worked on. I'll fix it up once I get AC'97 on my laptop to demonstrate
what I mean mixer-wise. Tim said he was working on an interface that
resembles what I've discussed publicly here.

This leads to:

3) James, if you don't like it, make your own, and kick our asses.

--dho

Re: [9fans] audio standards -- too many to choose from

[James Tomaschke]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Devon H. O'Dell wrote:
> This is starting to remind me of two things:
>
> 1) The case where this guy did a review of two different audio
> processors, and labeled the DAC of one as inferior to the other. He
> posted audio files of the resulting output from one and the other.
> Except he posted the exact same link for both of them. People
> commented about how right he was, after allegedly listening to both
> files. They were the same file. I'm an audiophile, but I know when to
> quit.
>
> 2) A la Tim Newsham and Federico Benavento, less talk more code. I
> have a mixerfs implementation from several years ago that Kris and I
> worked on. I'll fix it up once I get AC'97 on my laptop to demonstrate
> what I mean mixer-wise. Tim said he was working on an interface that
> resembles what I've discussed publicly here.
I don't know if this has been fixed yet:
http://jtomaschke.blogspot.com/2009/02/plan9-ac97-driver-bug.html

AC97 covers a wide range of chipsets and you might experience the same
problems.

>
> This leads to:
>
> 3) James, if you don't like it, make your own, and kick our asses.
>
I have already changed the audio and other interfaces such as keyboard
input to provide more access to the hardware.  I have no desire to
develop these things other than getting the program to work.  So when
someone brings up the subject I'm more than happy to try and get them to
do the work for me.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkqE/JkACgkQl3JZTBu3/9kZ8ACgjX0ialxITK+V9QvLn2vrJ0y5
qw8AnRH0Y8iRtkcKVFVT/ccpnyMS7/kg
=ql0O
-----END PGP SIGNATURE-----

Re: [9fans] audio standards -- too many to choose from

[matt]

Flogging this horse still

But as I got my old Edirol UA-3d working with plan9 again (i.e. I
plugged it in!) I thought I'd see what the USB audio landscape looked
like, my it has changed.

Has anyone tried one of these with plan9

http://www.roland.com/products/en/UA-101/

It also does MIDI which is something else I'd like to play with.

Re: [9fans] audio standards -- too many to choose from

[James Tomaschke]

Devon H. O'Dell wrote:
> If hardware is 24@192, #A is 24@192

I am not aware that #A allows for 24bit samples, I only see an option
"speed" to set sampling rates.  The man page says: "Each sample is a 16
bit little-endian two's complement integer".

I was only going by what the manpage said, perhaps it's out of date.

>
> I really don't understand why this isn't obvious. We're all smart
> here, why would you think we would suggest something that dumb?
>

Re: [9fans] audio standards -- too many to choose from

[Devon H. O'Dell]

2009/8/14 James Tomaschke <james@orcasystems.com>:
> Devon H. O'Dell wrote:
>> If hardware is 24@192, #A is 24@192
>
> I am not aware that #A allows for 24bit samples, I only see an option
> "speed" to set sampling rates.  The man page says: "Each sample is a 16
> bit little-endian two's complement integer".
>
> I was only going by what the manpage said, perhaps it's out of date.

Aha, this is where the misunderstanding comes in. We are talking about
a theoretical new interface to #A that can be created, not what exists
currently. I think we are all in agreement that #A needs changing to
move forward with modern hardware.

--dho

Re: [9fans] audio standards -- too many to choose from

[Paul Donnelly]

forsyth@terzarima.net (Charles Forsyth) writes:

>>Hardware 24bit@192kHz.
>
> the human ear can't hear as high as that
> still, it ought to please any passing bat!
> Hi-fi, hi-fi, ...

If you're recording doing it at 24-bit will pay off in the mixing
stage.

Re: [9fans] audio standards -- too many to choose from

[hiro]

> If you're recording doing it at 24-bit will pay off in the mixing
> stage.

Thanks. And there can be some other kinds of stages, too.

Re: [9fans] audio standards -- too many to choose from

[Tim Newsham]

>> By this logic, I need to have my application to convert CDROM-XA ADPCM
>> audio from a device into PCM just to talk to an interface, which in turn
>> must convert it back into ADPCM to play it back because the DMA
>> transfers to the audio hardware buffer require ADPCM.
>
> the problem with supporting everything the hardware will
> do is that it's quite expensive in terms of development time.
> and that is the scarce resource that needs to be optimized.

It can be, but in this case you're literally talking about
the ability to parse "enc <format type>", a table mapping
format names to numbers, code to poke that into some
card-specific register, and possibly the ability to list out
all of the supported formats...  More code, to be sure, but
not a lot of it.

> it also will make the interfaces much bulkier because you have
> to make accomidations for the quirks of n formats.

I haven't been able to think of any.  Perhaps just my lack
of imagination.

> - erik

Tim Newsham
http://www.thenewsh.com/~newsham/

Re: [9fans] audio standards -- too many to choose from

[sqweek]

On 12/08/2009, Tim Newsham <newsham@lava.net> wrote:
> Draw the line at what the hardware can be told to decode
> with a flip of a register?  The driver interface can easily
> accomodate arbitrary encoding names (see inferno's driver
> for an example).

 One thing I haven't seen mentioned (perhaps because I misunderstand)
is the added state that comes from allowing multiple formats.
Actually, I think this problem exists with the current interface
already - say I have a program playing music at 44kHz and then I get
an email or something that I've set up to cause an aural alert, but
the program that does that wants to work at 22kHz. This sort of
interaction sounds bad enough to me, I don't even want to think about
what it would sound like if the music was in mp3 and the alert changed
the kernel format to wav.
 I guess this is where mixerfs and the like enter the picture, but
I've come to think of the kernel as a hardware multiplexer so I'm
liking the idea of having the kernel support multiple formats.
 Then again, I also respect the argument that the kernel is not a
hardware multiplexer but a hardware abstractor - doesn't matter what's
down there the kernel provides the same interface. From this point of
view the simple approach is compelling, have /dev/audio exclusive
write to avoid the above sort of interaction and mix in userspace (or
is there still playback/recording sample rate interaction?). I think
everyone agrees it doesn't make sense to have an mp3 decoder
in-kernel...
-sqweek

Re: [9fans] audio standards -- too many to choose from

[Skip Tavakkolian]

an old interview with some relevance

http://www.wired.com/wired/archive/3.08/thompson.html

Re: [9fans] audio standards -- too many to choose from

[hiro]

Thanks, very interesting read.

I think he lied, he was perfectly aware of how the piracy issue would
turn out, and he had it all planned :D

On Fri, Aug 14, 2009 at 11:54 PM, Skip Tavakkolian<9nut@9netics.com> wrote:
> an old interview with some relevance
>
> http://www.wired.com/wired/archive/3.08/thompson.html
>
>
>

Re: [9fans] audio standards -- too many to choose from

[Bakul Shah]

On Sat, 15 Aug 2009 05:24:01 +0800 sqweek <sqweek@gmail.com>  wrote:
> On 12/08/2009, Tim Newsham <newsham@lava.net> wrote:
> > Draw the line at what the hardware can be told to decode
> > with a flip of a register?  The driver interface can easily
> > accomodate arbitrary encoding names (see inferno's driver
> > for an example).
>
>  One thing I haven't seen mentioned (perhaps because I misunderstand)
> is the added state that comes from allowing multiple formats.
> Actually, I think this problem exists with the current interface
> already - say I have a program playing music at 44kHz and then I get
> an email or something that I've set up to cause an aural alert, but
> the program that does that wants to work at 22kHz. This sort of
> interaction sounds bad enough to me, I don't even want to think about
> what it would sound like if the music was in mp3 and the alert changed
> the kernel format to wav.

The mixer would have to be smart.

>  I guess this is where mixerfs and the like enter the picture, but
> I've come to think of the kernel as a hardware multiplexer so I'm
> liking the idea of having the kernel support multiple formats.
>  Then again, I also respect the argument that the kernel is not a
> hardware multiplexer but a hardware abstractor - doesn't matter what's
> down there the kernel provides the same interface. From this point of
> view the simple approach is compelling, have /dev/audio exclusive
> write to avoid the above sort of interaction and mix in userspace (or
> is there still playback/recording sample rate interaction?). I think
> everyone agrees it doesn't make sense to have an mp3 decoder
> in-kernel...
> -sqweek

A decoder is just a resource (like a mouse, keyboard, disk,
network interface etc.).  If h/w provides an mp3 decoder it
would be nice to be able to use it.

One idea is to have a devicename for each format/samplerate.
Some would be backed by h/w, some provided solely by s/w.
Basically you need an ability to interpose some sort of
"stream" module to create a network.

Re: [9fans] audio standards -- too many to choose from

[Anthony Sorace]

Tim Newsham<newsham@lava.net> wrote:

// Yah, this format doesnt come up that often.. perhaps its not
// worth the effort, but then again the ability to switch a device's
// encoding isnt very much work either...  About as hard as
// changing the sampling rate or turning stereo on and off...

i'd argue that the primary cost is not the effort (of writing a
decoder vs. driving the
encoder to accept a different format), but the impact on the design.
being able to
say "this takes pcm" is a win.

my main question comes from "fancy" formats. on cards that do 5.1, for example,
do we simply write 6-channel pcm samples in a defined order? i have no idea how
the cards behave in this regard. if it requires more than that (say, write ac-3
directly), i'd suggest that's not /dev/audio.

Re: [9fans] audio standards -- too many to choose from

[Roman Shaposhnik]

On Aug 11, 2009, at 10:43 PM, Anthony Sorace wrote:
> Tim Newsham<newsham@lava.net> wrote:
>
> // Yah, this format doesnt come up that often.. perhaps its not
> // worth the effort, but then again the ability to switch a device's
> // encoding isnt very much work either...  About as hard as
> // changing the sampling rate or turning stereo on and off...
>
> i'd argue that the primary cost is not the effort (of writing a
> decoder vs. driving the
> encoder to accept a different format), but the impact on the design.
> being able to
> say "this takes pcm" is a win.

When I was toying with FFmpeg, I had this idea, that perhaps my
initial pull model (where an external application needs to pull raw
data from /n/ffmpeg and play it back either in rio window or over
the speakers) was flawed and the better model would be it push
model (where we would simply use a filter filesystem over an
existing raw device).

In fact, perhaps even the page(1) command is falwed. What should've
happened was a next layer over rio, where /dev/draw/n/data would
be able to accept any kind of image encoding.

> my main question comes from "fancy" formats. on cards that do 5.1,
> for example,
> do we simply write 6-channel pcm samples in a defined order? i have
> no idea how
> the cards behave in this regard. if it requires more than that (say,
> write ac-3
> directly), i'd suggest that's not /dev/audio.

With multichannel playback you have those two options:
     1. drive each channel separately with PCM
     2. do AC-3/DTC/MP3 passthrough

For audio recording it is mostly multichannel.

Thanks,
Roman.

Re: [9fans] audio standards -- too many to choose from

[Roman Shaposhnik]

On Aug 11, 2009, at 10:59 PM, Roman Shaposhnik wrote:
> With multichannel playback you have those two options:
>    1. drive each channel separately with PCM
>    2. do AC-3/DTC/MP3 passthrough

Meant to include a URL for the curious ones (was trigger-happy):
     http://www.mplayerhq.hu/DOCS/tech/hwac3.txt

Thanks,
Roman.

Re: [9fans] audio standards -- too many to choose from

[erik quanstrom]

> In fact, perhaps even the page(1) command is falwed. What should've
> happened was a next layer over rio, where /dev/draw/n/data would
> be able to accept any kind of image encoding.

i think page is a good thing.  pushing data
translation to the edges makes programs like
resample much simplier.  they can assume plan 9
images.  and you don't end up linking every
program to the same enormous library.

what makes page look crusty is the fact that
plan 9 images are uncommon on most systems.

it would be like having a plan 9 system with
10% utf-8, 40% 8259-1 and 50% sjis-encoded
text.

- erik

Re: [9fans] audio standards -- too many to choose from

[Roman Shaposhnik]

On Aug 12, 2009, at 4:18 AM, erik quanstrom wrote:
>> In fact, perhaps even the page(1) command is falwed. What should've
>> happened was a next layer over rio, where /dev/draw/n/data would
>> be able to accept any kind of image encoding.
>
> i think page is a good thing.  pushing data
> translation to the edges makes programs like
> resample much simplier.  they can assume plan 9
> images.  and you don't end up linking every
> program to the same enormous library.

Ok, I buy that -- you want a canonical format and you want
multiple frontends and backends cooperating using it.
But what would such an architecture look like from
an FS perspective?

Supposed I want resample to operate on movie frames.

Thanks,
Roman.

Re: [9fans] audio standards -- too many to choose from

[James Tomaschke]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Russ Cox wrote:
> I've done audio on a handful of operating systems
> and all I ever want to do with the card is set it up
> to play X kHz 16-bit little-endian PCM stereo and
> then control the volume.  The rest can be done from
> user space.  This is exactly what Plan 9's audio
> driver already does, and I wish the others were so
> simple.
It is simple, and it limits you simple applications.

Having to set the sampling format or rate has never been a problem, in
fact it is a requirement.  The sample and format are chosen by the
application based on what the hardware internally supports to eliminate
any sample or rate conversion overhead.  Low latency is very important
for many audio applications.

> By analogy, Plan 9's kernel gives user space access to the
> raw disk bytes.  It doesn't implement the interpretation
> of those bytes as 9660, fat, ext2, kfs, venti, or other disk
> "file formats".  That can be done better and more easily
> in user space.
This is an incorrect analogy, audio hardware (DACs/ADCs/DSPs) are
engineered to function on a unit of information (sample) which varies
between audio hardware.  Perhaps if all audio hardware were
SoundBlaster-16s then the audio(3) interface would make sense.

Then there's the whole topic on multiple hardware mix buffers or
multiple channels.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkqDPh8ACgkQl3JZTBu3/9mdKwCeMT59l/zBLLTB3SgjaIyZzguO
d+sAn0KnZzA4zURrQnz1xTQwbXuV90f8
=1h8x
-----END PGP SIGNATURE-----

Re: [9fans] audio standards -- too many to choose from

[Skip Tavakkolian]

>  but I argue it's exactly right.
> PCM is the native hardware sample format and is
> basically the "uncompress bitmap" of the audio world.

makes perfect sense.

Re: [9fans] audio standards -- too many to choose from

[Roman Shaposhnik]

On Aug 11, 2009, at 7:07 PM, Tim Newsham wrote:
>> i didn't mean translating from one /dev/audio to the next.
>> i ment dealing with azalia audio vs. ac97 vs. soundblaster.
>> and ogg/vorbis vs. mp3 vs pem vs. *law.
>
> I agree here.  I envision a separate codec server that
> sits on top of an audio server and encapsulates a bunch
> of this stuff.  It would be nice if it was practical
> to  "cat foo.mp3 >/dev/codec/mp3" or something like that..
> I haven't really thought this through much, just daydreaming
> this feature...

About 3 years ago (at the time when I was really active in the
FFmpeg community) I toyed with an idea to port ffmpeg to
Plan9 in a form of a media FS. IOW:
      % ffmpeg matrix.vob
      % page /n/ffmpeg/matrix.vob/video/1/1

My biggest problem (and the one that reoccurred on this list
a month or so ago) was coming up with the equivalent of
HTTP's 'content negotiation' process.

There was also an additional complication with audio codecs --
they don't lend themselves easily to *sane* packetization.

But I'm getting ahead of myself: the biggest trouble with
audio (and this is something that linux folks have been
struggling with a lot, especially during the days of 486s)
is that it places almost RT requirements on processing.
You can do drop frames in video and only those with 20/20
will notice, with audio -- you drop a packet and you're
screwed quality-wise.

May be its better to call this latency, since we can all appreciate
some of the shortcomings that 9P has when it comes to it.

To be fair, though, I'm pretty sure you won't have much issues
with RAW PCM when the local kernel does the multiplexing,
but when you do it over the wire -- there'll be trouble.

Thanks,
Roman.

P.S. The ffmpeg media FS can be resurrected if there's enough
interest in this approach *and* if we can come up with the
interface before the coding phase.

Re: [9fans] audio standards -- too many to choose from

[erik quanstrom]

> May be its better to call this latency, since we can all appreciate
> some of the shortcomings that 9P has when it comes to it.

i think you're drawing the wrong conclusion from a too-abstract
view of the facts.

9p is a ping-pong protocol.  this gives it *consistent* latency.
this is good for audio.

and if you're talking to a local kernel, it's the same, except
there is no 9p.

- erik

Re: [9fans] audio standards -- too many to choose from

[Roman Shaposhnik]

On Aug 11, 2009, at 9:25 PM, erik quanstrom wrote:
>> May be its better to call this latency, since we can all appreciate
>> some of the shortcomings that 9P has when it comes to it.
>
> i think you're drawing the wrong conclusion from a too-abstract
> view of the facts.

My ears begged to differ ;-)

> 9p is a ping-pong protocol.  this gives it *consistent* latency.
> this is good for audio.

I'm not sure either latency or RT is proper terminology here. But
I believe what I meant was clear: when you need overall latency
to be around 5ms you start to notice 9P.

> and if you're talking to a local kernel, it's the same, except
> there is no 9p.


Sure. And if you read my email to the very end I made a comment
of exactly the same nature.

Thanks,
Roman.

Re: [9fans] audio standards -- too many to choose from

[Tim Newsham]

> I'm not sure either latency or RT is proper terminology here. But
> I believe what I meant was clear: when you need overall latency
> to be around 5ms you start to notice 9P.

It sounds like you have a specific app in mind, and a real-time
one at that.  If you're using your audio device for live audio
(ie. adding effects to audio from your guitar)
you need pretty small latency.  You can go a bit higher than
5ms without noticing, though.

Most apps dont require realtime.  For example, streaming a song.
You dont care if the samples show up at year ear 2seconds
after they were sent from your hard drive, so long as all the
samples are delayed by the same amount..  You can stream this
clear across the country over all kinds of cut rate ISPs and
still get satisfactory results with enough buffering..

> Roman.

Tim Newsham
http://www.thenewsh.com/~newsham/

Re: [9fans] audio standards -- too many to choose from

[Roman Shaposhnik]

On Aug 11, 2009, at 10:10 PM, Tim Newsham wrote:
>> I'm not sure either latency or RT is proper terminology here. But
>> I believe what I meant was clear: when you need overall latency
>> to be around 5ms you start to notice 9P.
>
> It sounds like you have a specific app in mind, and a real-time
> one at that.  If you're using your audio device for live audio
> (ie. adding effects to audio from your guitar)
> you need pretty small latency.  You can go a bit higher than
> 5ms without noticing, though.

Not so much a special application (although I was doing VJing
at the time) but rather [multitrack] recording.

> Most apps dont require realtime.  For example, streaming a song.
> You dont care if the samples show up at year ear 2seconds
> after they were sent from your hard drive, so long as all the
> samples are delayed by the same amount..  You can stream this
> clear across the country over all kinds of cut rate ISPs and
> still get satisfactory results with enough buffering..

Sure. But if all you're aiming at with this thread is a driver that
supports simple audio playback, I guess I misread the subject
line.

Thanks,
Roman.

Re: [9fans] audio standards -- too many to choose from

[David Leimbach]

[-- Attachment #1: Type: text/plain, Size: 1204 bytes --]

On Tue, Aug 11, 2009 at 10:10 PM, Tim Newsham <newsham@lava.net> wrote:

> I'm not sure either latency or RT is proper terminology here. But
>> I believe what I meant was clear: when you need overall latency
>> to be around 5ms you start to notice 9P.
>>
>
> It sounds like you have a specific app in mind, and a real-time
> one at that.  If you're using your audio device for live audio
> (ie. adding effects to audio from your guitar)
> you need pretty small latency.  You can go a bit higher than
> 5ms without noticing, though.
>
> Most apps dont require realtime.  For example, streaming a song.
> You dont care if the samples show up at year ear 2seconds
> after they were sent from your hard drive, so long as all the
> samples are delayed by the same amount..  You can stream this
> clear across the country over all kinds of cut rate ISPs and
> still get satisfactory results with enough buffering..


At which point that constant latency could be considered just a transmission
delay :-).

VoIP seems to give consistent latency too, but latency for sure.... and it
works just fine.


>
>
>  Roman.
>>
>
> Tim Newsham
> http://www.thenewsh.com/~newsham/
>
>

[-- Attachment #2: Type: text/html, Size: 1964 bytes --]

Re: [9fans] audio standards -- too many to choose from

[Anthony Sorace]

i think russ has it exactly right: keep the kernel driver as simple as
is practical, do whatever else you want in user space. for /dev/audio,
i wouldn't suggest anything beyond plan 9's audio(3) as is. i'd
suggest some cleanup of the surround (kill /dev/volume, rationalize
/dev/audioctl), but the fundamentals are right.

having a user mode fs on top of that to do easy, standardized format
conversion would be pretty neat, and i wonder if it could be provided
in a backwards-compatible way without too many contortions. if we're
only being additive...

maybe not, though. maybe just keep the names clear and avoid the
compatibility trap all together. i don't exactly follow erik's upas/fs
analogy, but i do think a fs of sorts is very interesting. i think
that approach also lends itself well to mixers, which is a related set
of questions. i tend to think of it more analogous to /net than
upas/fs. "cat whatever > /mnt/mixer/new"?

Re: [9fans] audio standards -- too many to choose from

[Skip Tavakkolian]

> I'm not sure either latency or RT is proper terminology here. But
> I believe what I meant was clear: when you need overall latency
> to be around 5ms you start to notice 9P.

it needs to be isochronous.

Re: [9fans] audio standards -- too many to choose from

[erik quanstrom]

On Thu Aug 13 02:43:54 EDT 2009, 9nut@9netics.com wrote:
> > I'm not sure either latency or RT is proper terminology here. But
> > I believe what I meant was clear: when you need overall latency
> > to be around 5ms you start to notice 9P.

when you need the overall latency to be around 5ms,
aren't you need a suitable network?  (or none at all)

> it needs to be isochronous.

i believe it has that capability.  just keep multiple tags
outstanding.

- erik

Re: [9fans] audio standards -- too many to choose from

[Skip Tavakkolian]

>> it needs to be isochronous.
>
> i believe it has that capability.  just keep multiple tags
> outstanding.

at the device it needs to be isochronous; so if it's going over the
wire, you need to build some elasticity in.

or as media players would say: [ buffering... buffering... ] ☺

Re: [9fans] audio standards -- too many to choose from

[Devon H. O'Dell]

2009/8/13 erik quanstrom <quanstro@quanstro.net>:
> On Thu Aug 13 02:43:54 EDT 2009, 9nut@9netics.com wrote:
>> > I'm not sure either latency or RT is proper terminology here. But
>> > I believe what I meant was clear: when you need overall latency
>> > to be around 5ms you start to notice 9P.
>
> when you need the overall latency to be around 5ms,
> aren't you need a suitable network?  (or none at all)

When you're doing audio work, latency is an issue on playback when
you're playing with a recording. 5ms is acceptable, but getting much
higher is not. However, one of the other key points is that a
consistent latency is important. 50ms might be fine for playback with
video if it's a constant 50ms, but if you start drifting +- 10ms,
things can get wonky pretty quickly. This is easily demonstrable with
rhythm games (such as Rock Band or Guitar Hero) where latency induced
by a home audio system (mine at home is about 15ms induced by my
receiver and 5ms using the Xbox digital output) can have a very
significant negative impact on gameplay when one plays primarily by
sound. (Sound cues are easier to keep beat by than visual).

--dho

Re: [9fans] audio standards -- too many to choose from

[Anthony Sorace]

Devon H. O'Dell<devon.odell@gmail.com> wrote:
// This is easily demonstrable with rhythm games (such as Rock
// Band or Guitar Hero) where latency induced by a home audio
// system (mine at home is about 15ms induced by my receiver
// and 5ms using the Xbox digital output) can have a very
// significant negative impact on gameplay when one plays
// primarily by sound.

It's worth noting that the differences injected by typical home
components are significant enough that, at least in all the
versions of these games I've played, the designers have built in
a way for you to calibrate the audio against the video.

// (Sound cues are easier to keep beat by than visual).

The Rock and Roll Hall of Fame has a Guitar Hero set up in the
lobby, but you need to bring your own headphones. I didn't have
any on me, so tried playing by sight only. It went really poorly.

Re: [9fans] audio standards -- too many to choose from

[Devon H. O'Dell]

2009/8/13 Anthony Sorace <anothy@gmail.com>:
> Devon H. O'Dell<devon.odell@gmail.com> wrote:
> // This is easily demonstrable with rhythm games (such as Rock
> // Band or Guitar Hero) where latency induced by a home audio
> // system (mine at home is about 15ms induced by my receiver
> // and 5ms using the Xbox digital output) can have a very
> // significant negative impact on gameplay when one plays
> // primarily by sound.
>
> It's worth noting that the differences injected by typical home
> components are significant enough that, at least in all the
> versions of these games I've played, the designers have built in
> a way for you to calibrate the audio against the video.

Indeed. Early versions of the game only allowed you to do audio lag
calibration and made no video compensation (see: Guitar Hero 1 and 2;
Guitar Hero 3; Rock Band was the first to introduce both). This ended
up making the game even harder, because the gameplay is really a
combination of audio and video cues.

> // (Sound cues are easier to keep beat by than visual).
>
> The Rock and Roll Hall of Fame has a Guitar Hero set up in the
> lobby, but you need to bring your own headphones. I didn't have
> any on me, so tried playing by sight only. It went really poorly.

Even though I'm fairly adept at these games (probably top 100ish on
drums and I used to be top 50 on guitar; I'm much worse at that now),
playing on mute is noticeably more difficult. A good friend of mine
who was #1 on guitar for quite some time was able to do it almost
effortlessly, though. If you can keep time well, it's easy. However,
that assumes no video latency. If there is video latency, it sucks.

--dho

Re: [9fans] audio standards -- too many to choose from

[hiro]

> The Rock and Roll Hall of Fame has a Guitar Hero set up in the
> lobby, but you need to bring your own headphones. I didn't have
> any on me, so tried playing by sight only. It went really poorly.

Our visual perception is very unreliable, whereas our acoustic timing
can be very accurate.

Re: [9fans] audio standards -- too many to choose from

[hiro]

> 9p is a ping-pong protocol.  this gives it *consistent* latency.
> this is good for audio.

Some years ago when I set up the audio stuff in my house I had to
solve the task of streaming the output from mpd (a linux audio player)
running on my file server to the sound card in my room. I couldn't
find any low latency linux software, so I used inferno to export a pcm
fifo file. Mounting this I have stable <50ms end-to-end delay over
wireless, haven't found any easier/better working alternative yet.
It shows, the plan9/inferno audio implementation is working out great
so far for a general purpose os.
But if you want less latency I don't think 9p or even tcp is the way
to go. This is why people use udp and all those "simple" protocols for
streaming.

Re: [9fans] audio standards -- too many to choose from

[Lawrence E. Bakst]

You really have to ask yourself what are the goals for a new audio system and what "use cases" do you want to cover. I have some experience in this area, but I'm not a true expert. Here are 5 thoughts to be considered and an anecdote:

1. audio
I thought Russ's response was pretty good and about what I would do as a first step. "High end" or "Pro" audio is more than 16 bit these days, although 16 bits probably covers at least 80 or 90% of the "the market".
http://pro.sony.com/bbsc/ssr/cat-audio/cat-recorders/product-PCMD1/
http://pro.sony.com/bbsc/ssr/cat-audio/resource.latest.bbsccms-assets-cat-audio-latest-pcmd50.shtml

2. clocks
This is the most important thing to think about. What is your model for how the clocking will work? Beyond the simple case that Russ outlined of sending data to an old 16 bit Sound Blaster lay more complex cases that have to do with audio data and its associated clocks. If you desire to do complex things with audio at some point you will have to consider clocking.

3. video (when I say video I mean movies and not graphics)
If you think you are ever going to want to use your new audio system with a corresponding video system, you need to consider that from the outset. Audio and video need to be kept in sync with a common clock and perhaps adjustments need to be made for different latencies through each path. The lack of sync between audio and video that is so common today is a direct result of engineers that either don't understand the clocking issues, got it wrong, or ignored it. I see this over and over again. See the anecdote below.

4. muxing
Once you have audio and video the next thing you have to worry about is muxing. I think muxing can be kept out of the kernel but there are subtle interactions with clocking and other device stacks (see below) so I am not so sure.

5. complexity and modern life
It's one thing to blast some audio out to an old 16 bit Sound Blaster. It's another thing to capture A/V data from a Firewire camcorder and run the audio out to a set of powered USB speakers and the video to the screen or someplace else and have everything be in sync and work properly.

anecdote:
I once tried to fire an engineer that was writing and A/V driver for Windows. The CEO wouldn't let me. The engineer almost had it working but the root cause of all the audio artifacts from that driver was caused by his code keeping two separate clocks in software, one for video and one for audio. He was told that, told which lines of code to change, and even how to change them, but he refused to make the changes. He got so stressed out over all this that he walked off the job. He literally said, "I am going for out for a haircut" and never came back. We changed about 10-15 lines of code in his driver to derive one clock from the other and it all worked perfectly. Talk about timing.
--
leb@iridescent.org

Re: [9fans] audio standards -- too many to choose from

[Roman Shaposhnik]

On Aug 11, 2009, at 10:54 PM, Lawrence E. Bakst wrote:
> 3. video (when I say video I mean movies and not graphics)
> If you think you are ever going to want to use your new audio system
> with a corresponding video system, you need to consider that from
> the outset. Audio and video need to be kept in sync with a common
> clock and perhaps adjustments need to be made for different
> latencies through each path. The lack of sync between audio and
> video that is so common today is a direct result of engineers that
> either don't understand the clocking issues, got it wrong, or
> ignored it. I see this over and over again. See the anecdote below.

Unless you're doing pro stuff, it is pretty easy to treat audio as the
master clock
and adjust video playback accordingly.

Or am I not following?

Thanks,
Roman.

Re: [9fans] audio standards -- too many to choose from

[Tim Newsham]

Still would love to hear if anyone knows the answer to these:

> - What software exists for each of these formats?
> - Which format is the most "popular"?

Tim Newsham
http://www.thenewsh.com/~newsham/

Re: [9fans] audio standards -- too many to choose from

[Bakul Shah]

On Wed, 12 Aug 2009 09:50:13 -1000 Tim Newsham <newsham@lava.net>  wrote:
> Still would love to hear if anyone knows the answer to these:
>
> > - What software exists for each of these formats?

Are you asking about non p9 software? If so, have you looked
at SoX (Sound eXchange)? It is sort of like netpbm but for
audio formats. http://sox.sourceforge.net/

Re: [9fans] audio standards -- too many to choose from

[Roman Shaposhnik]

On Aug 12, 2009, at 12:50 PM, Tim Newsham wrote:
> Still would love to hear if anyone knows the answer to these:
>
>> - What software exists for each of these formats?

If you are asking about non Plan9 software I'd start with
ffmpeg.

>>
>> - Which format is the most "popular"?

I don't think I understand the question.

Thanks,
Roman.

Re: [9fans] audio standards -- too many to choose from

[Tim Newsham]

>>> - What software exists for each of these formats?
>
> If you are asking about non Plan9 software I'd start with
> ffmpeg.
>
>>> - Which format is the most "popular"?
> I don't think I understand the question.

Sorry, let me rephrase:
   - Of the different audio driver interface designs
     (audio(3), usb(4) and inferno usb(3)) which software
     (p9 and limbo) uses each?
   - Which of these interfaces is used the most?

> Roman.

Tim Newsham
http://www.thenewsh.com/~newsham/

Re: [9fans] audio standards -- too many to choose from

[Devon H. O'Dell]

2009/8/12 Tim Newsham <newsham@lava.net>:
>>>> - What software exists for each of these formats?
>>
>> If you are asking about non Plan9 software I'd start with
>> ffmpeg.
>>
>>>> - Which format is the most "popular"?
>>
>> I don't think I understand the question.
>
> Sorry, let me rephrase:
>  - Of the different audio driver interface designs
>    (audio(3), usb(4) and inferno usb(3)) which software
>    (p9 and limbo) uses each?
>  - Which of these interfaces is used the most?

I don't know which is used the most, but I don't think the Plan 9 ones
of them make particularly good sense to support multiple input formats
to multiple output formats. sox was mentioned here recently, and is a
great utility for doing the conversion. If we want a generic, reusable
audio layer, to *me* the Inferno one is best with:

audio
audioctl

It doesn't make sense to export files for volumes, codes, channels,
and other settings to me. This is because (at least with HDA cards)
you would end up with 5 bajillion files for controlling volume on each
individual channel. You'd end up with another file for reading/writing
codec settings. You'd end up with a file saying whether you preferred
digital output or analog output.

OSS relies heavily on ioctls for setting these things, but for good
reason. In our case, a standardized set of strings for the ctl file
seems best to me. If I want to change master volume, echo master
255.255 > /dev/audioctl. If I want to set digital out, echo output
channel digital > /dev/audioctl. etc. A more spread out filesystem
would make the ctl handler smaller, but would not reduce the amount of
code needed to support mixers / codecs / channels / whatnot (In fact,
you'd just have more code because you'd have to have functions for
reading/writing those files). Also, then you just need to come up with
strings -- you're just bit frobbing these things, and more complicated
filesystem hierarchy doesn't help explain it any better. echo 255 >
/dev/audio/mixer/channel/master/right; echo 255 >
/dev/audio/mixer/channel/master/left; just seems obtuse, and, as I
said, it's just adding more redundant code anyway.

To then play sound, you would probably have a sox-like converter
sitting on top of that (maybe even on top of /dev/audio?) that takes
input of a certain format and does either minimal conversion (i.e. a
card that supports XA ADPCM taking input from a playstation 2 sound
file not having any conversion at all, WAV going to the proper
byte-order PCM with the header stripped off, etc), or highest
resolution audio available (e.g. 5.1 flac getting converted to
whatever codec supports 5.1 audio).

Thoughts?

--dho

Re: [9fans] audio standards -- too many to choose from

[Christopher Nielsen]

Been reading the thread with interest, and I finally have a moment to comment.

I was thinking about this several years ago when I had a lot of spare
time on my hands and wanted to rethink and update the audio interface,
and I think a lot of what you are suggesting sounds similar to the
conclusions I had come to during the design phase. Comments inline.

On Thu, Aug 13, 2009 at 10:00, Devon H. O'Dell<devon.odell@gmail.com> wrote:
> 2009/8/12 Tim Newsham <newsham@lava.net>:
>>>>> - What software exists for each of these formats?
>>>
>>> If you are asking about non Plan9 software I'd start with
>>> ffmpeg.
>>>
>>>>> - Which format is the most "popular"?
>>>
>>> I don't think I understand the question.
>>
>> Sorry, let me rephrase:
>>  - Of the different audio driver interface designs
>>    (audio(3), usb(4) and inferno usb(3)) which software
>>    (p9 and limbo) uses each?
>>  - Which of these interfaces is used the most?
>
> I don't know which is used the most, but I don't think the Plan 9 ones
> of them make particularly good sense to support multiple input formats
> to multiple output formats. sox was mentioned here recently, and is a
> great utility for doing the conversion. If we want a generic, reusable
> audio layer, to *me* the Inferno one is best with:
>
> audio
> audioctl

I think this makes the most sense for the device driver interface. It's simple.

> It doesn't make sense to export files for volumes, codes, channels,
> and other settings to me. This is because (at least with HDA cards)
> you would end up with 5 bajillion files for controlling volume on each
> individual channel. You'd end up with another file for reading/writing
> codec settings. You'd end up with a file saying whether you preferred
> digital output or analog output.
>
> OSS relies heavily on ioctls for setting these things, but for good
> reason. In our case, a standardized set of strings for the ctl file
> seems best to me. If I want to change master volume, echo master
> 255.255 > /dev/audioctl. If I want to set digital out, echo output
> channel digital > /dev/audioctl. etc. A more spread out filesystem
> would make the ctl handler smaller, but would not reduce the amount of
> code needed to support mixers / codecs / channels / whatnot (In fact,
> you'd just have more code because you'd have to have functions for
> reading/writing those files). Also, then you just need to come up with
> strings -- you're just bit frobbing these things, and more complicated
> filesystem hierarchy doesn't help explain it any better. echo 255 >
> /dev/audio/mixer/channel/master/right; echo 255 >
> /dev/audio/mixer/channel/master/left; just seems obtuse, and, as I
> said, it's just adding more redundant code anyway.

I agree with this, too. Keep the interface simple and make the
audioctl file smart based on the type of audio card you have. A
complex file hierarchy just doesn't feel right to me. The simplest
interface you can get away with is generally the most powerful.

> To then play sound, you would probably have a sox-like converter
> sitting on top of that (maybe even on top of /dev/audio?) that takes
> input of a certain format and does either minimal conversion (i.e. a
> card that supports XA ADPCM taking input from a playstation 2 sound
> file not having any conversion at all, WAV going to the proper
> byte-order PCM with the header stripped off, etc), or highest
> resolution audio available (e.g. 5.1 flac getting converted to
> whatever codec supports 5.1 audio).

I haven't read up on sox, but I think the converter should be a file
server sitting on top of /dev/audio.

-- 
Christopher Nielsen
"They who can give up essential liberty for temporary
safety, deserve neither liberty nor safety." --Benjamin Franklin

Re: [9fans] audio standards -- too many to choose from

[erik quanstrom]

> Personally I can't hear over 9119 hz (audio), but I might want to record
> 1s of 192Khz (samples I presume) and stretch them by 100x to 9600 hz
> (audio) and have a (possibly) interesting time listening to the results
> without interpolating.

perfect customer for the Phone Company!  ☺

- erik