You are not logged in.
Lost Password?


Register To Post



 Bottom   Previous Topic   Next Topic

#1
PCM Audio And You
Posted on: 2013/5/12 13:52

Joined 2012/12/4
-1 Posts
CoderLong Time User (7 Years)
PCM, the common abbreviation of Pulse Code Modulation, is the simplest way to represent an audio stream digitally. It's effectively a two-dimensional graph, where the vertical axis is value and the horizontal axis is time. By plotting data points along this graph, called samples, a wave form can be expressed as data and later used in computer audio systems.

PCM on Virtual Boy

First things first, Virtual Boy was not designed with PCM audio in mind. It was designed as a feature-rich "chiptune" generator, so all PCM techniques are dark wizardry and not intended use of the hardware. That's not to say we can't do it, though.

The most obvious solution is to rapidly update the Virtual Boy's wave memory as a sort of small audio buffer. Unfortunately, this isn't feasible for two reasons:

* Wave memory cannot be updated while sound is being generated, even by channels that are using different wave buffers. Yeah. Updating wave memory, therefore, absolutely necessitates stopping audio entirely, which produces a dead spot every 32 samples.

* Virtual Boy samples are unsigned, meaning the minimum value is 0. The aforementioned dead spots, then, drop the output level to 0. Why does this matter? It matters because PCM streams situate "silence" in the middle of their range, which for Virtual Boy would be 31 or 32. If samples less than "silence" can be considered negative, then 0 is the maximum negative value: it will produce a high-frequency, low duty cycle pulse every time wave memory is updated.

If only Virtual Boy allowed you to update wave memory while sound is being generated... You know, like Mednafen does! But alas, it does not. However, there is an alternative.

Software Modulation

So using wave memory as PCM buffers might be a wash, but that doesn't mean we're out of options. At the end of the day, what we want is to modify the output level of the audio being generated, and modify it at some predetermined interval. As long as all the ups and downs occur at the right moments, that's what counts.

Here's what we can do: take only one wave pattern, and initialize all of its samples to 63, the maximum positive value. Then fire up a sound channel using that wave, with any frequency, and let it fly. This will produce a constant high signal, which is inaudible, but it lets us do funky stuff with it.

The high-frequency hardware timer can be used to schedule changes to the volume level of that sound channel. The channel never stops emitting its constant high signal; we simply change on our own exactly how loud that signal is. If we do this fast enough, we can replicate the effects of PCM audio, and all on hardware that doesn't natively support PCM.

The 16-Level Approach (4-bit)

I strongly suspect this is what Galactic Pinball does. But I haven't actually seen it in action to verify one way or the other. But hey, it's an option!

The volume level of a sound channel can be independently configured for both the left and right speakers. These levels are 4-bit, meaning there are 16 volume levels, with 0 being silence.

If a PCM stream is also 4 bits per sample, then it's a simple matter of changing the volume of a sound channel each time. This works for both mono and stereo streams, and it couldn't be easier!

The 46-Level Approach

4 bits might be fast and easy, but it's also rather restrictive. Not that sounds are bad at 4 bits, but they certainly aren't good either. Fortunately for us, we can step it up a bit.

A single sound channel can output levels from 0 to 15. Two sound channels mix together through addition. So using a second sound channel, we add levels 16 through 30. And using three channels, we add levels 31 through 45. Modifying three channels is much the same as modifying one; we just move onto the next channel if the value is above 15, or above 30.

Slight problem, though: Virtual Boy only has 5 wave channels, and we'd need 6 for 46-level stereo. We luck out again, because the noise channel can actually be used as that sixth channel. See, the way the noise algorithm works, the first 7 samples generated during a sequence will always be 63, the maximum value. And the sequence is reset whenever the tap location is configured. So if we just set the tap location every time we update the volume, we get a constant high signal out of the noise channel!

The 181-Level Approach

If you don't need stereo, you can use all 6 sound channels with additive mixing for a single audio stream. By adjusting volume alone, you have 181 levels of output, and they can get quite loud.

The 256-Level Approach (8-bit)

Sound channels have more than just volume: they also have an envelope setting. The envelope value is also 4-bit, so 0 to 15 for that one as well. When a sample is generated, the volume and the envelope are multiplied with one another, producing the output sample. This can range from 0 * 0 = 0, to 15 * 15 = 225.

The full list of unique products is attached to this post as Produts.txt.

As it turns out, there's only 90 of those. That's wedged between 6 and 7 bits of data, and they're not evenly-spaced. If you plot them out, the curve is roughly exponential. Not all that useful in its current form.

If you use two sound channels, on the other hand, it's like taking two of these products and adding them together. And if you do that, you can make a whole slew of different values. Every level from 0 to 255 can be expressed as some sum of two sound channels.

The full list of sums, from 0 to 255, is attached to this post as Sums.txt.

Let's take one as an example:

* Sample: 254
* Volume 1: 15
* Envelope 1: 14
* Volume 2: 11
* Envelope 2: 4

We know from earlier that each sound channel outputs a level that is the product of its volume and its envelope. That said, channel 1 here will output 15 * 14 = 210, and channel 2 will output 11 * 4 = 44. When mixed, the combined output will be 210 + 44 = 254.

By using 4 sound channels in this manner, you can have true 8-bit stereo coming out of the speakers.

Static!

While the high-pitched whine of the 0-based dead spots aren't a problem with using software modulation, static certainly is. I haven't put a great deal of research into why this happens, but I suspect it has more to do with the audio data I've been using and less to do with the techniques of software modulation. All I know is that when I did the 16- and 256-level methods, the static was prohibitively audible, but the 46-level method sounded okay. Go figure.

With some additional testing and experimentation, I'm hoping that we'll find that the 256-level approach is fully adequate for general-purpose PCM output on Virtual Boy.

Demo

Attached to this post are the following files:

* AtlasParkSector.ogg - A sound clip from City of Heroes
* pcm_46.zip - Contains a Virual Boy ROM showcasing the 46-level PCM technique
* pcm_46.ogg - A recording of that program, run on the actual hardware.

Mednafen has different timing than the Virtual Boy hardware, so its output will not be identical. You can adjust the delay between samples by pressing Up and Down on the left D-Pad.

Attach file:


txt Products.txt Size: 1.89 KB; Hits: 163
txt Sums.txt Size: 11.55 KB; Hits: 165
zip pcm_46.zip Size: 266.81 KB; Hits: 163
ogg AtlasParkSector.ogg Size: 396.23 KB; Hits: 164
ogg pcm_46.ogg Size: 407.14 KB; Hits: 177
Top

#2
Re: PCM Audio And You
Posted on: 2013/5/12 14:47

Joined 2007/10/26
USA
-1 Posts
ContributorTop10 PosterLong Time User (12 Years) Donator
Wow that is impressive. Quite a bit over my head..but the demo is extremely impressive.

Can these approaches also recreate voice ?

-Eric
Top

#3
Re: PCM Audio And You
Posted on: 2013/5/12 18:09

Joined 2012/12/4
-1 Posts
CoderLong Time User (7 Years)
After consulting with the scriptures, it would seem that 8-bit PCM doesn't want to happen.

* First of all, as it turns out, the product of volume * envelope actually loses its lower 3 bits. If not both of the operands were 0, then 1 is added to the result. This drops the total number of valid products from 90, which I thought it was, down to 27 on the hardware.

* It's still possible to generate 256 evenly-spaced levels, using all 5 wave patterns and combinations of volume and envelope. But the process of setting all of those values on the sound channels at run time is even more prohibitive than my previous attempt: the static almost completely drowns out the PCM clip.

So as far as 8-bit audio is concerned, bus latency is the limiting factor. It simply takes too long to set that many properties on the sound channels that the time between sets becomes audible.

Analyzing my resampled source clips, I discovered that the 16- and 46-level runs did indeed contain static in the source audio, and that it wasn't being caused by playing the sounds on Virtual Boy. I unfortunately haven't been able to craft a noiseless version of Atlas Park Sector for demonstration (once converting into 46-level samples), but other, less-noisy files sounded great.

All that being said, what should be possible is to generate sounds and music notes at run-time using a wave table, and producing sounds in that fashion is much more resilient when it comes to noise production. That's an experiment for another day, but I'm going to go on record saying that it IS possible to do noiseless PCM in this fashion, and doing so is a feasible means of providing PCM audio to games.
Top

#4
Re: PCM Audio And You
Posted on: 2013/5/12 22:04

Joined 2007/12/14
-1 Posts
CoderLong Time User (12 Years) App Coder
Quote:

Guy Perfect wrote:
We luck out again, because the noise channel can actually be used as that sixth channel. See, the way the noise algorithm works, the first 7 samples generated during a sequence will always be 63, the maximum value. And the sequence is reset whenever the tap location is configured. So if we just set the tap location every time we update the volume, we get a constant high signal out of the noise channel!


Nice. I had not realized this. Could definitely do six independent PCM voices then!
Top

#5
Re: PCM Audio And You
Posted on: 2013/5/13 1:09

Joined 2012/12/4
-1 Posts
CoderLong Time User (7 Years)
So I've been scrounging around on the internet to find out the mysterious source of the staticy noise in my sound clips. And I've found what it is, what causes it, and to an extent, what to do about it!

When reducing the sample size of a stream, a process called quantization, you won't always be able to land in good spots. Drawing samples down from 16 bits to 8 bits tends to work pretty well, since there's an evenly-spaced multiple there. But in this case, I'm drawing a 65536-level sample down to a 46-level sample, so there's going to be a lot of "in betweens" that get messed up due to rounding. This messed-up-ness is referred to as the quantization error.

There's a method called noise shaping that takes the quantization error for each sample in a stream and feeds it back into the input prior to quantization, which supposedly helps reduce the apparent noise in the output. Supposedly. I found all sorts of sources that were all "You can use it as feedback yaaaay!" but not a one of them told me how to apply it, where the sample rate figures in, etc. If anyone knows more about noise shaping, I'd love to have a pointer or two.

The other thing that will help, especially in conjunction with noise shaping, is to use a low-pass filter to blot out the noise in frequencies above the maximum signal frequency. Noise shaping, I'm told, can shove the quantization noise up into the high ranges, meaning a low-pass filter can snip it right off at the neck.

But that's neither here nor there. I won't be able to progress until I get some form of noise shaping working.

Quote:

blitter wrote:
Nice. I had not realized this. Could definitely do six independent PCM voices then!

If you control a PCM stream, you can put as many voices as you want in it. You won't need multiple sound channels in that regard. I'm only using them here to increase my sample size.
Top

#6
Re: PCM Audio And You
Posted on: 2013/5/13 5:12

Joined 2007/12/14
-1 Posts
CoderLong Time User (12 Years) App Coder
Quote:

Guy Perfect wrote:

Quote:

blitter wrote:
Nice. I had not realized this. Could definitely do six independent PCM voices then!

If you control a PCM stream, you can put as many voices as you want in it. You won't need multiple sound channels in that regard. I'm only using them here to increase my sample size.


Doing that in real-time though will significantly bump up the CPU usage, right? Probably better to leverage the VB's mixing hardware as much as possible. Six channels of 4-bit sound looks to me to be the most efficient way of doing multi-channel PCM audio on the VB.
Top

#7
Re: PCM Audio And You
Posted on: 2013/5/13 9:58
PVB Elite
Joined 2003/7/25
USA
1507 Posts
PVBCC 1stCoderContributor#3 PosterHOTY09 EntryLong Time User (15 Years) App CoderPVBCC 2010 EntryPVBCC 2013 Entry
Nice brain dump. A few comments:

First... yes, I believe Galactic Pinball and the T&E Soft games used the 4-bit approach, as does my wav converter.

Regarding the 46-level... you're saying you can get 46 levels in 3 channels, but you need 6 channels to do 46 levels? I quickly skimmed through it, so maybe I'm missing what you're saying, but are you saying because of stereo, you're using 3 channels for each? Why wouldn't you use the L/R levels for stereo (you get 4 bits for each channel independently, right)?

I took a different approach for more bits when I was experimenting with PCM years ago. I used varying levels of the waveform RAM, since 63 is (basically) 2x of 32, which is 2x of 16, and so on. I liked the waveform RAM, because it's later in the chain, so all 6-bits get applied (though they could get chopped later if the level is too low).

I tried a couple different methods... one was a dynamic method using a single channel, and always outputting just 5 bits (4-bits L/R, plus 1 bit of ENV), but sliding which 5 bits were output (for example, if the sample was 5, it'd output the lower 5 bits from the low-bit RAM... but if the sample was 250, it'd output the upper 5 bits from the high-bit RAM). This was pretty cool, because it gave more overall dynamic range, but still used the simple 5-bit resolution. Of course you could round, rather than just masking the bits, but that's more computation (it'd be better to do that ahead of time in the conversion process). The disadvantage of this is the small error, that it really only works for mono (stereo would need a second channel), and that it uses several waveform RAM slots.

The other method was using multiple channels with multiple RAM slots. This way is nice, because you CAN use it with stereo, and you can easily get more bits. Set the first channel's waveform to 63, and the second channel's waveform to 4... put the upper 4 bits in the first waveform and the lower 4 bits in the second channel. I guess if you really cared about perfection, you could do 32 and 2, but your low bits may get wiped away if you go too low.

The problem with any PCM though is that it's processor intensive, and ROM hungry. It's cool for short effects, where nothing else is happening... like "T&E Soft Presents", but it's not really practical for music or in-game special effects. I stuck with 4-bit for the wav converter because the sound is reasonable, you can do 2 samples/byte (5KB/s @ 10KSps mono), and it's (relatively) easy on the processor.

BTW, are you sure that the RAM is unsigned? It's been a long time since I've looked at any of this sound stuff (I'm pretty much going by memory on all of it), but I seem to remember looking at the output on the scope, and determining that it was signed. But I could be mistaken... IIRC, the dev manual didn't specify signed or unsigned (which is why I checked it out in the first place). If I could just find half my notes and test code, I'd probably have some better input. ;)

Oh, and I'm not sure that I'd jump to blame quantization error for the static sound. I'd run a few tests first. A simple one would be to make a wav file on your computer with the same 46 levels and see how it sounds. Then convert that, which should (in theory) map perfectly to the VB. Does it sound identical? Also, I'd try generating a tone and look at it on a scope to inspect the quality of the signal (does it look sinusoidal, are the samples evenly spaced, is there any strange behavior). Looking at the spectrum of that signal would probably tell you a lot as well.

DogP
Top

#8
Re: PCM Audio And You
Posted on: 2013/5/13 15:09

Joined 2012/12/4
-1 Posts
CoderLong Time User (7 Years)
Quote:
DogP wrote:
Regarding the 46-level... you're saying you can get 46 levels in 3 channels, but you need 6 channels to do 46 levels? I quickly skimmed through it, so maybe I'm missing what you're saying, but are you saying because of stereo, you're using 3 channels for each? Why wouldn't you use the L/R levels for stereo (you get 4 bits for each channel independently, right)?

You're right. My brain was still in "mutually exclusive" mode, which necessitating isolating left and right to their own channels when modifying the envelope and/or wave parameters. If only volume is being adjusted, then you can certainly use one sound channel for both left and right, since the input sample is always the same. Using all 6 sound channels, this gives us 91 levels in stereo!

Quote:
DogP wrote:
The problem with any PCM though is that it's processor intensive, and ROM hungry. It's cool for short effects, where nothing else is happening... like "T&E Soft Presents", but it's not really practical for music or in-game special effects. I stuck with 4-bit for the wav converter because the sound is reasonable, you can do 2 samples/byte (5KB/s @ 10KSps mono), and it's (relatively) easy on the processor.

I wouldn't say it's processor-intensive or ROM hungry. You're making a couple of assumptions that don't have to be the case. (-:

The physical upper bound on PCM sampling rate for Virtual Boy is 41700hz, which is the sampling frequency of the VSU when converting to an analog signal. Timing techniques notwithstanding, let's look at that in terms of CPU power. The processor runs at 20000000hz, which is 479.6 times the maximum theoretical rate of the sound samples.

Now, how much CPU does it take to output a sample? Let's say we're using the 46-level approach on three sound channels (which is sufficient for stereo!). That's three writes to the VSU, plus a handful of overhead cycles to prepare the samples. Let's say, in some terrible worst-case scenario, a sample requires 50 cycles to produce. That still leaves us with 9.5 times the CPU rate compared to sampling rate, meaning 41700hz output is still ~10.4% CPU, which isn't bad.

41700hz is approaching the rate of CD audio, 44100hz. Even chopping that in half to 20850hz, thus halving the CPU usage, still produces a good sound. Using a simpler sampling method, like 31- or 16-level, will further reduce CPU usage.

As for being ROM hungry, what novice tries to use uncompressed wave files for music on a system like Virtual Boy? It'd be far more effective to use sampled instruments and generate a PCM stream at run-time using the other 90% of the CPU. (-:

Quote:
DogP wrote:
BTW, are you sure that the RAM is unsigned? It's been a long time since I've looked at any of this sound stuff (I'm pretty much going by memory on all of it), but I seem to remember looking at the output on the scope, and determining that it was signed. But I could be mistaken...

Absolutely. Here's a wave pattern I just tested:

Open in new window


And here's the signal that the hardware produced:

Open in new window


Quote:
DogP wrote:
Oh, and I'm not sure that I'd jump to blame quantization error for the static sound. I'd run a few tests first. A simple one would be to make a wav file on your computer with the same 46 levels and see how it sounds.

Jumping to conclusions would be foolish. That's why I made a wav file on my computer with the same 46 levels to see how it sounded. It contained the same static that I heard on the Virtual Boy. (-:
Top

#9
Re: PCM Audio And You
Posted on: 2013/5/13 16:30
PVB Elite
Joined 2003/7/25
USA
1507 Posts
PVBCC 1stCoderContributor#3 PosterHOTY09 EntryLong Time User (15 Years) App CoderPVBCC 2010 EntryPVBCC 2013 Entry
Quote:

Guy Perfect wrote:
I wouldn't say it's processor-intensive or ROM hungry. You're making a couple of assumptions that don't have to be the case. (-:

What about the game? Does that get 0% of the CPU? ;) Like I said... if you're not doing anything else, it works great, but I don't want to spend anywhere near 10% of the CPU on audio in a real application. And 50 cycles/sample would be optimistic IMO, if you count all the overhead involved (assuming it's actually playing a game, firing an interrupt ~40,000x per second)... and if you're generating the samples on the fly, obviously that's even more overhead. Plus, didn't you say your 8-bit didn't work, because it took too long? IMO, the advantage of PCM is for realistic sounds that wouldn't be generated on the fly, like speech... not for creating a better VSU using the CPU). But in the end, if you make a game that uses PCM for everything, and it works well... that's all that matters.

Quote:

Guy Perfect wrote:
Absolutely. Here's a wave pattern I just tested:

Ah, good... I must have been thinking of something else I was working on.

Quote:

Guy Perfect wrote:
Jumping to conclusions would be foolish. That's why I made a wav file on my computer with the same 46 levels to see how it sounded. It contained the same static that I heard on the Virtual Boy. (-:

If 46 levels doesn't have sufficient quality, why even consider it? How many bits do you need before you're happy with the sound? It sounds like that's the number you should be shooting for.

DogP
Top

#10
Re: PCM Audio And You
Posted on: 2013/5/13 16:54

Joined 2012/12/4
-1 Posts
CoderLong Time User (7 Years)
Quote:
DogP wrote:
What about the game? Does that get 0% of the CPU? ;) Like I said... if you're not doing anything else, it works great, but I don't want to spend anywhere near 10% of the CPU on audio in a real application.

100% - 10% > 0%, you know. I'm not sure what you're trying to say.

Quote:
DogP wrote:
And 50 cycles/sample would be optimistic IMO, if you count all the overhead involved (assuming it's actually playing a game, firing an interrupt ~40,000x per second)... and if you're generating the samples on the fly, obviously that's even more overhead.

In the program attached to the first post, I was using the hardware timer interrupt, which proved to be less than what it said on the label in terms of speed. With the interrupt breaking, stack management and other overhead, it took more than 20 microseconds to get in and out of the interrupt handler.

But that's if you're using the hardware timer. My next experiment won't be using the hardware timer. (And don't go conjecturing at me that THAT won't work. I'll do it, and then you'll have to come up with an explanation. (-: )

Quote:
DogP wrote:
Plus, didn't you say your 8-bit didn't work, because it took too long?

I said it couldn't update four sound channel properties for each of the left and right signals without audible side-effects. This is an entirely separate issue from the sampling rate.

Quote:
DogP wrote:
IMO, the advantage of PCM is for realistic sounds that wouldn't be generated on the fly, like speech... not for creating a better VSU using the CPU). But in the end, if you make a game that uses PCM for everything, and it works well... that's all that matters.

Ah ah ah, no conjecturing allowed. That is not the scientific method! If you're creative enough, you'd be surprised just what can be accomplished, even when others say it can't be done.

Quote:
DogP wrote:
If 46 levels doesn't have sufficient quality, why even consider it? How many bits do you need before you're happy with the sound? It sounds like that's the number you should be shooting for.

Did someone say 46 levels had insufficient quality while I wasn't looking?

46 levels has beautiful quality. Check the sound file I attached to this post. The only problem is that quantization noise. If I can eliminate that, we're in business.

Attach file:


ogg Route26_46_40000.ogg Size: 1,631.49 KB; Hits: 123
Top

 Top   Previous Topic   Next Topic


Register To Post