Jump to content
  • entries
    14
  • comments
    81
  • views
    38,182

racing the beam and riding the speaker

Sign in to follow this  
RevEng

1,770 views

stella's voice

 

The 2600 isn't widely hailed for it's luxurious sound.

 

The TIA chip, which is responsible for the sound of the 2600 as well as the video, was designed with fairly minimal audio capabilities. On it's own it has the ability to make fairly simple tones, rumbles, and white noise type sounds.

 

Similar to how you need to "race the beam" to draw anything useful with a 2600, you also need to "ride the speaker" to add interesting audio texture to your sound effects, changing the frequency and volume content as the sound plays.

 

Further complicating the business of making interesting sounds is the fact that there are only a handful of frequencies to pick from in TIA's upper range, stemming from TIA's base-frequency-divider design. If you want a high pitched sound that shifts around, the transitions just aren't going to sound smooth.

 

One of the biggest hurdles to sound-coding on the 2600 surprisingly comes from wetware limitations, not hardware inadequacies. Our brains treat a sound more or less as a whole unit, rather than a collection of individual frequencies. Even when playing back a sound over and over, most people will have difficulty naming more than basic changes in the fundamental frequency, and even then they can do so only over a relatively large timescale.

 

In this sense, it's more intuitive for us to race the beam than it is to ride the speaker.

 

samples

 

One solution to this problem is playing back a sound sample. That way the coder remains blissfully unaware of the frequency content, but still can reproduce the sound.

 

The problem with sample playback is it needs to happen frequently and continuously. On the 2600 that translates to a complicated kernel design (or one that blanks while playing) and eats up valuable rom space.

 

I set out to investigate an alternative approach to sound reproduction by writing an FFT to TIA sound converter. The converter would generate lower-rate samples that would contain TIA frequency information. The plan was for this to be a kind of middle-ground between simple sound generation and expensive sample playback.

 

 

the converter

 

FFT stands for Fast Fourier Transform. Without going over a bunch of theory, it's a way of picking out what frequencies are present in a length of data, and what strengths they're present at.

 

So the converter opens the WAV file and performs a sliding window FFT on it - in other words, it figures out the frequencies are present in each 1/60th of a second chunk.

 

Then it looks at which frequency is loudest in each chunk, and finds a nearest TIA frequency match.

 

There are a few things that are wrong with this approach overall.

 

First, it completely ignores the fact that a sound is made up of more than one frequency. This makes the implementation easier, but limits the kind of sounds we can accurately represent.

 

Second, it uses square wave TIA frequencies to represent the sine waves in the FFT, which potentially changes the character of the sound a bit.

 

Third, it completely ignores the phase of the frequencies. Phase differences between related frequencies tend to provide a lot of texture to sounds.

 

But since my design goal wasn't to perfectly reproduce the sound, I figured it would be worth trying to see what kind of results I would get.

 

 

sound results

 

I've attached a zipfile below that contains 2600 binaries, batari Basic files, and the original WAV files. The 2600 binaries play the sounds on startup, and then play again if you press the fire button on the joystick.

 

The pacman-eats-a-ghost sample is pretty close to what I was hoping for as a result. Comparing the TIA sound playback in stella to the original WAV shows the converter nicely simplified the sound. It's not the same sound, but it's close in character.

 

Pacman-dies is interesting because it highlights TIA's weakness in the upper range.

 

The original sound sample starts off at a higher pitch, and our simplified tonal sample doesn't do a great job of representing it until the sound gets a fair bit lower, near the end of the sample.

 

Haha is a short sample of a laugh. The original sample had fundamental frequencies below the range of my FFT routine, so it picked up on upper harmonics. Because of that the result isn't accurate, but has a neat texture that would make a neat sound effect for an underwater animal or weird space alien. [edit - a low pass filter, as suggested by batari allowed the routine to find the lower fundamental!]

 

Dkintro is a sample of the Donkey Kong level intro tune. I didn't plan to use the converter as a tool for whole songs, but I think the result here shows that a specialized FFT-to-TIA-song-data converter might be feasible.

 

what next?

 

With a fair bit of extra code the converter should be able to compare FFT results with FFT data representing TIA's various possible outputs, instead of trying to match fundamental frequencies. This would allow the converter to better represent more complex sounds and noise.

 

It's a work in progress at this point, but if nothing else comes of it, hopefully I've inspired other 2600 coders to consider the texture of the sounds in their own creations. :)

 

FFT_to_TIA_results_2011.05.25.zip

 

[edit - low pass filtered laugh result added]

 

SFX-haha-lowpass.bin

Sign in to follow this  


10 Comments


Recommended Comments

I won't be able to try this out until later tonight, but I wonder why your FFT routine can't pick up fundamental frequencies below a certain point? The TIA is capable of low frequencies (I forget the AUDCx number, but the "Pitfall" tone can create a nice bass sound down to around 30 Hz.) Doing so could potentially create a richer sound rather than sounding tinny (I haven't heard it, but in my experience, representing only upper harmonics can do that.)

Share this comment


Link to comment

I'm no expert, but I'm pretty sure the lower-end frequency cutoff in an FFT is a function of the window size... A 1/60th second window can detect down to 120Hz, after which it's looking at partial waves, which will be interpreted as other frequencies.

 

I tried increasing the window size to 1/30th of a second, but the larger window smeared the dynamics in the resulting sound.

 

If I could get the lower frequencies in the FFT, there's no doubt TIA could play them back - plenty of frequencies to pick from in the lower end.

Share this comment


Link to comment

Hmmm, It looks like it isn't a cut-off issue.

 

I ran that same sample through Audacity's spectrum analysis, and it agrees more or less with the results from my app is outputting. But the original sounds like it has a much lower fundamental frequency.

 

I guess there's some phase interactions in the original sample that make it sound lower in pitch than it actually is, or maybe there's something else that's throwing both my and Audacity's analysis. :ponder:

Share this comment


Link to comment

I went looking for some FFT programs that would show the notes of something playing and they are fairly horrible:

 

http://www.rejc2.co.uk/programs/fft/

 

http://www.safe-install.com/programs/note-detector.html

 

http://www.pluto.dti.ne.jp/~araki/amazingmidi/

 

 

The first one I linked to is the best one I've found so far, but it looks like you have to close the program and open it every time you want to load a new wav file.

 

When you get done adjusting your FFT to TIA sound converter, I bet it will be better than those programs.

Share this comment


Link to comment

Nice Eric - I guess there really is nothing new under the sun! :D

 

I think with the addition of the other TIA waveforms the accuracy might be raised a bit... the trick is the FFT comparison code. Gonna have to think about that one for a bit.

 

RT - The nice part will be the ability to bypass the whole note conversion, and go directly to TIA values.

Share this comment


Link to comment
RT - The nice part will be the ability to bypass the whole note conversion, and go directly to TIA values.

That would be nice. If it will be able to output the values in a VbB Music and Sound Editor friendly way, it will be even nicer.

 

Single channel:

 

V,C,F

D

 

 

or both channels:

 

V,C,F (Channel 0)

V,C,F (Channel 1)

D

 

 

 

V - Volume (0 to 15)

 

C - Control [a.k.a. tone, voice, and distortion] (0 to 15)

 

F - Frequency (0 to 31)

 

D - Duration

Share this comment


Link to comment

In my blog archive you can find a fairly detailed description of the TIA audio generator which you can use to create precise waveforms. IIRC I kind used the waveforms like a frequency transform:

 

For each waveform & it's 90 degree phase-shifted version

Multiply the waveform against the block of samples (where bit=0 is *-1)

Calculate the RMS of the in phase & in quadrature

Compare the RMS for each waveform and select the two largest and calculate volumes

Repeat for next block of samples

 

In my case each block of samples was one TV frame long.

Share this comment


Link to comment

I had a chance to listen to these today, and that laugh clearly has a low fundamental frequency. The problem must be that it's a complex waveform, probably with some noise and other character. I tried messing around with the spectrum settings and found it does indeed have fundamentals around 100-130 Hz and harmonics, but either has a second fundamental around 1 kHz, or some noise that is adding to decayed harmonics which create an apparent peak here that has a larger amplitude.

 

Since we can't post pictures to blog comments, here's a screenshot of the analysis that I posted in the gallery, which may or may not work:

gallery_5792_375_72213.png

EDIT: I decided to run it through a low pass filter, 100 Hz cutoff, 12 dB per octave. Then, amplified by 39 dB. The laugh sounded similar as expected, but now has two lower peaks which might allow for a more effective conversion. I can't post a wav in the gallery, but I'm sure you could do the same in Audacity.

Share this comment


Link to comment

well done! The low pass filter as you described allowed the converter to find the lower fundamental.

 

The result is more accurate from a lower end point of view, but less interesting in texture... I think the original sample will be an interesting test for the next iteration.

 

I've attached the low-pass results to the bottom of the first post.

Share this comment


Link to comment
Guest
Add a comment...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
  • Recently Browsing   0 members

    No registered users viewing this page.

×
×
  • Create New...