Jump to content
IGNORED

7800 TIA SFX Library, sound2tia


RevEng

Recommended Posts

I've expanded the library of sound effects that come with the 7800basic "soundtest" program. Currently there's 116 fairly distinct sound effects. 

 

In the meantime, I figured I'd give you guys a preview... so coders have something new to borrow sounds from, and so non-coders have a new 7800 bin to play around with for a bit of their Saturday evening.

 

soundtest.bas.a78

soundtest.bas.bin

 

 

(Source code is at the 7800basic github repo)

 

You can also check this out in your web browser (with the most-excellent JS7800 emulator) by following this link.

 

cc65 playback routines and some sample sounds can be found here!

 

Anybody is free to do what they like with these... use them for basic, assembly, port them to the 2600, or whatever. Even if you don't know 7800basic, the TIA data format is pretty obvious.

 

Part of the reason for the growth in sound effects in this library, is I was able to make some improvements in my wave2tia program. There are a few organic-sounding sfx here that were the result of these improvements. In the case of some others, I got a good result from the program, and hand tweaked what I wanted. Other times the result wasn't really accurate, but was hella neat anyway.

 

Warning: nerdy theoretic technical talk ahead...

 

While wave2tia can create some neat conversions, the FFT approach I took is doomed to mediocrity, due to some basics about FFT analysis that I didn't understand when I started this thing. With FFT analysis you have one big trade-off - either you have good frequency accuracy over a very large period of time, or damn poor frequency accuracy over a relatively short period of time. There is no good frequency accuracy that can be performed over short windows. Unfortunately 1/60th of a second sized windows have terrible frequency coarseness. The situation is made worse because FFTs have more coarseness of frequency in the lower end, and TIA has more coarseness of frequency in the higher end.

 

In the end I had to implement a sliding window to get the detection as useful as it is. I analyse 2x 1/60th seconds together (for twice the frequency resolution), save the detected frequencies, advance a single 1/60th second chunk ahead, and then do the analysis on another 2x 1/60th seconds. The two frame sliding window is the sweet spot, and any larger of a window smears the results.

 

So wave2tia is mostly good for WAVs that consist of mostly pure tones, without a lot of overlapping tone layers, and don't change character super quick. Whole songs aren't great, and intelligible human voices are a no-go.

 

I also tried a more complicated implementation, where instead of picking detected fundamental frequencies, I compared the FFT of the WAV sound chunk to FFTs of the possible TIA sounds. While this one held the promise of using TIA noise effects, ultimately the result was even worse than my fundamental frequency approach. To get any sort of accuracy, I needed a sliding window of at least 8 frames. And FFT results tend to spread very differently to adjacent frequencies, depending on the phase of the frequency within the window, so in one phase the frequency in the FFT result would have distinct peaks, and in another phase you'd see those peaks more blurred across neighboring frequencies. I tried to account for this in the comparison scoring, but between that and the large window sizes, this method was beat.

 

I should point out that @EricBall was the first person I'm aware of to try WAV to TIA conversion. (unfortunately I wound up naming my program "wave2tia" before I learned of his "wav2tia" attempts). Interestingly he took a non-fft least-squares approach, which didn't fully pan out either. 60Hz tone samples combined with the limited TIA tone palette is a pretty rocky path.

 

So it's mostly the end of the road for the wave2tia code experiments. I'll likely rename it (to avoid collision with Eric) and post the fundamental-frequency version to github at some point. Of course, I still plan to use the program! The sound-effect library must grow!

 

[edit... wave2tia was renamed to sound2tia, and uploaded to github]

 

Where to get the sound2tia utility...

sound2tia can be found at github.

 

  • Like 13
  • Thanks 3
Link to comment
Share on other sites

I couldn't leave well enough alone, and had to take one more kick at the can. It seems that my FFT comparing version of sound2tia was greatly improved by generating additional FFT results from different phases of the TIA sounds. I think this is pretty much the best case result that FFT analysis and TIA tone playback will give you...

 

chicken.60hz.fft2.mp3

 

...while very quickly changing sounds under 60Hz are still a problem, I think for intelligible human voice the main problem is the tone palette available on TIA, rather than the 60Hz playback rate. I've made a 30Hz version of that sample that doesn't sound very different, and even made a 120Hz version without a better result.

  • Like 4
Link to comment
Share on other sites

The 7800 can do sample playback too, but to do so with either console, you need to change the voice control register to it's non-tone DC output setting, and then you update it's volume register during each and every scanline. i.e. during visible, overscan, and vblank time too.

 

To cope with that requirement, most games need to pause screen updates during sample playback. It's been my hope (and others, like Eric) that a less-intensive tone playback routine could substitute.

 

 

  • Like 2
Link to comment
Share on other sites

Thanks for the nod.

 

Yeah, the TIA audio generator is pretty limited in what it can create, so trying to use it to duplicate any normal audio is fraught with challenges.  That's one of the reasons I went with the brute force correlation rather than something like the FFT.  (Also given an FFT is just a way of calculating a Discrete Fourier Transform - which is essentially a correlation matrix between the input and the set of sine and cosine waves, I figured I could use the raw TIA waveforms to create a similar correlation matrix.)

 

Hmm... I wonder if what is required is to find the "closest" TIA waveform - so rather than looking for an exact match / correlation, instead somehow have each TIA waveform represent a range of frequencies.

 

 

Link to comment
Share on other sites

Yeah, in some ways your method is like a wavelet transform, with TIA shaped wavelets. I tried "closest" comparison in terms of checking near-tunings of the TIA frequencies, since that's just a matter of searching higher in the fft array, but that wasn't helpful to the end result.

 

If I was familiar with tensorflow, I'd throw in the towel on figuring out the best procedural approach, and sick a generative adversarial network on the problem. As it is, I've probably sunk enough hobby time into it, and at this point I don't think we can do much better.

Link to comment
Share on other sites

  • 3 months later...
On 2/6/2020 at 1:34 PM, RevEng said:

The 7800 can do sample playback too, but to do so with either console, you need to change the voice control register to it's non-tone DC output setting, and then you update it's volume register during each and every scanline. i.e. during visible, overscan, and vblank time too.

 

To cope with that requirement, most games need to pause screen updates during sample playback. It's been my hope (and others, like Eric) that a less-intensive tone playback routine could substitute.

 

 

Space Harrier on the 130XE uses POKEY sampling without halting gameplay.

  • Like 1
Link to comment
Share on other sites

Full disclosure.
This library has been the ONLY way I can get close to making anything listenable with the TIA.
I page through, find a sound, take the data, and then start playing around with it until I get what I want.

I'm sure if I use it enough I'll figure out how to so things from scratch, but for right now that is the only way.

I hope that is okay with you.  If not, please tell me a how you would like it used.

 

-Steve

Link to comment
Share on other sites

Help yourself, that's what it's here for. :) The reason I put this out is so we don't need to have silent demos, when a coder hasn't gotten around to sounds yet. It's also why I added a standard sound effect format to 7800basic - so we could share and tweak. (note: assembly coders are welcome too)

 

I'm totally fine if the sounds from the library are taken and used as-is. Your find-something-and-tinker approach is great, since it means your game sounds will be unique.

 

Also, there's no shame in tweaking existing sounds to get new ones. The first 30 sounds in the library were created from scratch because I didn't have an alternative, but the rest of them started off by running sound2tia on downloaded WAV files. If sound2tia generated anything interesting, I'd tweak the sounds by hand to remove bits I didn't like, or making slight changes. It's a much easier process than designing from scratch.

  • Like 3
Link to comment
Share on other sites

13 minutes ago, RevEng said:

Also, there's no shame in tweaking existing sounds to get new ones. The first 30 sounds in the library were created from scratch because I didn't have an alternative, but the rest of them started off by running sound2tia on downloaded WAV files. If sound2tia generated anything interesting, I'd tweak the sounds by hand to remove bits I didn't like, or making slight changes. It's a much easier process than designing from scratch.

Wow. thanks!!

Wait, is sound2tia is a utility we can use ourselves?   

(scrolls up)
Oh hey!  I missed that!  Going to try it.  I've got ALOT of sound files! 

Edited by fultonbot
  • Like 1
Link to comment
Share on other sites

You're totally welcome. :)

 

Just a heads up that sound2tia works best on simpler sounds, though sometimes complex ones will create an interesting non-speech result that's tweakable. Listen for cool stuff, even just in the middle of the sound, and trim away what you don't need. I'd guess that about a quarter of my conversion attempts turned into something usable, which is actually pretty decent, considering how little time it takes to try.

  • Like 3
Link to comment
Share on other sites

38 minutes ago, RevEng said:

You're totally welcome. :)

 

Just a heads up that sound2tia works best on simpler sounds, though sometimes complex ones will create an interesting non-speech result that's tweakable. Listen for cool stuff, even just in the middle of the sound, and trim away what you don't need. I'd guess that about a quarter of my conversion attempts turned into something usable, which is actually pretty decent, considering how little time it takes to try.

Can the output be used with playsong instead of playsfx?  I tried but ran into a lot of issues.  playsfx works, but i'd I can't stop the sound to try a new one and I'd  like to loop and change the tempo too.

Link to comment
Share on other sites

Unfortunately no, the data formats for song/instrument data is very different from the sound effect format. For loops and tempo changes, you'd need to roll your own loop, so to speak.

 

I might look at giving the ability to trigger SFX within songs, but it's probably going to be quite a while before I can get to that.

  • Like 1
Link to comment
Share on other sites

34 minutes ago, RevEng said:

Unfortunately no, the data formats for song/instrument data is very different from the sound effect format. For loops and tempo changes, you'd need to roll your own loop, so to speak.

 

I might look at giving the ability to trigger SFX within songs, but it's probably going to be quite a while before I can get to that.

Okay, I'll pivot to taking very short loops and sfx and converting for TIA. This is very very cool. Thanks!!

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...