Jump to content
Tursi

VGM Compression Tool

Recommended Posts

2 hours ago, OLD CS1 said:

Hey, how do I run this?  E/A?

Yeah, it's an EA#5 file with a TIFILES header.

 

  • Like 1

Share this post


Link to post
Share on other sites

I was getting a file error trying to run it.  I think it was an incomplete transfer over HDX as after a second copy attempt it worked fine.

 

I had to noise-reduce the recording as I think my Triple Tech is putting noise into the sound channel, but the volumes of both the SID (6581R4AR) and 9919 are definitely not equal when examined (look at the last two seconds of the Audacity project, which contains the TI's beep,) but that could be more about the volume setting of the SID.  Dunno.

 

Clipboard02.thumb.png.18b5a4bdd9d16df1c4812a1ddb125153.png

 

M4A recording:

Tursi Final Takeoff PSG-SID.m4a

 

Audacity project:

Tursi Final Takeoff PSG-SID (Audacity Project).zip

Share this post


Link to post
Share on other sites

Cool, thanks! Yeah, there's no reason to expect the two chips to have the same output level. There will be tools for balancing manually, I guess, but I'd also guess few people will use the combination. But I figured I'd provide it.

 

Share this post


Link to post
Share on other sites
On 6/23/2020 at 6:17 PM, Tursi said:

That guy is crazy good. He and Rushjet1 need to get into a PSG battle. ;)

 

(Edit: LaurenX is my fav, though Witch's Den also made me say "damn" out loud. ;) )

 

Rushjet doesn't have a page of just his PSG tunes, though... I did buy a bunch from him for Super Space Acer though. More of his retro chip tunes are NES, though.

 

https://rushjet1.com/

 

I discovered Rushjet through his soundtrack on Rikki and Vikki for 7800.

 

I also found this recent post from Tomy where he talks about how he generates the instruments on the PSG.  He talks about using VGM Music Maker for windows, but are there any better trackers out there now?

  • Like 1

Share this post


Link to post
Share on other sites
3 hours ago, PeteE said:

I discovered Rushjet through his soundtrack on Rikki and Vikki for 7800.

 

I also found this recent post from Tomy where he talks about how he generates the instruments on the PSG.  He talks about using VGM Music Maker for windows, but are there any better trackers out there now?

The guy FarmerPotato posted says he's using Shiru's VGM Music Maker, I wonder if that's the same one? I haven't looked at it for many years but I was under the impression it was really good. However it doesn't seem to be on his download page anymore... according to a forum post from 2012:

 

"Looks like Shiru removed both the VGM Music Maker and TFM Music Maker due to a misunderstanding in a topic and hasn't posted since July.
viewtopic.php?p=16101&highlight=#16101"

 

People ruin everything. ;) 

 

  • Like 2

Share this post


Link to post
Share on other sites

On Friday I ran my test suite and discovered that the various tweaks and improvements I'd made had broken pretty much everything, so today I finished an overhaul of the RLE compression and tested on all the songs, finally working again. Most of the bugs were in the test code, which is why it worked, but as the test code got tighter more things were tripped. Anyway, all working now.

 

I made two small changes to the file format. One probably was obvious but the tools didn't really support it before - you can have an empty stream simply by setting the stream pointer to 0. This means songs that (for instance) don't have a drum beat don't need to waste 4 bytes per second ticking off a silent volume channel. The improvement is small but it's there.

 

The second was to change the reference of the backref offset. Rather than being an offset from the beginning of the file, it's now an offset from the current file position. This let me remove the need to pass the start of the file around just for this single case, and made the TI code slightly faster to boot (adding a register rather than adding the value at an address ;) ), and let me remove the last piece of code that was tying the "common" code block to a particular player.

 

I added a sample project showing the SN and SID playing together - Wonder Boy rather than Castlevania, but I don't think I'll take it any further. Same concept would let you run on the ForTI though, if you had a song that needed it. Anyway, it was a bit discordant so I don't think I'll force anyone to run it, but it shows the concept works and it's in the repo. Also a video here:

 

https://lbry.tv/@tursilion:1/VGMComp-Progress-SN-and-SID:0

 

  • Like 5

Share this post


Link to post
Share on other sites

And now I've finished a major re-org of the code and I think it's ready to call first release, though maybe beta since nobody but me has used it yet. ;)

 

First, the SFX player for the TI SN is complete, so it now has feature parity with the old toolchain. Getting SFXs working smoothly required a few little tweaks here and there, as well as in the compressor to make multiple songs work correctly with the new empty streams, but it's all doing the right thing now. I ported the old test app to verify SFX.

 

With the basic TI functionality in place, I created a linkable library and moved all the example projects into a 'samples' folder. The Makefile for the library will go out and build all the samples as well, and I cleaned up the spare files in each folder to make the example more clear. I also wrote up a quick readme file to document the methods in the library.

 

All up on github, you'll find the lib here: https://github.com/tursilion/vgmcomp2/tree/master/Players/libtivgm2

Sample programs: https://github.com/tursilion/vgmcomp2/tree/master/Players/Samples

EXEs: https://github.com/tursilion/vgmcomp2/tree/master/dist

 

Getting there. The TODO list is still long, but really only the ColecoVision player is a big one, the rest should overwhelm me merely with numbers. ;)

 

 

  • Like 5

Share this post


Link to post
Share on other sites

Been a little tougher, but I got MOD conversion mostly working. It's somewhat random how well the automatic process works, but I think it's possible to massage most MODs into something usable. Don't expect to not have to do SOME work, but for a sample of what can be done with NO effect, I posted a short video of the current status.

 

https://open.lbry.com/@tursilion:1/VGMComp-Mod-Test:5?r=J8w7SGxXLD3UUrHVxe5d5g1o5rwKT1Wh

 

  • Like 5

Share this post


Link to post
Share on other sites

I pushed an updated version since that video - I think the tinniness (spell check says that's a word!) was due to a mis-scale of frequency. I halved the tone frequencies and things sounded a heck of a lot better. Probably need to do a new video. ;)

 

  • Like 3

Share this post


Link to post
Share on other sites

Reached the point where I could fully process a song and so finished off the new visualizer demo I started in /2017/. ;)

 

It didn't turn out as well as I'd have liked, but it works. I still like the concept but the layout clearly needs more thought. Anyway, new tune!

 

Program and source attached, or just have a look at the video. ;)

 

https://open.lbry.com/@tursilion:1/2020-07-18-05-09-29:4?r=J8w7SGxXLD3UUrHVxe5d5g1o5rwKT1Wh

 

vgmcplay3r.zip

  • Like 6

Share this post


Link to post
Share on other sites

Ahhh, nice work!

It´s a swedish song from the mid 80´s with the Herreys Brothers. Have never been my favourite but I enjoy your conversion.

Here´s the original:

 

Share this post


Link to post
Share on other sites
8 hours ago, Nick99 said:

Ahhh, nice work!

It´s a swedish song from the mid 80´s with the Herreys Brothers. Have never been my favourite but I enjoy your conversion.

Here´s the original

 

Yeah, I first heard it on an Atari XL conversion. This version is converted from a Protracker MOD which was in turn inspired by that Atari 8-bit version. ;)

 

  • Like 1

Share this post


Link to post
Share on other sites

Added the volume smoothing tool - overuse definitely causes notes to blend together, so I guess it's working, but a single pass improved GalaxyII quite a bit, and saved 4k on the resulting file to boot! I plugged it into the old keyboard viz to make another demo, or you can watch the video here:

 

https://open.lbry.com/@tursilion:1/2020-07-21-02-06-47:e?r=J8w7SGxXLD3UUrHVxe5d5g1o5rwKT1Wh

 

piano.zip

  • Like 9

Share this post


Link to post
Share on other sites

Wow Tursi!
nice result!

 

i have some old .MOD i created when i was younger 😛
I would be very happy to try to convert and listen a TI99 version of them :) :) 
 

  • Like 1

Share this post


Link to post
Share on other sites
3 hours ago, ti99iuc said:

i have some old .MOD i created when i was younger 😛
I would be very happy to try to convert and listen a TI99 version of them :) :) 

Right now you need programming to get them over to the TI (though I'll get a quickplayer like the old tool had before it's done). But you can test the converter on the PC side. The tools are all over at https://github.com/tursilion/vgmcomp2 - you can download as a zip file. Binaries for Windows are in the "dist" folder. You would start with "mod2psg", and I recommend the "-d" flag to see what it autodetects the samples as, cause there's probably only a 50% chance it'll guess right. ;) (so: mod2psg -d mymod.mod).

 

That'll output a separate file for every final track with tones and noise separated. A 4 channel Protracker can output up to 8 files for that reason. You can test it immediately with "testplayer" and the filename prefix (in this case: testplayer mymod.mod). Then you can go back to mod2psg and adjust anything it got really wrong (use mod2psg.exe with no inputs to see the options). To just play around and see what they might sound like, this is as far as you need to go.

 

To get it ready for the TI you need to get down to three tone channels and 1 noise channel. "arptones" will combine two tone channels into one by alternating the tones, and "mergenoise" will combine two noise channels into one by keeping the loudest one. For other needs I'm just starting on those tools, "smoothvolume" is the newest one which cleans up some of the noise on the volume channels. The trickiest part of this phase is keeping the filenames straightforward.

 

After that, "prepare4sn" will do the final conversion, and you pass it your four channels. And then "vgmcomp2" compresses the output of that.

 

It's a lot of little steps, but the intent is that the old process, while very simple, had very little flexibility. Every time I wanted to add the ability to tweak the result, I needed to insert it into the middle of a big bloated program. This lets you chain together the steps that are appropriate to a particular song - this should make better results possible. In theory once you decide what you need you can batch file them.

 

GalaxyII would have been something like this (not exactly what I actually did, because I did it over several days):

 

rem (I can't remember if I had to manually set any noise channels, and I didn't write it down!)
mod2psg -d galaxyII.mod

rem this generated 6 channels - 4 tone and 2 noise
smoothvolume galaxyII.mod_ton00.60hz galaxyII.modA_ton00.60hz
smoothvolume galaxyII.mod_ton01.60hz galaxyII.modA_ton01.60hz
smoothvolume galaxyII.mod_ton02.60hz galaxyII.modA_ton02.60hz
smoothvolume galaxyII.mod_ton03.60hz galaxyII.modA_ton03.60hz
smoothvolume galaxyII.mod_noi04.60hz galaxyII.modA_noi04.60hz
smoothvolume galaxyII.mod_noi05.60hz galaxyII.modA_noi05.60hz

rem merge channel 3 into 0 (determined by playing and watching)
rename galaxyII.modA_ton00.60hz galaxyII.modA_ton00.old
rename galaxyII.modA_ton03.60hz galaxyII.modA_ton03.old
arpnotes galaxyII.modA_ton00.old galaxyII.modA_ton03.old galaxyII.modA_ton00.60hz
 
rem merge channel 5 into channel 4 (all noises)
rename galaxyII.modA_noi04.60hz galaxyII.modA_noi04.old
rename galaxyII.modA_noi05.60hz galaxyII.modA_noi05.old
mergenoise galaxyII.modA_noi04.old galaxyII.modA_noi05.old galaxyII.modA_noi04.60hz

rem prepare and pack
prepare4sn galaxyII.mod_ton00.60hz galaxyII.mod_ton01.60hz galaxyII.mod_ton02.60hz galaxyII.mod_noi04.60hz galaxy2.snpsg
vgmcomp2 -v galaxy2.snpsg galaxy2.snsbf

 

So, a lot of steps, but hopefully they follow logically. Some of it could be automated but most of the time you want the human influence at each one to determine what is required.

 

I feel like I rambled offtrack a bit... for testing your old songs you can start by just playing with mod2psg and testplayer. ;)

 

Edited by Tursi
stupid editor inserts blank lines into code blocks if you use the tag instead of the editor tools to create it.
  • Like 3

Share this post


Link to post
Share on other sites

.. just looking at that sequence there, I think I'll do away with the filename confusion and just have the tools rename the original files automatically. That was a pain to keep straight the first time around doing it every time looks tedious and error-prone.

 

  • Like 2

Share this post


Link to post
Share on other sites

To re-test the toolchain, I reconverted Galaxy2.mod and saved off the steps. The result is attached. It's slightly better than the previous because I did the smoothing before merging channels - so when noises and tones were interleaved they preserve what they were doing instead of blending with the alternating note. Not too noticeable, but it was bugging me so I'm glad to have done it.

 

The steps were as follows. Again, determining what I wanted to do was at least some verification with testplayer between steps. ;)

D:\work\TI\vgmcomp2\galaxy>..\release\mod2psg -d GALAXYII.MOD
D:\work\TI\vgmcomp2\galaxy>..\release\smoothvolume.exe GALAXYII.MOD_ton00.60hz
D:\work\TI\vgmcomp2\galaxy>..\release\smoothvolume.exe GALAXYII.MOD_ton01.60hz
D:\work\TI\vgmcomp2\galaxy>..\release\smoothvolume.exe GALAXYII.MOD_ton02.60hz
D:\work\TI\vgmcomp2\galaxy>..\release\smoothvolume.exe GALAXYII.MOD_ton03.60hz
D:\work\TI\vgmcomp2\galaxy>..\release\smoothvolume.exe GALAXYII.MOD_noi04.60hz
D:\work\TI\vgmcomp2\galaxy>..\release\smoothvolume.exe GALAXYII.MOD_noi05.60hz
D:\work\TI\vgmcomp2\galaxy>..\release\smoothvolume.exe GALAXYII.MOD_noi06.60hz
D:\work\TI\vgmcomp2\galaxy>..\release\smoothvolume.exe GALAXYII.MOD_noi07.60hz
D:\work\TI\vgmcomp2\galaxy>..\release\arptones.exe GALAXYII.MOD_ton00.60hz GALAXYII.MOD_ton03.60hz
D:\work\TI\vgmcomp2\galaxy>..\release\mergenoise.exe GALAXYII.MOD_noi04.60hz GALAXYII.MOD_noi05.60hz
D:\work\TI\vgmcomp2\galaxy>..\release\mergenoise.exe GALAXYII.MOD_noi04.60hz GALAXYII.MOD_noi06.60hz
D:\work\TI\vgmcomp2\galaxy>..\release\mergenoise.exe GALAXYII.MOD_noi04.60hz GALAXYII.MOD_noi07.60hz
D:\work\TI\vgmcomp2\galaxy>del *.old
D:\work\TI\vgmcomp2\galaxy>..\release\prepare4SN.exe GALAXYII.MOD_ton00.60hz GALAXYII.MOD_ton01.60hz GALAXYII.MOD_ton02.60hz GALAXYII.MOD_noi04.60hz galaxy2.snpsg
D:\work\TI\vgmcomp2\galaxy>rem (manually patched a couple of lines in the snpsg)
D:\work\TI\vgmcomp2\galaxy>..\release\vgmcomp2.exe -sn -v galaxy2.snpsg galaxy2.snsbf

There were actually 4 noise channels output rather than 2. I had to move the intermediate merge.old files between them, though I did that in Explorer. There were a few rows right in the first tone that were dropping to zero, making a little static, so I manually patched those (about 6 lines - this is why the intermediate formats are in text!)

 

Resulting program is here. Again, improvement is minor, but the arpeggio at the beginning and some of the thunderclaps are more crisp.

 

I'll stop spamming now for a bit. ;)

 

piano.zip

  • Like 3

Share this post


Link to post
Share on other sites

Not much for this side, but tonight I pushed the first version of all the ColecoVision versions of the players. This took a bit of tweaking to get everything building clean on both sides.

 

Performance of the C code is surprisingly close to what the 9900 GCC version side - code size too. Despite the clock differences the performance of the Z80 in the ColecoVision is remarkably close to the TI... that said, a hand-optimization pass of the Z80 code sure wouldn't hurt. ;)

 

Share this post


Link to post
Share on other sites

I did an optimization to the Coleco libraries and they are roughly 20% faster... I'd like to get more but the C code is probably pretty good now.

 

I also finished and pushed up the "QuickPlayer" tool for this generation, it'll take your compressed file and spit out either an EA#5 (TIFILES) program for the TI or a 32k cartridge ROM for the ColecoVision. It can handle all the combinations that I've presented demos for with a simple Windows GUI (a command line version is also available). It's in the dist folder with the rest of the EXEs. ;)

 

image.thumb.png.01719d18ee8c0458f4b679fb68fd6a39.png

 

A couple of challenging tools left then most of the rest should go pretty quick.

 

  • Like 5

Share this post


Link to post
Share on other sites

Pushed up the SID converter today - this converts SID files (the ones based on emulating a C64!) into PSG. I used a package called csid and to my surprise the results are quite good - they had some talented musicians over there. Even losing the waveforms most of the music I converted still sounded really good. (Not all. But most!)

 

Anyway, that's in the usual place, and once it's done uploading a demo video is here:

https://open.lbry.com/@tursilion:1/2020-07-29-04-25-13:6

 

 

 

Edited by Tursi
  • Like 4

Share this post


Link to post
Share on other sites
13 hours ago, Tursi said:

Pushed up the SID converter today - this converts SID files (the ones based on emulating a C64!) into PSG. I used a package called csid and to my surprise the results are quite good - they had some talented musicians over there. Even losing the waveforms most of the music I converted still sounded really good. (Not all. But most!)

 

Anyway, that's in the usual place, and once it's done uploading a demo video is here:

https://open.lbry.com/2020-07-29-04-25-13:6

 

This is some seriously impressive translation to our humble SN76489!

 

In flashitback.sid in particular, around 21:14:

 

This song has a lot of notes in the 3rd voice that bend up/down while echoing 4 times.

 

Here is one that goes F-4 G#4 C-5 D#5 four times, while the volume fades from 9F to 04.

 

Since you have a .sid file, is the volume data from a register capture? What if the
SID's envelope generator was loaded to produce this effect? In that case there would
not be all the volume values in the capture. Do those have to be emulated?

 

(I don't know much about SID, but it has a volume envelope generator onboard. Just wondering.)

 

Also, I see 8 bit volume values. What's the scale for these?

 

image.png.3460eb66e0c44d1aa2b71054eb26f68f.png

 

Share this post


Link to post
Share on other sites

Without the envelope generator, the SID has only a single master volume (for all channels). So you have to include the envelope in any volume calculation. It doesn't have per-channel control like the SN does, just per-channel envelope. The converter runs a full SID emulation (indeed, there's also a 6510 running CPU code!) then samples the emulation state (tone comes from the period registers, but volume comes from the emulation's output) every frame.

 

VGMComp2's convert tools convert all music to split channels with 12-bit tone and 8-bit volume for the sake of later tools - having more resolution to work with reduces the loss when you manipulate, and means we don't cut out features that other chips might be able to use. When you use the "prepare" tool, then the limitations of the target chip (which may be SID, AY-3-8910 or SN76489) are taken into account. So the test play does necessarily sound a little better than the final product, but the goal is to show you got a clean conversion. In the case of these SID tunes, the noise channels (if more than one) are combined and the volumes mapped from 8-bit linear to 4-bit logarithmic (and if there is any extremely low notes, those need to be trimmed, clipped, or scaled). There's some loss, but the Galaxy2 demo shows that it still sounds decent, it really abuses the slow volume ramps. 

 

You can also run the prepare4sn tool and then use testplayer to see what it sounds like after limitations are enforced on the SN.

 

I decided to walk the rest of the way through the process. The noise channels merged nicely and even use custom frequencies where needed for the varying sounds. The only issue was the first channel having bass notes that were too low... after trying a few things I eventually went with a single octave higher, allowing a little bit of clipping, and play out the result. It almost makes me happy - I wanted to change the volume of the re-tuned channel, and I haven't written that tool yet. ;) But you can hear that the sequences you called out still work fine with the reduced volume resolution.

 

https://open.lbry.com/@tursilion:1/2020-07-30-15-07-40:1?r=J8w7SGxXLD3UUrHVxe5d5g1o5rwKT1Wh

(Skip ahead to 22:05 to bypass most of my rambling and experimenting ;) )

 

  • Like 2

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...

  • Recently Browsing   0 members

    No registered users viewing this page.

×
×
  • Create New...