Jump to content
IGNORED

RMT2LZSS: convert RMT tunes to LZSS for fast playback!


rensoup

Recommended Posts

4 hours ago, rensoup said:

Good to hear that someone else found a use for it (on the A8)!

 

That's an interesting diagnostic given that dmsc's code works in Altirra and on Vinscool's real unmodified Atari 800XL but not on your modified 600XL ?

 

It seems to me there's an issue with your extension ?

 

I don't know if SKCTL needs to be initialized but I never touched because it seems to be useful for producing different kind of sounds (like synthpopalooza demonstrated ). It was discussed a while back that it could possibly be stored as an extra channel but only a single tune ever changed it during sound playback... so it was left alone.

 

I figured I could add an option to set the SKCTL value in RMT2LZSS so that it would be possible to try different modes like synthpopalooza but... too lazy so far.

 

...Perhaps for the next version then!

 

As for 2. you mentioned that your simple stereo is not switchable ?? Doesn't that renders it incompatible with every A8 software ? (Your fix would make 50+hz tunes DLIs slower too)

 

 

All books I read about sound on POKEY are telling me that it has to be initialized first. Here's the chapter in @phaeron's 'Altirra hardware reference manual':

Quote

5.2 Initialization

POKEY does not have a RESET line and therefore powers up in indeterminate state. IRQEN must be reset prior to clearing the I bit on the CPU to avoid stray IRQs.

Bits 0 and 1 of SKCTL normally control the keyboard scan and debounce features. However, clearing both of those bits also activates another initialization function, which causes the 15KHz clock, 64KHz clock, serial port hardware, and polynomial noise generators to be reset.

Setting the serial clock selection bits SKCTL (bits 3-5) to 0 resets the serial port circuitry. Therefore, SKCTL should be set to $00 to initialize all POKEY functions.

I think there might be some issues when other programms, e.g. a game dos, might have changed something before your own programme starts. On emulator most people won't get this issue, cause they often start .xex direct from host fs.

But I will retry the tests with an unmodified 130 XE later today.

 

The only issue I had so far with the simple stereo in my 600 XL, is that every programme that does not know about second pokey plays just on left channel, but full. Very few programmes resulted in a knocking sound on right channel - maybe they write into wrong registers.

 

Of course writing into the registers of second POKEY will cost some extra time, but this is optionally and not needed to have at least all 4 channels of one POKEY, when SKCTL has been set.

Actually @dmsc's routine doesn't care about second POKEY at all, so tunes, that could run in RMT in stereo are mono only here. So this small addition will please dual POKEY owners - that have older stereo boards build in, some newer automaticly recognize mono output and switch to dual output then - at least a bit.

  • Like 1
Link to comment
Share on other sites

Ok, don't have to test on ATARI anymore. Test yourselves in Altirra (set to stereo) or on real machine.

 

In the atr is a bootable disc with wrong and corrected xex files and you will find my sources to both plus used song.

lzss_player_issues.zip

 

PS: when you're ready with stereo, try to listen to it when Altirra is set to mono...

Edited by pps
added a PS
  • Like 1
Link to comment
Share on other sites

17 hours ago, pps said:

Ok, don't have to test on ATARI anymore. Test yourselves in Altirra (set to stereo) or on real machine.

 

In the atr is a bootable disc with wrong and corrected xex files and you will find my sources to both plus used song.

yeah but that's not fair, you're using a disc loader which messes with SKCTL ?. That wouldn't happen if you'd booted from a cart, would it ?

 

Anyway, I've updated the player so that SKCTL can be set to any value so anybody can experiment with it. Upload will follow shortly.

  • Like 1
Link to comment
Share on other sites

new release!

 

1.63 Apr 2021
    -added initial SKCTL field
    -slightly tweaked frequency reduction option
    -added flag for outputting a single tune line
    -added Vinscool's custom notetables (V21)

 

@VinsCool I've changed the frequency reduction to output the even Pokey data register sets instead of odd ones (or the opposite?) and it helps with your BS tune a lot but it's still not as good as your manual tweaks... not sure why...

You can also output the first song line only, which should help with iteration times hopefully?

 

The SKCTL value isn't part of the LZSS format so it still needs to be set manually in your app.

  • Like 2
  • Thanks 2
Link to comment
Share on other sites

3 hours ago, rensoup said:

ok maybe it is the problem then ?

It's honestly so minor that I never thought it was worth being nitpicky about it, please don't mind me in this case :D 

2 hours ago, rensoup said:

yeah but that's not fair, you're using a disc loader which messes with SKCTL ?. That wouldn't happen if you'd booted from a cart, would it ?

Ahhh good point there, I have been using a flashcard for my hardware tests, so no wonder thing worked properly, and that just made me realise something.

A friend of mine actually tried to run my tunes on a setup that was different from mine, I think it used some sort of disc drive emulation, and was also a stereo machine.
For some reason things just sounded wrong on it, like the entire high pass filter modulation I did was missing, it played like it was entirely disabled.
Maybe this was all related? I've actually thought it was a setup problem, since again, it worked on my machines.

1 hour ago, rensoup said:

new release!

 

1.63 Apr 2021
    -added initial SKCTL field
    -slightly tweaked frequency reduction option
    -added flag for outputting a single tune line
    -added Vinscool's custom notetables (V21)

Awesome! Thanks for the new update, SKCTL surely will be interesting to try based on synthpopalooza's researches.

1 hour ago, rensoup said:

The SKCTL value isn't part of the LZSS format so it still needs to be set manually in your app.

Oh, nevermind, maybe another time, if I can learn programming :D 

1 hour ago, rensoup said:

@VinsCool I've changed the frequency reduction to output the even Pokey data register sets instead of odd ones (or the opposite?) and it helps with your BS tune a lot but it's still not as good as your manual tweaks... not sure why...

Interesting tweak idea! The only reason why my manual tweaks in the tune worked well enough was simply because I made sure to get the most important stuff always on the even number of frames, so the less significant would go on the odd ones, and sound relatively good still.
Could it be possible to have a setting to chose whichever frames are picked at conversion time? That may very useful for testing purposes, and if we're lucky, make a tune sound correct first try as well, if something was ordered differently

2 hours ago, rensoup said:

You can also output the first song line only, which should help with iteration times hopefully?

Pretty cool, I don't exactly see much time save from this addition, but why not :P 

Link to comment
Share on other sites

7 hours ago, rensoup said:

yeah but that's not fair, you're using a disc loader which messes with SKCTL ?. That wouldn't happen if you'd booted from a cart, would it ?

 

Anyway, I've updated the player so that SKCTL can be set to any value so anybody can experiment with it. Upload will follow shortly.

Sorry, if I confused you with this. Disc loading is imho the normal way on Atari. At least, that's what I do.

As I don't know much about making sounds, I am happy that people develop cool tools and release their sources for common use, so that I can put some music into my productions. This time I got this issue and so I wanted to get the reason.

 

To say it clear to the people: Every .obx that has been created by your tool in earlier versions has this problem, if someone put them onto disc. That's a pity. Many people (loading via SIO device) will get shocked by the sound, when they should get impressed.

 

I stay tuned, what cool things will be discovered in future. I learned tons of things by all the releases and have to learn a lot more :)

Edited by pps
  • Like 3
Link to comment
Share on other sites

20 hours ago, VinsCool said:

A friend of mine actually tried to run my tunes on a setup that was different from mine, I think it used some sort of disc drive emulation, and was also a stereo machine.

So it's all good now, right ?

 

20 hours ago, VinsCool said:

Awesome! Thanks for the new update, SKCTL surely will be interesting to try based on synthpopalooza's researches.

22 hours ago, rensoup said:

The SKCTL value isn't part of the LZSS format so it still needs to be set manually in your app.

Oh, nevermind, maybe another time, if I can learn programming :D 

 

I mean you should now be able to use 2 tone and other funky SKCTL mode with the RMT2LZSS player.

 

If you wanted to play the .lzs16 file inside your own app however, you would have to set SKCTL manually because it isn't saved inside the .lzs16 file.

 

20 hours ago, VinsCool said:

Could it be possible to have a setting to chose whichever frames are picked at conversion time? That may very useful for testing purposes, and if we're lucky, make a tune sound correct first try as well, if something was ordered differently

It's doable of course, I just have to be a little more cautious with the other playback frequencies. Speaking of which... I probably introduced a bug by adding this so while 100->50hz works, others might not anymore?

 

20 hours ago, VinsCool said:

Pretty cool, I don't exactly see much time save from this addition, but why not :P 

Well the idea is to be nearly as fast as pressing play inside RMT, I don't know what kind of beast of a computer you own but on my machine, compressing a LZSS tune like BS takes a second or two! 

Edited by rensoup
  • Like 1
Link to comment
Share on other sites

16 hours ago, pps said:

To say it clear to the people: Every .obx that has been created by your tool in earlier versions has this problem, if someone put them onto disc. That's a pity. Many people (loading via SIO device) will get shocked by the sound, when they should get impressed.

But it's all good with the new 1.63 release, right ???

Link to comment
Share on other sites

3 minutes ago, rensoup said:

So it's all good now, right ?

Possibly? I don't have any way to confirm that myself but I'd imagine yes :P  

4 minutes ago, rensoup said:

I mean you should now be able to use 2 tone and other funky SKCTL mode with the RMT2LZSS player.

 

If you wanted to play the .lzs16 file inside your own app however, you would have to set SKCTL manually because it isn't saved inside the .lzs16 file.

Ohhhhh okay! Awesome!

4 minutes ago, rensoup said:

It's doable of course, I just have to be a little more cautious with the other playback frequencies. Speaking of which... I probably introduced a bug by adding this so while 100->50hz works, other might not anymore?

Woops :D it happens 

4 minutes ago, rensoup said:

Well the idea is to be nearly as fast as pressing play inside RMT, I don't know what kind of beast of a computer you own but on my machine, compressing a LZSS tune like BS takes a second or two! 

That's about the time it takes in general for me too, yeah. for most tunes it's half a second to maybe 5 seconds if the tune is really big, like the 200hz ones.

Link to comment
Share on other sites

3 hours ago, emkay said:

But particular the 16 bit stuff is still totally underdeveloped.

Seems, it still the main problem due to RMT not supporting it at all. 

 

 

Not sure what you mean with 16bit, you have full AUDCTL control per envelope step if you use extended instruments with .erti files

  • Like 1
Link to comment
Share on other sites

1 minute ago, rensoup said:
 

Not sure what you mean with 16bit, you have full AUDCTL control per envelope step if you use extended instruments with .erti files

Probably the fact that the 16-bit tables in the customnotestable.txt are currently placeholder and/or out of tune due to the mismatch, that's something I planned to tackle eventually too :D 

  • Like 2
Link to comment
Share on other sites

51 minutes ago, VinsCool said:

Probably the fact that the 16-bit tables in the customnotestable.txt are currently placeholder and/or out of tune due to the mismatch, that's something I planned to tackle eventually too :D 

I'm thinking I should change the original RMT code to use the 6th 16bit table (instead of the 1st one) when using dist 6... so that everything is consistent...

 

I'm always afraid to make changes though because I have no idea if they're right or not!

Edited by rensoup
  • Like 1
Link to comment
Share on other sites

12 minutes ago, rensoup said:

I'm thinking I should change the original RMT code to use the 6th 16bit table (instead of the 1st one) when using dist 6... so that everything is consistent...

 

I'm always afraid ot make changes though because I have no idea if they're right or not!

Using the 6th position would make more sense in my opinion.
This was why my own comments in the .txt file were a little confused, because I was unsure about the order but at the same time it would have been logical to follow the same order used for 8-bit tables, at least as it is so far, stuff seems to work as expected, hahaha

  • Like 1
Link to comment
Share on other sites

1 hour ago, VinsCool said:

Using the 6th position would make more sense in my opinion.
This was why my own comments in the .txt file were a little confused, because I was unsure about the order but at the same time it would have been logical to follow the same order used for 8-bit tables, at least as it is so far, stuff seems to work as expected, hahaha

I checked the code and hmm there's probably another glitch... 

if you select RMT upatched, it's all good.

if you select RMT with Custom tables, the behaviour might change depending on whether you're using custom audctl (dist 6 16bit is the 6th table) or not (dist 6 16 bit is the first table), so the table has to be in 2 places... I think...

 

Need to fix, but as long as you're using tables other than the 1st one you should be ok... 

  • Like 1
Link to comment
Share on other sites

1 hour ago, rensoup said:

I checked the code and hmm there's probably another glitch... 

if you select RMT upatched, it's all good.

if you select RMT with Custom tables, the behaviour might change depending on whether you're using custom audctl (dist 6 16bit is the 6th table) or not (dist 6 16 bit is the first table), so the table has to be in 2 places... I think...

 

Need to fix, but as long as you're using tables other than the 1st one you should be ok... 

16 Bit means 16 bit handling on two channels. If 16 bit is used, the frequency of both channels have  to be fixed together, to have the recommended voice stable at 16 bit. 

This also means that vibrato and portamento have to be build on 16 bit , too.

 And, then , it must be assured to have the 2nd voice still available for volume and chosing the wanted generator. 

And, well, it seems to be a tremendously crazy task of interweaving, to put something 16 bit to the LZSS converter.

 

If you don't hear in direct response what you were doing, the result might be technically impressive, but the essence of the music still might be missing. 

 

Link to comment
Share on other sites

19 hours ago, rensoup said:

I checked the code and hmm there's probably another glitch... 

if you select RMT upatched, it's all good.

if you select RMT with Custom tables, the behaviour might change depending on whether you're using custom audctl (dist 6 16bit is the 6th table) or not (dist 6 16 bit is the first table), so the table has to be in 2 places... I think...

 

Need to fix, but as long as you're using tables other than the 1st one you should be ok... 

I think it would be a good idea to use the 6th one for both cases, so it will still be the one used by default, as expected, and the 6th slot wouldn't need to be changed since it would have been assigned to Distortion 6 anyway.
That would then leave every other slots follow the same order, and land on the matching number used in the 8-bit tables.

In my opinion, that would work the best, considering how 8-bit had already been moved around and is 100% confirmed to work as expected by following the same order used in the tracker itself.
That would make 16-bit do exactly what is expected as well when 16-bit is enabled, or it will simply use the matching 8-bit tables when not.

Do you think it would be possible to add a certain form of individual instrument check, now that I think of it?


Something such as:

"if, and only if, instrumentXX uses AUDCTL bits for 16-bit ($50 for channel 2 output or $28 for channel 4 output), then 16-bit tables get used, else use the 8-bit tables like normal"

 I'm not sure if it's already the case but I'm asking anyway ? 


That could save a lot of potential issues if conflicting settings happen to be used at the same time, for example:

"1 channel has an instrument at 1.79mhz (or not), and another channel has an instrument using "Join x+x", playing on the same row, they could be using different tables for different purposes, while still "technically" output 16-bit sound"

 

This one case is a bit interesting, because that would make possible certain effects currently disabled in RMT when 16-bit sound (Distortion 6, specifically) is output, such as vibrato or portamento, where one channel handles most of the fine tuning and the other does most of the global note output.
The downside would be the requirement of 2 different 8-bit tables, but once they (eventually) exist as 16-bit tables, it shouldn't be difficult to copy and paste elsewhere... I was able to prove it is possible to do in some of my test tunes...

 

Tables truly are a blessing to have since you don't have to manually set the frequencies in the tracker every time, the same situation applies to the Sawtooth waveform I used recently, except that one was a "use 2 different 8 bit tables" situation, no 16-bit was necessary :D 

 

Does that even make any sense? lol, sorry if not, I'm rambling once more about "what could possibly work", hahahaha ?
 I'm still really amazed how well the customnotestables.txt + custominstrument.erti system works so far by the way, you have my respects for making a lot of POKEY music making better ?

Link to comment
Share on other sites

On 4/10/2021 at 5:05 AM, emkay said:

16 Bit means 16 bit handling on two channels. If 16 bit is used, the frequency of both channels have  to be fixed together, to have the recommended voice stable at 16 bit. 

This also means that vibrato and portamento have to be build on 16 bit , too.

 And, then , it must be assured to have the 2nd voice still available for volume and chosing the wanted generator. 

And, well, it seems to be a tremendously crazy task of interweaving, to put something 16 bit to the LZSS converter.

hmm... that's pretty hazy...  

 

both channels have  to be fixed together, to have the recommended voice stable at 16 bit.

 

another way of saying that ?

  • Like 1
Link to comment
Share on other sites

On 4/10/2021 at 11:10 PM, VinsCool said:

I think it would be a good idea to use the 6th one for both cases, so it will still be the one used by default, as expected, and the 6th slot wouldn't need to be changed since it would have been assigned to Distortion 6 anyway.
That would then leave every other slots follow the same order, and land on the matching number used in the 8-bit tables.

I split the code so that RMT unpatched does its usual thing and RMT custom used custom tables in that predictable way you're mentioning but that broke your Freeze version ?

 

It seems you're using custom AUDCTL from time to time and the regular RMT code at other times (where it checks for the filter,...)

 

This is the code for custom tables, this is done for all 4 channel:

 

if ((m_songRuntime.TrackLine[CH].Envelope.bUseAUDCTL) && ((m_songRuntime.AudCtl & AUDCTL_CH1CH2_LINK) != 0))
{
  int frqTable16Offset = tabbeganddistor[m_songRuntime.TrackLine[CH].Envelope.distortion] << 1;

  byte outNote = m_songRuntime.TrackLine[CH].OutNote;
  m_songRuntime.AudF[ChBaseIdx] = frqtabbasshi[frqTable16Offset + outNote + frqtabbassloOffset];
  m_songRuntime.AudF[ChBaseIdx+1] = frqtabbasshi[frqTable16Offset + outNote];
}

which means:

 

(note is set from 8bit table initially)

 

if the instrument envelope has AUDCTL set and the channel has 16bit mode set 

   ->override note from the 16bit table assigned to the current distortion and set both AUDF

 

So that's pretty simple...

 

On 4/10/2021 at 11:10 PM, VinsCool said:

Do you think it would be possible to add a certain form of individual instrument check, now that I think of it?


Something such as:

"if, and only if, instrumentXX uses AUDCTL bits for 16-bit ($50 for channel 2 output or $28 for channel 4 output), then 16-bit tables get used, else use the 8-bit tables like normal"

 I'm not sure if it's already the case but I'm asking anyway ? 

 

So that's what it does above except that it checks all 4 channels, it is unnecessary (should be like you said 2&4 or 1&3) but that way you have full control.

In the code above It means that AUDCTL 16 bit mode should be set on channel 1 & 3 to work as expected because the 16bit note is set on the current channel (lo byte) and current channel+1 (hi byte)

 

It seems that in the original RMT code, 16bit mode is set on channel 2 & 4 and the 16bit note is set on the previous channel (lo byte) and the current one (hi byte).

Perhaps I should just mimick that ?

 

On 4/10/2021 at 11:10 PM, VinsCool said:

That could save a lot of potential issues if conflicting settings happen to be used at the same time, for example:

"1 channel has an instrument at 1.79mhz (or not), and another channel has an instrument using "Join x+x", playing on the same row, they could be using different tables for different purposes, while still "technically" output 16-bit sound"

basically 16bit mode is checked last so any 8 bit mode stuff is overriden... so that's good right ?

 

On 4/10/2021 at 11:10 PM, VinsCool said:

This one case is a bit interesting, because that would make possible certain effects currently disabled in RMT when 16-bit sound (Distortion 6, specifically) is output, such as vibrato or portamento, where one channel handles most of the fine tuning and the other does most of the global note output.
The downside would be the requirement of 2 different 8-bit tables, but once they (eventually) exist as 16-bit tables, it shouldn't be difficult to copy and paste elsewhere... I was able to prove it is possible to do in some of my test tunes...

? not sure what you mean...

 

On 4/10/2021 at 11:10 PM, VinsCool said:

Does that even make any sense? lol, sorry if not, I'm rambling once more about "what could possibly work", hahahaha ?
 I'm still really amazed how well the customnotestables.txt + custominstrument.erti system works so far by the way, you have my respects for making a lot of POKEY music making better ?

Mostly ?

I have no idea what I'm doing... but your tunes sound awesome so I'll just go with the flow ?

 

But first we should fix the Freeze tune so that it works with the custom code only, to clear any misunderstanding... I'll be putting up a new version shortly!

 

 

  • Thanks 1
Link to comment
Share on other sites

Ow, sorry to know things broke like this.
I did indeed use a mix of custom and regular RMT stuff in the tune, because some cases needed it and some didn't ;) 

I personally like that behaviour, if a custom instrument entry is present: use the custom crafty data in the file, else it simply works exactly like originally coded.
That works pretty well for me :) 

For what you mentioned above, yeah, that's what I meant to say, to simply mimick the way RMT handled 16-bit with the custom entries, I am not sure exactly how it behaves as I have not 100% experimented with it, but either way I know something will work out anyway, and like I mentioned in my more rambly part of the post, I still have many ideas I wanted to experiment with so don't worry about what I said, it will make sense... sometime when I get to try something new, or old :D 

Analmux's experiments are for example a great source of inspiration, emkay's cryptic posts also give me good clues for many things, and there's several ideas I have yet to try out on my own, also inspired by synthpopalooza's own impossible sounds, hahaa

Link to comment
Share on other sites

38 minutes ago, VinsCool said:

Ow, sorry to know things broke like this.
I did indeed use a mix of custom and regular RMT stuff in the tune, because some cases needed it and some didn't ;) 

alright I took a look... I thought it was a problem with 16bit tables but you aren't using them at all... yeah I guess you mentioned that earlier ?

 

But you're using the filter all the time which is supposed to have to be set manually in the new custom code. I'm pretty sure we discussed having each audctl feature as separate table in the .erti file but you liked having all the audctl bits as a single value.

 

42 minutes ago, VinsCool said:

I personally like that behaviour, if a custom instrument entry is present: use the custom crafty data in the file, else it simply works exactly like originally coded.
That works pretty well for me :) 

Well, you're going to have to make up your mind because otherwise that 's going to cause a lot of headaches ?

 

So can you try setting the filter directly into the envelope audctl for Freeze.rmt ?

 

I'm putting the new 1.64 version here because I don't know if it's the direction we should take, so give it a shot and let me know...

 

RMT2LZSS164.zip

  • Thanks 1
Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...