Jump to content
IGNORED

RMT2LZSS: convert RMT tunes to LZSS for fast playback!


rensoup

Recommended Posts

25 minutes ago, rensoup said:

All that NTSC vs PAL is rather cryptic... I'm not sure what you're getting at. As far as I understand that stability issue doesn't seem related to it...

 

Right.

 

25 minutes ago, rensoup said:

The LZSS player just plays raw pokey streams but RMT2LZSS is just like the title says, it plays RMT tunes in the exact same way as the original RMT player, byte for byte...

 

Which is good. But, as it seems, more exact than the RMT play itself.

 

25 minutes ago, rensoup said:

...Well that's the idea anyway but, on a slightly different topic,  it turns out that initial release has a bug (at least) seemingly related to the portamento (whatever that is ?) which I fixed when cleaning up the code. To verify that all is good after I make some changes, I reoutput all 1070 tunes and compare with my initial release and after that cleanup 4 out of 1070 failed the compare test...

Believe it or not, all 4 were Emkay edits ?, so congratulations on using the portamento in ways nobody else has and exposing a bug in my code ?

 

A new release should be out soon...

 

Seems, I missed something here. 

As I'm always pointing out, I'm really shaping the resulting waves on sharp timing programming. 

As long as RMT and any emulation isn't 100% corrected, there will be slight differences in the result.

 

I wouldn't take care too much about it. 

 

 

 

Link to comment
Share on other sites

9 hours ago, rensoup said:

that's right but that's more of a lucky guess ? ?

 

The RMT XEX does the same as the RMT player source code, so that XEX is fine...

 

I Looked at the Pokey state and it's identical in both except for interrupts flags which don't have anything to do with the problem in this case... noticed something however. Here's another test... what do you think?

Sketch 32 V37_testC.obx 16.71 kB · 7 downloads

Oh there's something indeed! I noticed everytime something reached volume 0, it would usually show as using White Noise in the Audio Monitor of Altirra... Seems like it's no longer the case, now it keeps whatever is in the register as is until it's actually changed, close to how the RMT player XEX did it, and how I believe it should be doing it?
I just listened quickly between 2 outputs with only 1 channel playing at random times and seems like Test C makes it even better now.
Getting rid of some clicks, sounds like it.
I'm not exactly sure what was done, but it sounds about as good as before, except a little less clicky?

Good job for fixing a portamento bug btw! I wonder if that may be related to a few of my other songs behaviour too, I haven't really noticed anything wrong for portamento related effects, except maybe for 2-3 songs that were very very slightly different, but barely made any impact overall.

 

Link to comment
Share on other sites

10 hours ago, emkay said:

Which 4 tunes were the problematic ones for the LZSS Player?

 

I don't remember all 4 but these were 2 of them:

 

Xmas_Synth
Green_Beret_Remix
 

but the bugs were in my portamento translation of the code anyway

Edited by rensoup
Link to comment
Share on other sites

9 hours ago, VinsCool said:

Oh there's something indeed! I noticed everytime something reached volume 0, it would usually show as using White Noise in the Audio Monitor of Altirra... Seems like it's no longer the case, now it keeps whatever is in the register as is until it's actually changed, close to how the RMT player XEX did it, and how I believe it should be doing it?
I just listened quickly between 2 outputs with only 1 channel playing at random times and seems like Test C makes it even better now.
Getting rid of some clicks, sounds like it.
I'm not exactly sure what was done, but it sounds about as good as before, except a little less clicky?

 

Regarding testC, I remembered @dmsc added an optimization to help the compressor and I thought it may have had the side effect of "stabilizing" playback, so I took it out hoping it would be as jittery as RMT but you guys seem to say it actually improves it ???

 

this is the code:

 

Quote

    int vol  = buf & 0x0F;
    int dist = buf & 0xF0;
    if( vol == 0 )
        buf = 0;

    else if(( dist & 0x10 )!=0)
        buf &= 0x1F;     // volume-only, ignore other bits
    else if(( dist & 0x20 )!=0)
        buf &= 0xBF;     // no noise, ignore noise type bit

 

the lines in red force these settings if volume is 0:

 

 Normal operation
 Sample noise source
 Sample 9-bit or 17-bit polynomial generator (see AUDCTL bit 7)
 Mask out clock pulses using 5-bit polynomial generator

 

So I guess that's not a good idea and I should take them out ? 

 

What about the other lines ?
 

Edited by rensoup
Link to comment
Share on other sites

8 hours ago, sack-c0s said:

nice work!

I suppose the next step is to investigate writing an editor. If we're outputting a register stream then we're not necessarily tied to the whole concept of tracker patterns and instrument editing, so that should give a lot of freedom to lay out an editor in a new way...

tdk_2.obx 6.94 kB · 8 downloads

thanks, yes it may be time to look beyond RMT but writing an editor from scratch doesn't seem realistic for such a small user base... I think it may be better to look at what already exists and would be be suited ... but that's probably best left for another thread because it could be bloody ?

 

Link to comment
Share on other sites

2 hours ago, rensoup said:

 

Regarding testC, I remembered @dmsc added an optimization to help the compressor and I thought it may have had the side effect of "stabilizing" playback, so I took it out hoping it would be as jittery as RMT but you guys seem to say it actually improves it ???

 

this is the code:

 

 

the lines in red force these settings if volume is 0:

 

 Normal operation
 Sample noise source
 Sample 9-bit or 17-bit polynomial generator (see AUDCTL bit 7)
 Mask out clock pulses using 5-bit polynomial generator

 

So I guess that's not a good idea and I should take them out ? 

 

What about the other lines ?
 

Honestly, I really don't know what should or should not be added/removed, so that is not a question I can answer with the very little understanding of Atari 8 bit hardware I have :D 
I personally do think that keeping registers at the same state they were before they reach volume 0 would be the most logical choice, but maybe there was a reason for this specific design in the first place, so I cannot say much in that regard ?

Either way, playback after LZSS compression sounds great regardless of A, B or C changes in my opinion, so I think only you can decide :P 

Link to comment
Share on other sites

3 hours ago, rensoup said:

I think it may be better to look at what already exists and would be be suited ...

QFT

No intention to start blood flowing, but as we are talking about streams and not patterns here, IMHO it would make sense to create "Pokey-Sound-Fonts" and import them together with std-MIDI files into an converter to produce the streams. Real musicians are not tied to fixed tracker patterns...

Link to comment
Share on other sites

6 hours ago, sack-c0s said:

With this playback technique there's no need for blood though - we have a shared playroutine and potentially we can all go do our thing in terms of preferred editor. 

It's especially fantastic considering we are not limited to what a tracker would support and not support feature wise; all that matters is how the POKEY chip programming was done to produce unique results, so that means anything could be done if we know what we are doing :D

 

7 hours ago, Irgendwer said:

QFT

No intention to start blood flowing, but as we are talking about streams and not patterns here, IMHO it would make sense to create "Pokey-Sound-Fonts" and import them together with std-MIDI files into an converter to produce the streams. Real musicians are not tied to fixed tracker patterns...

Actually I think I can understand what you mean...

At the end what matters is the resulting output of anything a musician (or a very clever sound programmer!) could do, so the method to achieve it can certainly be narrowed down to personal preferences. 

Link to comment
Share on other sites

7 hours ago, Irgendwer said:

QFT

No intention to start blood flowing, but as we are talking about streams and not patterns here, IMHO it would make sense to create "Pokey-Sound-Fonts" and import them together with std-MIDI files into an converter to produce the streams. Real musicians are not tied to fixed tracker patterns...

I'm not sure what technique that is but personally I'd try to no stray too far from the tracker design because everybody's using it for a reason... or at least I would have an option to switch between whatever fancy view and tracker view.

 

7 hours ago, sack-c0s said:

With this playback technique there's no need for blood though - we have a shared playroutine and potentially we can all go do our thing in terms of preferred editor. 

Of course. I still think it's too big a job though and I doubt anyone will want to work long enough for it to reach maturity...

  • Like 1
Link to comment
Share on other sites

7 hours ago, Irgendwer said:

QFT

No intention to start blood flowing, but as we are talking about streams and not patterns here, IMHO it would make sense to create "Pokey-Sound-Fonts" and import them together with std-MIDI files into an converter to produce the streams. Real musicians are not tied to fixed tracker patterns...

I think creating Pokey soundfonts is more work than creating a new editor :)

 

But any editor could, in theory, could be your editor. I envision vi(m) (or emacs, leafpad, notepad++) editing a text file with a simple syntax, shortcut to run it though simplesyntax2LZSS and replay, etc... With the possibility to just check an instrument, check a single pattern, start at a specified pattern in the song, play whole song, etc...

  • Like 1
Link to comment
Share on other sites

3 minutes ago, ivop said:

But any editor could, in theory, could be your editor. I envision vi(m) (or emacs, leafpad, notepad++) editing a text file with a simple syntax, shortcut to run it though simplesyntax2LZSS and replay, etc... With the possibility to just check an instrument, check a single pattern, start at a specified pattern in the song, play whole song, etc...

Agreed, I wondered if a spreadsheet could do the job at some point ?

 

  • Haha 1
Link to comment
Share on other sites

New release!

 

Get it from the first post !

 

1.1 Jan 2021: 
    -fix for portamento
    -debug output: can output SAPR with empty header for easy comparison with Altirra produced SAPR
    -added flag for merging all selected tunes into one
    -added tune time to A8 player
    -added volume offset for each channel
    -added replay frequency reduction 
    -added tweakable VU-meters colors to A8 player
    -added switch for disabling LZSS volume optimization
 

  • Like 5
  • Thanks 2
Link to comment
Share on other sites

One interesting feature about this release is the "replay frequency reduction" which simply discards pokey updates for tunes above 50hz... I find it very hard to hear a difference between a 200hz tune and its reduced 50hz version for every tune I tried.

Makes me wonder if there's a problem with RMT or if it's just not worth it ?

  • Like 1
Link to comment
Share on other sites

49 minutes ago, rensoup said:

Agreed, I wondered if a spreadsheet could do the job at some point ?

In theory, it could. Export to CSV or something, and have a tool convert that to LZSS. Or VBScript ?  Or whatever LibreOffice uses, and reimplement RMT2LZSS  ;) 

 

But, I think it will be hard to beat simple ASCII text editor that can run binaries and shell scripts by the press of a single key (combination).

Edited by ivop
typo
  • Like 1
Link to comment
Share on other sites

1 minute ago, rensoup said:

One interesting feature about this release is the "replay frequency reduction" which simply discards pokey updates for tunes above 50hz... I find it very hard to hear a difference between a 200hz tune and its reduced 50hz version for every tune I tried.

Makes me wonder if there's a problem with RMT or if it's just not worth it ?

It seems that most 100+Hz tunes use the increased update rate mostly for drums, and not regular instruments. The drums are definitely better at a higher replay rate IMHO, but the rest is more or less the same at 50Hz.

  • Like 1
Link to comment
Share on other sites

40 minutes ago, ivop said:

In theory, it could. Export to CSV or something, and have a tool convert that to LZSS. Or VBScript ?  Or whatever LibreOffice uses, and reimplement RMT2LZSS  ;) 

 

But, I think it will be hard to beat simple ASCII text editor that can run binaries and shell scripts by the press of a single key (combination).

it's more about navigation: with a text editor you'd have to format the text yourself which would probably be painful, unless there are text editors which are column based... not exactly an expert in that area.

 

41 minutes ago, ivop said:

It seems that most 100+Hz tunes use the increased update rate mostly for drums, and not regular instruments. The drums are definitely better at a higher replay rate IMHO, but the rest is more or less the same at 50Hz.

yes that's how I understand it but even for drums I can't really tell the difference using headphones (not fancy ones though)... Have you got an example where you can clearly hear it ?

 

There were a few 100hz tunes which I thought had very clear sound and when I tried reducing them, the sound felt the same, so it seemed to be more about good instrument design

  • Like 1
Link to comment
Share on other sites

Found a bug ;)

 

If the loop is not jumping to Zero, the converter tries to create several tunes. 

But it doesn't recognize a loop back or jump forward. 

A loop back means the end of a tune, jump forward not.

In the case of a jump forward it also produces only several "lz16" files. But no obx file.

Link to comment
Share on other sites

3 hours ago, rensoup said:

yes that's how I understand it but even for drums I can't really tell the difference using headphones (not fancy ones though)... Have you got an example where you can clearly hear it ?

I think this one does the job? ;) The nice thing about 2x speed is that you can more easily work around the 32-40 accumulated volume limit.

 

100Hz-drum-test.rmt

 

 

Edit: here are the .obx files of your latest converter:

 

100Hz-drum-test.obx

 

and down to 50Hz

 

100Hz-drum-test-down-to-50Hz.obx

 

The shuffle and hat sounds are mostly missing. And the kick and snare drum sound nothing like the 100Hz version.

 

Edited by ivop
  • Like 3
Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...