[RELEASE] Raster Music Tracker, Unofficial version 1.33 BETA by VinsCool

[ivop]

On 3/19/2022 at 2:45 AM, VinsCool said:

Well, there we have it. Thank you Mathy for the reply, and confirmation.

As long as this is the request given, I must oblige, and not share the sources, with all the work I added into.

 

Maybe time will tell, but for now, the most I can do is share occasional progress and updates in this thread, when I am able to show something new  

While it is true I do not enjoy being restricted with what I can share, I respect the decision, and I hope I will still be able to make something good out of what I can do!

I really think it's a pity that you are not allowed to share the source code and your improvements. In the first place, two questions: Why? And what is the Štěrba family stance in this?

 

Many people on this forum could help you with C++. In your posts, I already noticed you mentioned a couple of "rookie" mistakes, that you fixed, and learned from, but it could have progressed faster if you didn't have to reinvent the wheel, so to speak.

 

And I don't mean myself specifically. I'm mostly a C99 programmer, and use C++ as "C with classes" if I'm forced to use C++ (Qt5 for example). I don't know what a protected abstract
virtual base pure virtual private destructor is, and I never needed one

 

But off the top of my head, I can easily name four or five programmers on AtariAge that have 30+ years of experience in C++ and the Visual Studio environment. Why not use that potential?

 

Edited March 23 by ivop

[VinsCool]

19 minutes ago, ivop said:

Many people on this forum could help you with C++. In your posts, I already noticed you mentioned a couple of "rookie" mistakes, that you fixed, and learned from, but it could have progressed faster if you didn't have to reinvent the wheel, so to speak.

I agree with that.

It was especially difficult for me to do the things I wanted to do so far for the fact that I literally did not know how to code until the month of July.

I am not exaggerating right now, I learned C++ out of necessity on the spot around the month of October just to be able to add something into RMT, so obviously the rookie mistakes are plentiful since then, lol

 

But hey, I learn something new every day!

Despite the limitations I was facing, I still have found a way or another to get things to work, that's still something!

 

At the moment, I mostly have everything I wanted to get working, for now I need to learn a bit more about the concept of dialog boxes for the options I am editing.

It's just a matter of time, really.

I mean, even from lack of experience, there are still many things I was able to find and improve, including bugs that have existed for a very long time, it's a nice feeling to have! Hah!

 

Did you know for example that the communication between sa_pokey.dll/apokeysnd.dll and RMT relied on simply being timed, and wouldn't even take care of if it was reading invalid data?

I did not know that either, and just by handling it slightly differently, random chances of crashing from reading an invalid pointer when the sound was reset for setting PAL/NTSC and/or MONO/STEREO output disappeared almost entirely!

If the process was done too quickly, it would even try to read from a NULL pointer, and would immediately crash, specifically.

This makes me believe the way things were initialised in the first place was actually deliberate, in order to work around that situation.

 

Well anyway, yeah, I still have a lot to learn, and that's part of the fun, right?

 

 

[VinsCool]

real time memory display anyone?

I actually wrote that function for editing displaying the tuning lookup tables, but I just realised nothing stops me from using it to display any parts of the emulated Atari memory, so why not? hah!

 

 

[EDIT]

So this seems to work pretty well now

 

 

 

 

 

Edited March 24 by VinsCool

[emkay]

Currently I don't find a better place to post this. 

But my own thread has been ... you might know

 

I used some "special tuning" within this tune. Do you get it?

 

 

 

 

[VinsCool]

I don't have anything to show tonight, but I got the memory viewer tweaked significantly, now it's a nice and clean debugger-like display, and all the data is updated in real time when things are edited in memory  

It's also it's own void function now, and I can feed to it 2 parameters: the g_atarimem[address] offset and the number of bytes I want to copy, then all of the desired bytes will be copied to a loop, and processed to make an array of strings in the "table" format. Trust me, I am going somewhere with this, the formatting is deliberate, and will have many more uses later.  

 

I also finally have gotten the dialog box tweaked further and now the "test" option seems to work, at last, this makes all of the things mentioned above done in real-er time, and cancelling the dialog box would revert to the previous options used.

I have not yet coded the reset button, but now that I actually know what class member I had to call and where I had to call it, this won't take longer to update.

 

Not more to show tonight, however, I'm getting tired, and I'm starting to hallucinate columns of hexadecimal numbers if I keep going like this lol

[VinsCool]

3 hours ago, VinsCool said:

Not more to show tonight, however, I'm getting tired, and I'm starting to hallucinate columns of hexadecimal numbers if I keep going like this lol

Nevermind, I'm a fucking liar, of course I couldn't get off my computer when I want to test some ideas 😹

 

What will come next is only more exciting! Stay tuned (hehe I'm funny not)

[ilmenit]

@VinsCool - I didn't have time yet to look at the recent changes in your fork of RMT, but the most missing functionality from the developer perspective was "automation of export" so RMT could be included into "one click build system". Now after every single change done by musician you had to manually start the RMT, select exporting to a stripped version, writing manually an address, making a copy-paste of the configuration file etc. Could you add a command line automation for this, please?

[VinsCool]

14 hours ago, ilmenit said:

@VinsCool - I didn't have time yet to look at the recent changes in your fork of RMT, but the most missing functionality from the developer perspective was "automation of export" so RMT could be included into "one click build system". Now after every single change done by musician you had to manually start the RMT, select exporting to a stripped version, writing manually an address, making a copy-paste of the configuration file etc. Could you add a command line automation for this, please?

I think this is a really good idea, actually!

Now I have no idea how this could be done, but I will keep this in mind, because this honestly could be very useful for batch conversions too.

 

At the moment I have kind of broken the export format, but the tables generator I have been working on is one of the many things that will be part of it, so you won't have to manually edit that stuff every time.

 

Another idea I had was to pair a .txt file output with the stripped module export specifically for the same reason you mentioned, so I imagine there is a way to combine this into a command line argument and save a lot of time for everyone

 

[VinsCool]

I told you all it was only a matter of time and figuring it out before it happened  

 

Unofficial RMT 1.32 + SAPR Dumper + Future LZSS Converter

 

So far, SAPR dumper appears to work perfectly, I had to convert to LZSS using RMT2LZSS however, but I actually plan to write my own conversion procedure directly from RMT itself  

So far this was a lot easier than I expected... I'm a bit scared right now lol

[VinsCool]

Not yet demonstrated here, but yesterday's test version was then used on a large amount of modules I had around.

Honestly, I am SHOCKED.

 

Everything I tested, and I mean, EVERYTHING I tested, was dumped to SAPR with perfect success rate, and the resulting LZSS compression was a 100% perfect match as a result.

 

I am very excited to push all of this to the next level.

And before someone asks me, yes, I do plan to not rely on a limited 6502 driver anymore once all of this stuff is done.

 

There are so many things I want to do with the .rmt module format, it's making my everything resonate with ecstatic motivation, I'm not going to lie about this!

 

[ivop]

From a demo/game perspective where space is sometimes limited, it would still be nice to have a 6502 player.

 

Do you plan a release shortly? I would really like to play with all the tuning options and create a new tune for SV2K22.

 

[VinsCool]

3 hours ago, ivop said:

From a demo/game perspective where space is sometimes limited, it would still be nice to have a 6502 player.

 

Do you plan a release shortly? I would really like to play with all the tuning options and create a new tune for SV2K22.

 

Well honestly I could just release test versions more often... at the risk of being overwhelmed by a lot of potential bugs people will find.

 

I actually did plan to release a new version as soon as I got my Tuner code completed, which is not very far from being the case, but I feel a bit conflicted as well... 

A lot of things need work, and I just feel bad if I release things in their incomplete state, while showing off all I have been doing in the meantime.

 

Please tell me, what do you want?  

Should I post new interim versions more often, at the risk of being buggy and/or unfinished, and thus, likely to make unfamiliar users frustrated?

Or

Make a new version release when I feel it is ready?

 

There are also some other elements to consider right now...

The 6502 driver still need work, I had some bugs I noticed recently that require my attention... and also a lot of the optimisations have yet to be done as well.

The VUPlayer code is also unfinished, and lacks many of the planned features, and the code for itself has a lot to be desired, (for my defense, I did write that entire player from scratch, VU Meter parts included, except from the timing related loop that was borrowed from the reverse-engineered-then-improved RMT XEX player...)

 

That being said, unfinished/bugs apart, the tracker in its current form does work quite well still, so if you really want an interim release, I have no problem with it.

I'm just conflicted over the buggy/incomplete functions I have been working on, but if that can be ignored, things still work nicely, as shown in my recent Sketch tunes I recorded the other days  

 

All and all, everyone is allowed to call me stubborn or ungrateful, but this is precisely why I wish I had the permission to make the entire project open source... I would not feel bad for seeing builds every commit, if that's what people wanted!

[Beeblebrox]

@VinsCool IMHO I'd release when you are ready. 😁 Releasing on a regular basis, as you say, will likely see you overwhelmed with bug reports and sidetracked with requests and likely you'll spend more time trying to keep up with it all than actually working on it.

 

I guess what you could do is have a couple coder/musician testers whom you could run things by via PM's. 

 

Keep up good work. 😉

[VinsCool]

@Beeblebrox Thank you!

I mean if people want to get versions more often I have absolutely no problem with that  

I am just trying to do exactly what you said, to actually work on things themselves as we speak!  

 

If anyone wants to get to test features early, feel free to PM me, or if the demand is high, I would just drop more test builds in this thread, I have no problem with either option.

Maybe I should make a Poll in the thread's header...

[VinsCool]

come one youtube you can do better than 360p after 2 hours
but here's a SAPR dumper testrun, with pretty direct LZSS conversion going as well  

if you have a spare 3 hours here's me screwing around
I promise there is good tunes

[VinsCool]

Work in Progress: full SAPR dumper is now functional, loop detection is functional as well! frame perfect registers dump straight from RMT, conversion to LZSS is then a piece of cake  

[ivop]

On 4/5/2022 at 1:47 AM, VinsCool said:

Please tell me, what do you want?  

I'm not in a hurry yet, as SV2K22 is many moons in the future, but what I would like is all the cool stuff you showed in videos, like all the tuning stuff and easy use of sawtooths, and then export a stripped RMT file, a header file with the options (un)set, and then assemble a minimal player.

 

For a 16kB entry, which will hopefully sport four to six effects, and lasts around three minutes, it is not feasible to use the LZSS player. It uses less cycles per frame, but a lot of memory for 180 seconds of music.

Edited April 6 by ivop

[VinsCool]

1 hour ago, ivop said:

I'm not in a hurry yet, as SV2K22 is many moons in the future

Alrighty!

1 hour ago, ivop said:

but what I would like is all the cool stuff you showed in videos, like all the tuning stuff and easy use of sawtooths, and then export a stripped RMT file, a header file with the options (un)set, and then assemble a minimal player.

This should all be done, in this case, when it is ready 

Speaking of sawtooth, this is currently the only table I have not yet generated in my tuning code, but this is more due to a circumstantial reason, that one will also be added later.

Everything else that already did have a table at some point is already fully functional as we speak 😊

1 hour ago, ivop said:

For a 16kB entry, which will hopefully sport four to six effects, and lasts around three minutes, it is not feasible to use the LZSS player. It uses less cycles per frame, but a lot of memory for 180 seconds of music.

I understand!

It really is great for large tunes, but smaller ones won't get compressed that good, as far as I could tell.

 

---

 

Currently, here's my deal with the POC SAPR/LZSS export from RMT as of this message:

 

- SAPR dump is fully functional, but currently I had to map the function calling to a keyboard shortcut until I write the proper dialog box code/class for it.

That means what is shown in my video was literally what happened in real time, for testing purposes. A pretty crude method but for quick tests that did a fine job at least  

 

- My dumper's loop point detection is very primitive.

But it is more than good enough to at least get a full tune play and stop at a frame perfect position with identical output every times I run it!

I am not exactly sure how rensoupp achieved his own looping method, but I'm studying the LZSS player sources to find out!

I can at least manipulate the playback sequence, but I don't exactly like "having" to add an extra section of data just when the previous data exists and is 99% the same... I will find a way!

 

- Only SAPR dump from RMT with optional SAP header infos being written.

But that is party related to what I stated above, that's just because it's not necessary right now for my tests.

A proper handler for all of that will be a simple enough task to add since there's already plenty of SAP/XEX export code at my disposal  

 

- LZSS compression is not yet done from RMT's side.

But I plan to add it up, hopefully that shouldn't be too hard to do  

 

So what can i say? It does work pretty well so far if SAPR is the only data needed, so that's a pretty good step done for sure.

 

---

 

With this in mind, I have plenty of things to polish and finish properly.

 

The Tuning dialog box has several things incomplete, but that's really not the most urgent matter, it's just a handful of things to finish now!

The Test and Reset buttons, specifically, are not finished, due to random issues related to the Dialog class stuff, but I am learning more about programming so that's just a matter of time to get it done.

 

As far as tables generation goes, this is a success for sure!

Temperament/Ratio tuning is also working quite well.

Non 12-Tones scales aren't that great in this, but my tests for simpe things like Pentatonic or Greek Diatonic scales do work, which is certainly a good sign.

I also need to find a way to "offset" what notes and positions every tables start, since my approach for it was rather dumb, involving generating a reference in memory, and have every notes tuned from that reference, which is then converted into a table for the g_atarimem[0xB000] block, where I put all my tables.

 

Exporting them in XEX/SAP/Whatever is then very easy since I can simply do a quick memcpy to whichever location I find use for them  

Obviously SAPR recording doesn't need all that extra bitbanging orgy since it will literally just record whatever the player does "in real time" but with the fastest possible timer, with respect to VBI rate as well, hehe

 

🦆 Anyway, I got sidetracked replying to your post, sorry about that 😅

 

[ivop]

21 hours ago, VinsCool said:

It really is great for large tunes, but smaller ones won't get compressed that good, as far as I could tell.

The main thing is repeating patterns. Like, my Lullaby interpretation will compress pretty good. But Procrastinate probably a lot less, as it are basically three songs in one. I'm aiming for at least four effects and music progressing from one song/effect to the other. I have a few songs in mind that could work well with the RMT player (which in itself is a form of compression), but probably not so much with the LZSS player.

 

21 hours ago, VinsCool said:

🦆 Anyway, I got sidetracked replying to your post, sorry about that 😅

Rubber 🦆 is always useful

 

I'm a big fan of https://en.wikipedia.org/wiki/Release_early,_release_often

 

I might not be the proper tester/user at this moment, but others might be. Like emkay, miker, synthpopalooza, makery, etc...

 

Something I'd like to see in the future is better MIDI input. Right now it is step by step, but it would be cool if you can just loop a song line, and set MIDI record to a certain pattern and then play your bass line, or chords. Or even record to a certain column of the song list, and improvise cool synth solos that span more than four bars

 

Also, a mixture of LZSS and a song player. Each pattern could be LZSS compressed separately, and then played in the right order according to the RMT song data. That might be a good compromise between LZSS and a plain RMT player.

 

Edited April 7 by ivop

[VinsCool]

1 hour ago, ivop said:

Also, a mixture of LZSS and a song player. Each pattern could be LZSS compressed separately, and then played in the right order according to the RMT song data. That might be a good compromise between LZSS and a plain RMT player.

This is precisely what I had in mind too, based on my earlier tests.

There's a really good potential to take this approach, if I combine my initial loop point detection, and add case by case patterns into it.

I'm certain there's a way to get chunks of tunes to gain a drastic amount of size reduction that way, but I can see the drawback of having to keep track of an index for each ones of them during play time...

 

Now I won't get ahead of myself too soon, but I believe this may be possible to do. 

As far as I could tell, that is what rensoupp already did for the tune + looped tune player he made, so now it would simply be a matter of scaling it to several tiny tunes, then mix and match the data for the patterns accordingly, keep track of the playback index, and roll around with each of the tunes pointers whenever they are required during the playback.

This approach could save up on the long run if the same patterns keep coming during playback, I imagine.

 

However, I have no confirmation yet if that could even be considered since I barely have dabbed into the LZSS player driver as we speak.

Plus out of nowhere I have gone back to the RMT driver code to try fixing a bunch of the things I did poorly so yeah, I cannot stay focused at 1 thing at the time can I? 😹

[ivop]

58 minutes ago, VinsCool said:

This is precisely what I had in mind too, based on my earlier tests.

There's a really good potential to take this approach, if I combine my initial loop point detection, and add case by case patterns into it.

I'm certain there's a way to get chunks of tunes to gain a drastic amount of size reduction that way, but I can see the drawback of having to keep track of an index for each ones of them during play time...

I think you don't need any loop detection anymore. Just LZSS compress each pattern, and play them according to the song data. Loops are done through the song lines. And each pattern is only compressed once, instead of each time it is heard in the background.

 

Quote

Plus out of nowhere I have gone back to the RMT driver code to try fixing a bunch of the things I did poorly so yeah, I cannot stay focused at 1 thing at the time can I? 😹

No worries. You are doing great stuff

Edited April 7 by ivop

[DerpDerp3001]

Does anybody know what a note key is? I am having difficulty with the software.

Edited April 8 by DerpDerp3001

[VinsCool]

6 hours ago, ivop said:

I think you don't need any loop detection anymore. Just LZSS compress each pattern, and play them according to the song data. Loops are done through the song lines. And each pattern is only compressed once, instead of each time it is heard in the background.

Yeah that was exactly what I meant to say!  

I referred to loop point as to where the song actually comes to the "end" of data sequence needed in this case.

6 hours ago, ivop said:

No worries. You are doing great stuff

Thanks a lot!

Tonight I finally fixed the 16-bit generation code I added a while ago to truly work as 16-bit numbers now, this fixed a lot of bugs as well!

Found a lot of new improvements in many places I had edited in the driver so far, so this feels really good to see I can find some new shortcuts to progressively debloat or repair what I broke inside the hacked driver!  

1 hour ago, DerpDerp3001 said:

Does anybody know what a note key is? I am having difficulty with the software.

Hello, a Note Key refers to all of the keyboard keys used to input notes, typically the layout would look like something like this:

I hope this helps you  

[DerpDerp3001]

19 minutes ago, VinsCool said:

Yeah that was exactly what I meant to say!  

I referred to loop point as to where the song actually comes to the "end" of data sequence needed in this case.

Thanks a lot!

Tonight I finally fixed the 16-bit generation code I added a while ago to truly work as 16-bit numbers now, this fixed a lot of bugs as well!

Found a lot of new improvements in many places I had edited in the driver so far, so this feels really good to see I can find some new shortcuts to progressively debloat or repair what I broke inside the hacked driver!  

Hello, a Note Key refers to all of the keyboard keys used to input notes, typically the layout would look like something like this:

I hope this helps you  

But how exactly do I set up those keys? Since when I initially tested the application, none of the keys on my keyboard would insert notes.

Edited April 8 by DerpDerp3001

[VinsCool]

22 hours ago, DerpDerp3001 said:

But how exactly do I set up those keys? Since when I initially tested the application, none of the keys on my keyboard would insert notes.

Did you create a new song line pattern, and tried to input notes in the tracks that way? No pattern would not accept any note.

Or maybe were you accidentally in "JAM MODE" (formerly known as PROVE MODE)? In this case CTRL+SPACEBAR would go back to EDIT MODE.