Trying to make some PoKEY music!

[ivop]

Playing a mono file, which shows it writes to a stereo pokey.

Pokey_PutByte: addr=0 byte=31
Pokey_PutByte: addr=16 byte=31
Pokey_PutByte: addr=1 byte=0
Pokey_PutByte: addr=17 byte=0
Pokey_PutByte: addr=2 byte=0
Pokey_PutByte: addr=18 byte=0
Pokey_PutByte: addr=3 byte=0
Pokey_PutByte: addr=19 byte=0
Pokey_PutByte: addr=4 byte=243
Pokey_PutByte: addr=20 byte=243
Pokey_PutByte: addr=5 byte=0
Pokey_PutByte: addr=21 byte=0
Pokey_PutByte: addr=6 byte=152
Pokey_PutByte: addr=22 byte=152
Pokey_PutByte: addr=7 byte=192
Pokey_PutByte: addr=23 byte=192
Pokey_PutByte: addr=8 byte=0
Pokey_PutByte: addr=24 byte=0

 

Edited August 12, 2021 by ivop

[rensoup]

3 hours ago, ivop said:

Conclusion: RMT + sa_pokey.dll is not cycle exact, and will never be. Unless, maybe, a new sa_c6502.dll is created, too, that comunicates the exact cycle when the PutByte() is done to the sa_pokey.dll library with functions that were not part of the original API.

Does it really matter ?

 

Surely the comands can be buffered and then when Process is called you just send them the way you want to the pokey engine ?

 

I did hook up the wav generator to RMT2LZSS a while back(never finalized the details though), I think I just turned the whole song into Pokey commands and when it was done, I sent all the commands to the virtual Pokey... seemed to work ok.

[VinsCool]

Very damn awesome stuff ivop!

I think I may have some explanation regarding a few of the observations, based on how the rmtplayr.a65 was made in the first place and how it is then possible to hack into the rmt.exe.

3 hours ago, ivop said:

- The Pokey engine runs constantly, even if there is no song playing.

That is expected, from what I could tell. Easily noticed when the 'stop' button is pressed, there will still be remaining values left in the POKEY registers (AUDC, AUDF, AUDCTL, maybe a few more?)
As far as I understand, the setting to 'clear registers' was made specifically for that purpose, to make sure all registers are reset to 0, to avoid playback problems when leftover data is still in memory.

So why does it always run? Most likely so we could input notes when the player is stopped, or in 'Jam' mode, where no data is written in patterns, so that way we always get a real time feedback of the sounds we want to create.
There is a setting to always clear registers pressing Escape is pretty much a given for that purpose, because we can get very constant output that would differ very slightly between playbacks (I think for example of polycounters Distortion C timbres, for example, never being 100% the same each time).

 

So to my understanding, it is initialised once, then good to go, with optional 'clear registers' Escape key when needed.
I have no idea what else is affected by resetting the values, I suppose certain things like writing to STIMER during a brief moment after SKCTL is initialised to 0 before being set to 3 again would be a really good way to get a near 100% identical output from initial state every time it is being cleared.

3 hours ago, ivop said:

- BUG1 RMT initializes Pokey_SoundInit only once(!) with the NTSC clock speed (Pokey_SoundInit: freq17=1789790 playback_freq=44100 num_pokeys=2)

Now that's an interesting observation, which also explains another similar observation I have made before.

I noticed RMT initialises the POKEY to PAL clock speed, and regardless of what playback speed was set in the settings, the PAL frequencies will be the ones that play, so there will always be a slight tuning difference between what I was doing in the tracker vs the export itself.
Hardly a major problem in my opinion, but a slightly annoying observation for sure, so I suppose there is something missing regarding how regions are handled there.
As far as I could tell, NTSC may just be a last minute addition using existing parameters and was called good enough, which, in fact, is kinda true actually.

That was good enough, even if not very accurate.

3 hours ago, ivop said:

- BUG2 RMT always initializes 2 pokeys and writes to both, even if the song is mono(!).

I actually know why!
The rmtplayr engine exists in RMT.exe as 3 different entities.

- 1 Stereo rmtplayr for the tracker itself, will switch to Mono when required, which I *think* is simply done by forcing the Left side to be panned Center, then the Right side to be stopped and muted, but never actually disabled!

- 1 Mono rmtplayr for exports (SAP and XEX)

- 1 Stereo rmtplayr for exports (SAP and XEX)

 

This is actually pretty funny, because I think we can both agree that with a little bit of code, there is definitely a way to combine all 3 of them into a single 1. 

It turns out the 2 Stereo rmtplayr in the executable are identical, except for some memory pointers at the start of the file, which is like 1 or 2 bytes very easy to edit.
The Mono rmtplayr on the other hand was specifically assembled with all Stereo features stripped away, but at the core itself, it's almost exactly the same thing too.

I suppose there is to be a more in-depth explanation of this design choice, but this is the conclusion I have come to after a lot of time I have spent hacking the program a little while ago.

3 hours ago, ivop said:

Conclusion: RMT + sa_pokey.dll is not cycle exact, and will never be. Unless, maybe, a new sa_c6502.dll is created, too, that comunicates the exact cycle when the PutByte() is done to the sa_pokey.dll library with functions that were not part of the original API.

This may be the reason why playing back tunes at speed faster than 200hz actually works perfectly fine in the tracker itself, but not in exported data.
Because it isn't cycle accurate, it never suffered from major slowdowns!

[ivop]

25 minutes ago, rensoup said:

Does it really matter ?

 

Surely the comands can be buffered and then when Process is called you just send them the way you want to the pokey engine ?

 

I did hook up the wav generator to RMT2LZSS a while back(never finalized the details though), I think I just turned the whole song into Pokey commands and when it was done, I sent all the commands to the virtual Pokey... seemed to work ok.

It might seem ok, but it really all depends on the timing. Running RMT2LZSS on Altirra has cycle exact Pokey emulation. Playing the .rmt file with RMT has not. I can hear the difference.

Edited August 12, 2021 by ivop

[ivop]

35 minutes ago, VinsCool said:

Very damn awesome stuff ivop!

Thanks.

Quote

I think I may have some explanation regarding a few of the observations, based on how the rmtplayr.a65 was made in the first place and how it is then possible to hack into the rmt.exe.

That is expected, from what I could tell. Easily noticed when the 'stop' button is pressed, there will still be remaining values left in the POKEY registers (AUDC, AUDF, AUDCTL, maybe a few more?)
As far as I understand, the setting to 'clear registers' was made specifically for that purpose, to make sure all registers are reset to 0, to avoid playback problems when leftover data is still in memory.

So why does it always run? Most likely so we could input notes when the player is stopped, or in 'Jam' mode, where no data is written in patterns, so that way we always get a real time feedback of the sounds we want to create.
There is a setting to always clear registers pressing Escape is pretty much a given for that purpose, because we can get very constant output that would differ very slightly between playbacks (I think for example of polycounters Distortion C timbres, for example, never being 100% the same each time).

Yes, it makes sense. It probably runs in a separate thread, and is controlled by the main program.

 

Quote

I noticed RMT initialises the POKEY to PAL clock speed, and regardless of what playback speed was set in the settings, the PAL frequencies will be the ones that play, so there will always be a slight tuning difference between what I was doing in the tracker vs the export itself.

Hardly a major problem in my opinion, but a slightly annoying observation for sure, so I suppose there is something missing regarding how regions are handled there.
As far as I could tell, NTSC may just be a last minute addition using existing parameters and was called good enough, which, in fact, is kinda true actually.

You are right that the RMT editor and player are initialised to PAL (you need to select NTSC 60Hz specifically), but the Pokey backend DLL is not! It always uses the NTSC clock speed and resampling parameters, no matter if your song is PAL or NTSC, which is a bug

 

Quote

That was good enough, even if not very accurate.

I actually know why!
The rmtplayr engine exists in RMT.exe as 3 different entities.

- 1 Stereo rmtplayr for the tracker itself, will switch to Mono when required, which I *think* is simply done by forcing the Left side to be panned Center, then the Right side to be stopped and muted, but never actually disabled!

This is what you see in the output of the dummy sa_pokey.dll. This rmtplayr engines writes the same values to left and right.

Quote

- 1 Mono rmtplayr for exports (SAP and XEX)

- 1 Stereo rmtplayr for exports (SAP and XEX)

 

This is actually pretty funny, because I think we can both agree that with a little bit of code, there is definitely a way to combine all 3 of them into a single 1. 

It turns out the 2 Stereo rmtplayr in the executable are identical, except for some memory pointers at the start of the file, which is like 1 or 2 bytes very easy to edit.
The Mono rmtplayr on the other hand was specifically assembled with all Stereo features stripped away, but at the core itself, it's almost exactly the same thing too.

Yes, and here is where it goes wrong. An exported mono song has a different native player compared to the in-editor player RMT provides.

Quote

I suppose there is to be a more in-depth explanation of this design choice, but this is the conclusion I have come to after a lot of time I have spent hacking the program a little while ago.

This may be the reason why playing back tunes at speed faster than 200hz actually works perfectly fine in the tracker itself, but not in exported data.
Because it isn't cycle accurate, it never suffered from major slowdowns!

Funny how similar conclusions can sometimes be made through a totally different form of "hacking". And together they make even more sense. Nice!

Edited August 12, 2021 by ivop
typoo

[rensoup]

13 minutes ago, ivop said:

It might seem ok, but it really all depends on the timing. Running RMT2LZSS on Altirra has cycle exact Pokey emulation. Playing the .rmt file with RMT has not. I can hear the difference.

Well yeah, isn't it why you're doing this in the first place ( because the current Pokey emulation isn't perfect ) ? 

 

The only thing that is timing dependent is sending the register values to Pokey, and I just batch send them (the rmt xex in Altirra does the same but not in the exact same order if you recall)

 

So -with your upcoming perfect pokey emulation-, once all 9 Pokey_PutByte() have been sent, in process just send the values to the virtual pokey:

 

reg0, StartCycle

reg1, StartCycle+10 (roughly the time for a lda $----,x + sta $---- )

...

reg8, StartCycle+10*8

 

Would be the same as what I do in the RMT2LZSS player. Yeah it's obvious, not sure what the issue is.

[ivop]

6 minutes ago, rensoup said:

Would be the same as what I do in the RMT2LZSS player. Yeah it's obvious, not sure what the issue is.

The problem is that within the sa_pokey.dll you don't know StartCycle and StartCycle+10 or StartCycle+10*8.

 

Even though it is finally resampled to 44100Hz 16-bit stereo, the source signal at CPU clock speed (which is wrongly always NTSC) should be modified and have an effect immediately exactly how it does on real hardware. A communication between sa_c6502.dll and sa_pokey.dll is needed to do cycle exact emulation.

 

17 minutes ago, rensoup said:

Well yeah, isn't it why you're doing this in the first place ( because the current Pokey emulation isn't perfect ) ?

Certainly! But I'm pointing out now why, with the current API, it is not possible to do cycle exact 6502 and Pokey emulation, the way Altirra does it.

[VinsCool]

29 minutes ago, ivop said:

You are right that the RMT editor and player are initialised to PAL (you need to select NTSC 60Hz specifically), but the Pokey backend DLL is not! It always uses the NTSC clock speed and resampling parameters, no matter if your song is PAL or NTSC, which is a bug

 

I mean, I agree with you there!
What I am saying, for sure, PAL tuning is what currently is output from RMT.exe using either sapokey.dll or apokeysnd.dll, so if it's also running with NTSC clock speed... that's a pretty cursed combination, to say the least ?
I guess this may also explains why RMT seems to run at a slightly slower speed than Altirra, both PAL and NTSC speeds, too.

30 minutes ago, ivop said:

Yes, and here is where it goes wrong. An exported mono song has a different native player compared to the in-editor player RMT provides.

It's actually running with the exact same player, except the Mono version was assembled with stereo features stripped away.
I am still playing around the visual player code these days, and since rmtplayr is tightly bonded to it, I am pretty sure there is a simple way to define what mode would be used based on the RMT4 or RMT8 header, for example.
The way rmtplayr.a65 is currently made depends on a lot of IFT ELI ELS EIF statements, but I believe I could easily work around this using a different approach, combined to a method to detect a second POKEY.

I have not yet attempted this, but I am pretty sure that can work out, since I have implemented the region detection already, with the adjustments to the other when necessary, or vice versa, and that uses pretty much the same approach too.  

38 minutes ago, ivop said:

Funny how similar conclusions can sometimes be made through a totally different form of "hacking". And together they make even more sense. Nice!

This is pretty nice how things seem to correlate indeed.
That is also nice because I know we're on the same track too   

 

[rensoup]

3 hours ago, ivop said:

Certainly! But I'm pointing out now why, with the current API, it is not possible to do cycle exact 6502 and Pokey emulation, the way Altirra does it.

I just checked the ASAP code and there is a cycle count (seems obvious I guess)

 

 

Just curious, that tune where you can tell the difference, does it play properly with WASAP given that it's the same Pokey core as RMT ?

 

[ivop]

15 hours ago, VinsCool said:

It's actually running with the exact same player, except the Mono version was assembled with stereo features stripped away.

Hence it's not the same player, timingwise

15 hours ago, VinsCool said:

I am still playing around the visual player code these days, and since rmtplayr is tightly bonded to it, I am pretty sure there is a simple way to define what mode would be used based on the RMT4 or RMT8 header, for example.

Yes, that would be great. A single player for both mono and stereo (selected at runtime by the player itself), and use the exact same player for exports.

12 hours ago, rensoup said:

Just curious, that tune where you can tell the difference, does it play properly with WASAP given that it's the same Pokey core as RMT ?

It's not a specific tune. Sometimes I hear the left speaker cancel the right somewhat, because they are slightly out of phase. Happens mostly at high frequencies. Playing the exported .xex file with a proper mono player on Altirra or real hardware does not have that issue.

[ivop]

New observation, sa_pokey.dll outputs 44100Hz stereo at 8-bits(!). So a Process() with sndn of 1804 is actually 902 8-bit sample pairs. 902/44100*1000 = 20.45ms maximum delay before a PutByte() might actually take effect.

 

Edit: BTW most of the problems we experience with RMT are a problem of RMT and/or its embedded 6502 code, not the sa_pokey.dll per sé. I'm just exploring the possibilities to improve the situation with Vinscool and perhaps Phaeron('s code ). My "upcoming perfect pokey emulation" is not on my todo list yet, but it might be later on. Who knows.

 

But the bugs in RMT have to fixed first or a workaround has to be established. For example, one could create 4 separate sa_pokey.dll's that ignore the resample rate and are either fixed PAL or NTSC (right now it's always NTSC, even with PAL tuning). And on top of that, there's mono and stereo version of both. The mono version should then ignore writes to the second pokey, even though it was "initialised" with two pokeys (which RMT always does), and then only renders the mono channel and copies it to the right channel without extra delay.

 

Edited August 13, 2021 by ivop

[ivop]

From hexdit Rmt.exe: (the only occurence in the whole file)

 

...

00026840   44 00 55 68  44 AC 00 00  68 5E 4F 1B  00 FF 15 9C

...

 

0x001b4f5e equals 1789790, which is the NTSC clock speed. Rmt.exe could be patched to two versions to either use the NTSC clock speed resampling, or the PAL clock speed resampling.

 

 

Edit:

 

changed to

 

...

00026840   44 00 55 68  44 AC 00 00  68 87 0F 1B  00 FF 15 9C

...

 

0x001b0f87 equals 1773447, the PAL clock speed.

 

$ wine Rmt.exe

Pokey_Initialise: argc=0
Pokey_SoundInit: freq17=1773447 playback_freq=44100 num_pokeys=2
 

 

Edit2: if you use this patched Rmt.exe with a proper sa_pokey.dll, you hear the music resampled with freq17=1773447, i.e. PAL clock

Edited August 13, 2021 by ivop

[VinsCool]

Very interesting findings!

Do you know about how apokeysnd.dll works too? Does it use the same structure for arguments, etc?
This one, specifically:

 

apokeysnd.dll

[ivop]

objdump -x apokeysnd.dll | grep APokeySound
	[   0] APokeySound_About
	[   1] APokeySound_Generate
	[   2] APokeySound_GetRandom
	[   3] APokeySound_Initialize
	[   4] APokeySound_PutByte

A totally different API. You can't use this DLL with RMT now, can you? Oh, I see you can!

 

Edit: The API is slightly different (no generic GetByte, but only GetRandom), and Generate is probably called to deliver 44100/8-bit/stereo. The rest can be mapped 1:1 to sa_pokey.dll.

 

Edit2: Really, I never knew you could use this DLL, too. But the API has the same problems as sa_pokey.dll. Alas.

Edited August 13, 2021 by ivop

[VinsCool]

1 hour ago, ivop said:

objdump -x apokeysnd.dll | grep APokeySound
	[   0] APokeySound_About
	[   1] APokeySound_Generate
	[   2] APokeySound_GetRandom
	[   3] APokeySound_Initialize
	[   4] APokeySound_PutByte

A totally different API. You can't use this DLL with RMT now, can you? Oh, I see you can!

 

Edit: The API is slightly different (no generic GetByte, but only GetRandom), and Generate is probably called to deliver 44100/8-bit/stereo. The rest can be mapped 1:1 to sa_pokey.dll.

 

Edit2: Really, I never knew you could use this DLL, too. But the API has the same problems as sa_pokey.dll. Alas.

I see, thanks!
This is the .dll I have been using for a long time now because it's what offers the most accurate POKEY output, as far as I could tell.
Something I am unsure of is, is it the same as the apokeysnd.dll offered in the ASAP repository? It just feels way too different to be the same thing.

I don't wanna bother you but @antrykot if you could enlighten us regarding this .dll it would be really appreciated  
That's the plugin I personally think is the best currently but its origin remains a mystery to me ?

[VinsCool]

I can finally display notes with decent accuracy for the most common combination.
Currently Distortion A and C 64khz are good, 0 and 8 trigger 'Noise' to be displayed instead, everything else use a fallback to display the tracker notes instead, which is ok but octave may be off.
Most of the code right now is pretty ugly but there's a lot of room for improvement later

 

 

Scene 2 - Volcano.obx

[emkay]

6 hours ago, VinsCool said:

I can finally display notes with decent accuracy for the most common combination.
Currently Distortion A and C 64khz are good, 0 and 8 trigger 'Noise' to be displayed instead, everything else use a fallback to display the tracker notes instead, which is ok but octave may be off.
Most of the code right now is pretty ugly but there's a lot of room for improvement later

 

 

Scene 2 - Volcano.obx 8.75 kB · 3 downloads

The progress is nice. 

But I don't understand why always the main voice is totally over the rest of the tune?

On standard speakers and less  volume there seems to be just one channel. 

 

[_The Doctor__]

that happens with lots of music in real life, bands and orchestras do indeed have sections where one section may be purposely louder than the rest, if I adjust my master volume on the stereo... the other sections can disappear , It's organic, qute pleasant and refreshing to hear on a pokey tune

[VinsCool]

Important to point out this is an old tune I made.

One of my earliest, in 2018, actually.

So the volume balance isn't quite the best.

I only used it as an example for the graphical display of musical notes.

 

The only benefit the module got over the old upload was the better tuning, but that's all, nothing else was edited.

 

Edited August 14, 2021 by VinsCool

[ivop]

The same happens with apokeysnd.dll, playing a mono file:

APokeySound_PutByte: addr=0 data=23
APokeySound_PutByte: addr=16 data=23
APokeySound_PutByte: addr=1 data=0
APokeySound_PutByte: addr=17 data=0
APokeySound_PutByte: addr=2 data=0
APokeySound_PutByte: addr=18 data=0
APokeySound_PutByte: addr=3 data=0
APokeySound_PutByte: addr=19 data=0
APokeySound_PutByte: addr=4 data=0
APokeySound_PutByte: addr=20 data=0
APokeySound_PutByte: addr=5 data=128
APokeySound_PutByte: addr=21 data=128
APokeySound_PutByte: addr=6 data=152
APokeySound_PutByte: addr=22 data=152
APokeySound_PutByte: addr=7 data=192
APokeySound_PutByte: addr=23 data=192
APokeySound_PutByte: addr=8 data=0
APokeySound_PutByte: addr=24 data=0

First and only init: (no PAL or NTSC clock speed???)

APokeySound_Initialize: stereo=1

Calling Generate():

APokeySound_Generate: cycles=35468 depth=8
APokeySound_Generate: cycles=35468 depth=8
APokeySound_Generate: cycles=2051 depth=8
APokeySound_Generate: cycles=35468 depth=8
APokeySound_Generate: cycles=3258 depth=8
APokeySound_Generate: cycles=35468 depth=8
APokeySound_Generate: cycles=5429 depth=8
APokeySound_Generate: cycles=35468 depth=8
APokeySound_Generate: cycles=35468 depth=8
APokeySound_Generate: cycles=1046 depth=8
APokeySound_Generate: cycles=35468 depth=8
APokeySound_Generate: cycles=3378 depth=8

Larger values for cycles instead of sndn values (clock cycles vs 1/44100 units), but still no cycle exact PutByte().

 

apokeysnd.c

#include <stdio.h>
  
void __declspec(dllexport) APokeySound_Initialize(int stereo) {
    fprintf(stderr, "%s: stereo=%i\n", __func__, stereo);
}

void __declspec(dllexport) APokeySound_PutByte(int addr, int data) {
//    fprintf(stderr, "%s: addr=%i data=%i\n", __func__, addr, data);
}

int __declspec(dllexport) APokeySound_GetRandom(int addr, int cycle) {
    fprintf(stderr, "%s: addr=%i cycle=%i\n", __func__, addr, cycle);
    return 0;
}

int __declspec(dllexport) APokeySound_Generate(int cycles,
                                               unsigned char *buffer,
                                               int depth) {
//    fprintf(stderr, "%s: cycles=%i depth=%i\n", __func__, cycles, depth);
    return cycles;
}

void __declspec(dllexport) APokeySound_About(const char **name,
                                             const char **author,
                                             const char **description) {
    *name = "apokeysnd dummy";
    *author = "me";
    *description = "dummy";
}

Compile with:

i686-w64-mingw32-gcc -o apokeysnd.dll -shared apokeysnd.c -s -Wl,--subsystem,windows

Put apokeysnd.dll in your (test) RMT directory.

Edited August 14, 2021 by ivop

[VinsCool]

I finally gave a shot at real hardware, thankfully this appears to work perfectly so far
Return to DOS pressing Escape sends me to the Self Test screen, but I accept this over a crash lol

 

 

[ivop]

Nice CRT afterglow on channel 2

Edited August 14, 2021 by ivop

[VinsCool]

28 minutes ago, ivop said:

Nice CRT afterglow on channel 2

Thanks!
Just tested my PAL machine too, it had no problem either.
Unfortunately my monitor is not compatible with PAL colours  

 

[VinsCool]

stuff on screen makes a little more sense now!

 

Marble Zone.obx

[emkay]

59 minutes ago, VinsCool said:

stuff on screen makes a little more sense now!

 

Marble Zone.obx 8.11 kB · 2 downloads

Very nice. I'm missing the dying cats, which is a great thing