rb+ tutorial #3: rb+ and audio - untangling a mess!

[ggn]

At its conception rb+ was very closely tied to raptor, which uses the U235 Sound Engine. Raptor has its own way of doing things and audio is no exception. Including a tune meant opening rapapp.s and adding it there, adding samples would mean converting them to the right format by hand and then feeding it to your project AND editing another .s file... Let's say that early adopters found all this bewildering and counter intuitive .

 

Gradually and with user feedback things improved: rb+ commands were added to give better control for playing music and triggering sfx, assets.txt was added so importing became less of a pain (you can now even feed it an mp3 and it'll convert to the format you need). In general, things improved - not perfect but still.

 

Then a second audio engine was added! People got sick of .mod files and wanted to stream some audio instead. The new audio engine by Zerosquare offers μLaw samples playback and while it does waste a lot of space, the results are quite impressive!

 

A major component of both sound engines is that they both handle joypad/rotary/etc input.

 

Add all this to the mix - confusion ensues!

 

So let's try to de-confuse things a little. First, a comparison between the two engines.

 

U-235

Can play .mod files: yes (4 channel ones)

Can play samples: yes, up to 4 raw PCM samples alongside with the .mod

Fine control of samples: yes - pitch and volume

Supports μLaw compressed samples: no

Supports mouse input: no (it probably can, never tested)

 

Zerosquare

Can play .mod files: no

Can play samples: yes, up to 4 simultaneously

Fine control of samples: no - fire and forget affair

Supports μLaw compressed samples: yes

Supports mouse input: yes

 

Which one to choose? Well, that's up to you. Main difference is of course .mod files vs μLaw music playing. As for the rest... read on.

 

Selecting a sound engine in your project

 

This is quite easy to do. For legacy reasons the U235 engine is selected by default in every new project. In your project folder open rapapp.s and locate the following line:

 

player equ 1      ;0=Zerosquare's player, 1=U-235 player

As instructed, replace the 1 with 0 if you want to use Zerosquare's engine.

 

U235 usage

 

In order to play a .mod file you first have to import it as an asset. Same goes for sound samples. Have a look at the assets.txt article for this.

 

Especially for samples, raptor (or U235) (or both IDK, LOL) require the use of sample banks. This is in essence a list where you have to define your samples. It is defined in file rapu235.s of your project. Typically an entry in that table looks like this:

 

sample0:	dc.l	explode_sam		; start of sample
		dc.l	explode_sam_end	; end of sample
		dc.l	0				; repeat offset
		dc.l	0				; repeat length
		dc.w	0				; <NULL>
		dc.b	0				; fine tune
s2_vol:		dc.b	192				; volume
		dc.l	8000			; default play rate

The first two parameters are the sample's start and end addresses. These are automatically available for you from assets.txt. In this example if you import a sample and assign it the name explode_sam, rb+ will give you the addresses explode_sam and explode_sam_end for free. The next two parameters control sample looping: which sample from the start you would like looping to occur and how many samples it should be. NULL - well, just leave it to 0 . volume is a value from 0 to 255 and finally play rate is the replay frequency. This probably goes up to 32000 but I'm not sure really - just try it out if you're curious.

 

To play a module from rb+ it pretty straightforward. Just use MODPLAY(mod_address) and replace mod_address with the label you used to import the module in assets.txt. Stopping a module and restarting is a bit more tricky as the SE doesn't have a clear cut mechanism for it. This snippet of code might work but it's not fully tested on real hardware yet:

        MODPLAY(0)
        SNDKILL(0)
        SNDKILL(1)
        SNDKILL(2)
        SNDKILL(3)
        VSYNC
        U235SE_modregdump[0]=0
        U235SE_modregdump[2]=0
        U235SE_modregdump[3]=0
        U235SE_modregdump[4]=0
        U235SE_modregdump[5]=0
        MODPLAY((int)strptr(Module2))

To play a sound sample with the default values just use sndplay(voice,x) where voice is the channel you want to use (4 to 7, 0-3 are used for .mod playing) and x is the sample number from the start of the sample bank. If you want to play the sample at a different frequency you can use SNDPLAYFREQ(voice,x,y) which uses channel voice to play sample x at frequency y (Hz).

 

For more info on sample commands have a look at the list below (which is copied from rb_quickref.txt).

 

To handle the jagpads, first of all you have to set rotary_mode1 and rotary_mode2 to correspond to your controllers (1=rotary, -1=jagpad). For classic d-pad handling you first have to use getpad(x) where x=0,1 the joypad you wish to read (alternatively you can just read the variables U235SE_pad1 and U235SE_pad2 directly). These values contain the states for all buttons, so to detect just one button you need to mask out the rest. So something like

if x band (PAD_U) then           'checks for up
if x band (PAD_U bor PAD_L) then 'checks for up+left

Have a look at the mask values below for the full set.

 

Here's the full command set for U235:

 

MODPLAY(mod_address) plays a .mod file at address "mod_address"

MODPLAY(0) stops .mod playing

GETPAD(x) returns values from either pad 1 or 2. (Can be replaced by reading U235SE_pad1 and U235SE_pad2 directly.)

SNDPLAYFREQ(voice,x,y) triggers sample x from module into channel "voice" at y frequency in Hz

MODVOL(x) sets mod music volume (x in range 0 - 63)

SNDVOL(x) sets global sfx volume (x in range 0 - 63)

SNDKILL(x) stops playing sample at channel number x

SNDVOLRESET(x) resets volume of current sample on channel x

SNDFREQRESET(x) resets frequency of current sample on channel x

SNDDELTA(x,y) set or adjust the volume on channel x to y (0 to 63)

SNDFREQ(x,y) sets frequency of channel x to y (0 to 65535)

rotary_mode1 +1 = rotary, - 1 = jagpad (Port 1) (.l)

rotary_mode2 +1 = rotary, - 1 = jagpad (Port 2) (.l)

turn_direction1 rotary value (bigger is faster) - +=left / 0=nothing / - =right (Port 1) (.l)

turn_direction2 rotary value (bigger is faster) - +=left / 0=nothing / - =right (Port 2) (.l)

rotary_interval1 trim value for rotary (Port 1) (.l)

rotary_interval2 trim value for rotary (Port 2) (.l)

spin_delta1 value to add to turn_direction per increment (Port 1) (.l)

spin_delta2 value to add to turn_direction per increment (Port 2) (.l)

Direction pad masks PAD_UP, PAD_U, PAD_DOWN, PAD_D, PAD_LEFT, PAD_L, PAD_RIGHT, PAD_R, PAD_HASH, PAD_9, PAD_6, PAD_3, PAD_PAUSE, PAD_A, PAD_OPTION, PAD_STAR, PAD_7, PAD_4, PAD_1, PAD_0, PAD_8, PAD_5, PAD_2, PAD_B, PAD_C.

 

Zerosquare

 

Zerosquare's engine is much more simple in comparison, but is a bit less flexible.

 

You can play a sample in each of the 4 channels it has available using SNDZEROPLAY chan, start_address, len, frequency, params where chan=1 to 4, start_address and len can be obtained from assets.txt like above, frequency is an integer divisor to the main clock of 46168Hz and params sets the replay mode. The syntax is slightly different if playing a sample from RAM or ROM:

 

Example for playing samples from RAM (ABS):

SNDZEROPLAY(1, strptr(sample_start), (strptr(sample_end)-strptr(sample_start)+3) and 0xfffffffc, 46168/9233, Zero_Audio_8bit_muLaw|Zero_Audio_Looping)

This will play a sample in channel 1 starting from sample_start, with length sample_end-sample_start rounded up to 4 bytes, with a channel divisor of 5 (i.e. 46168/9233) in order to play 9233Hz. The sample will be μLaw packed, so you need to import it as such. Finally we ask the engine to loop the sound forever.

Example for playing samples from ROM:

SNDZEROPLAY(1, (void *)sample_start, (sample_end-sample_start+3) and 0xfffffffc, 46168/9233, Zero_Audio_8bit_muLaw|Zero_Audio_Looping)

This does exactly the same as above, only the sample is in ROM.

 

To silence a channel just play a zero sized sample (len=0).

 

Finally, to read inputs, first choose what inputs you need (d-pad, rotary, mouse) by calling the routines Input_SetNormalPadMode, Input_SetJoyPort1, Input_SetJoyPort2, Input_SetRotaryMode, Input_SetAtariMouseMode, Input_SetAmigaMouseMode. Then each time you need to read input just call ZEROPAD subroutine. Then variables zero_left_pad, zero_right_pad, zero_mousex_delta, zero_mousey_delta and zero_rotary_delta are filled accordgingly. Again you'll need to mask out the status for the buttons you don't need for pad, so something like

if zero_left_pad band Input_Pad_Star then

will check left pad if the star button is pressed.

 

And here's the list of variables and functions exposed when you use the Zerosquare engine:

 

ZEROPAD() reads both pad ports and sends results back to variables zero_left_pad, zero_right_pad, zero_mousex_delta, zero_mousey_delta and zero_rotary_delta. By default the engine is configured to assume 2 joypads connected. This command needs to be used in capitals.

Input_SetNormalPadMode sets up the engine to read two pads (enabled by default). zero_left_pad and zero_right_pad can be read using these constants: Input_Pad_Pause, Input_Pad_A, Input_Pad_Up, Input_Pad_Down, Input_Pad_Left, Input_Pad_Right, Input_Pad_C1, Input_Pad_B, Input_Pad_Star, Input_Pad_7, Input_Pad_4, Input_Pad_1, Input_Pad_C2, Input_Pad_C, Input_Pad_0, Input_Pad_8, Input_Pad_5, Input_Pad_2, Input_Pad_C3, Input_Pad_Option, Input_Pad_Sharp, Input_Pad_9, Input_Pad_6, Input_Pad_3. example: "if zero_left_pad band Input_Pad_Star" will check left pad for star button press.

Input_SetJoyPort1 enables joypad port 1 to be used for rotary/mouse input. This doesn't enable rotary/mouse mode. zero_rotary_delta will give the number of rotary ticks since the last read command.

Input_SetJoyPort2 enables joypad port 2 to be used for rotary/mouse input. This doesn't enable rotary/mouse mode. zero_rotary_delta will give the number of rotary ticks since the last read command.

Input_SetRotaryMode enables rotary mode.

Input_SetAtariMouseMode enables Atari mouse mode. Input_Mouse_Left and Input_Mouse_Right are the masks to check for button presses zero_mousex_delta, zero_mousey_delta are the number of mouse ticks in x and y axis since the last read command.

Input_SetAmigaMouseMode enables Amiga mouse mode. Input_Mouse_Left and Input_Mouse_Right are the masks to check for button presses zero_mousex_delta, zero_mousey_delta are the number of mouse ticks in x and y axis since the last read command.

SNDZEROPLAY chan, start_address, len, frequency, params plays a sample starting from start_address with length len to channel chan with speed frequency and with flags params. *Channel* should be from 1 to 4. start_address should be aligned to 4 bytes. len should be a multiple of 4. frequency should be an integer that divides the base frequency of 46168Hz. So for example if it's set to 1, it'll play a sample at 46168Hz, a 2 will play a sample at 23084Hz etc. Available flags are: Zero_Audio_8bit_Signed (plays an 8-bit signed sample), Zero_Audio_8bit_Unsigned (plays an 8 - bit unsigned sample), Zero_Audio_8bit_muLaw (plays a 8 - bit compressed ?w sample) - Zero_Audio_Looping (enables sample loop). Example for playing samples from RAM (ABS): SNDZEROPLAY(1, strptr(sample_start), (strptr(sample_end)-strptr(sample_start)+3) and 0xfffffffc, 46168/9233, Zero_Audio_8bit_muLaw|Zero_Audio_Looping). Example for playing samples from ROM: SNDZEROPLAY(1, (void *)sample_start, (sample_end-sample_start+3) and 0xfffffffc, 46168/9233, Zero_Audio_8bit_muLaw|Zero_Audio_Looping)

Edited March 29, 2017 by ggn

[ggn]

Just added a first draft of this - hopefully it makes some sense to people. If there are things unclear - just reply here!

[Zerosquare]

Zerosquare's engine is much more simple in comparison, but is a bit less flexible.

Let's be honest: it's a lot less flexible than U235

You can't set the volume, there is no pitch control either (besides the very coarse divisor selection), there's no stereo support....

 

That's because it wasn't originally designed as a general-purpose audio engine, but as an experiment. Basically "let's see if it's possible to have 4 channels of high-quality audio on the Jaguar". Hence everything that had an impact on performance or wasn't absolutely required got left out. The only reason why an experiment got integrated into the ST ports and rB+ is because CJ and ggn liked the results

 

The downside to this, besides the limitations that have been mentioned above, is that the code is a complete mess. If you look at it and go crazy, don't say you weren't warned

Seriously though, the codebase really makes evolutions difficult or impossible. There are a lot of things I'd like to add: volume/pitch/panning control, samples chaining, more channels, dynamic channels allocation, automatic fade-outs for clickless "cuts", better audio compression algorithms... (and some other stuff I won't mention because they're even more speculative), But I can't do that without starting from scratch ; as that's a lot of work, I don't know when (or if) it will be done.

 

As a guide, consider using the player if:

- you want high audio quality - with 46 kHz sample rate, interpolation and µLaw encoding support, I believe it's the best available option, short of using CD Audio tracks on the JagCD.

- you use streamed music and don't need real-time tweaking of your sound effects.

- you need mouse support. It's completely unrelated to the audio (and could be done with other audio players as well), but it needs no additional work and is known to work fine.

 

It is not suitable if:

- your music is in MOD format (you could convert the music to a sample, but that would use tons of memory).

- you want to change the pitch and/or volume of your sound effects in real time.

Edited March 28, 2017 by Zerosquare

[Zerosquare]

 

strptr(sample_end)-strptr(sample_start)+3) and 0xfffffffc

 

Don't do that, please -- it can introduce nasty clicks at the end of samples (or at regular intervals if the sample is looped)!

 

Instead, make sure your audio length is a multiple of four samples.

Or add a few silent samples to bring the audio length to a multiple of four before feeding it to the rB+ converter.

Or ask ggn to modify the converter so that it does that automatically.

Or ask ggn to ask me to write some code to do that.

 

But please don't do that. Glitchy audio makes me sad

Edited March 28, 2017 by Zerosquare

[ggn]

As a guide, consider using the player if:

- you want high audio quality - with 46 kHz sample rate, interpolation and µLaw encoding support, I believe it's the best available option, short of using CD Audio tracks on the JagCD.

- you use streamed music and don't need real-time tweaking of your sound effects.

- you need mouse support. It's completely unrelated to the audio (and could be done with other audio players as well), but it needs no additional work and is known to work fine.

 

It is not suitable if:

- your music is in MOD format (you could convert the music to a sample, but that would use tons of memory).

- you want to change the pitch and/or volume of your sound effects in real time.

 

Thanks for this, will update the top post on the players comparison.

 

 

Don't do that, please -- it can introduce nasty clicks at the end of samples (or at regular intervals if the sample is looped)!

 

Instead, make sure your audio length is a multiple of four samples.

Or add a few silent samples to bring the audio length to a multiple of four before feeding it to the rB+ converter.

Or ask ggn to modify the converter so that it does that automatically.

Or ask ggn to ask me to write some code to do that.

 

But please don't do that. Glitchy audio makes me sad

 

Believe it or not I've got you covered on this (more or less). If you read my ramblings on build.bat you'll notice that during building some extra files are generated, including a file that contains the asset imports (ramassets.inc). Let me paste a snippet from this from an actual generated file:

 

 

.dphrase
BMP_PLAYER: incbin "build/_nyancat.bmp.gfxdata"
BMP_PLAYER_end:
BMP_PLAYER_clut: incbin "build/_nyancat.bmp.clut"
BMP_PLAYER_clut_end:
.extern BMP_PLAYER
.extern BMP_PLAYER_clut
.dphrase
BMP_ENEMY: incbin "build/_ufo.bmp.gfxdata"
BMP_ENEMY_end:
BMP_ENEMY_clut: incbin "build/_ufo.bmp.clut"
BMP_ENEMY_clut_end:
.extern BMP_ENEMY
.extern BMP_ENEMY_clut
.dphrase

See? After every file the .dphrase will insert zeros to pad the files - so there will always be silence between samples . Telling the people to add silence would have been tricky - what if people supplied a mp3 file to import for example?

 

(well ok, there's a corner case here but I'll make it insert a dc.b 0,0,0 after each incbin so it'll sort itself out. Happy now? - [EDIT] There you go, committed!)

Edited March 29, 2017 by ggn

[Zerosquare]

See? After every file the .dphrase will insert zeros to pad the files - so there will always be silence between samples

That's fine for 8-bit signed, but it's broken for 8-bit unsigned and muLaw, and will cause a loud click for sounds whose length isn't a multiple of four (zero doesn't map to silence for those formats).

You can either:

- pad the length before conversion

- pad the length after conversion (use $00 for 8-bit signed, $80 for 8-bit unsigned, and $FF for mulaw)

- ask me to sort it out

 

Your choice

Edited March 29, 2017 by Zerosquare

[sh3-rg]

I had some clicky problems when looping ulaw samples, I fiddled with the end position and it vanished. Never mentioned this to anyone, probs should have.

[ggn]

That's fine for 8-bit signed, but it's broken for 8-bit unsigned and muLaw, and will cause a loud click for sounds whose length isn't a multiple of four (zero doesn't map to silence for those formats).

You can either:

- pad the length before conversion

- pad the length after conversion (use $00 for 8-bit signed, $80 for 8-bit unsigned, and $FF for mulaw)

- ask me to sort it out

 

Your choice

8-bit unsigned isn't supported by the import tool at all, so it's not an issue. μLaw is now padded with $ff.

 

...next?

[Zerosquare]

[+Gemintronic]

Stupid question: can I swap these two sound systems in and out of memory? Meaning, can I play back the music to Degz and then switch over to digital samples for a cutscene?

[+CyranoJ]

Stupid question: can I swap these two sound systems in and out of memory? Meaning, can I play back the music to Degz and then switch over to digital samples for a cutscene?

 

No, it is a decision you have to make when building your project as to the direction you want to go in. Input and RNG paths are also tied to the audio system.

 

It *is* possible to swap the sound systems out, but the DSP is a fickle beast.

[ggn]

Stupid question: can I swap these two sound systems in and out of memory? Meaning, can I play back the music to Degz and then switch over to digital samples for a cutscene?

Stock answer: Start doing things. If you get something working and you're really stuck with this - give a shout here. Maybe we could work something out!

[Clint Thompson]

Ok - I've got a small loop for background music going and it's clicking or popping at the end but is dead silent on both the front and end of the mp3... what exactly is it I need to do to rid this funk?

 

SNDZEROPLAY(1, strptr(EMERGE), (strptr(EMERGE_end)-strptr(EMERGE)+3) and 0xfffffffc, 46168/15389, Zero_Audio_8bit_muLaw)

[ggn]

Try telling the sound routine to play the exact amount of bytes instead of rounding them up, see if that helps:

 

SNDZEROPLAY(1, strptr(EMERGE), strptr(EMERGE_end)-strptr(EMERGE), 46168/15389, Zero_Audio_8bit_muLaw)

[Clint Thompson]

Try telling the sound routine to play the exact amount of bytes instead of rounding them up, see if that helps:

 

SNDZEROPLAY(1, strptr(EMERGE), strptr(EMERGE_end)-strptr(EMERGE), 46168/15389, Zero_Audio_8bit_muLaw)

 

Yeah, I toyed around with deleting things first and trying different things before asking because I hate to ask but the exact posted above will play the loop once and then make bezerk glitch sounds afterwards instead of looping so I just shut it off... I should also mention the sound is looping, which I didn't have here blindly pasted the wrong thing:

 

 

Doing it like this will make it glitch bad:

SNDZEROPLAY(4, strptr(LEVEL1MUSIC), strptr(LEVEL1MUSIC_end)-strptr(LEVEL1MUSIC), 46168/15389, Zero_Audio_8bit_muLaw|Zero_Audio_Looping)

 

and this makes it pop each loop even though it's clean:

 

SNDZEROPLAY(4, strptr(LEVEL1MUSIC), (strptr(LEVEL1MUSIC_end)-strptr(LEVEL1MUSIC)+3) and 0xfffffffc, 46168/15389, Zero_Audio_8bit_muLaw|Zero_Audio_Looping)

 

I guess my question too is what is the+3 doing?

[+CyranoJ]

I'm guessing its not starting / ending on a long boundary, so the clipping is causing a sample or 2 to skip.

[ggn]

I'm guessing its not starting / ending on a long boundary, so the clipping is causing a sample or 2 to skip.

"not starting" is not possible as each imported asset is aligned to multiples of 16. Probably the later then.

 

I guess my question too is what is the+3 doing?

It's not just the "+3", it's used in conjunction with "and 0xfffffffc". To explain what happens:

 

- We want to ensure that a number is a multiple of 4 rounded up, i.e. 1 will be rounded to 4, 2 will be rounded to 4, 3 will be rounded to 4, 4 will be rounded to 4, 5 will be rounded to 8 etc.

- One classic numerical recipe to do the rounding up is to do integer((number+round_value-1)/round_value)*round_value (or something to that extent).

- If we break down to the binary level, we notice that 0 is 0000, 1 is 0001, 2 is 0010, 3 is 0011, 4 is 0100, 5 is 0101, 6 is 0110, 7 is 0111, 8 is 1000, 9 is 1001, etc etc. If you haven't noticed already every multiple of 4 has the two last bits zeroed.

- So if we combine all the above and notice that we want to round up to a power of two, we see that to round up to multiples of 4 it's simply a matter of adding 3 to our number and clear the last 2 bits of the result.

- Binary ANDing of two numbers is simply taking each number's binary representation and check each bit from one number with the corresponding bit of the other. Then produce 1 only if both bits are 1. So 1 AND 0 is 0, 0 AND 1 is 0, 1 AND 1 is 1, 0 AND 0 is 0. (4 AND 5 is 4 - left as an exercise to the reader).

- 0xfffffffc is a 32bit value and in binary it is 11111111111111111111111111111100, conveniently keeping all bits to 1 except the last 2.

 

So to sum up, add 3 to the number we want to round up and zero the last 2 bits using and 0xfffffffc. Hope that makes things more clear!

 

 

 

Now, as to your problem. A few posts above an issue with μLaw packed samples was addressed and a fix has been pushed to the repository. Is your rb+ install up to date?

 

If yes then if you don't mind could you upload the offending mp3 file here or somewhere and give a link?

Edited January 3, 2018 by ggn

[+CyranoJ]

- We want to ensure that a number is a multiple of 4 rounded up, i.e. 1 will be rounded to 4, 2 will be rounded to 4, 3 will be rounded to 4, 4 will be rounded to 4, 5 will be rounded to 8 etc.

 

 

ah, that's the problem.

 

Don't round up, just drop the low 2 bits.

 

Rounding up on MuLaw could put 0 in the output stream, which isn't silence.

[ggn]

ah, that's the problem.

 

Don't round up, just drop the low 2 bits.

 

Rounding up on MuLaw could put 0 in the output stream, which isn't silence.

Good observation but the sample is already padded with $ff to avoid that problem if you see the above posts. Unless Clint doesn't have the latest rb+, in which case try downloading the latest from github, clear your build folder, rebuild and see if that works.

[Clint Thompson]

Good observation but the sample is already padded with $ff to avoid that problem if you see the above posts. Unless Clint doesn't have the latest rb+, in which case try downloading the latest from github, clear your build folder, rebuild and see if that works.

 

Will try that, thanks guys!

[Clint Thompson]

Try telling the sound routine to play the exact amount of bytes instead of rounding them up, see if that helps:

 

SNDZEROPLAY(1, strptr(EMERGE), strptr(EMERGE_end)-strptr(EMERGE), 46168/15389, Zero_Audio_8bit_muLaw)

After downloading the 100% new and improved version of Rb+ and dropping the +3, this has finally resolved my audio click-pop issues. Thanks!

[Clint Thompson]

Back with a question in regards to SNDVOL - assuming this works with all sounds/music played by the Zeroplayer, does it only change volume before and/or after playing a song or sound but unable to change sound during playback? I was looking to do a fade out sound by having the SNDVOL decrease slowly over the course of like 10 seconds but it's not working and maybe because it isn't setup to work that way?

[Zerosquare]

The Zeroplayer doesn't have a volume control, so SNDVOL is probably only for the U-235 player.

Unfortunately this means there's no easy way to do fade-outs/fade-ins.