rb+ tutorial #5: build.bat - just a build script?

ggn · March 16, 2017

build.bat is unfortunately rb+'s front-end. It's not very fancy or cool like an IDE, it's just something that gets the job done. It's responsible for compiling all sources and binaries into a single file, ready to be executed in emulators or real hardware. However it has slowly evolved into a much more sophisticated (read: complex) system. And here's why.

Invoking build.bat without any parameters will print something like this





Usage: build.bat projectname
       build.bat projectname sendy
       build.bat projectname ROM
       build.bat projectname ROM sendy
       build.bat projectname ROM UNPACKED
       build.bat projectname ROM UNPACKED sendy
       build.bat projectname sendy bjl
       build.bat projectname new

Folder "projectname" must exist inside folder "projects"
and have a file called "projectname.bas" inside.

When building a ROM it is assumed that there will exist
a file called assets.txt inside the project folder
and will contain all assets to be included in ROM (if any).
By default the rb+ main program is packed into the ROM
and depacked during boot. This can be avoided by passing
the UNPACKED switch.

if you specify "new" then a new project will be created
from the "include\template" folder. No project will be
created if folder exists.

If one of the parameters is BOSSMODE then virtualjaguar is not run.

If one of the parameters is CLEAN, the build folder will be wiped.
Otherwise, an incremental build is preformed

if you specify "sendy" the project, upon succesful compilation
will be sent to Skunkboard (caution - when using ROM, bank
#1 will be erased without any warning!)

"bjl" will upload the binary to Jaguar via BJL. Note that
you might need to edit the command line parameters to the uploader
in this file (just search for "lo_inp.exe")

Current projects:
(list of project directories)

Let's go through all the options.

- To build a new project just type build.bat myproject new. This will create a folder inside your projects directory and copy a few files that are absolutely needed for a project to build. These are:

rapapp.s - raptor skeleton assembly file. You generally don't need to mess around with this file, with a single exception when you choose your audio engine code.
rapinit.s - raptor init file. You definitely need to mess around with this file as it contains the definitions for all raptor objects and lists.
rapu235.s - contains the audio sample definitions when using the U-235 sound engine. Not discussed in this post.
<projectname>.bas - Very bare source file that contians some needed initialisations. Generally keep all lines before the first RLOCATE and delete the rest.
assets.txt - imported assets (graphics/sound/etc). Covered in another post.
assets\partipal.bmp - this bmp file essentially contains the palette values used for the particle/print layer. Edit this for an easy way to change the colours.
assets\fonts\f_16x16.bmp / f_8x16.bmp / f_8x8.bmp - these files contain 3 different sized fonts in 4bpp format. Edit these to change the default fonts.

- To build a project as an .abs file (i.e. runs from RAM) and send it to virtualjaguar type build.bat myproject. If all goes well, virtualjaguar will start up with your compiled binary. Otherwise the build log will be printed to the console so you can see what went wrong.

- To build a project as an .abs file (i.e. runs from RAM) and send it to the jaguar via skunkboard type build.bat myproject sendy. This assumes that a skunkboard is actually connected to your dev machine, working and ready to upload a binary. No checks are performed for this - it's your responsibility!

- To build a project as an .abs file (i.e. runs from RAM) and send it to the jaguar via BJL type build.bat myproject sendy bjl. This assumes that a BJL cable is actually connected to your dev machine, working and ready to upload a binary in 8bit mode. No checks are performed for this - it's your responsibility!

- To build a project as a .rom file (i.e. runs from ROM) and send it to virtualjaguar type build.bat myproject rom. If all goes well, virtualjaguar will start up with your compiled binary. Otherwise the build log will be printed to the console so you can see what went wrong.

- To build a project as an unpacked .rom file (i.e. runs from ROM) and send it to virtualjaguar type build.bat myproject rom unpacked. If all goes well, virtualjaguar will start up with your compiled binary. Otherwise the build log will be printed to the console so you can see what went wrong.

- To build a project as a .rom file (i.e. runs from ROM) and send it to the jaguar via skunkboard type build.bat myproject rom sendy. This assumes that a skunkboard is actually connected to your dev machine, working and ready to upload a binary. No checks are performed for this - it's your responsibility!

- To build a project as an unpacked .rom file (i.e. runs from ROM) and send it to the jaguar via skunkboard type build.bat myproject rom sendy unpacked. This assumes that a skunkboard is actually connected to your dev machine, working and ready to upload a binary. No checks are performed for this - it's your responsibility!

- In order to speed build times up the build system caches imported assets. The cache folder is build and resides in your project folder. The rule to re-convert an asset is simple, if the converted file inside the build folder is newer than the source file, then the conversion is skipped. However this might go wrong for whatever reasons (clock skew, changing machines, timezones, maybe you download a file with a date set into the future etc etc) so this mechanism can screw up. There's not much to do in this case except clean the cache folder. You can do this either manually or by adding clean to all the above command lines, for example build.bat myproject clean, build.bat myproject sendy clean etc. If you don't want to ever want to worry about this issue at the expense of slower build times (this can vary depending on the files to be converted) then you can always build cleanly.

- Finally if you want to simply build the project and not run it, prepend your command line with bossmode. For example build.bat myproject bossmode, build.bat myproject clean bossmode etc.

Build process explained

At first building rb+ binaries sounds pretty straightforward - i.e. just compile and link the damn code! However, as we realised it is a bit more complex than this.

But let's cover the bare essentials first. rb+ consists of a few building blocks:

bcx, a basic to C translator. This enables you to type code in Basic without having to code in C which is scary for most beginners.
gcc 4.6.4 which translates C code to 68000 machine language.
Raptor engine, which covers the graphics side of things (also audio and input but we'll get to that).
U-235 Sound Engine and Zerosquare Audio Engine, which handle audio and input.
rmac which assembles 68000 and Tom/Jerry code into bytecode.
rln which links binaries produced by rmac and gcc together into a final binary.

A simple build system would do the following in order:

Assembles raptor skeleton code (rapapp.s)
Translates the .bas file into .c
Compiles the resulting .c file
Link the above produced files together with raptor and u-235 (which are provided as libraries, as they're close sourced)

That's all fine if we want to compile the .bas file provided by the template when we create a new project. But what happens when we want to import graphics and audio? In this case first of all the assets have to be converted into native format data. Then they have to be exposed to the code somehow. Both issues are solved by inserting an extra step to run a specialised program to read assets.txt and produce the converted files, as well as a file called ramassets.inc which is then read by rapapp.s and a file called romassets.h (a bit wrongly named, but bear with me) to be read by gcc when compiling the code. So our build process now becomes:

Runs buildlink to do the asset conversion (if any) and produce include source files
Assembles raptor skeleton code (rapapp.s)
Translates the .bas file into .c
Compiles the resulting .c file
Link the above produced files together with raptor and u-235 (which are provided as libraries, as they're close sourced)

And that would be all that would be required... if we weren't building ROM files...

Up till now we didn't have any problem on where assets are placed - we just let rmac and rln deal with it. rmac reads all assets so it knows where inside the binary they will reside, so it marks down their addresses from the start of the binary. Then rln reads all those marked addresses, and since it knows where in RAM the binary will run (we use $4000 always) it can calculate all addresses and update all marks. This is great since we can change the order of the assets or place them wherever we want inside the .s file and together with rln they will find a solution.

There are two ways to produce a ROM. One is to place all assets in RAM and just create a ROM that copies its contents to RAM and runs them. In this case nothing is changed: you mark all assets as "abs" in assets.txt, the same prodedure as above happens with the extra step of making a ROM file, which includes writing a valid ROM header, then a small piece of code that copies ROM to $4000 and then moves the execution pointer (or Program Counter) there.

In the other mode it's probably not feasible to have all assets in RAM so you also have to use some space in ROM. After all a ROM can be up to 6MB (although 4 is the usual maximum) which is bigger than the 2MB of RAM. But now a problem arises: where do we put the ROM assets? At this stage I could have just shrugged and said: "it's yours, folks!", leaving you to sort out the mess and hardcode addresses. For people that ask "why not just place them after the rb+ binary?" consider that we don't know how much the binary will be before compiling and linking. So with this idea there is absolutely no way to do a 1 pass build and have this info known during building and not after.

The solution was very simple: just place assets at the start of the ROM, then the rb+ binary - after all this does get copied to RAM so we don't care much where it will end up. Then their addresses will be fixed and we can expose them to rb+ and raptor. Then we can create a new file called romassets.inc which will contain all these new addresses. romassets.h will now double up to contain all these addresses for gcc too. One extra thing that requires attention here is that all assets are placed aligned to 16 byte boundary (i.e. two phrases), just to be on the safe side (we don't know which of the assets will be used by the OP or the Sound Engine). Finally, all the assets are placed into one large file so we don't have to deal with alignment again. So our build process now becomes:

Runs buildlink to do the asset conversion (if any) and produce include source files. Also create a linkfile with all assets and produce extra include files in the case of ROM building.
Assembles raptor skeleton code (rapapp.s)
Translates the .bas file into .c
Compiles the resulting .c file
Link the above produced files together with raptor and u-235 (which are provided as libraries, as they're close sourced)

Back to ROM building, it did sound like an awful waste to have a binary file in ROM that just gets copied to RAM and then executed. So by default the build process will automatically pack the RAM binary before placing it into ROM. When the ROM boots it will unpack the binary to RAM instead of just copying it. This way quite some memory can be claimed back. The build process now looks like this:

Runs buildlink to do the asset conversion (if any) and produce include source files. Also create a linkfile with all assets and produce extra include files in the case of ROM building.
Assembles raptor skeleton code (rapapp.s)
Translates the .bas file into .c
Compiles the resulting .c file
Link the above produced files together with raptor and u-235 (which are provided as libraries, as they're close sourced)
When creating a ROM, pack the resulting binary
When creating a ROM, call makearom to create the ROM

The special program simply writes a file with the cartridge header, the boot/unpack code, the ROM assets (if any), the rb+ binary and finally pads the file to 1,2,4 or 6MB. However, people complained that the unpack sometimes takes too long and think that the system has crashed. So the command unpacked was added so it's possible to create an unpacked ROM for testing (or final). For example, build.bat projectname ROM unpacked or build.bat projectname ROM UNPACKED sendy.

And that's it! Hopefully next time you open build.bat instead of going "WTF is going on here???" you can appreciate all the hard work rb+ does for you .

Edited June 2, 2017 by ggn