Cartridge Programming

[snicklin]

I've seen various discussions around here on what I think might be talking about programming for cartridges, but I don't really know much about it at all.

 

Due to this, a lot of it goes over my head somewhat and I am lost under a lot of acronyms.

 

I am considering programming for cartridges and need to know the differences between the different cartridge types and what my limitations would be.

 

Is it quick/slow to swap a bank of memory? Is it different between the different cartridge types? What are the rough costs of each cartridge type?

 

Is there a guide anywhere that could help me with this?

 

Or perhaps stick to disk programming???

[Rybags]

It can be safely assumed that the common bankswitch types are instantaneous or at least fast enough such that a read on subsequent cycle will hit the freshly selected bank.

 

There's all manner of bankswitch types around, in fact the possibilities could go into billions but most work by either specific write to a $D5xx address or range or simply to an access to a $D5xx address or range.

e.g. some types you store the bank # into a register address, others you simply access a certain address to switch banks.

 

A good description of most banking schemes is in the Atari800Win+ Help system.

 

In a way, cart programming is easier than disk. With a boot disk there's nothing to do extra if it's a single stage boot but there's work to do if multi-stage boot or if it loads stuff as the game or demo goes along.

Generally with a cart it's "all there" though one methodology of doing stuff on cart is to just treat the thing as a big virtual disk and load parts into Ram as needed.

Or do it both ways, have a sort of fixed kernal area then load stuff into Ram. Of course copying and running from Ram will usually assume a 32K or greater system.

 

What can be tricky with carts is dealing with programming in that with bankswitching you have multiple sets of data and program that occupy the same addresses and also aren't all available at once. That can play havoc at times but does encourage programming with certain structures and protocols in mind.

[snicklin]

Thank you for your (as ever) informative response there.

 

I was considering whether I could have software with a lot of resources (differing music / graphics) per level and then just use the bank switching as a loading mechanism. In an emulator that would be quick, but then I want to consider a user who uses a normal Atari with a cartridge in there.

 

For example, if someone was to use a SIDE2 (one of the many, many alternatives), is it able to address as much space as there is on the compact flash card? i.e. (not that I would), in theory, could I write a game that is 8gb in size and then just switch through the banks? I presume that there is a limit, perhaps 16mb? But then is there a web site that will tell me the limitations of each cartridge type? Perhaps I could write my code to be compatible with multiple cartridge types or I could release differing versions?

 

Sorry if I am asking daft questions, I have never gone beyond 64K.

[snicklin]

What can be tricky with carts is dealing with programming in that with bankswitching you have multiple sets of data and program that occupy the same addresses and also aren't all available at once. That can play havoc at times but does encourage programming with certain structures and protocols in mind.

 

I reckon that if I get that right in my mind in the first place, I should be OK here. I'd just consider keeping the main code in normal memory and then switching in music / graphics / text etc as required.

[Rybags]

SIDE I don't think has any FlashRom dedicated to user programs, it just has the onboard SpartaDos and the CF card media is accessed at the sector level as per most other media types from Atari.

 

I think the Atarimax IDE cart might have some user area.

 

Personally, I'd be looking at supporting one or at best a few cart types. Atarimax flash types and maybe SIC and another 1-2 types. If you aim for too many it has potential to become a maintenance headache.

Also note of course there's different banking schemes like 8K only that's variable Rom bank or switched off completely, then ones where the top 8K is fixed and lower 8K bankable.

To do programming to support those different types can become cumbersome. Of course if you're doing so using the "virtual disk" philosophy then it can be easy, all you need then is a resident driver/loader depending on what cart type is in use.

 

So I suppose bottom line is - choose the programming philosophy first, then base how extensive you want to support carts based on that. The virtual disk idea has it's pitfalls but a big advantage is you can easily support practically any cart type, the only limitation is if it has enough Rom space to suit you.

[flashjazzcat]

SIDE has 256KB of flash ROM on the "external" cart: this is usually occupied by the XEX loader, which takes only a couple of banks. You can put anything you like there, and either boot it directly with the switch in the loader position, or enter it from SDX via "CAR" on the command line.

[phaeron]

One thing about writing for cartridges is that you can't use self-modifying code unless you copy code to RAM, which can be annoying if you're used to using it everywhere.

 

You can't address CompactFlash devices as memory. They're essentially IDE devices and have to be driven through read/write commands to a sector at a time through a small set of memory-mapped ports. The parallel ATA protocol is pretty simple, but it's still not as direct or as fast as mapped ROM. The CompactFlash adapter carts out there also have varying levels of stability, i.e. some older designs had a habit of corrupting data. A very cool game could be done with the huge amount of storage on a CF card, but not that many people have a suitable cart.

 

A gotcha with bankswitched carts is the #&*($ cartridge lockup code in the XL/XE OS, which IIRC is a pain on warm reset and when temporarily unmapping the cartridge. Then there's the extra problems on SECAM machines, which apparently actually exist.

[Rybags]

Yep, the cart interlock + checksum is a pain. Simply disabling a cart to get to the Ram can lock up the machine thanks to the OS VBlank keeping watchdog on the insert status.

The checksum, they got it wrong in more than one way. Only the last 16 bytes of the cart is used, for whatever crazy reason the first 240 bytes of the OS is also involved though I feel it's likely a typo that never got fixed in development.

 

Interlock can be gotten around by either not using the OS Stage 2 VBlank or using your own code that ensures the interlock register in Ram reflects what the state is.

Checksum can be a tricky one... IIRC the OS does that check before it calls any cart or Dos init vectors, so one way around it can be to keep the last 16 bytes of all banks the same. In any case you need to maintain the flag/vector area at $BFFA-BFFF for each bank anyway, so doing another 10 bytes isn't too much extra work.

[Island2Live]

Regarding the »checksum«, it's now getting also interesting for me, because I am also developing a game for cartridge. Can you put some light on this checksum-thing please? Or maybe provide a link where I can further investigate by myself?

 

Kind regards,

Henrik (Island2Live)

[Wrathchild]

The sources of some very early work I did when converting Gauntlet to different cartridge formats is available here. Might make a useful study

[snicklin]

SIDE I don't think has any FlashRom dedicated to user programs, it just has the onboard SpartaDos and the CF card media is accessed at the sector level as per most other media types from Atari.

 

Lets say for example that you have an 8gb card, can't an 8gb file (or 2gb if there are file size limitations on CF) be placed on the CF card and then the program be accessed like any other game on the card? So it sort of like a big disk file but in a cartridge? This is opposed to having a "real" cartridge.

[Wrathchild]

Maybe relevant, is there any equivalent of ProWizard's work on the r16 format employed for cart conversions running from the MyIDE2?

[flashjazzcat]

 

Lets say for example that you have an 8gb card, can't an 8gb file (or 2gb if there are file size limitations on CF) be placed on the CF card and then the program be accessed like any other game on the card? So it sort of like a big disk file but in a cartridge? This is opposed to having a "real" cartridge.

 

You can do pretty much what you like with the CF card: you could even write your own filesystem driver, or fill 8GB with raw sectors full of data. But the thing about the HDD is that - as Avery wrote - it's not something you can "bank" in and out. You need RAM to hold the data you're pulling from the CF card. This is one of the advantages of using a banked ROM for your project: hundreds of KB of code space right there in the cart window, instantaneously accessible.

[snicklin]

You can't address CompactFlash devices as memory. They're essentially IDE devices and have to be driven through read/write commands to a sector at a time through a small set of memory-mapped ports.

 

Thanks Avery, I didn't notice this comment before.

 

And thanks Jon for point out the above comment.

 

I see now. One thing that I was thinking that was possible (although I didn't want to implement) was the ability to have a massive graphics mode 8 video. Each frame would be on an 8K boundary and then the banks would be switched every frame to point to the next frame. As I have to copy the data through, this isn't possible.

 

Thank you all for your comments, they are helping me to understand this subject better.

[phaeron]

Videos can definitely be played off of CompactFlash, but the CPU can only copy around 110KB/sec max. The fastest I've done CPU-based video+audio is about 86K/sec and around 9 fps. There are very dirty tricks that can be used to play video faster, but you'd have to be insane to use them in a production.

The Cart! has a massive 1Gbit flash, which could be used for a decent length bank-switched video... but again, you would be targeting a very small user base.

 

Regarding the »checksum«, it's now getting also interesting for me, because I am also developing a game for cartridge. Can you put some light on this checksum-thing please? Or maybe provide a link where I can further investigate by myself?

 

The XL/XE OS detects cartridges via TRIG3 and copies its value to GINTLK ($03FA) during early OS init. It also computes the checksum of $BFF0-C0EF and stores it in CARTCK ($03EB). These two variables cause the following pain:

• Stage 2 VBLANK checks TRIG3 against GINTLK, and if they don't match, it jumps to an infinite loop at $C0DF to hang the machine. TRIG3 is 0 if $A000-BFFF is unmapped and 1 if it is mapped. Therefore, switching banks won't cause problems, but disabling or enabling the cartridge will. Note that built-in BASIC doesn't affect TRIG3.
• Warm start recomputes the cartridge checksum, and if it doesn't match CARTCK, forces a cold start.

The checksum isn't that big of a deal, because not every cart needs warm start capability. Even if you did, you could probably defeat it just by using a single byte to make the checksums the same for all banks (it's the same as the SIO checksum and thus trivial to manipulate).

 

SECAM introduces an additional problem since the FGTIA chip only reads triggers on horizontal blank, and therefore plain code that tries to copy TRIG3 to GINTLK after a cartridge change won't work reliably, since sometimes the code gets a stale value from TRIG3.

 

Not sure why Atari bothered with these checks. The GINTLK lockup code isn't even very safe, since it leaves NMIs enabled, so stage 1 VBI still keeps running and can still light up fireworks.

Edited March 25, 2015 by phaeron

[snicklin]

Videos can definitely be played off of CompactFlash, but the CPU can only copy around 110KB/sec max.

 

Well, one of the other options that I was thinking about was to have advertisement boards within a game. Alternatively, TV sets that are a few characters wide and high. Then the data would be taken from a cartridge/side 2 (etc!) in order to provide a large amount of data for the display. That data could be copied into the font RAM. However I don't really want to take a lot of CPU time away in order to fulfil an extravagance.

[pirx]

I see now. One thing that I was thinking that was possible (although I didn't want to implement) was the ability to have a massive graphics mode 8 video. Each frame would be on an 8K boundary and then the banks would be switched every frame to point to the next frame. As I have to copy the data through, this isn't possible.

 

A bit over 10 years ago I was investigating porting American Laser Games products to XL/XE (contacts with IP holders failed, but I was possibly not persistent enough).

Anyway, for this project I conjured "pirxompression" - a scheme for compressing moving pictures to roughly 2KiB (and then later storing them on a CF in a manner similar to MyIDE).

 

Imagine this - convert the graphics to character graphics and then find most similar characters.

Create 128 classes of abstraction where each class is a group of similar characters.

Replace characters in a class with a character most similar to others.

Finally you get 1 character set (1024 bytes) plus a character map (~1000 bytes) per frame. Round it to 2KiB for some additional information like collision maps, sound registers data, etc. Much more feasible to copy this amount of data than 8KiB.

 

You can look how does it work in action in my little "NO SIGNAL" intro.

 

There are several ways to tweak this idea to get nice visuals - I was using simple Pythagorean distance formula where each character is a vector of coloured pixels and this can be definitely improved (e.g. rastaconverter uses different distance formulas to a great effect).

In the intro above I was just choosing the first character in a group to replace all subsequent ones, but this is definitely not necessarily the best choice possible.

A simple fix would be to start converting graphics to character data starting not from the upper left corner, but from the middle of the screen and spiral outwards. Basically the centre of the screen would get the best characters and the more peripheral characters would get what is left over.

 

Now, for the moving pictures it is also possible to reuse some (if not all) character set data in a following frame, so finally a kind of simplistic 8-bit "mpeg" would be possible, getting the size of the frame to something like 1.5KiB on average (a ballpark value).

 

I have actually coded this those years ago. Just for laugh looked at the code and it is so awful it hurts Anyway, this is perfectly doable.

 

I have also looked at Dragon's Lair(s) and Space Ace and these are even better candidates for such a conversion. Another possible games are Megarace (a pre-rendered racing track and real-time animated cars on it), but the one I liked the most was "Solar Crusade" from Infogrames.

 

Nowadays I would ditch Compact Flash for its size, price, unreliability and power requirements and would go for SD with some (very) simple atmega or other interface chip on a cart. More tinkering with hardware and software but definitely more power without the price.

 

Best,

 

pirx

Edited April 11, 2015 by pirx

[snicklin]

Imagine this - convert the graphics to character graphics and then find most similar characters.

Create 128 classes of abstraction where each class is a group of similar characters.

Replace characters in a class with a character most similar to others.

Finally you get 1 character set (1024 bytes) plus a character map (~1000 bytes) per frame....Much more feasible to copy this amount of data than 8KiB.

 

 

Snap! I've been considering something similar myself. It is similar to how JPEG works, by having 64 different 8x8 blocks and then selecting the closest one.

 

You could also utilise some Run Length Encoding compression on the blocks to keep the size down.

 

For lower bandwidth, you could always reuse the character set from frame to frame. Perhaps update it every so many frames, maybe 8 for example. Or perhaps only update it when the "distance" from the real picture has surpassed a threshold. Perhaps even you could keep a cache of the last few character sets used and then select the best one from what is in memory?

[pirx]

Reusing characters works, but the quality deteriorates very quickly, maybe when very little moves on the screen it would last for more than a couple of frames.

Anyway, this is lossy compression and may be as bad as one wishes

[dmsc]

Hi!,

 

A bit over 10 years ago I was investigating porting American Laser Games products to XL/XE (contacts with IP holders failed, but I was possibly not persistent enough).

Anyway, for this project I conjured "pirxompression" - a scheme for compressing moving pictures to roughly 2KiB (and then later storing them on a CF in a manner similar to MyIDE).

 

Imagine this - convert the graphics to character graphics and then find most similar characters.

Create 128 classes of abstraction where each class is a group of similar characters.

Replace characters in a class with a character most similar to others.

Finally you get 1 character set (1024 bytes) plus a character map (~1000 bytes) per frame. Round it to 2KiB for some additional information like collision maps, sound registers data, etc. Much more feasible to copy this amount of data than 8KiB.

This is a type of vector quantization, many video codecs employ a similar technique where you have a cedebook (your character data) and the indices to the codebook (your screen data).

 

In FFMPEGs libavcodec (the best multi-codec compression library) there is an implementation of the ELBG algorithm to find the best codebook approximating a source image/video; and a codec for the A64 commodore 64 video format that uses this exact technique. This could be adapted (with monochrome and 128 chars only) for the atari.

[snicklin]

Here is a decent example of it, it is somewhat lossy but it still looks good.

 

(26s to 51s)

[pirx]

It is nice to know my ideas were quite clever

 

 

I have another idea for hardware assisted decompression making it even more similar to mpeg ^_^ When panning happens - compensate sub-character panning with smooth horizontal and vertical scroll. It would be perfect for some scrolling backgrounds, especially in drawn animation type films like Dragon's Lair.

Normally with panning the whole character set is busted each frame. If we can compensate for the movement with soft scroll, we could save some bytes.

With the complication of the algorithms the gain might be minimal though ;]

 

pirx

Edited April 14, 2015 by pirx

[snicklin]

I think that compensating for movement will make the algorithm very difficult.

 

For example, with this scrolling lady, if you are moving 1 pixel at a time upwards, you are changing everything by one pixel row. If it was moving 8 pixels at a time, we would see re-use of characters. Considering the limited time for the Atari to decode this, catering for movement may be difficult. And as the lady is spinning, those characters below are changing all the time.