Altirra & xBIOS 4.3 loading

[flashjazzcat]

Burst reading isn't particularly difficult and plenty of source code examples exist. The SIDE loader burst reads and keeps everything including the IDE and 32-bit FAT cluster code inside of three pages of RAM.

[dmsc]

Hi!

Quote

Burst reading isn't particularly difficult and plenty of source code examples exist. The SIDE loader burst reads and keeps everything including the IDE and 32-bit FAT cluster code inside of three pages of RAM.

Yes, and my own boot-loader (for Sparta DOS disk format, using only 355 bytes including headers) also does burst reading in a very simply way: in the buffer refill routine you check if the remaining bytes to read are >= the sector size, and then simply read the sector to the destination address and continue, see at: https://github.com/dmsc/mkatr/blob/master/src/asm/boot256.asm#L238

 

I suspect adding that capability to XBIOS would mean using about 30 extra bytes.

 

Edited July 27, 2019 by dmsc

[rensoup]

On 7/27/2019 at 6:18 PM, flashjazzcat said:

Burst reading isn't particularly difficult and plenty of source code examples exist. The SIDE loader burst reads and keeps everything including the IDE and 32-bit FAT cluster code inside of three pages of RAM.

Is the source available ?

[flashjazzcat]

1 minute ago, rensoup said:

Is the source available ?

No, but the meat and bones of burst IO are perfectly illustrated by @dmsc's source code.

[rensoup]

23 hours ago, dmsc said:

Hi!

Yes, and my own boot-loader (for Sparta DOS disk format, using only 355 bytes including headers) also does burst reading in a very simply way: in the buffer refill routine you check if the remaining bytes to read are >= the sector size, and then simply read the sector to the destination address and continue, see at: https://github.com/dmsc/mkatr/blob/master/src/asm/boot256.asm#L238

 

I suspect adding that capability to XBIOS would mean using about 30 extra bytes.

 

That's very cool... it uses SIOV though. Fair enough for the boot sector but what if I want to load files without the OS ?

 

I've been trying to avoid using the OS because of the memory cost but I initially thought that switching off the OS then switching it back on would result in losing the content of last 16K of RAM, but it seems that its just a bank switch and it's still there if you switch it off again?

If I wanted to use the OS, I'm guessing I would still need to restore a lot of the ZP, make sure the DCB area is usable, and probably restore interrupts handlers/NMI ? Not really a fan of the idea... but perhaps It's less messy thant it seems ?

 

What about using custom SIOV routines in conjunction with your code ? It seems some are available like in Thor's OS++ ? or Altirra OS (although I'm not sure if the code is available) ? also I found one on AA

 

I'm guessing I would lose SIO patching and it wouldn't work on modern devices ?

 

 

[rensoup]

47 minutes ago, flashjazzcat said:

No, but the meat and bones of burst IO are perfectly illustrated by @dmsc's source code.

No worries I thought maybe you weren't using the OS.

[flashjazzcat]

18 minutes ago, rensoup said:

I thought maybe you weren't using the OS

I'm not using the OS, since the OS has no native ability to read sectors from a Compact Flash card. If it had, I'd be able to replace all the code which communicates with the hardware with 'JSR SIOV', and the XEX loader module would be about 2/3 of the size.

 

While it's reasonable for the SIDE loader to communicate directly with SIDE hardware (since it solely targets said hardware) and work only with the FAT file system, it's helpful if a boot loader intended for general use or one integral to a binary calls the SIO. In so doing, it will automatically have the ability to load from any device, including PBI-attached hard disks (which may have the ability to host bootable ATRs).

[phaeron]

38 minutes ago, rensoup said:

I've been trying to avoid using the OS because of the memory cost but I initially thought that switching off the OS then switching it back on would result in losing the content of last 16K of RAM, but it seems that its just a bank switch and it's still there if you switch it off again?

No, toggling the OS ROM just switches the chip enables between RAM and ROM. RAM is not modified when the ROM is switched in, and writes do not go to the RAM when stores hit the ROM (unlike the C64, notably, where they do).

38 minutes ago, rensoup said:

If I wanted to use the OS, I'm guessing I would still need to restore a lot of the ZP, make sure the DCB area is usable, and probably restore interrupts handlers/NMI ? Not really a fan of the idea... but perhaps It's less messy thant it seems ?

For full compatibility, you should save the entire $00-7F and $200-3FF OS variable regions. The OS variables are extensively documented by Atari, but custom OSes and especially PBI device handlers do not have to conform to that and can and will reuse memory locations for different variables. This takes a bit of RAM, but it is RAM that you would otherwise need for the code SIO code anyway.

 

38 minutes ago, rensoup said:

What about using custom SIOV routines in conjunction with your code ? It seems some are available like in Thor's OS++ ? or Altirra OS (although I'm not sure if the code is available) ? also I found one on AA

Avoid Os++ if you have a requirement to stay clear of Atari copyrights, as it contains copied routines from the Atari OS.

 

AltirraOS source code is available, in src/kernel/source/shared in the Altirra source code archive. It would need some stripping however to remove cassette and PBI support, though, it's constrained to compatibility with DOS's interrupt routines, and it doesn't have high speed support. It's not the best starting point for an embedded SIO loader.

 

When choosing a custom SIO routine, consider whether you want high-speed and PBI support, and then check whether the SIO routine has proper handling of overrun/framing errors, timeouts, and does retries. Several demos use SIO routines that don't handle the latter properly, a particular case being a demo that had its SIO timeout so short it couldn't actually load off of a real disk drive.

 

But really, unless you're looking to do something fancy like an animated load screen, consider just swapping the OS back in and using SIOV. Then people won't be staring at your load screen all day, as fancy as it might be.

 

38 minutes ago, rensoup said:

I'm guessing I would lose SIO patching and it wouldn't work on modern devices ?

In Atari800, you must use either the OS ROM or a copy of the ROM for SIO patch acceleration. In Altirra, using the OS ROM or supporting PBI devices is required; copying the OS ROM will not work and will disable acceleration.

 

Cartridge-based loaders will not work either, but those typically are pretty fragile anyway, having to either copy-and-modify the OS in RAM or trying to scan for JSR SIOV calls. These may already be a lost cause if you have a multi-stage load that uses all RAM including RAM under the OS.

 

SIO2SD style devices will work as long as you have a well behaved SIO routine that properly respects timing and retry protocol requirements. A custom SIO routine will also disable high-speed, however, unless your custom routine implements it, and there are a few types. You need at least the OS ROM and OS interrupt architecture for PBI devices, so if you're looking to support those you might as well just use SIOV.

[xxl]

@rensoup

 

0EAB: A9 20             LDA #$20
0EAD: 2C A9 00          BIT $00A9

 

means:

 

 lda #$20
 .byte { BIT $ffff }
 lda $00

 

yes, xBIOS is optimized in terms of code length and not speed of code, thanks to this it takes only 1KB with so many functions.

All operations (also binary load) are buffered, it has its consequences but also advantages - let's be a binary loader with LZ4 decompressor?:

stream decompresor (part of binary loader not loaded file) starts working when in binary load semgment stop adres = 0:

 

source binary file (21 KB):
0. 2000-2005
1. Init 2000
2. 2010-459F
3. 4800-4AD2
4. 5400-8303
5. Run 5800

 

compreset segments (12 KB):
0. 2000-2005
1. Init 2000
2. 2010-0000  (LZ4)
3. 4800-4AD2
4. 5400-0000  (LZ4)
5. Run 5800

 

http://www.atari.org.pl/forum/viewtopic.php?pid=245899#p245899

 

if you want other I/O than SIO try your own and use: xBIOS_SET_IOMODULE (e.g. like I did with RAMCART)

but you must know that this will allow you to communicate with any imaginary device but there must be AtariDOS filesystem - not Sparta, nor FAT - if you want use eg.FAT use devices that perform translation on the fly to the AtariDOS family file system (eg SIOCart , WiFiPrime
 

[dmsc]

Hi!

6 hours ago, rensoup said:

That's very cool... it uses SIOV though. Fair enough for the boot sector but what if I want to load files without the OS ?

 

I've been trying to avoid using the OS because of the memory cost but I initially thought that switching off the OS then switching it back on would result in losing the content of last 16K of RAM, but it seems that its just a bank switch and it's still there if you switch it off again?

If I wanted to use the OS, I'm guessing I would still need to restore a lot of the ZP, make sure the DCB area is usable, and probably restore interrupts handlers/NMI ? Not really a fan of the idea... but perhaps It's less messy thant it seems ?

 

IMHO, please try to not use custom SIO routines, they will make your program incompatible with many accelerators, PBI devices and any other means to load data. Swapping the OS is easy, and saving the ~640 bytes needed to make the OS work is not difficult, and would make the lives of your users a lot easier.

 

 

 

[rensoup]

On 7/28/2019 at 9:46 PM, xxl said:

 lda #$20

 .byte { BIT $ffff }
 lda $00

I see, it's that space saving optimization...

 

On 7/28/2019 at 9:46 PM, xxl said:

yes, xBIOS is optimized in terms of code length and not speed of code, thanks to this it takes only 1KB with so many functions.

All operations (also binary load) are buffered, it has its consequences but also advantages - let's be a binary loader with LZ4 decompressor?:

stream decompresor (part of binary loader not loaded file) starts working when in binary load semgment stop adres = 0:

But is that really useful when you can just exomize your xex ?

 

 

 

 

I'm currently testing xBios and it works fine but it seems to have a lot of features which are unexplained.

 

xSPEED          equ xBIOS+$3e6 ; STD SPEED
xHSPEED         equ xBIOS+$3e7 ; ULTRA SPEED

 

I still don't understand how to set high speed mode

xIRQEN          equ xBIOS+$3e8 ; User IRQ (1 byte)

 

Does this mean my own IRQ handler will run if set to 1 ? I have no idea what it's used for...

xAUDCTL         equ xBIOS+$3e9 ; AUDCTL

 

?

 

I wish I could remove the write to disk code with a setting in the CFG file to save some space.

 

As for a custom IO module, I wouldn't know where to start...

 

 

[rensoup]

Lots of good info, Thanks!

 

My project specs:  128KB RAM / 1x 130KB ATR (or 2x 90KB ATR?) everything compressed on disk

 

I'd like some minimalistic loading info, I don't want a black screen and I'd rather not have an infinite looping anim. At least some form of progress bar.

 

At first I thought xBios would be fine and cover all my needs but it seems that there are so many new storage options that it's not possible...

 

I'm not going to consider Custom SIO code, that seems doomed to fail.

 

I've been thinking I could do 2 versions:

 

1)xBios: runs on 128KB machines with real drives and some modern devices

 

2)RAMdisk with early OS switch, would need 256KB machines, loads/decompresses everything in extended mem. switch off OS permanently. Run.

 

This leaves out those with 128KB machines and devices incompatibles with xBios, is that a lot of people ?

 

Though I'm starting to warm up to the idea of doing OS switches whenever needed.If I wanted to go that road, what's involved exactly besides saving those 2 RAM regions ?

 

Do I need to save the stack content ?
Do I need to save registers (A X Y P SP) ?
What about Ithe RQ vector ? I don't recall it even being set when reaching my code.
NMI Vector surely ?

If I restore the NMI before switching the OS back on, surely it's going to start poking into memory ? like putting shadow registers into HW registers ? or some other crazy stuff ?

 

 

[phaeron]

6 minutes ago, rensoup said:

Do I need to save the stack content ?

No, the OS does not rely on the contents of the stack across calls into it.

 

You may want to avoid using the lower portion of the stack page ($100-13F or so), as I seems to recall that some patched OSes may try to stash data there.

6 minutes ago, rensoup said:

Do I need to save registers (A X Y P SP) ?

Nope. SIOV should always be called with D=0 and I=0 in the P register, however.

 

Avoid wrapping the stack by calling into the OS with a very low S register value, such as S=$02. The VBI handler has a $0104,X check for the I flag that fails if the stack wraps around page one. This should not ordinarily be a problem unless you are playing tricks with the stack.

6 minutes ago, rensoup said:

What about Ithe RQ vector ? I don't recall it even being set when reaching my code.
NMI Vector surely ?

If I restore the NMI before switching the OS back on, surely it's going to start poking into memory ? like putting shadow registers into HW registers ? or some other crazy stuff ?

You need to restore all of the OS vectors and variables. The IRQ vector is needed for SIO interrupts, and the NMI vector is needed for the VBI to update OS timer 1 for SIO timeouts. The SIO routines may in turn access any variables in the OS database, including ones not documented in the OS manual. If a PBI device is present, all of the PBI vectors and variables will be needed as well. You'll also need to re-enable the VBI and write POKMSK back to IRQEN.

 

This does mean that the OS will overwrite some hardware variables with its own values, yes. This includes all of POKEY as well as parts of ANTIC and GTIA. The latter will be limited while SIO is in operation, but unfortunately there is an issue with it always writing CRITIC=0 on exit which forcibly exposes you to a short time when VBI stage 2 can run and write the full set of registers. Some PBI devices and IDE-based disk emulators will also rely on this for feedback effects by writing to color registers, even if you try replacing the VBI handler. Thus, it's best to make sure your entire loading screen is compatible with the OS NMI handlers and that all the appropriate shadow variables like COLOR1 and SDLSTL/SDLSTH are set accordingly.

 

[xxl]

> But is that really useful when you can just exomize your xex ?

 

no.

 

 

> I still don't understand how to set high speed mode

 

there are several types of turbo for disk drives, xB supports US. xHSPEED. If there is a different value than $ff, this means that the flop drive supports US and if you used xBIOS_SET_DEFAULT_IOMODULE (or .cfg) you can request communication at that particular speed.

 lda xHSPEED
 bmi no_US
 sta xSPEED
no_US

 

If you use AtariOS for communication, the HSPEED variable does not matter because xBIOS to read data refers to the OS and not to the hardware - it can be any device working at any speed.

 

 

> Does this mean my own IRQ handler will run if set to 1 ? I have no idea what it's used for...

 

no, if you use xBIOS_SET_DEFAULT_IOMODULE, you can use IRQ when communicating with the device. This is for advanced users - look at $D20E

 

 

> xAUDCTL

 

for advanced users, communication can affect music playback.

 

> I wish I could remove the write to disk code with a setting in the CFG file to save some space.

 

it does not take much space

 

> As? for a custom I?O module, I wo?uldn't know wher?e to start

 

for users who want to create xB for a specific device, for example, cartridges, ramdisks, etc.

 

It seems to me that you should not change the I/O module, use that one xBIOS has after launch - you will not have to worry about turbo systems or various devices like HDD or cartridges, they will all be supported - however, the price of such a solution - you must have enabled OS ROM during communication.

 

If you write under the 80's standards, then you have a simple .atr (Atari - peripheral) .car (Atari - cartridge) if you write under the user's "standards" from 2019 then do the .xex version for 256KB ram: D

[+Nezgar]

On 7/29/2019 at 8:40 PM, rensoup said:

I'd like some minimalistic loading info, I don't want a black screen and I'd rather not have an infinite looping anim. At least some form of progress bar.

With ANTIC switched off, normally a blank screen, you can still alter the background colour and use player-missile graphics. The simplest would be to display progress by simply moving a solid vertical line from left to right, but you have all 4 players and 4 missiles at your disposal. Some simple text or numbers indicating progress would be possible with more work. Altering the BGcolour seems common for depackers and highspeed tape loaders.

 

Maybe with ANTIC off, it will also help increase CPU cycles available for buffered I/O too.

 

Edit: Another method I just remembered was like SuperArc/SuperUnArc - it would use a display list of only 1 text line for status during operation, which gave you most of the benefit of having ANTIC off, plus 1 line of text still visible.

[flashjazzcat]

12 hours ago, rensoup said:

At least some form of progress bar

One extremely safe means of providing a progress bar is to split your main binary into similar sized chunks and maintain the animation between segment loads (i.e. call your animation code - which was loaded right at the start - via the INIT address between each segment). This way you don't have to care a damn about what the loader does with POKEY, NMIs, etc, when actually loading sectors from the storage device.

[rensoup]

Thanks again, I'm still unsure which way to go but I'll definitely investigate OS switching.