TOS Versions and HD Boot

[pixelmischief]

What levels of HD boot support are available in what TOS versions and using what drivers or preparation methods? I am imagining an information matrix with TOS versions as rows and features as columns; or vice versus.

 

In my case, I have a 1040 STe with rainbow, a Supra 20 controller, and a SaTanDisk. I have both Uwe's HDDRIVER and PPera's drivers, too. What do I need to do to auto-boot from either or both of the hard disks?

[+SoulBuster]

Excellent idea to have this somewhere. I wish I had the answers to help fill it out. It should include the Atari and ICD drivers also.

 

If some one is willing to feed the information in this thread, I can create the matrix.

[ParanoidLittleMan]

TOS 1.xx have only ACSI autoboot support, what consists of loading MBR from attached disk/media (1, very first sector on it), checking checksum and if is OK executing code on it.

 

TOS 2.0x, 3.xx, 4.xx have IDE autoboot code too, and they will not autoboot further possible attached devices after first successful autoboot - in case of 1.xx driver code needs to prevent it . There is code for ACSI and IDE sector read too in 2.0x, but that's not used normally, because is not so fast as good driver code. Can be good to make MBR code shorter, but then it will not autoboot with 1.xx .

TOS 3.xx and 4.xx have SCSI autoboot code too. Falcon (4.xx) has no ACSI port, so no code for boot from it.

 

Satandisk, UltraSatan and other regular ACSI devices will autoboot with any TOS version. In case of multiple devices on ACSI port, they need to be set properly, so to different target IDs , otherwise likely none will boot. It is not recommended that 2 or more devices autoboot one after other, since multiple dirvers may not cowork well. If data exchange is wanted, it is best to load driver directly, as runnable file. But it works not normally with Hddriver, which will kill existing driver in RAM, and then partitions from first device will be not accessible.

Sadly, Atari did not define many things well, or at all, so there may be diverse problems with multiple devices attached.

 

Matrix should be like:

 

TOS 1.xx - ACSI autoboot

TOS 2.xx - ACSI and IDE autoboot

TOS 3.xx - ACSI and SCSI autoboot. There is IDE autoboot too, even if it has no IDE port by factory.

TOS 4.xx - IDE and SCSI autoboot.

Of course, there are some patches for 1.xx TOS versions for IDE autoboot available.

[pixelmischief]

TOS 1.xx have only ACSI autoboot support, what consists of loading MBR from attached disk/media (1, very first sector on it), checking checksum and if is OK executing code on it.

Can I assume that driver packages like yours will properly write this magic first sector to my ACSI hard disks? After it has been so conditioned, can I simply turn the TOS 1.X machine on and it will boot from the hard disk?

 

Satandisk, UltraSatan and other regular ACSI devices will autoboot with any TOS version.

But only after they have been prepared using a hard disk partitioning tool that also writes the sector/driver to the disk; correct?

 

If data exchange is wanted, it is best to load driver directly, as runnable file...But it works not normally with Hddriver, which will kill existing driver in RAM, and then partitions from first device will be not accessible.

If two drives are chained together, but only one of them is bootable and loads a driver, will partitions on both still be available? Will HDDRIVER also work properly in this way?

 

Matrix should be like:

 

TOS 1.xx - ACSI autoboot

TOS 2.xx - ACSI and IDE autoboot

TOS 3.xx - ACSI and SCSI autoboot. There is IDE autoboot too, even if it has no IDE port by factory.

TOS 4.xx - IDE and SCSI autoboot.

Of course, there are some patches for 1.xx TOS versions for IDE autoboot available.

So, am I right in understanding that TOS versions bootstrap hard disks as you specify in your matrix. Is it also true that they can only be recognized by this minimal bootstrap if they have some magic sector written? Can you please explain what the role of a driver installed on the hard disk plays and what the process is from power-on to GEM desktop with partition icons? As always, you are a wealth of valuable information and if we can use this as a bit of a starting point, I think we could get together something of a definitive guide to mass storage on Atari ST/TT/Falcon computers.

Edited November 1, 2018 by pixelmischief

[ParanoidLittleMan]

Yes, my driver SW will write 'magic' sector - called normally MBR (master boot record) with code what works on all TOS versions from 1.00 .

To add that it will write low level ACSI code in case of ACSI device/driver. In case of IDE will write IDE low level code. That's what all decent driver SW writes normally.

But there is not much space in MBR, since it needs to hold partition information too, so there is about 400 bytes only available.

 

Yes, autoboot will happen only when proper Atari autoboot code is installed on disk. Some old PC hard disk, even if it is with Atari compatible partitions will not autoboot - only after that install.

 

Will other disk be available depends on driver self. If it checks for different adapter type attached disk and can handle partition types on it, that it will happen.

Hddriver will do it, but not with all disks - some may have partitions what it recognizes not. There is no rule in this case. And additionally, you may run out from max partition count limit (14) . More chances for success are with running driver directly for second disk . Well, this may need some experimenting.

If Atari defined some things better, that would be more clear.

 

Boot process is pretty much simple: TOS will load very first sector in disk buffer, then calculate it's checksum - simple adding of 256 words. If that sum is $1234 then it will perform jsr (jump to subrutine) to code in disk buffer . Then driver SW needs to do lot of things - well, first step is to load longer code part, since it can not fit in 1 sector. Allocating RAM for disk buffers, code, tables, Setting hard disk vectors of TOS, some system variables + setting C: as boot device - what is actually 2 steps - C: as boot drive (will be active at next autoboot) + C: as default drive, from which AUTO and Desktop.inf will load. As extra steps it may perform installing Atari PUN structure - what is small RAM area containing useful info about installed partitions and adapter device type. And may install XHDI structure too - later is not established by Atari, and it still does not resolve all possible situations. But all it was really long time ago.

After all it, it simply returns to caller - TOS, what then will execute AUTO folder PRGs - from default drive - normally C: , if there is such, then start AES, load Desktop.inf (or Newdesk.inf in case of 2.06), then may again do Auto run of GEM SW (at 1.04) .

 

To add that people having TOS 1.00 and 1.02 need to care about that max part. size is 256 MB, unlike 512 MB in case of later. So, it want to use same disk/media on multiple Ataris, need to use max partition sizes as it is with lowest TOS version . And in case of multiple medias, like 2 slot UltraSatan - max total partition count is 14 (stays for all regular TOS versions) it may happen that will run out of that - 10 partitions on first, and 8 on second - will be not able to access last 4 on second.

Well, actually, TOS is the real limit here, I must say. That made me to go in long process of improving it's FAT16 filesystem code.

[pixelmischief]

But there is not much space in MBR, since it needs to hold partition information too, so there is about 400 bytes only available.

Is the limited space of the MBR the reason for the 14 partition limit? What is the structure of an MBR? What is the structure of a partition table record?

 

Some old PC hard disk, even if it is with Atari compatible partitions will not autoboot - only after that install.

Can Atari compatibility be wedged into an existing PC partition table, non-destructively? If so, does any software currently do that?

 

Will other disk be available depends on driver self. If it checks for different adapter type attached disk and can handle partition types on it, that it will happen.

Perhaps we can test the current state of compatibility for the known, common factors; adapter and partition types.

 

More chances for success are with running driver directly for second disk.

What does this mean? Are you talking about running the MBR installer for the second disk so that it can be recognized at startup, or are you talking about a second execution of the driver at start-up, targeted to the second disk? Perhaps you could walk me through how this would be done.

 

Boot Process:

- TOS load very first sector in disk buffer. This is a bootstrap driver written to disk by a hard disk preparation and partitioning utility; i.e. PPera's Hard Disk SW, Uwe's HDDRIVER, ICD Pro Disk Utilities, etc.).

- TOS calculates buffered first sector checksum; simply adding 256 words of the bootstrap driver.

- TOS checks if that sum is $1234. If it is, TOS performs a JSR (jump to subroutine) to the bootstrap driver code in the disk buffer.

- Bootstrap driver loads the complete hard disk driver, which will always be too big to fit in just the first sector of the disk.

- Hard Disk Driver allocates RAM for disk buffers, code, tables.

- Hard Disk Driver sets hard disk vectors of TOS and some system variables.

- Hard Disk Driver sets C: as the default boot drive. This setting will be active at next autoboot.

- Hard Disk Driver sets C: as the default drive from which AUTO and Desktop.inf will be loaded.

- Hard Disk Driver may also populate the PUN structure in the Atari's BIOS. On systems that support it, this a small RAM area containing useful info about installed partitions and adapter device type.

- Hard Disk Driver may also populate the XHDI structure. This is a non-Atari extension of the PUN structure, established to provide a more complete source of hard disk and partition information.

- Hard Disk Driver returns control to TOS

- TOS executes any files with the PRG extension stored in the AUTO folder of the default drive set in a previous step; normally C:.

- TOS starts the AES.

- AES loads Desktop.inf on TOS versions below 2.06 and Newdesk.inf on 2.06 and above.

- AES may again do Auto run of GEM SW (at 1.04).

Does this look correct to you? I am particularly unsure about the last step. Does AES pass control to GEM there? What do you mean by "again do Auto run of GEM SW". And, are you saying that the statement is only applicable to TOS 1.04?

 

To add that people having TOS 1.00 and 1.02 need to care about that max part. size is 256 MB, unlike 512 MB in case of later.

Is this limitation only on the boot partition, or are all partitions greater than 256MB unavailable to TOS 1.00 and 1.02?

 

That made me to go in long process of improving it's FAT16 filesystem code.

Can you say more about what problems this solves and how people can take advantage of that now?

 

WOW! Thank you so much! This stuff has always been mystery to me and I really appreciate the enlightenment!

[pixelmischief]

This also appears to be useful; even if a bit inaccessible.

 

https://info-coach.fr/atari/documents/_mydoc/Atari_HD_File_Sytem_Reference_Guide.pdf

[ParanoidLittleMan]

I can not go here in too long explanations of all questioned. You can look after MBR structure and about FAT16 - there are many sites covering it. But that will be mostly for PC (DOS) type MBR and FAT16. Atari's is not same, even if it is partially compatible.

So, first to explain that difference: FAT16 means 16-bit file allocation table record (FAT) - so it can have 65536 values. If 1 record is 1 sector long then capacity of that partition can be max 32 MB. It's called usualy cluster size. And in that case TOS (Atari) FAT16 is same as DOS FAT16. TOS uses even Intel byte order.

But 32 MB was too short even in 80-es, so they went on larger partitions, while staying at FAT16 - introducing multiple sector clusters. So, DOS has max partition size of 2 GB, where 1 cluster is 64 sectors. And that needs 32-bit sector addressing too (32 MB can be addressed with 16-bit) .

Unfortunately, by Atari they went not so good way - and that is still little mystery for me why ? Probably it was because used C-compiler, what had problems with 32-bit variables and unsigned 16-bit variables - and that's the reason for some limitations. So, TOS stays at 16-bit sector addressing by larger partitions too, and instead many sectors in 1 cluster using so called large sectors: in case of 512 MB partition it is 32 sectors (16 KB) - and that's minimal value what can handle - driver will get only sector count what to load/write, and that means dealing with all 16 KB (and multiple of it) even if only few bytes are needed to transfer. Cluster is normally 2 large sectors. So, how that then partition size limit is not 1GB ? (16KB x 65536) - because it is not real FAT16, just some kind of FAT15 - max FAT record count is 2 POW 15 (32768) - and that would be mentioned lack of proper work with unsigned 16-bit variables.

 

Above means that TOS FAT16 large partitions (above 32 MB) are not compatible with DOS FAT16 large partitions. But they can be made compatible by using special parameters for cluster count + giving proper count of reserved sectors etc. That can do only Hddriver and my Driver/partitioner SW. And no other SW can access such partitions on Atari. On PC, Linux, MAC it is well accessible since is DOS compatible.

 

Considering MBR structure: TOS actually has nothing with it. Atari defined little different MBR partition record structure than DOS, but nothing forces us to use it. Why ? Because TOS deals not at all with it. Only hard disk driver and partitioner SW is what deals with it. So, it is simply better to use DOS partition table type - good for compatibility. And even if not attaching to some new computer it will not harm.

 

"Can Atari compatibility be wedged into an existing PC partition table, non-destructively? If so, does any software currently do that?"

As said above, TOS/DOS compatibility means special partition parameters, so answer is: no. You must create new partition - so deleting old, DOS one first if need space on disk. I don't see it as big flaw - saving all from couple GB disk on PC now takes only couple minutes. Then can copy all it back on compatible part. by need.

 

"Perhaps we can test the current state of compatibility for the known, common factors; adapter and partition types."

 

You are welcome to do it. I can say that I don't support different type adapter in my current driver SW. The reason is that it will then need some 5-10 KB extra RAM space per adapter type, while such usage is not spread. I started to code universal driver, with support for all known adapter types (and that means more than ACSI + IDE - since I have some my design adapters), but that was never finished because no interest for it.

 

"What does this mean? Are you talking about running the MBR installer for the second disk so that it can be recognized at startup, or are you talking about a second execution of the driver at start-up, targeted to the second disk? Perhaps you could walk me through how this would be done."

 

No, no need to install anything. You boot from first disk - and first is what TOS executes it's autoboot (ACSI in case of 2.06) . Then you simply run not MBR installer (actually it is just autoboot code installer, what installs short code in MBR, sets checksum + installs longer part of code) - but run directly hard disk driver program. That may be same as autoboot installer (Hddriver - it will know that executes not from MBR, so will act like runnable driver). By my driver SW there is separated code for autoboot install and for runnable driver (R at filename end) . You can run it from floppy or hard disk.

 

"Does this look correct to you? I am particularly unsure about the last step. Does AES pass control to GEM there? What do you mean by "again do Auto run of GEM SW". And, are you saying that the statement is only applicable to TOS 1.04?'

 

Well, that was not best formulated. AES and GEM are basically same thing. Actually can say that AES pass control to Desktop . And at TOS 1.04 (not only, but it and later ones) there is possibility to automatically run (after reset) some GEM PRG by setting it in Options, Install Application - after what need to Save Desktop, since that will be stored there for next boot. Normal AUTO run (subdir AUTO + PRGs in it) can not run GEM/AES SW because it is not initialized yet.

I just meant that can do again some kind of Auto(matic) run .

 

"Is this limitation only on the boot partition, or are all partitions greater than 256MB unavailable to TOS 1.00 and 1.02?"

 

All partitions. What causes confusion is that by some hard disk driver SW there is special limit for first partition only (caused by short space for code in MBR and not to smart idea how to store and access main code part of driver) .

 

"Can you say more about what problems this solves and how people can take advantage of that now?"

 

That's described little here: http://atari.8bitchip.info/tos105.html

 

As said there is cluster count limit in TOS - 32768 instead 65536 - so one advantage is max partition size of 1 GB instead 512 MB.

Then, since logical sector size is always 1 physical sector (512 bytes) access to disk is more efficient and faster.

Now can access directly DOS partitions up to 1GB, without program BigDOS. And even no need for extra TOS disk buffers, which are of min. size: 4 x logical sector size. Existing 2 KB buffers are enough. Less RAM needed . But can add buffers if want - then will work little faster. With some old slow disk more faster.

 

And max partition count is 30 instead 14. It needs little explan. : 14 is actually 16 bits - because there is A + B . 30 is 32 bits. Atari programmers decided that 14 is enough, and that was OK for 1989. Now we may need more, considering available Flash card capacities for instance. And nicest thing is that all those improvement done by me do not require more RAM for TOS workspace, so TOS 1.04 and 1.62 low RAM layout is same.

[pixelmischief]

Thank you very, very much.

[ParanoidLittleMan]

I made some mistakes in previous post, and can not edit it now.

So, large sectors are in case of 257-512 MB partitions of size 8 KB = 16 sectors. 2 of such gives 16 KB in 1 cluster - for 1 FAT record.

16KB x 32768 gives 512 MB (FAT15 in fact) .

Part. of size 32-64MB have 1 KB large sector size, 65-128MB 2 KB, 129-256 4KB. Cluster size is always 2 sectors. TOS can handle other values of sector counts in clusters in some cases, but that's not used.

FAT16 can have 65535 slots (not values) - values in slots are number of next FAT record (slot) for next cluster of file, and that may be not in order (file fragmentation). $FFFF means that it is last cluster of file.

[pixelmischief]

Thanks for the correction. I am going to take all of what is here and distill it into a "A User's Guide to Mass Storage on the Atari ST/TT/Falcon". The more technical parts, like those you have corrected, will go into an Appendix. Hopefully we can get this pinned and new users will be able to easily avoid the many and various pitfalls there are when attaching a hard disk to a 16/32 bit Atari.