Databyte disks on Atari 810

[Farb]

Hi all,

 

Does someone have an Atari 810 drive and one or more of the following Databyte releases:

 

Mr. Robot and His Robot Factory

Boulder Dash II

Spy vs. Spy II: The Island Caper

 

It would be really helpful to know if any of these titles boot correctly on a real Atari 810. Any info someone can provide would be appreciated!

 

Thanks.

[phaeron]

Don't have a real 810 to check, but looking at the first one in emulation, it's using a nasty timing check on phantom sectors with long sectors in the mix. Specifically, it reads a sequence of logical sectors and checks the first byte received from each to verify the physical sector order. The 810 and 1050 handle long sectors differently and the difference in timing would probably affect this check.

[BigBen]

I can do the test for you but first have to search and fix my 810. It will take some time.

[ebiguy]

I can not help as I don't have these titles but I wanted to add a reminder for anyone doing the test :

Always remove the disk before turning off the 810 drive. I had some titles garbaged by not following this advice, regardless the fact that the disk was write protected or not !

I am not a drive expert but it is as if the bytes on the disk are garbaged where the head is located when you turn the drive off.

[_The Doctor__]

as a general rule you don't want to leave a disk in any drive as you power up or shut down the drive itself... we used to insert disks into x86 machines during post/bios splash display to boot on those machines as well... the post/bios/splash screens allowed to insert floppy for boot or hard drives to spool up... it was often noted for almost all machines not to leave disks in drives during power up or off.... warnings about that existed for 8 inch drives also when using the head always engaged option. Most drives only but the heads on the disk during read write but some folks wanted to save time and leave them touching the disk at all times..

Edited September 10, 2018 by _The Doctor__

[Fred_M]

I have just tested the original disks of Mr Robot and Spy vs Spy II by Databyte in an 810. Sadly, I don't have Boulderdash II.

 

First I have loaded the 2 games in my XF551 (speedy disabled) and both loaded okay without any problems.

 

Then the 810:

 

Mr Robot loads and runs like the XF551. No problem at all.

 

It is a lot different for Spy vs Spy II. I tried to load it 7 times. You can hear that the 810 has a lot of difficulty with the protection. Out of the 7 times it loaded once quite okay (it took ages). The game started but there were some visual problems (some mess on the screens). The other 6 times the game did not start. Everytime I see the loading text of Databyte. 2 times the loading reached the animated title screen but did not load the actual game.

 

After the 810 test, I loaded both disks with my XF551 again, and both started the game without any problems (and a lot faster than the 810).

 

Hope this helps

Edited September 10, 2018 by Fred_M

[ijor]

Don't have a real 810 to check, but looking at the first one in emulation, it's using a nasty timing check on phantom sectors with long sectors in the mix. Specifically, it reads a sequence of logical sectors and checks the first byte received from each to verify the physical sector order. The 810 and 1050 handle long sectors differently and the difference in timing would probably affect this check.

Phaeron, seems Altirra emulation is wrong here. Mr Robot does load from a real 810, but it fails on Altirra with full 810 disk emulation, at least with Altirra version 3.1.

 

The problem is that these titles have dup sectors that are both long and have a CRC error. As you already know, the 810 is "blind" to CRC errors on long sectors because it interrupts the FDC long before performing the CRC check.

 

You are correctly ignoring the CRC error for the purpose of returning Complete or Error to the CPU. But you are not ignoring it for the purpose of selecting which dup sector to return.

[phaeron]

Phaeron, seems Altirra emulation is wrong here. Mr Robot does load from a real 810, but it fails on Altirra with full 810 disk emulation, at least with Altirra version 3.1.

 

The problem is that these titles have dup sectors that are both long and have a CRC error. As you already know, the 810 is "blind" to CRC errors on long sectors because it interrupts the FDC long before performing the CRC check.

 

You are correctly ignoring the CRC error for the purpose of returning Complete or Error to the CPU. But you are not ignoring it for the purpose of selecting which dup sector to return.

 

I'm confused here, because Altirra doesn't take CRC errors into account when selecting the next sector. It simply returns the next sector in physical order that has matching density. From what I can tell, the problem is that enough time is passing for the DRQ bit to get set and this is causing the firmware to retry enough times to often get the clean sector out of the three physical sectors assigned to vsec 671. We may be looking at different images, though.

[Farb]

I have just tested the original disks of Mr Robot and Spy vs Spy II by Databyte in an 810.

 

Thank you for taking the time to do this, Fred!

[ijor]

 

I'm confused here, because Altirra doesn't take CRC errors into account when selecting the next sector. It simply returns the next sector in physical order that has matching density. From what I can tell, the problem is that enough time is passing for the DRQ bit to get set and this is causing the firmware to retry enough times to often get the clean sector out of the three physical sectors assigned to vsec 671. We may be looking at different images, though.

 

You are right. I missed the fact that the 810 will retry on any status bit set, not just on the "error" bits. And even if DRQ is not set, BUSY should also be set and that seems enough to provoke a retry.

 

So Altirra is always returning the good dup sector (not just often as I can see, but always it seems). But for some reason this title does work on the 810 as you can see from Fred_M test. And the protection expects to read one of the other dup sectors with a CRC error.

 

May be your retry timing is not exactly accurate so that a real 810 happens sometimes to return one of the other dup sectors? It is possible that we are looking at different images, but I am testing the image that in theory was generated from Fred_M's original disk.

 

We really need somebody with an 810 to perform some kind of test suite

Edited September 11, 2018 by ijor

[Fred_M]

The disk I tried is the same disk I sent to Farb for dumping last year or so

 

I have some other original Databyte disk releases (Aztec, Boulderdash construction kit, Polar Pierre). I can test these too, if it will help.

Edited September 11, 2018 by Fred_M

[ijor]

The disk I tried is the same disk I sent to Farb for dumping last year or so

 

Hi Fred, thank you very much for your testing and for all the original disks you dumped!

 

The 810 you have is a stock one, right? Because a different firmware might be relevant here.

 

[Fred_M]

Hi Ijor,

 

As far as I know it is a stock 810. I have bought it (among other 800 series stuff) at a small Atari 8-bit computer show in Hanau Germany in the early nineties. The drive was used in a high-school at a US military base in Karlsruhe Germany. The stickers on the bottom suggest that it was serviced by Atari themselves. I have never modified the drive.

 

I tried the other titles too:

 

Aztec: success

Boulderdash constr. kit: no success

Polar Pierre: no succes

 

I have uploaded a video to Youtube of the loading process of Mr. Robot. Maybe some of you can see what is happening there

 

https://youtu.be/E8E7KtDoAas

Edited September 11, 2018 by Fred_M

[ijor]

I watched the video, many thanks once more ! It's difficult to deduce exactly what's happening. But I have a possible theory ...

 

What's happening here is that the good dup sector is hiding the other dup ones that have error. So the 810 firmware never returns the "bad" sectors, always the good ones (the retry logic on the 1050 is quite different, so it sometimes returns one of the bad sectors). The protection needs to get one of the bad ones. And it will keep retrying until getting a dup sector with the expected content. In theory, it might seems this will never happen, and this is the behavior we see with Altirra.

 

May be there is an emulation inaccuracy here, but there is another possible explanation. May be the "good" dup sector reads bad once every so many times. That might perfectly happen on the real world as opposed to the emulation realm. Now, if the "good" sector eventually reads bad, then this might "unhide" the other dup sectors. And because the software protection retries constantly, the disk might eventually load. Just a theory ...

Edited September 11, 2018 by ijor

[+DjayBee]

I will prepare a tool which gives more information about what actually is returned by the OS.

[phaeron]

I watched the video, many thanks once more ! It's difficult to deduce exactly what's happening. But I have a possible theory ...

 

What's happening here is that the good dup sector is hiding the other dup ones that have error. So the 810 firmware never returns the "bad" sectors, always the good ones (the retry logic on the 1050 is quite different, so it sometimes returns one of the bad sectors). The protection needs to get one of the bad ones. And it will keep retrying until getting a dup sector with the expected content. In theory, it might seems this will never happen, and this is the behavior we see with Altirra.

 

May be there is an emulation inaccuracy here, but there is another possible explanation. May be the "good" dup sector reads bad once every so many times. That might perfectly happen on the real world as opposed to the emulation realm. Now, if the "good" sector eventually reads bad, then this might "unhide" the other dup sectors. And because the software protection retries constantly, the disk might eventually load. Just a theory ...

 

I took a more detailed look at what's going on in track 37 and in the 810 firmware, and I can't see how this would ever work, unless the Mr. Robot image I'm looking at is materially different than the one that booted successfully (MD5 0a790975bb76494ceb004ac1e7a7042c).

 

The logic that's running in the copy protection routine is simple: it simply checks one sector at a time in a list. If a sector fails, it retries just that sector with random delays. No restarting the sequence, no multiple sector pattern dependencies. Furthermore, it isn't looking for the good sector, it's only looking for the two long ones. It also only checks the first byte returned and ignores the SIO status, so whether the drive returns C/E or what error it returns is irrelevant.

 

Now, there is a very significant difference between the 1050 and 810 firmwares: the 1050 only does one retry, while the 810 does four. These retries are all internal to the drive, so no computer or SIO transfer delays apply. Furthermore, the sectors are all spaced apart by several sectors' distance and the 810 firmware only has delays of a few milliseconds in the retry path, so it should easily see the next sector. That's also not counting the device error retry that the OS will do in response, kicking off another five attempts. This means that short of something bizarre like the drive AGC consistently getting knocked out of whack or data bytes getting stuck in the FDC and returned on the next command, I don't see how the 810 isn't going to find the good sector eventually. Drive speed variation doesn't seem plausible, the speed would have to be way off to introduce enough deviation in timing and the bit lengths in the flux dump don't reflect that.

[Fred_M]

I don't know much about the technical details, but my disk was verified in the a8preservation project. The MD5 number seems to be different though

 

More details on the website of a8preservation: http://a8preservation.com/#/software/dump/3564

 

This disk is in the a8preservation torrent.

[ijor]

unless the Mr. Robot image I'm looking at is materially different than the one that booted successfully (MD5 0a790975bb76494ceb004ac1e7a7042c).

The MD5 is not the same as the image at the torrent, but probably doesn't matter and it's basically the same because what you describe matches what I see on Altirra with the other images.

 

Furthermore, it isn't looking for the good sector, it's only looking for the two long ones. It also only checks the first byte returned and ignores the SIO status, so whether the drive returns C/E or what error it returns is irrelevant.

It is actually looking for the good sector and one of the "bad" ones. This is relevant because then the chances to read the expected sectors are better, or should I say, slightly less worse. But yes, it ignores the status altogether. The status is only relevant for the drive and SIO retries.

 

Now, there is a very significant difference between the 1050 and 810 firmwares: the 1050 only does one retry, while the 810 does four ... That's also not counting the device error retry that the OS will do in response, kicking off another five attempts.

 

If I follow the firmware correctly (you should implement debugging on the drive's CPU! ) the 810 actually seems to retry three times, reading the sector up to four times total, not five. This means that if the "good" sector reads bad just once, then, depending on the initial head position, the drive will return one of the "bad" ones. I don't think this is that unlikely at all. But you are correct that in this case SIO will retry. So for the actual software to get the expected sector, we need the "good" sector to read with error twice in a row. And this is indeed much less likely, especially when we know the sector normally reads ok. Possibly it might happen, eventually, after a very long time. But according to the video, loading seems to be delayed by some seconds, not by minutes ...

 

I now suspect that what is happening here is something else. ebiguy was kind enough to run some tests on his 810. I am waiting for some additional test to confirm my theory.

Edited September 12, 2018 by ijor

[ijor]

Finally, and thanks to ebiguy performing some tests on his 810, we could confirm how and why these releases load on 810 drives. It is quite amazing ... long read ahead on this post, and probably very interesting for the technically minded ...

 

To recap: We are talking about some original releases by Databyte, like Mr. Robot. They load fine, as expected, from a 1050 drive. They should not pass the protection check and then don't load, on an 810 drive though. At least that's what it seems. Yet, and as reported by Fred_M in this very same thread, they do load, even when the load is slightly longer.

 

The copy protection check has several phases. I don't know all the details of all the checks performed by the software, DjayBee is the expert about that. But at one stage the protection check for the presence of several double (duplicated) sectors. It uses a rather simple mechanism for this purpose. It has a list of sectors to read and which data (just the first byte, actually) to expect for each sector. If the data is not the expected value it performs a random delay and retry. It retries indefinitely until getting the expected data. Timing is not much relevant here, and the protection doesn't check the status returned by the drive when reading the sector. It only verifies the data and nothing else.

 

Most double sectors can be verified ok on an 810. Some need several retries until reading the expected sector. But one sector is problematic. It is logical sector 671, or physical sector 5 on track 37. If you try to load the ATX image on Altirra, it will never pass this part of the protection. It will keep trying to read this sector for ever.

 

The problem is that this is a triple sector. There are three different physical sectors on this track that have the sector number 5. That wouldn't be so much of a problem by itself. Except that one copy of the sectors is a plain, normal, good, sector; and the two other copies are long sectors marked at the header to contain 256 bytes (as a sector on a DD disk), and besides, these two copies also have a CRC error. The problem is that because of how the drive firmware works, a good sector usually hides other dup sectors with error. The firmware retries whenever a sector reads with error as reported by the FDC. So the firmware skips sectors with errors and always return sectors that read without any error. Then, if the software expects to get the data of one the "bad" sectors, as it happens here, it might never be able to get them depending on the exact behavior of the firmware.

 

On the 1050, which was obviously the drive targeted by the protection, and after a few retries on the worst case, the expected sector, even with errors, is eventually returned to the computer. The 810 firmware has a different retry logic, it retries more times, and it behaves differently with long sectors. Normally it will always return the good dup sector, and never one of the long sectors with error. To decide if the sector was read ok or not, and then if a retry should be performed or not, the 810 checks all the bits on the FDC status register. If any bit is set, it will assume an error and it will retry, meaning it will skip the sector we want. The other difference is that the 810, and contrary to the 1050, it doesn't wait for the FDC to complete the operation when the sector is long. The loop is coded in such a way that if the FDC doesn't stop after 128 bytes (as would happen on a normal sector), the firmware will exit the loop, read the status register and interrupt the FDC.

 

There are four possible bits that could be set here:

 

- Bit 3 indicates a CRC error. This is indeed is set on the 1050 because "our" sector has a CRC error. But not on the 810 because the firmware doesn't wait, and it reads the status register long before the FDC performs the CRC check.

- Bit 2 indicates a Data Lost error. This is also set on the 1050. But again, and for the same reason, it won't be set on the 810.

- Bit 1 is the DRQ bit. Once again, this bit is always set on long sectors on the 1050. But on the 810, it might be set only sometimes. The FDC asserts DRQ once for every byte. At nominal bitrate this is once every 64us. And in some cases, depending on the loop position, the CPU would take less time since the previous DRQ until reading the status register. This was confirmed by the tests performed by ebiguy.

- Bit 0 is the BUSY bit. DRQ might be set only sometimes. But, on the 810, the BUSY bit should always be set because as we described, the 810 firmware reads the FDC status register long before it completes the operation.

 

Then in the best case, bit 0 would be the only bit set. But this is enough to provoke a retry and to skip "our" sector. So how it is not skipped? Well, the reason is something in the firmware code that I didn't note before. Let's look at the 810 loop:

       LDX #$FF
       ....
       LDA #~$88      ; Read sector command
       STA $00        ; FDC command register
0C19   BVS $0C2A      ; FDC done or timeout
       LDA $039C      ; RIOT timer
0C1E   BIT PORTA
       BPL $0C19      ; DRQ not yet set
       LDA $03        ; FDC data register
       INX
       STA $80,X      ; Save byte on sector buffer
       BPL $0C1E      ; Read next byte
0C2A   JSR $0C74      ; Check status

 

The last "BPL" branch is actually intended as unconditional. On a normal sector, the FDC reads 128 bytes only, so the X register will always be positive (remember X is $FF at the start of the loop), and the loop will exit at the top when the FDC would signal that it's done. But on long sectors the FDC will keep delivering bytes. The firmware will read a 129th bytes and exit the loop at the bottom when X is negative, after saving the 129th byte on RAM ... Wait, on RAM? The CPU would be executing the instruction STA $80,X with the X register having the value $80. The 650X wraps around on Zero Page addressing modes, so it will write to address $00. BUT, RAM starts at address $80 on the 810. The rest of zero page is the hardware space and address $00 happens to be the address of the FDC command register. So the firmware is inadvertently writing the 129th byte of the sector to the FDC command register !!! On a modern platform this could open a very serious security exploit. LOL.

 

Well, not so bad. Normally the FDC ignores writes to the command register at the middle of a read sector like in this case. Unless ... unless the command happens to be a Force Interrupt one. Force Interrupt commands terminate the current operations by the design. Then the FDC will complete the command and shortly will clear the BUSY bit! And yes, believe it or not, and surely by an amazing coincidence, the 129th byte of the sector is one of the Force Interrupt variants.

 

It is not easy at all to see the 129th byte on the ATX image. The ATX image only stores 128 bytes, even for long sectors. On almost every case it doesn't matter. Except in this case. I could confirm the full sector content from the low level RAW dump. But in this particular case it is actually possible to get it from the ATX image as well. This is because the relevant sectors are overlapped. When the FDC is reading the 129th byte of "our" sector" it will actually be at the middle of the next one. Reading the data of the next sector on the ATX image we could see that is it completely filled with the value $25. As you might know, Atari drives write data to the disk inverted. So the $25 is written to the disk as $DA ($25 inverted). And any byte with $D on the high nibble is a Force Interrupt command.

 

An almost unbelievable coincide, right? But there is more. The FDC is rather slow. It doesn't clear the BUSY bit immediately after receiving a Force Interrupt command. It takes several cycles. The timing on the 810 firmware is just long enough so that sometimes, but not always, it happens to read a clean FDC status register without any status bit set. This was also confirmed by tests performed by ebiguy.

 

So, because of the exact code of the 810, because of the overlapped sectors and the content of the next sector, the firmware will interrupt the FDC before reading the status register. And sometimes the status register will read as $00. Then the FDC will interpret the sector is ok. It won't retry. It will return the expected sector to the computer and without any error. The protection software would then verify the data of the double sector it was expecting. The protection check will pass. And finally, the Databyte release and the 810 would live happily ever after

 

Wow, Isn't this amazing?!

Edited September 14, 2018 by ijor

[phaeron]

Very, very nice detective work from the both of you. Force Interrupt is one of the most important and criminally undocumented commands in the 177X/277X FDC series. I'd say about half a dozen of the most recent bugs I've had to track down in full drive emulation have been related to bizarre behaviors with this command, including the XF551's unusual status codes. Do you know what kind of timing is involved in Force Interrupt completing?

 

BTW, Altirra does support debugging drive coprocessors. The ~s command is used to switch processor contexts (~1s, ~2s, etc).

[+Spaced Cowboy]

 

BTW, Altirra does support debugging drive coprocessors. The ~s command is used to switch processor contexts (~1s, ~2s, etc).

 

 

[sigh] Of course it does. Phaeron, I really (*really*) wish you'd come over to the dark side (Mac)

[Farb]

 

 

Wow, Isn't this amazing?!

I'm not sure what is more amazing - the phenomenon itself or the work you guys did to figure it out!

 

Sent from my ONEPLUS A6003 using Tapatalk

[ebiguy]

Very, very nice detective work from the both of you. Force Interrupt is one of the most important and criminally undocumented commands in the 177X/277X FDC series. I'd say about half a dozen of the most recent bugs I've had to track down in full drive emulation have been related to bizarre behaviors with this command, including the XF551's unusual status codes. Do you know what kind of timing is involved in Force Interrupt completing?

 

BTW, Altirra does support debugging drive coprocessors. The ~s command is used to switch processor contexts (~1s, ~2s, etc).

 

Hats off for ijor.

To be honest, I was the arm manipulating the 810 and he was the brain !

But that's the best way to learn !!!!!

One cool thing with the Atari is that we still discover some secret behaviours or features almost 35 years later.

That's what keeps me on this computer forever.

Edited September 14, 2018 by ebiguy

[phaeron]

Does this mean it's time to extend ATX to store the full long sector? Previously it didn't look like this was necessary for standard disk drives, but this is the first case we've seen where a stock drive does actually need more than 128 bytes even though the computer can't directly see them. The format has a way to indicate the size of the physical sector but not that more than 128 bytes are stored.

[_The Doctor__]

Does this mean it's time to extend ATX to store the full long sector? Previously it didn't look like this was necessary for standard disk drives, but this is the first case we've seen where a stock drive does actually need more than 128 bytes even though the computer can't directly see them. The format has a way to indicate the size of the physical sector but not that more than 128 bytes are stored.

Yes this is exactly where the scene is going, this will allow atx capable emulation both in Altira and sdrive max atx to do what they are suppose to do. And this may be another step towards reconstituting disk from atx using bit writer etc.... all data should be there... eventually we end up needing it so it would appear