Myarc HFDC DSR and allocation error

[+InsaneMultitasker]

I did some more testing and confirmed that there is indeed a problem with the HFDC DSR. Deleting a file that is fractured will almost always free one or more incorrect sectors. Subsequent writes to these freed sectors will result in corruption, usually by overwriting sectors allocated to other files.

 

According to the letter from Geoff Trott to Beery many years ago:

 

"Item 0545699 94/01/16 19:56
From: GEOFF_TROTT@UOW.EDU.AU@INTERNET#
To: BW.MILLER Beery W. Miller

-snip-

For those that do have HFDCCs and would like a copy of the new EPROM (version 13), if they can get a blank EPROM (27C128 or 27C256) to me at the next meeting or by mail I will be able to program it for them. I will be sending this to Beery Miller for him to distribute for others and the same conditions will hold for people overseas, except I will expe ct something to cover the cost of the postage."

 

Does anyone have a copy of this H13 EPROM from 1994?

 

I have identified the problem routine and came up with a fix that requires four additional bytes (two instructions). Alas, there is no space in the EPROM where I can inject a branch, execute the code, and return. I'm thinking Geoff identified a simpler way to fix this?

 

For the record, the DSR problem shows up when a fractured file is deleted. During the step that frees sectors from the bitmap, the counter that keeps track of the most recent sector (within the cluster) is calculated incorrectly. The value must first be stored in a temporary register then copied for the next iteration. This problem is particularly nasty because every removed cluster compounds the corruption.

 

Edit: To clarify, this DSR problem only affects floppy drives and the DSK1 emulate file. The Geneve is unaffected as its master DSR uses a different routine.

Edited May 18, 2018 by InsaneMultitasker

[atrax27407]

I don't have that one. I have Vn 1.1, Vn 1.4, and Vn 1.5. Would any of these be of help to you?

[+InsaneMultitasker]

I don't have that one. I have Vn 1.1, Vn 1.4, and Vn 1.5. Would any of these be of help to you?

 

Are those the same as H11,H14,and H15? I have dumps of the 'best' H11 and it contains the error. The H14 dump has the error and if I recall correctly H15 is a derivative of H11. (I use the "H" versus "Vn" only because that us how they used to be labeled).

 

H12 reportedly had bugs and was avoided like the plague. I neither have a dump of that version nor any documentation explaining the differences between H11 and H12.

[atrax27407]

Yes. They are H11, H14, and H15. Let me know if you need them and I can send them on. BTW, what are the offending bytes?

[+OLD CS1]

Is there a validation utility for TI file systems?

[+InsaneMultitasker]

Fred wrote some programs to validate file IO back when he was updating the IDE card's DSR. They might be usable with other devices, I've never checked.

 

Most of the existing DSRs have been vetted fairly well however this HFDC floppy issue seems to have slipped through the cracks because it only shows up when a file is fractured and then only if that file is deleted and another written. Many people fill their disks and don't look back. And since the corruption isn't obvious you wouldn't know unless you inspected all files or a catalog entry (FDR) was trashed in the process.

 

Even during copying you would not know the file was corrupted, because the sector content itself is valid even if it doesn't belong to the original file. Very naughty.

[+mizapf]

I wonder which version is used in MAME. The HFDC has been the default floppy controller from the start, and I don't remember any issues with file corruptions. So I should look out for a fragmented file, delete it, and write a new file?

[+InsaneMultitasker]

I wonder which version is used in MAME. The HFDC has been the default floppy controller from the start, and I don't remember any issues with file corruptions. So I should look out for a fragmented file, delete it, and write a new file?

 

I'm not next to my system right now so this XB program is from memory. Run this on a 360K or smaller disk to avoid the extra complications of multiple sectors/AU when inspecting the FDR and sectors.

 

(Use a blank disk. Save TEST1 and TEST2 on a different disk)

 

1 !SAVE DSK2.TEST1

100 OPEN #1:"DSK1.FILE1",OUTPUT, DISPLAY, FIXED 255

110 OPEN #2:"DSK1.FILE2",OUTPUT,DISPLAY, FIXED 255

120 FOR X=1 to 3::PRINT #1:"File 1"&RPT$(" ",200)::NEXT X

130 FOR X=1 to 3::PRINT #2:"File 2"&RPT$(" ",200)::NEXT X

140 Y=Y+1::IF Y<10 THEN 120

150 CLOSE #1::CLOSE #2

 

Upon completion, FILE1 and FILE2 should be fractured. On disk, the files will be written 3 sectors at a time, in the following repeating pattern: "111222111222...111222". Each file's FDR will contain 9 or 10 data chain pointer blocks based on the value of "Y" tested in 140. You can increase this number, just don't exceed the maximum cluster number for a floppy (76, IIRC).

 

Next, inspect the clusters starting at FDR byte 28d. Inspect the files. I used Birdwell's Disk Utilities to generate a file report showing the cluster allocations. View the files to visually confirm the files do not overlap at this time.

 

In emulation, you can make a copy of the .DSK file at this point.

 

Next, delete FILE1. This should free up the sectors from the bitmap.

 

Next run this program from XB. I chose FILE3 so that you can visually inspect the sectors and know whether it 'belongs' to file1, file2, or file3.

 

1 !SAVE DSK2.TEST2

100 OPEN #3:"DSK1.FILE3",OUTPUT,DISPLAY, FIXED 255

120 FOR X=1 to 3::PRINT #3:"File 3"&RPT$(" ",200)::NEXT X

140 Y=Y+1::IF Y<10 THEN 120

150 CLOSE #3

 

We expect the DSR to re-use the sectors originally allocated to FILE1. Alas, the delete process frees the wrong sectors and some FILE2 content will now be overwritten by FILE3.

 

1. View FILE3, all records should contain "File 3" records as expected.

2. View FILE2 (the file we did not delete). The first records should contain "File 2" but as you continue to view the file, File3 will show up in increasing frequency.

3. If you inspect the FDR cluster pointers and/or run a file report, File2 and File3 now overlap.

[+InsaneMultitasker]

Section of code that needs to be fixed with a possible solution (untested). Documenting for review.

 

 

 

* NOW JUST DELETE ALL THE CLUSTERS IN THE FDR
; each data chain pointer block is comprised of 3 bytes, which are then split into 2 sets of 3 nybbles.

; (1) The first sector of the cluster and (2) the highest sector offset within the file [NOT the sector itself!]

; from WINDS3S

;

BLWP @GETVIB
JEQ FRSER0 ERROR
AI R6,CLUSTR
CLR R10
LI R5,76
FNTPR1 MOVB *R6+,R1 get first 2 data chain nybbles
SWPB R1
MOVB *R6+,R1 get 3rd nybble
MOVB R1,R2
ANDI R1,>0FFF mask the three nybbles. Now we have the first sector of this cluster
JEQ DONCLS if 0, no more sectors have been allocated in this FDR
SWPB R2 one nybble for the offset
MOVB *R6+,R2 two more nybbles
SRL R2,4 mask off 4 bits; now we have the offset

;

; R1- holds starting sector to free for this cluster

; R2- holds total sectors to free, base 0

; R10 - should track the sector offset within the file for each iteration

;

; unfortunately, as the routine works through each cluster, R10 no longer

; points to the current high sector. We need to store the value and correct

; for the next iteration.

*FNTPR2 S R10,R2
* MOV R2,R10
;

; possible solution? Must validate.

;

mov r2,@hold ;save copy of last highest sector
s r10,r2 ; calculate sectors this cluster
mov @hold,r10 ;restore R10 for our counter
inc r10 ;and adjust for next pass. Must be done AFTEr first pass, since it will already be base 0.

 

BLWP @FRBITM

DEC R5
JNE FNTPR1

 

 

[+mizapf]

Thinking about that, we should try to collect all available ROM dumps for use in MAME; you would then be able to select one of it by a configuration switch. This would allow us to trace issues due to bugs in the DSR. We could then also add a new variant that you seem to propose here.

[+acadiel]

Thinking about that, we should try to collect all available ROM dumps for use in MAME; you would then be able to select one of it by a configuration switch. This would allow us to trace issues due to bugs in the DSR. We could then also add a new variant that you seem to propose here.

 

Just for my own knowledge; how hard is it to get another one of the controller cards working like the WD1773 Corcomp one?

[+mizapf]

Just for my own knowledge; how hard is it to get another one of the controller cards working like the WD1773 Corcomp one?

 

Almost there, we have high-quality PCB photos, the ROMs, and the set of circuits on it. However, we are still lacking the schematics, if I remember correctly. I started to draw schematics almost three years ago. It would take some weeks of focused work on that one to get it running, but it should be possible. I put it back on lower priority and continued with the 99/8 instead.

[+Ksarul]

Once you do get a raw schematic for it, I can take that and turn it into the Visio format I've been using for all of the other cards that I've done new schematics for. Slowly but surely, we have schematics for all of the FDC boards. I think the only one completely missing is the Atronic controller, and I still need to do the updated diagrams for the HFDC and the TI Sidecar (but at least we have something for both of them).

 

Actually, of you have the photos of a bare board and a comparison photo set of the same board populated, I can make a run at doing the schematic, Michael. I did the same thing for the Myarc FDC.

Edited May 20, 2018 by Ksarul

[+mizapf]

Actually, of you have the photos of a bare board and a comparison photo set of the same board populated, I can make a run at doing the schematic, Michael. I did the same thing for the Myarc FDC.

 

I decided to push the complete set of photographs (230 MiB) on my server, before I start to sort out possibly unneeded ones, which could be right those ones that would have helped you most. The package contains populated and unpopulated Corcomp boards, with or without the MG extensions.

 

Please get them from http://www.mizapf.eu/files/corcomp.zip

 

The xcf files are GIMP files (with multiple layers), they contribute most of the size. If you prefer to get a smaller set, just tell me.

[+Ksarul]

These should get me most of the way to where I need to be, Michael. I have downloaded the whole set already. . .

[+InsaneMultitasker]

Bank 00 holds the offending code. I compared H11 to the assembled H12 code. The first 1k seems to match. A routine to save and restore the FAC was modified in H12, presumably to extend the range and save a few bytes. The code is off by 4 bytes as a result, so I can't truly compare the code and reconcile it without removing 4 bytes, reassembling, and comparing. I'll save the other 3 banks for some other time.

 

Fortunately, RAG linker was able to create the bank 00 image file though it added a 6-byte header to the file. I haven't figured out how to tell the linker to drop the header. Minor inconvenience at this point. In H11 there are six available bytes at the end of the bank; in H12, there are 10, the exact number of bytes I need for a 'hack' test. I don't remember an easy way to combine the four banks together into one image from the TI side.

[+InsaneMultitasker]

Had a chance to assemble the other three banks. Bank01 assembles with two unknown jump labels and one unknown data label. Banks 10 and 11 assemble properly. Unfortunately, there are differences in each bank when compared with version H11. Reconciling this will not be as simple as I first thought and without understanding all the changes, I'm not sure that using this version is a good idea. I will dig into this more when time permits.

[+InsaneMultitasker]

Since the H11 and H12 EPROMs are so different, I decided to abandon the H12 source and tackle this another way.

 

I disassembled H11 bank 00 then reassembled the resulting source code. After a few tweaks I was able to successfully match the EPROM bank to the reconstituted image file. Next I went through the code and fixed the absolute addressing I could find (hopefully I caught them all) and again verified that everything assembled properly. I next updated the floppy cluster routine and reassembled. Because the paging routines for the HFDC are located after the fixed code, there are differences across the DSR. Ah well. A little bit of Winhex magic later and I had replaced bank 00 with my updated routines, and created a new HFDC.bin file.

 

The Geneve booted fine, which in theory meant the HFDC powerup routine worked. However, when I used ROMPAGE, the HFDC won't complete its powerup and locks the system. So something is not quite right and I hope I just missed an absolute address, courtesy of Diskassembler. I may play around with this a bit more tonight.

 

Edit: I realized that all of the DSR linked lists and jump tables contain DATA statements with pointers to routines. Diskassembler isn't smart enough to deal with this type of code. oops.

Edited May 26, 2018 by InsaneMultitasker

[+InsaneMultitasker]

Today I worked through bank 00 to identify all of the linked lists and BLWP vectors. I then re-assembled the code to compare with the original, corrected a few typos, and ended up with a matching image file.

 

Next I added my corrections, reassembled, and created a new HFDC DSR binary. Within MAME the DSR Powerup worked properly and I was able to load files from the floppy drive. Next I ran my XB test program to intertwine/fracture two files called TEST1 and TEST2. I reviewed the file reports via Disk Utilities and wrote down the used sectors. I then deleted FILE1 and recreated it as FILE3. The new file was written to the same sectors previously allocated to FILE1. No corruption!

 

The challenges here are two-fold: (1) to further verify that my code change works and does not impact some other operation and (2) to verify that I didn't miss some hard-coded address. I don't want to introduce a new problem.

 

I am cautiously optimistic...

 

Edit: attached the H91 test DSR. Use at your own risk.

hfdc-H11-H91.bin

Edited May 28, 2018 by InsaneMultitasker

[+Ksarul]

Thanks, O Insane One! I'll have to try this out soon. . .

[+InsaneMultitasker]

Thanks, O Insane One! I'll have to try this out soon. . .

I ran into one problem that I so far have not been able to reproduce. I copied files from DSK1 to DSK2. Although the files copied OK, their file descriptor records were wrong. The word that holds the allocated level 3 records was set to 3 sectors for every file. Seems too much of a coincidence to not be related to my changes. I was playing with MDM5 and Disk Utilities at the time and didn't realize the issue right away. I am trying to recall what I did in the hopes of reproducing the error.

[Swim]

Yes. They are H11, H14, and H15. Let me know if you need them and I can send them on. BTW, what are the offending bytes?

 

I think that H14 had head step speeds changed/added? What does H15 offer over H14?

[atrax27407]

H15 (and H16, I think) were creations from "The Great Gazoo". H15 removed floppy drive control for those who had RAMdisks as DSK4-8 and H16 removed hard drive functionality for those who wanted to use the HFDC strictly as a floppy drive controller.

[Swim]

H15 (and H16, I think) were creations from "The Great Gazoo". H15 removed floppy drive control for those who had RAMdisks as DSK4-8 and H16 removed hard drive functionality for those who wanted to use the HFDC strictly as a floppy drive controller.

 

Ah, I didn't realize they got their own version label. Thanks.

[atrax27407]

Those are Tony's designations to differentiate them from existing versions.