Gesswein MFM hard disk emulator

[jbdigriz]

This is a neat device that emulates 1-2 ST506/ST412 or other MFM hard disks for reading and or writing. Can be used to pull data off disks or to emulate disks for that old computer you don't have drives for. I think I read somewhere it can be passed thru to a VM. Possibly this could work for MAME/MESS emulated systems?

 

TI-99 has been tested with the system and works for reading WDS-100 disks. Emulation works for reading, but writing fails this point. Probably have to be patched for the TI/Myarc sector layouts, directory structure, bitmaps, etc. (not enough info yet to correct, it says), but the actual writing appears to work. Works with a wide variety of other systems. List here:  http://www.pdp8online.com/mfm/status.shtml

 

There is a group buy under way right now if you want to check it out. Pricing seems reasonable. http://www.pdp8online.com/mfm/

 

Emulator uses a BeagleBone Black/BeagleBone Green Wireless for the actual processing. Interesting. Go team, TI! ? 

 

Edited August 11, 2020 by jbdigriz

[+Ksarul]

Looks like an interesting alternate to the DREM, if they get it working with TI disk formats. The DREM is already fully compatible with the TI, but choices are always a good thing. . .

[+9640News]

It's great there is something else out there.  I just hope there is someone out there that make it fully compatible with the TI-99 as it took Oleksandr a lot of time and back and forth testing to get the DREM fully functional.  I seriously doubt it will be just a few tweaks as the HDC9234 chip did have enough differences from the WDS emulation it will take some effort on someone's part.  At least with the DREM, it is fully functional on hard drive capability.  I'm running 2 x 64 MB images with my setup.

 

Beery

[+arcadeshopper]

I own one of these and was able to format it on my wds100 setup.. got some errors later though during copying things to it but wasn't sure if it was my controller or the emu.. so sounds like it was the emu

 

 

[jbdigriz]

7 hours ago, Ksarul said:

Looks like an interesting alternate to the DREM, if they get it working with TI disk formats. The DREM is already fully compatible with the TI, but choices are always a good thing. . .

 

7 hours ago, jbdigriz said:

There is a group buy under way right now if you want to check it out. Pricing seems reasonable. http://www.pdp8online.com/mfm/

 

 

Wups, the URL for the current group buy is actually: https://decromancer.ooo/mfm-emulator.html  You can buy one at the link above but the decromancer on has a few mods IIRC.

[jbdigriz]

7 hours ago, Ksarul said:

Looks like an interesting alternate to the DREM, if they get it working with TI disk formats. The DREM is already fully compatible with the TI, but choices are always a good thing. . .

Yes, I was familiar with the DREM (and the HxC, Gotek, etc for floppies) but this is the first emu I've seen that can read physical hard disks, unless I'm missing something here. Also the Gesswein emulator uses the BeagleBone, which I need to get anyway. Yes, you could hack the DREM for all kinds of things and I see it has an expansion port. So, yes, choices are good. Not meaning to salt anyone's cornflakes. ;-)

[jbdigriz]

6 hours ago, arcadeshopper said:

I own one of these and was able to format it on my wds100 setup.. got some errors later though during copying things to it but wasn't sure if it was my controller or the emu.. so sounds like it was the emu

 

 

Are your test results, logs, etc. written up anywhere?

[jbdigriz]

7 hours ago, BeeryMiller said:

It's great there is something else out there.  I just hope there is someone out there that make it fully compatible with the TI-99 as it took Oleksandr a lot of time and back and forth testing to get the DREM fully functional.  I seriously doubt it will be just a few tweaks as the HDC9234 chip did have enough differences from the WDS emulation it will take some effort on someone's part.  At least with the DREM, it is fully functional on hard drive capability.  I'm running 2 x 64 MB images with my setup.

 

Beery

Point well taken, but I'd be willing to take a shot at it, so I'll probably buy at least a bare PCB. Reminds me, though, do you know if the tape interface on the SMC chipset was ever gotten to work reliably? I've gotten different answers over the years, but never found detailed instructions for the HFDC.

Edited August 12, 2020 by jbdigriz

[+arcadeshopper]

Are your test results, logs, etc. written up anywhere?

Nope just what I said it formatted fine then let me copy some files then it came up with errors

Sent from my LM-V600 using Tapatalk

[jbdigriz]

Mostly off-topic here, but what I'm really looking for is an SMD disk emulator that will do 990 drive emulation. DS10, and  DS50 drives mainly, since that's what I have controllers for. There is a Fujitsu Eagle emulator being worked on that might be usable with some work, so there's hope.

[jbdigriz]

16 minutes ago, arcadeshopper said:

Nope just what I said it formatted fine then let me copy some files then it came up with errors

Sent from my LM-V600 using Tapatalk
 

Wish I knew more about the WDS100. Which chipset does it use on the controller? And thanks for the feedback.

 

Edited August 12, 2020 by jbdigriz

[RickyDean]

I have three of these boards from pdponline, but have not soldered the first component on them yet. Been building a Sams, two horizon 4000b's, and repairing other ti cards as well as a couple of tv's, a Chevy Duramax, and changing engines in a 05 Durango, plus changing out the air conditioning compressor in my 5 ton unit and recharging it. So been a busy little boy. That's not counting the countless other things I'm doing.

[jbdigriz]

9 minutes ago, RickyDean said:

I have three of these boards from pdponline, but have not soldered the first component on them yet. Been building a Sams, two horizon 4000b's, and repairing other ti cards as well as a couple of tv's, a Chevy Duramax, and changing engines in a 05 Durango, plus changing out the air conditioning compressor in my 5 ton unit and recharging it. So been a busy little boy. That's not counting the countless other things I'm doing.

I know the feeling, trust me. Please keep us posted with your experiences with the boards, when you get to them. Thanks!

 

[+9640News]

10 hours ago, jbdigriz said:

Point well taken, but I'd be willing to take a shot at it, so I'll probably buy at least a bare PCB. Reminds me, though, do you know if the tape interface on the SMC chipset was ever gotten to work reliably? I've gotten different answers over the years, but never found detailed instructions for the HFDC.

The Tape interface never had software/etc. that was developed to work with it, if it even worked at all.  Some of the later HFDC's had some of the chips for the tape interface removed as well.

 

With the DREM, you just power down your system, pull the SD card, insert it into the PC, and copy your files.  Takes about 20 seconds to backup 2 x 64 MB images.

 

Beery

[jbdigriz]

3 hours ago, BeeryMiller said:

The Tape interface never had software/etc. that was developed to work with it, if it even worked at all.  Some of the later HFDC's had some of the chips for the tape interface removed as well.

Hmm. Another project, maybe, but later.

3 hours ago, BeeryMiller said:

With the DREM, you just power down your system, pull the SD card, insert it into the PC, and copy your files.  Takes about 20 seconds to backup 2 x 64 MB images.

Straightforward enough. Of course, if you back up using a TiPi or serial transfer, you wouldn't even have to shut down.
I always used MagicFM for things like this, but does the DREM allow you to treat the entire emulated disk as an image file of some kind? That would be less tedious.

3 hours ago, BeeryMiller said:

Beery

 

[+9640News]

3 minutes ago, jbdigriz said:

Hmm. Another project, maybe, but later.

Straightforward enough. Of course, if you back up using a TiPi or serial transfer, you wouldn't even have to shut down.
I always used MagicFM for things like this, but does the DREM allow you to treat the entire emulated disk as an image file of some kind? That would be less tedious.

 

With the DREM, each image whether it be a hard drive image (supports 2 for our systems) or floppy image (supports 2 as well) has a CFG and a DSK file.  The CFG file gives information about the image including the controller type it is interfaced to, and all of the disk geometry.  The DSK file is basically a sector by sector dump of the image.

 

I've got an image setup for 64 MB which is 8 heads, 1024 cylinders, 32 sectors/track,  256 bytes/sector.  

 

I have had issues stepping up to 2048 cylinders, but that I believe is on the formatting side of things, not with the DREM itself.  At the time, I do not believe there was anything on the market with 2048 cylinders to allow us to step to 128 MB even though MDM5 allows you to enter a higher cylinder count.

 

There was another way to step up to 128 MB, and that was to go to 16 heads, but that requires a hardware modification to the HFDC that Barry Boone had published in Micropendium.  At this point, I really do not need anything above 64 MB anyways.

 

I do recall back in the 80's, there were a couple of users that indicated they had 80 MB drives.  Not sure if they ever filled the drive that full to use most of the space or not.

 

Myarc did claim the HFDC could do either 120 or 128 MB, but do not know if they had an actual drive and proved it, or just assumed it.

 

Beery

 

[webdeck]

I pulled the trigger and ordered one of these to use it to try to get a raw image of my dying hard drive.  Using the ST-225's recovery mode, I am close to a full read:

Found cyl 0 to 614, head 0 to 3, sector 0 to 31                                
Expected 78720 sectors got 78455 good sectors, 0 bad header, 265 bad data
0 sectors marked bad or spare
149 sectors corrected with ECC. Max bits in burst corrected 8
Track read time in ms min 29.553666 max 1776.017709 avg 38.902760

I have corresponded back and forth with David and he was able to add initial support for the HFDC format to be read based on the raw data from my drive.  There is still more work to do to get emulation working properly.  You can see the code changes in this commit: https://github.com/dgesswein/mfm/commit/da54ff8cef77b1a1416edcdfea1fca40ac6ff6cf

 

The format he was able to reverse-engineer:

//   CONTROLLER_MYARC_HFDC
//   No manual found describing low level format
//   6 byte header + 2 byte CRC
//      byte 0 0xa1
//      byte 1 0xfe ^ upper 2? bits of cylinder
//      byte 2 cylinder low
//      byte 3 cylinder high in upper 4? bits, low 4 bits head
//      byte 4 sector number
//      byte 5 unknown, 1 for sample I have
//      byte 6-7 CRC
//
//   Data
//      byte 0 0xa1
//      byte 1 0xfb or 0xf8
//      Sector data for sector size
//      ECC code (4 byte)

I'm wondering if anyone has some details on the low level specification for the format used by the HFDC.  Some key areas to understand:

1. Apparently, at least based on the raw read of my drive, the deleted data mark 0xf8 is used in some data sectors, which is different from other controllers.  Anyone know details on this?
2. For the dump of my drive, he saw that the last cylinder was in normal WD format. It has 256 byte sectors with a different polynomial. I wonder if my drive didn't format the last cylinder and that is old data from a prior format? Or is this to be expected?

If anyone can point me to some docs I can work with David to try and get his lower cost MFM emulator to fully support the Myarc HFDC.

 

Thanks,

-Mike

[+9640News]

1 hour ago, webdeck said:

I pulled the trigger and ordered one of these to use it to try to get a raw image of my dying hard drive.  Using the ST-225's recovery mode, I am close to a full read:

Found cyl 0 to 614, head 0 to 3, sector 0 to 31                                
Expected 78720 sectors got 78455 good sectors, 0 bad header, 265 bad data
0 sectors marked bad or spare
149 sectors corrected with ECC. Max bits in burst corrected 8
Track read time in ms min 29.553666 max 1776.017709 avg 38.902760

I have corresponded back and forth with David and he was able to add initial support for the HFDC format to be read based on the raw data from my drive.  There is still more work to do to get emulation working properly.  You can see the code changes in this commit: https://github.com/dgesswein/mfm/commit/da54ff8cef77b1a1416edcdfea1fca40ac6ff6cf

 

The format he was able to reverse-engineer:

//   CONTROLLER_MYARC_HFDC
//   No manual found describing low level format
//   6 byte header + 2 byte CRC
//      byte 0 0xa1
//      byte 1 0xfe ^ upper 2? bits of cylinder
//      byte 2 cylinder low
//      byte 3 cylinder high in upper 4? bits, low 4 bits head
//      byte 4 sector number
//      byte 5 unknown, 1 for sample I have
//      byte 6-7 CRC
//
//   Data
//      byte 0 0xa1
//      byte 1 0xfb or 0xf8
//      Sector data for sector size
//      ECC code (4 byte)

I'm wondering if anyone has some details on the low level specification for the format used by the HFDC.  Some key areas to understand:

1. Apparently, at least based on the raw read of my drive, the deleted data mark 0xf8 is used in some data sectors, which is different from other controllers.  Anyone know details on this?
2. For the dump of my drive, he saw that the last cylinder was in normal WD format. It has 256 byte sectors with a different polynomial. I wonder if my drive didn't format the last cylinder and that is old data from a prior format? Or is this to be expected?

If anyone can point me to some docs I can work with David to try and get his lower cost MFM emulator to fully support the Myarc HFDC.

 

Thanks,

-Mike

 

I posted the Myarc MDM5 source code that includes the MFM hard drive formatting code about a month ago here on Atariage.  The source for the MFM format code is in the archive file.

 

 

MDM5-V129-V160.zip

Edited October 24, 2020 by BeeryMiller

[+9640News]

The source file you are looking for in the above zip file is FW2-S.  I think that has the details you are seeking.

 

Beery

 

[webdeck]

Okay, #2 turned out to be a mistake and wasn't actually the case with the raw read of my drive.

 

As for #1, not clear on what the deleted data mark is used for.  Is the source code for the HFDC DSR available?

 

The good news is that I have been able to use the emulator now with a TI console, PEB, HFDC, FG99, and TIPI, and the files that don't have errors are comparing byte for byte with the ones I copied off my hard drive previously.  Force command is making it a breeze to copy files over to the TIPI where I can compare them with linux diff.

 

As for the 256 bad sectors from the original drive, while they are mostly concentrated in one area the drive and mostly on head 3, they are impacting a fair number of files, sadly.  This does confirm that it is my physical hard drive that is having issues and not the HFDC at least.

 

I was also able to edit a file successfully, but I haven't tested writes extensively yet.

[+mizapf]

The "deleted data mark" is a special data address mark (DAM) that says that the contents of the sector shall be ignored. It is specified in the encoding of FM and MFM, but I don't know whether this is really used with any floppy drive system.

 

However, as far as I remember, the HFDC DSR does it the wrong way, it initializes sectors with the DDAM instead of the DAM. As it is ignored by all other controller, no harm is done.

[webdeck]

Yes, the FW-2 code that Beery pointed me to uses command >62 to format, which, according to http://www.whtech.com/ftp/datasheets and manuals/Datasheets - Misc/SMC HDC9234.pdf writes the deleted data mark instead of the normal data mark because bit 4 is zero (>72 would write the normal data mark.)  Sounds like a bug in MDM5.  I would need to see the DSR code to know what it is doing when it writes out a sector to see what command it is sending to the 9234.  The meaning of bit 4 is inverted for the write commands vs. the format command, which probably caused confusion.

[+mizapf]

I noticed that issue when debugging the HFDC emulation in MAME. If I remember correctly, we could verify this with a real HFDC and a Lotharek/Gotek.

 

In fact, the WD17xx and the SMC92x4 controllers just set a status bit which indicates whether it is a DAM or DDAM. It works normally in all other respects, so it seems as if the DSRs don't check the status bit.

 

18 minutes ago, webdeck said:

I would need to see the DSR code to know what it is doing when it writes out a sector to see what command it is sending to the 9234.

The quickest way would be to turn on the logging in MAME. I can do that in a few minutes. Do you have a particular test that I should run?

[+9640News]

1 hour ago, webdeck said:

Yes, the FW-2 code that Beery pointed me to uses command >62 to format, which, according to http://www.whtech.com/ftp/datasheets and manuals/Datasheets - Misc/SMC HDC9234.pdf writes the deleted data mark instead of the normal data mark because bit 4 is zero (>72 would write the normal data mark.)  Sounds like a bug in MDM5.  I would need to see the DSR code to know what it is doing when it writes out a sector to see what command it is sending to the 9234.  The meaning of bit 4 is inverted for the write commands vs. the format command, which probably caused confusion.

I found the HFDC Eprom source code.  There has been concern this code may or may not have a bug, but I forgot what that may have been.  @InsaneMultitasker I know has commented on this subject somewhere here onAtariage in the past.  This code is the only code I am aware that exists outside of any code someone may have disassembled from the eprom.  I haven't chased down the specific file that has the code you are seeking.

HFDC.ARK

[webdeck]

2 hours ago, mizapf said:

I noticed that issue when debugging the HFDC emulation in MAME. If I remember correctly, we could verify this with a real HFDC and a Lotharek/Gotek.

 

In fact, the WD17xx and the SMC92x4 controllers just set a status bit which indicates whether it is a DAM or DDAM. It works normally in all other respects, so it seems as if the DSRs don't check the status bit.

 

The quickest way would be to turn on the logging in MAME. I can do that in a few minutes. Do you have a particular test that I should run?

I guess the test would be to see if the DSR code cares at all if a sector has DAM or DDAM.  From what I can tell is formatted sectors have DDAM and then whenever data is written it changes to DAM.  It would simplify things for the emulator code as written if all the DDAMs were replaced with DAMs, but unsure if that would break anything in the real world when using it.