SDrive at 126kbit/sec - please help testing

[HiassofT]

Hi!

 

I modified the SDrive firmware so that Pokey divisor 0 should work fine on both PAL and NTSC Ataris (for example using my highspeed SIO patch).

 

http://www.horus.com/~hias/tmp/sdrive-hias-090602.zip

 

Please report back if this version works fine on your Ataris/SDrives so that Raster can add the changes in the next official release.

 

Some background information:

 

I replaced the USART_Transmit_Bytes routine with a bit-banging implementation so that we have complete control over transmission timing. This way we can work around Pokey strangeness (it samples 5 Atari clock cycles too late) and clock/crystal speed differences (SDrive uses an 14.31818MHz crystal which matches NTSC speeds exactly, but is approx. 1% too fast for PAL Ataris).

 

so long,

 

Hias

[a8isa1]

Hi!

 

I modified the SDrive firmware so that Pokey divisor 0 should work fine on both PAL and NTSC Ataris (for example using my highspeed SIO patch).

 

http://www.horus.com/~hias/tmp/sdrive-hias-090602.zip

 

Please report back if this version works fine on your Ataris/SDrives so that Raster can add the changes in the next official release.

 

Some background information:

 

I replaced the USART_Transmit_Bytes routine with a bit-banging implementation so that we have complete control over transmission timing. This way we can work around Pokey strangeness (it samples 5 Atari clock cycles too late) and clock/crystal speed differences (SDrive uses an 14.31818MHz crystal which matches NTSC speeds exactly, but is approx. 1% too fast for PAL Ataris).

 

so long,

 

Hias

Hi, Hias!

 

I wanted to test out the higher speeds with SDrive but my 800XL is in need of repair/replacement and my 800 doesn't seem to be up to the task.

 

Pokey divisor of 2 appeared to work with your basic diagnostic but when I tried the extended diagnostic the test stops with error 43.

 

The H option in SDRIVE.COM does report that the new firmware is in place.

 

I'll try again when I have a working XL.

 

Regards,

 

Steve Sheppard

[HiassofT]

Hi Steve!

 

I wanted to test out the higher speeds with SDrive but my 800XL is in need of repair/replacement and my 800 doesn't seem to be up to the task.

 

Pokey divisor of 2 appeared to work with your basic diagnostic but when I tried the extended diagnostic the test stops with error 43.

Error 43? Strange...

 

On the Atari 800 you have to use "diag-nonmi.atr" or "diag-ext-nomni.atr". The other versions try to copy the OS ROM into RAM and install a faster NMI handler (which will fail on the Atari 800). The "nonmi" versions simply use a faster immediate VBI handler and should also work with the Atari 800. Divisor 0 might cause occasional problems, but divisor 1 should work fine.

 

so long,

 

Hias

[a8isa1]

Hi Steve!

 

I wanted to test out the higher speeds with SDrive but my 800XL is in need of repair/replacement and my 800 doesn't seem to be up to the task.

 

Pokey divisor of 2 appeared to work with your basic diagnostic but when I tried the extended diagnostic the test stops with error 43.

Error 43? Strange...

 

On the Atari 800 you have to use "diag-nonmi.atr" or "diag-ext-nomni.atr". The other versions try to copy the OS ROM into RAM and install a faster NMI handler (which will fail on the Atari 800). The "nonmi" versions simply use a faster immediate VBI handler and should also work with the Atari 800. Divisor 0 might cause occasional problems, but divisor 1 should work fine.

 

so long,

 

Hias

Oops!

 

I was using older diagnostics. With the current versions SDrive does indeed test successfully with Pokey divisor of 1.

 

With divisor of 0 the test stops almost immediately.

 

- Steve Sheppard

[HiassofT]

I was using older diagnostics. With the current versions SDrive does indeed test successfully with Pokey divisor of 1.

Great, thanks for testing!

 

With divisor of 0 the test stops almost immediately.

This is normal. I did some more tests with diag-ext-nonmi and compared the XL OS with the "old" REV.A OS:

 

The NMI handler of the old OS needs slightly more CPU time (it has to check for the "system reset" key) and therefore almost every transmission fails at divisor 0.

 

With the XL OS the chances of a transmission succeeding are a little bit better. diag-ext-nonmi is usually able to read some 5-50 sectors before failing.

 

In my current development version of MyPicoDos I use a slightly modified highspeed code (the fast VBI handler doesn't increment 18..20), which works fine at divisor 0 with an unpatched XL OS. But on the old OS divisor 0 also causes frequent "hiccups" (every 20-40 sectors or so).

 

so long,

 

Hias

[a8isa1]

I was using older diagnostics. With the current versions SDrive does indeed test successfully with Pokey divisor of 1.

Great, thanks for testing!

 

With divisor of 0 the test stops almost immediately.

This is normal. I did some more tests with diag-ext-nonmi and compared the XL OS with the "old" REV.A OS:

 

The NMI handler of the old OS needs slightly more CPU time (it has to check for the "system reset" key) and therefore almost every transmission fails at divisor 0.

 

With the XL OS the chances of a transmission succeeding are a little bit better. diag-ext-nonmi is usually able to read some 5-50 sectors before failing.

 

In my current development version of MyPicoDos I use a slightly modified highspeed code (the fast VBI handler doesn't increment 18..20), which works fine at divisor 0 with an unpatched XL OS. But on the old OS divisor 0 also causes frequent "hiccups" (every 20-40 sectors or so).

 

so long,

 

Hias

Hi Hias!

 

Thanks for explaining that.

 

I'm looking forward to trying the revised MyPicoDos.

 

Regards,

 

Steve Sheppard

[HiassofT]

Thanks for explaining that.

 

I'm looking forward to trying the revised MyPicoDos.

I've uploaded a preview version to my website: http://www.horus.com/~hias/tmp/mypdos-090603.zip

 

You can now choose in the initializer if highspeed SIO should be enable during the boot process or, as before, after booting or not at all.

 

I've also added a special "SDrive version" (MYINITS.COM) which auto-configures the SDrive to use divisor 0 or 1 (selectable in the initializer). Of course, this version only makes sense with the SDrive :-)

 

Please drop me a line if pokey divisor 0 works on your system(s).

 

so long,

 

Hias

[a8isa1]

Thanks for explaining that.

 

I'm looking forward to trying the revised MyPicoDos.

I've uploaded a preview version to my website: http://www.horus.com/~hias/tmp/mypdos-090603.zip

 

You can now choose in the initializer if highspeed SIO should be enable during the boot process or, as before, after booting or not at all.

 

I've also added a special "SDrive version" (MYINITS.COM) which auto-configures the SDrive to use divisor 0 or 1 (selectable in the initializer). Of course, this version only makes sense with the SDrive :-)

 

Please drop me a line if pokey divisor 0 works on your system(s).

 

so long,

 

Hias

Hias, this is great!

 

Pokey divisor=0 is working perfectly. No stuttering or other problems.

 

Thanks again!

 

- Steve Sheppard

[HiassofT]

Here's an updated version of the SDrive firmware: http://www.horus.com/~hias/tmp/sdrive-hias-090609.zip

 

I changed the SIO code a little bit, the new version should work fine with QMEG 3 OS again (thanks to Raster for reporting the bug).

 

And I added another feature: the SDrive now honors the "read only" file attribute of (image-) files and won't allow formatting or writing to protected images from the Atari. This means you can have both write protected (important) and writable (for example data) drives at the same time.

 

so long,

 

Hias

[raster/c.p.u.]

Here's an updated version of the SDrive firmware: http://www.horus.com/~hias/tmp/sdrive-hias-090609.zip

I changed the SIO code a little bit, the new version should work fine with QMEG 3 OS again (thanks to Raster for reporting the bug).

Thanks. I will test it.

And I added another feature: the SDrive now honors the "read only" file attribute of (image-) files and won't allow formatting or writing to protected images from the Atari. This means you can have both write protected (important) and writable (for example data) drives at the same time.

Hmm, I'm sorry but I think it isn't useful feature.

The reason is in SDrive control philosophy. You haven't any possibility for change "file read only attribute" by SDrive control program not even during your program is running. So, you can use some game/program/tool and if you need to save some unique created state/config/image/music/data, you discover that it can't be stored anyhow. It is the point why SDrive firmware ignore read-only file attribute and it respects manual R/W switch only.

[HiassofT]

And I added another feature: the SDrive now honors the "read only" file attribute of (image-) files and won't allow formatting or writing to protected images from the Atari. This means you can have both write protected (important) and writable (for example data) drives at the same time.

Hmm, I'm sorry but I think it isn't useful feature.

The reason is in SDrive control philosophy. You haven't any possibility for change "file read only attribute" by SDrive control program not even during your program is running. So, you can use some game/program/tool and if you need to save some unique created state/config/image/music/data, you discover that it can't be stored anyhow. It is the point why SDrive firmware ignore read-only file attribute and it respects manual R/W switch only.

And if we add an option (ignore/obey read only flags) to the control program? It should be possible to squeeze this into the sdrive firmware and then the user can choose which behaviour he/she prefers.

 

The reason I added this feature was because I liked to have some finer grained R/W control and to be able to protect images from accidentally changing/trashing/... them.

 

What do you think?

 

so long,

 

Hias

[raster/c.p.u.]

And if we add an option (ignore/obey read only flags) to the control program? It should be possible to squeeze this into the sdrive firmware and then the user can choose which behaviour he/she prefers.

Unfortunately it doesn't solve the problem with manual switching for write enable. Again you have to decide in advance, if you want to write or not to read-only images without possibility to change your choice when you really need it.

And it would increase size of control program and configuration again. I don't like this way...

 

The reason I added this feature was because I liked to have some finer grained R/W control and to be able to protect images from accidentally changing/trashing/... them. What do you think?

I fully understand the advantages of read-only protection, but I think that this feature could cause a lot of troubles for SDrive users (and for me of course ).

Now I have the SDrive in read-only mode default (for protection of all data/programs/etc...) and I'm enabling write-enable switch only when I need it.

Edited June 10, 2009 by raster/c.p.u.

[HiassofT]

Hi Raster!

 

I fully understand the advantages of read-only protection, but I think that this feature could cause a lot of troubles for SDrive users (and for me of course ).

Now I have the SDrive in read-only mode default (for protection of all data/programs/etc...) and I'm enabling write-enable switch only when I need it.

I guess this has a lot to do with personal preferences and the usual work-flow one is used to. Actually, I prefer it the other way round :-)

 

IMHO the concept of a write-protect/read-only file attribute or switch (on the SD card) is to protect the data, mainly from user errors. This is a very safe method and a lot less error prone than manually having to toggle a switch. So, basically, write access is only allowed if the card itself is unlocked AND if the (image-) file is marked read/write. In all other cases the data is read-only.

 

I use the read-only file attribute only on a few occasions: only in the case where I want a certain (original) disk to be protected. In these cases I usually have a writable working copy of the disk (image) and/or separate data disks.

 

BTW: of course you don't need to include the feature into your mainline firmware if you don't like it :-)

 

so long,

 

Hias

[raster/c.p.u.]

I guess this has a lot to do with personal preferences and the usual work-flow one is used to.

Of course.

I'm not sure if I explain my main reason truly.

It's the situation when I (some user) have created some new unique data (music in TMC, picture in graphics editor, etc.) and then I (he) _need_ to write it for sure.

There will be too many obstacles:

* SDcard LOCK - you can't eject the SDcard and unlock the card (loss of all SDrive drives' settings), that's why SDrive has unlock manual switch.

* SDrive write switch disabled - you can enable it manually.

* file readonly attribute is set - you can't eject SDcard and permit write to file (loss of all SDrive drives' settings), so there will not be any solution to write data => You have to loss your new created data, sorry.

 

I think you are certain of all read-only file attributes states on all your mediums and all files always. But I'm sorry, I'm not.

So, It's my resulting reason for clear and any time manually changeable switch - read-only (always) / read-write (always).

[HiassofT]

Here's another update:

http://www.horus.com/~hias/tmp/sdrive-hias-090619.zip

 

This version fixes the formatting issues with MyDos. Previously only SD and ED images worked, SDrive didn't like the "GetStatus" command between "PercomPut" and "Format".

 

Hint: to format a large disk image from MyDos first use the "O" command to configure the drive as a high capacity drive and enter the total number of sectors (for example 1440 for a QD image) and then use "I" to format the drive.

 

so long,

 

Hias

[HiassofT]

I just noticed that formatting in enhanced density (using the $22 command) was broken in my last 2 versions :-(

 

Here's a fixed version (I also managed to get AVR Studio running in VMware and updated the project file):

http://www.horus.com/~hias/tmp/sdrive-hias-090620.zip

 

so long,

 

Hias

[a8isa1]

Here's another update:

http://www.horus.com/~hias/tmp/sdrive-hias-090619.zip

 

This version fixes the formatting issues with MyDos. Previously only SD and ED images worked, SDrive didn't like the "GetStatus" command between "PercomPut" and "Format".

 

Hint: to format a large disk image from MyDos first use the "O" command to configure the drive as a high capacity drive and enter the total number of sectors (for example 1440 for a QD image) and then use "I" to format the drive.

 

so long,

 

Hias

Formatting has never worked for me with SDrive.

 

I have tried all of your recent beta firmwares. The red LED stays lit as does yellow #1 and SDrive locks up, ie. doesn't respond to the Atari nor to buttons BOOT, LEFT or RIGHT.

 

For MyDOS on SDrive I can use the /n option, recreate file structure, but not an actual format.

 

Also with the latest firmware I noticed something odd. In the past for a large ATR (e.g 16MB) if I forget to use the 'O' option to set the disk parameters and do a format (with '/n') I would get a default 720 sectors. With the new firmware the default now seems to be 1040 formatted sectors. Doesn't matter if the ATR was created SD or DD. I never use ED, btw.

 

If I do use the 'O' option and set the size parameters correctly then there is no problem. I get the correct number of sectors formatted.

 

Since the issuse only occurs due a user mistake the problem is not very significant. It's just something I noticed.

 

The bigger problem, not being able to issue a format command, has me a little concerned since now I know SDrive was intended to have this ability and my SDrive won't allow it. However, this is also not a major problem since I have the workaround which I described above.

 

- Steve Sheppard

[HiassofT]

Hi Steve!

 

Formatting has never worked for me with SDrive.

 

I have tried all of your recent beta firmwares. The red LED stays lit as does yellow #1 and SDrive locks up, ie. doesn't respond to the Atari nor to buttons BOOT, LEFT or RIGHT.

What kind of image did you try to format?

 

The SDrive zeroes out all data when formatting an image, which takes a few seconds for SD/ED/DD images. A 16MB image could take a few minutes then. During this process the red LED lights up.

 

Also with the latest firmware I noticed something odd. In the past for a large ATR (e.g 16MB) if I forget to use the 'O' option to set the disk parameters and do a format (with '/n') I would get a default 720 sectors. With the new firmware the default now seems to be 1040 formatted sectors. Doesn't matter if the ATR was created SD or DD. I never use ED, btw.

Strange. I'll have a look at this.

 

BTW: which MyDos version are you using?

 

IIRC MyDos 4.53 also reported 1040 sectors if I configured the drive (with 'O') as a standard drive, for example 2 sides 40 tracks or 1 side 80 tracks (QD). Configuring it as a high capacity drive and entering the number of sectors worked fine. I'll recheck this the next days.

 

The bigger problem, not being able to issue a format command, has me a little concerned since now I know SDrive was intended to have this ability and my SDrive won't allow it. However, this is also not a major problem since I have the workaround which I described above.

Could you try formatting a smaller image (DD or 1MB or so)?

 

so long,

 

Hias

[a8isa1]

Hi Steve!

 

Formatting has never worked for me with SDrive.

 

I have tried all of your recent beta firmwares. The red LED stays lit as does yellow #1 and SDrive locks up, ie. doesn't respond to the Atari nor to buttons BOOT, LEFT or RIGHT.

What kind of image did you try to format?

 

The SDrive zeroes out all data when formatting an image, which takes a few seconds for SD/ED/DD images. A 16MB image could take a few minutes then. During this process the red LED lights up.

 

Also with the latest firmware I noticed something odd. In the past for a large ATR (e.g 16MB) if I forget to use the 'O' option to set the disk parameters and do a format (with '/n') I would get a default 720 sectors. With the new firmware the default now seems to be 1040 formatted sectors. Doesn't matter if the ATR was created SD or DD. I never use ED, btw.

Strange. I'll have a look at this.

 

BTW: which MyDos version are you using?

 

IIRC MyDos 4.53 also reported 1040 sectors if I configured the drive (with 'O') as a standard drive, for example 2 sides 40 tracks or 1 side 80 tracks (QD). Configuring it as a high capacity drive and entering the number of sectors worked fine. I'll recheck this the next days.

 

The bigger problem, not being able to issue a format command, has me a little concerned since now I know SDrive was intended to have this ability and my SDrive won't allow it. However, this is also not a major problem since I have the workaround which I described above.

Could you try formatting a smaller image (DD or 1MB or so)?

 

so long,

 

Hias

 

I could be mistaken but I am fairly certain that MyDOS (4.53/4) has always defaulted to 720 sectors. I will confirm later with SIO2PC, SDrive, and 1.2 MB floppies drives where the image or media is actually larger than 720K

 

I'll try waiting longer on the formatting. Pretty sure I waited several minutes for standard DD and SD images but I will try again.

 

- Steve Sheppard

[a8isa1]

Using SDrive formatting of 1MB and 720 sector images were both successful.

 

Formatting of 16MB images still fails.

 

I waited a long time. After some minutes I get "Error -- 138". Both the red and yellow #1 LEDS remain lit. Buttons BOOT, LEFT, and RIGHT do not repsond. If I cold boot a directory listing of the image shows "0000 SECTORS FREE".

 

SIO2PC and real floppy testing will have to wait. Too much equipment to move around and too late at night to do that.

 

In an emulator formatting large images without performing MyDOS 'O' option yields 720 sectors formatted. With SDrive it's 1040.

 

- Steve Sheppard

[HiassofT]

I could be mistaken but I am fairly certain that MyDOS (4.53/4) has always defaulted to 720 sectors. I will confirm later with SIO2PC, SDrive, and 1.2 MB floppies drives where the image or media is actually larger than 720K

I just did some tests with the original SDrive firmware, formatting a large image with "/N" also results in a 1040 sectors (1028 free) image.

 

Then I had a look at the source and found the reason: The "get status" ($53) SIO command always sets the enhanced density bit (bit 7 of first byte) if the image has more than 720 sectors (regardless of SD or DD). I guess it would be better to change the code so that this flag is only set if the image contains 1040 SD sectors (I do it this way in AtariSIO).

 

Another strange thing: When formatting with "/N" MyDos sends a percom put command to the SDrive and tries to configure it to standard 720 sectors SD/DD, but the SDrive responds with an OK (I had suspected to get an error here). Internally, the SDrive just marks the received percom config as invalid so it won't allow to format the image with $21.

 

@Raster: is there a special reason that the "goto Send_ERR_and_Delay" is commented out and replaced with "goto Send_CMPL_and_Delay"?

 

I did another test with AtariSIO and 65535 sector, SD and DD images formatted with "/N" result in 720 sector images.

 

Yet another test: formatting a 1MB image (4096 DD sectors) took approx. 30 seconds with my SDrive. So formatting a 16MB image should take some 8 minutes. This is too long for the maximum allowed SIO timeout (approx. 255 seconds, the get status command of the SDrive returnes a recommended value of 240 seconds timeout for format command). So, currently the only possibility is to first format the drive, then wait for the timeout to occurr and the red LED to go out and then use "format /N" to write the directory/VTOC.

 

@Raster: is it possible to speed up the blanking process a little bit? I haven't looked at the particular source code, but 30 seconds for 1MB seems a bit long to me.

 

so long,

 

Hias

[raster/c.p.u.]

Hi.

Another strange thing: When formatting with "/N" MyDos sends a percom put command to the SDrive and tries to configure it to standard 720 sectors SD/DD, but the SDrive responds with an OK (I had suspected to get an error here). Internally, the SDrive just marks the received percom config as invalid so it won't allow to format the image with $21.

@Raster: is there a special reason that the "goto Send_ERR_and_Delay" is commented out and replaced with "goto Send_CMPL_and_Delay"?

We had an experiences that Percom Write command should be confirmed OK always and Error should appears not until following format command.

(Are we wrong?)

 

@Raster: is it possible to speed up the blanking process a little bit? I haven't looked at the particular source code, but 30 seconds for 1MB seems a bit long to me.

There is used one identical method for blanking as for writing ('faccess_offset').

Clear_atari_sector_buffer_256();

faccess_offset(FILE_ACCESS_WRITE,maz,(u16)psize);

IMHO the only way could be bypass the blanking in full, but I don't like such idea at all.

[HiassofT]

Hi Raster!

 

Another strange thing: When formatting with "/N" MyDos sends a percom put command to the SDrive and tries to configure it to standard 720 sectors SD/DD, but the SDrive responds with an OK (I had suspected to get an error here). Internally, the SDrive just marks the received percom config as invalid so it won't allow to format the image with $21.

@Raster: is there a special reason that the "goto Send_ERR_and_Delay" is commented out and replaced with "goto Send_CMPL_and_Delay"?

We had an experiences that Percom Write command should be confirmed OK always and Error should appears not until following format command.

(Are we wrong?)

No, sorry, I was mistaken. I just checked with my 1050 Speedy and 1050 Happy: They happily accept an invalid percom block (for example 32 tracks or 2 sides). So this is prefectly fine.

 

@Raster: is it possible to speed up the blanking process a little bit? I haven't looked at the particular source code, but 30 seconds for 1MB seems a bit long to me.

There is used one identical method for blanking as for writing ('faccess_offset').

Clear_atari_sector_buffer_256();

faccess_offset(FILE_ACCESS_WRITE,maz,(u16)psize);

IMHO the only way could be bypass the blanking in full, but I don't like such idea at all.

IMO the blanking must be kept, even if it causes troubles on images larger than ~8MB. Several programs (like sector copiers) expect that the disk is blanked after formatting, not blanking the image would be a severe bug (again, IMO).

 

I was thinking about if the blanking process could be accelerated a little bit. I just had a look at the code:

 

faccess_offset() calls mmcReadCached() before zeroing each sector. This call could be skipped, except for the first sector of ATR images (we need to keep the ATR header :-) and for the last sector. The sectors in between could be written directly. This would save some time and maybe we could stay below ~4 minutes.

 

I'll have a closer look at the code and see if it's possible to squeeze the additional code into the firmware.

 

so long,

 

Hias

[raster/c.p.u.]

faccess_offset() calls mmcReadCached() before zeroing each sector. This call could be skipped, except for the first sector of ATR images (we need to keep the ATR header :-) and for the last sector. The sectors in between could be written directly. This would save some time and maybe we could stay below ~4 minutes.

I'll have a closer look at the code and see if it's possible to squeeze the additional code into the firmware.

Please, take care of changes in main R/W operations. We really don't want to meet some disasters with destroyed data.

Also I'm a bit afraid of source code lucidity with some special exceptions in main methods...

[HiassofT]

Please, take care of changes in main R/W operations. We really don't want to meet some disasters with destroyed data.

Also I'm a bit afraid of source code lucidity with some special exceptions in main methods...

Yes, of course. I'll be very careful here. My plan was to keep the original faccess_offset() unchanged and implement a separate method to handle blanking full 512byte sectors - if there's enough space left in the flash ROM.

 

so long,

 

Hias