Writing a sector copier cart, need some info.

[tschak909]

The sector copier will try to read sector 4 from the target disk, and subsequently ask for a percom block, which should successfully detect the disk.

 

-Thom

[sanny]

On 10/25/2019 at 10:48 PM, flashjazzcat said:

I think if an application requests a raw sector from a quad-density volume without allocating sufficient memory for the data (having established disk density using the proper methods), it deserves everything it gets. What would be your proposed alternative?

 

Proposed alternative: honor the "buffer size" parameter. Why is it there if it's ignored?

One could want to read just the first 10 bytes of a sector or something....

[flashjazzcat]

22 minutes ago, sanny said:

Proposed alternative: honor the "buffer size" parameter. Why is it there if it's ignored?

One could want to read just the first 10 bytes of a sector or something....

Great question. I wonder, then, why every hard disk controller firmware I ever looked at completely ignores the buffer size and just returns a full sector of whatever size is stipulated by the density of the partition, and why no-one previously decided this kind of irresponsible software design represented a critical flaw which required a solution. Presumably (and I clutch at straws here to find plausible reasons for such negligence) no disk operating system or application which deals directly with logical sectors has yet needed to read only the first ten bytes of a hard disk sector into a buffer only ten bytes long on pain of crashing if said buffer overflows. I have no doubt that the advantages of being able to do this would far outweigh the benefits of being able to use simple, partially unrolled loops to transfer sector data (whether interleaved, padded, or a full 512 bytes) at the highest possible speed when loading programs and data via an FMS.

Edited October 29, 2019 by flashjazzcat
typo

[sanny]

Thanks for your divine insights, flashjazzcat...

 

I've been working before, not with hard disk controller firmwares, but with floppy controllers. And, surely, they don't get a "buffer size" argument when reading, just "buffer address" and "number of sectors" parameters.

 

And, yes, I was wrong. I've re-read the description in "Mapping the Atari", and the buffer size variable seems to be used only when writing and when returning the number of bytes read. So it's not an input argument. So there is in fact no "buffer size" argument when reading. This voids my comment.

[tschak909]

work on this is progressing, am currently debugging the UI. 

 

I have also written a first pass of sector to bank/address calculations (as well as determining how many sectors can be done in one pass), given the number of banks detected in system (and in relation to the detected source sector size)..

 

Basically, low-memory is used to store the first 40K or so of sectors. I won't use the OSRAM, as flipping the cartridge in and out will prove to be problematic, BUT, it WILL use the full complement of PORTB banked ram that is there, so if you have 1088K of RAM, you'll be able to do all floppy types in one pass.

 

-Thom

[flashjazzcat]

1 hour ago, tschak909 said:

as flipping the cartridge in and out will prove to be problematic

Can you elaborate?

[tschak909]

well, I mean, it's essentially an 8K cartridge, all the program code is there. would be pulling the rug out from under me.

 

-Thom

[flashjazzcat]

1 minute ago, tschak909 said:

well, I mean, it's essentially an 8K cartridge, all the program code is there. would be pulling the rug out from under me.

Ah right... I was puzzled since you mentioned the cartridge space in the context of 'OS RAM', which I assume to mean the shadow RAM under the OS.

 

You can use the 8K under the cartridge by running the IO and cartridge management code out of low RAM (copy it to $700 on initialisation, for example); this is how stuff like the SIDE loader disables and re-enables its own ROM. In the case of the copier, you'd effectively be toggling BASIC on and off using bit 1 of PORTB.

 

I can fully appreciate why using RAM under the OS wouldn't be desirable, however, if you wanted to use it as buffer space for SIO transfers. But you could get around the OS having to be off in order to access RAM under it by always reading sectors into main memory (with the OS on, obviously), then turning off the OS, copying the data into or out of the space under the OS ROM, then turning the OS ROM on again before transferring the next sector.

[HiassofT]

On 10/31/2019 at 3:04 AM, sanny said:

And, yes, I was wrong. I've re-read the description in "Mapping the Atari", and the buffer size variable seems to be used only when writing and when returning the number of bytes read. So it's not an input argument. So there is in fact no "buffer size" argument when reading. This voids my comment.

That's not correct.

 

For (serial) SIO communication DBYT determines the number of bytes to receive. After that SIO will receive the checksum byte and compare it to the calculated checksum. So if you set DBYTLO/HI too short you'll mostly get a checksum error (chances are 255:1) or if you set it too long you'll get a timeout.

 

The important thing is that SIO will never trample on memory beyond what you set in DBYT, if some PBI devices don't check DBYT and return an error if it doesn't match the phyiscal sector size their firmware is broken and needs fixing (it's a really simple check that doesn't cost much).

 

BTW: it's also perfectly vaild to issue a read sector command with DSTAT set to 0 (meaning an immediate SIO command), then SIO won't need a buffer at all. You can use that eg to read sector 4 to work around the XF551 density detection problems and then just wait a bit - this way you won't run into a checksum error (if you set DBYT to 128 and XF551 sends 256 bytes), and then have to wait a bit to recover or run into the SIO timeout (if you set DBYT to 256 but XF551 sends 128 bytes).

 

so long,

 

Hias

[flashjazzcat]

I can probably live with that, and in so doing be the only kid on the block to bother with such checks.  Returning an error is certainly preferable to being tasked with returning an arbitrary number of bytes as specified in the DCB.

 

This would prevent the only crash scenario I have witnessed resulting from a DBYT mismatch: attempting to read 512bps SDFS partitions from SpartaDOS 3.2 or some other DOS unaware of 512 byte sectors (or nonsensically boot said DOS from such a partition).

[Rybags]

Do any of the IDE type devices read directly to Ram at the low-level driver layer?  Or do they present a buffer to IO space e.g. $D500 which then gets copied from at a higher software level?

[flashjazzcat]

The IDE controller presents a data register which indexes the device's internal sector buffer. The driver transfers a full sector of data to and from the buffer address specified in the DCB. This may be a burst IO address or a DOS buffer.

[tschak909]

1 hour ago, HiassofT said:

That's not correct.

 

For (serial) SIO communication DBYT determines the number of bytes to receive. After that SIO will receive the checksum byte and compare it to the calculated checksum. So if you set DBYTLO/HI too short you'll mostly get a checksum error (chances are 255:1) or if you set it too long you'll get a timeout.

 

The important thing is that SIO will never trample on memory beyond what you set in DBYT, if some PBI devices don't check DBYT and return an error if it doesn't match the phyiscal sector size their firmware is broken and needs fixing (it's a really simple check that doesn't cost much).

 

BTW: it's also perfectly vaild to issue a read sector command with DSTAT set to 0 (meaning an immediate SIO command), then SIO won't need a buffer at all. You can use that eg to read sector 4 to work around the XF551 density detection problems and then just wait a bit - this way you won't run into a checksum error (if you set DBYT to 128 and XF551 sends 256 bytes), and then have to wait a bit to recover or run into the SIO timeout (if you set DBYT to 256 but XF551 sends 128 bytes).

 

so long,

 

Hias

Dude, seriously, THANK YOU. The fact that you can set dbyt=0 was the missing piece I needed.

 

I was trying to use sector 4 to check for density, and I was getting checksum errors.

 

-Thom

[flashjazzcat]

Ah yes: DBTY checking depends on explicit support for DSTAT=0 otherwise sector 4 tests which previously worked on DD partitions with no DBYT checks will fail with the size check in place.

[HiassofT]

Well, with DSTATS=0 you should simply ignore what's in DBYT (and DBUF) as there's no data to transmit ?

 

Not 100% sure what'll happen with DSTAT set to $40 or $80 and DBYT set to 0 - could well be that SIO expects a checksum for the zero-bytes block or goes completely crazy

 

so long,

 

Hias

Edited November 3, 2019 by HiassofT

[pirx]

On 10/25/2019 at 7:02 AM, tschak909 said:

Are there any other sector copiers out there that can handle the 512 byte sectors? 

Our TOMS Copy supports 512 sectors, but as this is supposed to work with IBM compatible disks I doubt it would work with Atari formatted 512 sectors. 

[flashjazzcat]

20 hours ago, HiassofT said:

Well, with DSTATS=0 you should simply ignore what's in DBYT (and DBUF) as there's no data to transmit

Of course: that's understood. I just mean that explicit support for DSTATS=0 is required if DBTY checking is not to cause breakage on its own (in the context of the sector 4 read test previously discussed).

 

Implementing both was an interesting exercise, not least because free bytes in the U1MB PBI BIOS ROM tend to be counted in single digits. Fortunately the internal 'density' variable corresponds to DBYTHI, and one need then only check for DBYTLO = $80 if DBYTHI = $00. Immediate mode (DSTATS = 0) is checked for last, and will return status = OK without transferring any data at all for any command providing other parameters (sector number, device, DUNIT, etc) did not themselves generate out-of-range errors.

	ldy Density	; get density MSB
	cpy dbythi
	bne LockedOrBadSector
	lda dbytlo	; this must be $80 for SD, and zero for everything else
	cpy #0
	bne Notsingle
	cmp #$80
	.byte $24	; skip 'tay'
Notsingle
	tay
	bne LockedOrBadSector

	ldy dstats
	bne @+
	iny	; say OK
	rts
@
	[transfer code]

Hopefully this will suffice... and hopefully it will not result in some bizarre corner-case breakage.  This will work on FAT-hosted ATRs as well as partitions and is not especially intrusive. The returned error code is somewhat arbitrary but it will do.  

Edited November 3, 2019 by flashjazzcat

[HiassofT]

54 minutes ago, flashjazzcat said:

Of course: that's understood. I just mean that explicit support for DSTATS=0 is required if DBTY checking is not to cause breakage on its own (in the context of the sector 4 read test previously discussed).

Those XF551 checks (with DSTATS=0) usually ignore the SIO error status, it's a simple "fire and forget" command to work around the XF551 firmware quirks. So don't worry too much about that.

 

Just checked my MyPicoDos code and in getdens.src I set DBYTLO/HI and DSTATS to 0, start the SIO call and then wait 20 frames (time enough for 512 bytes at 19k2) for the the sector data to be transmitted before proceeding.

 

I think it should be perfectly fine to implement some simple strict checking, eg check if DBYT matches the sector/status/percom length and if DSTATS is set right ($80 or $00 doesn't make much sense for "read" as does $40 or $80 for "write" commands) and if the parameters are odd return eg a $90 error.

 

I'd say it's also probably not worth spending too much time and thoughts about the actual error code numbers. most of the code out there doesn't use the error status for much more than at maximum some error text printout on the screen.

 

so long,

 

Hias

[flashjazzcat]

2 minutes ago, HiassofT said:

I think it should be perfectly fine to implement some simple strict checking, eg check if DBYT matches the sector/status/percom length and if DSTATS is set right ($80 or $00 doesn't make much sense for "read" as does $40 or $80 for "write" commands) and if the parameters are odd return eg a $90 error.

Yes: that was already done (obviously it would be pretty dangerous to allow a sector write when DSTATS = $40). I wasn't always well versed in writing mass storage drivers and learned a lot from studying the behaviours of other devices, such as the KMK/JZ interface and IDE Plus 2.0 (indeed anything by Drac030 seemed pretty exemplary, and we conversed a lot by email over the years in order to ensure that all APT firmwares were compatible to the last detail). It seemed reasonable to follow suit regarding the DBTY check (or lack thereof) as well, but I at least now see an opportunity to avoid crashing - say - MYDOS when inadvertently using it to pull a directory from a partition with 512 byte sectors. Immediate mode SIO (DSTATS = 0) was absolutely news to me, on the other hand.

9 minutes ago, HiassofT said:

I'd say it's also probably not worth spending too much time and thoughts about the actual error code numbers. most of the code out there doesn't use the error status for much more than at maximum some error text printout on the screen.

This makes sense. There are a couple of existing escape routes which return errors and are reached via branches, so exploiting the same code as an exit path for the new checks saves valuable bytes.

[flashjazzcat]

Quoting myself here, but spot the... erm... deliberate mistake:

        cpy #0
	bne Notsingle
	cmp #$80
	.byte $24	; skip 'tay'
Notsingle
	tay
	bne LockedOrBadSector

The BIT instruction ($24) is a useful method of implementing an unconditional branch over one (BIT zp) or two (BIT abs) subsequent bytes, providing care is taken not to access touch-sensitive hardware registers via the resulting operation. In the code shown above, when the branch to 'Notsingle' is not taken, a BIT $A8 instruction is executed (TAY becoming the operand of the BIT instruction), and then the BNE instruction is executed. The problem with this code is that BIT alters the Z flag as well as the N and V flags, so the state of the Z flag here depends not on the result of CMP #$80, but the result of a logical AND between the contents of $A8 and the accumulator.

 

Ironically enough, the corrected code does not use BIT at all and is simpler and (as a whole) shorter:

	lda #0
	ldy Density
	bne @+
	lda #$80
@
	cmp dbytlo
	bne ReturnNAK
	cpy dbythi
	bne ReturnNAK

The BIT method is most useful in scenarios like this:

Write
	lda #$30			; write
	.byte $2C
Read					; internal sector read routine
	lda #$20
rwsect
	sta rwcmd

The carry flag is unaffected by BIT, so we can do things like this:

Label1
	sec
	.byte $24
Label2
	clc
	ror Flag

Anyway: I thought it worth pointing out that even after years of employing such coding techniques, it's possible to screw up (well, here at least).

Edited November 9, 2019 by flashjazzcat
typos

[dmsc]

Hi!

On 11/9/2019 at 6:51 AM, flashjazzcat said:

 

The BIT method is most useful in scenarios like this:

Write
	lda #$30			; write
	.byte $2C
Read					; internal sector read routine
	lda #$20
rwsect
	sta rwcmd

The carry flag is unaffected by BIT, so we can do things like this:

Label1
	sec
	.byte $24
Label2
	clc
	ror Flag

I try to avoid using BIT for skip, as if the memory location is a hardware register you can modify hardware state and it modifies Z, N and V flags.

 

For the CLC case, it is faster to do this (not taken BCC is 2 cycles, BIT is 3 cycles):

label1:
	sec
	.byte	{bcc}	; $90
label2:
	clc

The trick of using not taken branches to skip one byte also works for some special cases of loading values, like:

label3:
	lda	#0
	.byte	{bne}	; $D0
label4:
	lda     #{nop}  ; $EA

Here the first case takes 6 cycles (LDA #0 \ BNE * \ NOP), this is the same as using BIT to skip two bytes. It works when you need to load a flag with zero or any non-zero value, and after the skip the Z flag value depends on the accumulator, unlike the BIT case.

 

The above works for loading various constants: $0A (ASL), $18 (CLC), $38 (SEC), $4A (LSR), $58 (CLI), $B8 (CLV), $D8 (CLD), $EA (NOP). And it works for LDX /LDY.

 

Have Fun!

 

[Rybags]

Nice use of the never taken branch but I don't think there's a non modifying instruction that can perform the 3 byte NOP ?  (ignoring illegals)

[flashjazzcat]

12 minutes ago, dmsc said:

I try to avoid using BIT for skip, as if the memory location is a hardware register you can modify hardware state and it modifies Z, N and V flags.

I mentioned the caveat regarding hardware registers, but it's rarely an issue in real world situations.

13 minutes ago, dmsc said:

For the CLC case, it is faster to do this (not taken BCC is 2 cycles, BIT is 3 cycles):

Interesting! The advantage of BIT is that it doesn't depend on any flags being in a known state, but in this scenario the branch idea looks good.  Probably a good way to introduce more bugs if not used carefully, of course.

18 minutes ago, dmsc said:

Here the first case takes 6 cycles (LDA #0 \ BNE * \ NOP), this is the same as using BIT to skip two bytes.

Cool!

 

In terms of BRK examples, I should probably have included this code (from Hias' high-speed SIO driver) which shows a cascading sequence:

ErrNak
	lda #ERR.NAK		; NAK or any other error: set code $8B
	.byte $2C		; BIT xxx (skip next 2 bytes)
ErrFrm	lda #ERR.Frame		; framing error
	.byte $2C		; BIT xxx (skip next 2 bytes)
ErrOvr	lda #ERR.Overrun	; data input overrun
	.byte $2C		; BIT xxx (skip next 2 bytes)
ErrTO	lda #ERR.Timeout	; timeout error

Error	; general error routine, code in A
	sta STATUS

Brevity matters much more than performance here.

[dmsc]

Hi!

25 minutes ago, flashjazzcat said:

I mentioned the caveat regarding hardware registers, but it's rarely an issue in real world situations.

In my mini-simulator I had to add special tracking of flags after BIT because this usage was interfering with the detection of uninitialized memory usage. Now it allows BIT to read from uninitialized memory but sets the Z, N and V flags as "uninitialized" and reports any instruction that uses those flags.

 

25 minutes ago, flashjazzcat said:

Interesting! The advantage of BIT is that it doesn't depend on any flags being in a known state, but in this scenario the branch idea looks good.  Probably a good way to introduce more bugs if not used carefully, of course.

Cool!

 

In terms of BRK examples, I should probably have included this code (from Hias' high-speed SIO driver) which shows a cascading sequence:

ErrNak
	lda #ERR.NAK		; NAK or any other error: set code $8B
	.byte $2C		; BIT xxx (skip next 2 bytes)
ErrFrm	lda #ERR.Frame		; framing error
	.byte $2C		; BIT xxx (skip next 2 bytes)
ErrOvr	lda #ERR.Overrun	; data input overrun
	.byte $2C		; BIT xxx (skip next 2 bytes)
ErrTO	lda #ERR.Timeout	; timeout error

Error	; general error routine, code in A
	sta STATUS

Brevity matters much more than performance here.

Yes, that is a good use case, as the error path does not need to be fast. You need to remember to not use ERR values from $D0 to $D7, specially the $D5 page that can cause page flips on some cartridges at read  

 

Have Fun!

[flashjazzcat]

26 minutes ago, dmsc said:

You need to remember to not use ERR values from $D0 to $D7, specially the $D5 page that can cause page flips on some cartridges at read

Absolutely, but I never yet hit a situation where the opportunity to use BRK coincided with an operand which would cause memory access in the IO region (although of course such cases are bound to exist). I can well imagine that the technique can cause problems with debuggers and simulators.

 

I'm not surprised you have a lot of cycle and byte-saving techniques up your sleeve, if FastBASIC is anything to go by.