Jump to content
IGNORED

Horizon RAMdisk ROS and CFG Development


Recommended Posts

FYI, I've had one report of CFG834 "squealing" during the peripheral test. As I don't recall any major changes between 8.34 and what was posted here, I'll wait for more feedback before a code review. (resolved: bad file transfer to the TI hardware).

 

Update: I used DSKU to compare CFG832e (July 31 2015) with CFG834. The only differences are the version text and "Axiom" vs. Axion".

 

Update: Added task to add a CRC self-test to CFG, to help protect against corruption.

Edited by InsaneMultitasker
  • Like 1
Link to comment
Share on other sites

Post #200 edited to reflect modified ROS834 (padded file from 8188 to 8192 bytes).

 

DSR problem identified in this thread:

 

http://atariage.com/forums/topic/256351-hrd-setup-questions-mame-specificmaybe/page-2?do=findComment&comment=3583501

 

Edit: noticed ROS CRC detection did not fail when file was padded to 8192 bytes. Routine is presently hard-coded to 8188 bytes based on the source code. Will review change to routine to account for final 4 bytes or dynamic file size.

 

File CFG-S1

 

       CI   R0,>1000+8188     end of ROS?
       JNE  CRC11             no, read next byte and calculate
Edited by InsaneMultitasker
  • Like 2
Link to comment
Share on other sites

  • 6 months later...

Came across this bug today. Both Atrax and I have confirmed that in some cases, you can write beyond the currently allocated sectors, corrupting other drives in the process.

 

When ROS looks for free sectors to write to a file, it does so based on the bitmap in sector 0. When it scans the bitmap, it compares the scanned (current) allocation unit number with the total sectors of the disk. With a 3200 sector disk, the total sectors must be divided by 2, to ensure each allocation unit represents two sectors. In the current code the division never takes place, so ROS thinks there are up to 6400 sectors to scan.

 

The offending routine is listed below. I'll fix this as time permits.

 

* FIND FIRST FREE SECTOR      OUT: R1= Sector number (ZERO if none)
*
FNDSEC LI   R6,>5838          Point to the start of the BITMAP
       MOV  @OFSET,R7         Page # of the first page
       CB   @MASK+1,@LINK+1   8 bit or 16 bit BUS
       JNE  FNDSED            Jump if 16 bit
       SWPB R7                8 bit, so swap it
FNDSED LDCR R7,0              Write the page # to the card
       CLR  R1                Start searching at sector 0
       JMP  NXWRD             Jump and get the first WORD
NXCHK  DEC  R2                Finished with this WORD
       JNE  NXSEC             Nope, so shift some more
NXWRD  MOV  *R6+,R0           Get another WORD of the BITMAP
       SWPB R0                Put it in the right order
       LI   R2,16             16 sectors per WORD
NXSEC  SRL  R0,1              Shift 1 sector out
       JNC  FDSEC             If no carry, then it's free
       INC  R1                Not free, so check next sector
       C    R1,@FORSEC        Are we at the end of the drive (BUG! Must /2  IFF disk total allocation is greater than 1600 sectors, i.e., 2 AUs/sector)
       JLT  NXCHK             Nope, so check next sector
       CLR  R1                Yep, so send >0000 back
***2-19-2012tt    FDSEC  RT               RETURN TO THE CALLING PROGRAM
* Test for >1600 and return proper sector#
FDSEC  C    @FORSEC,@D1600    2-19, disk size>1600?
       JLE  FDRT11            no
       SLA  R1,1              yes, adjust x2
FDRT11 RT
  • Like 5
Link to comment
Share on other sites

  • 1 month later...

While testing the allocation bug fix I ran into another odd scenario.

 

When writing to a physical floppy disk on a Myarc FDC using level 2 IO, if there is insufficient space for the file, a remnant is left behind that consumes the remaining space. The same things happens when saving a program to disk using XB and the level 3 SAVE opcode. I haven't tested the same processes with a TI or CorComp disk controller, but I assume the file stub is written or the remnant is deleted.

 

When writing to a Horizon RAMdisk, the filename and its pointer are written as expected. The filename shows up in a disk catalog. However, the DSR does not write an FDR cluster (chains of 3 bytes which point to the used sectors) for the first group of allocated sectors, leaving the cluster value as >00,>00,>00. This has the unfortunate side effect of redirecting the file to the disk's VIB, sector 0. As many of us well know, a destroyed sector 0 means the disk contents will not be accessible.

 

I need to complete some additional testing and compare controller responses before I (try to?) fix this issue. As far as I can tell, the bug has been lurking in the ROS since the 90s so a while longer won't matter too much. ;) The offending code snip can be found below.

 

 

 

* CREATE DATA CLUSTERS
*
CRE BL @ADDRF Bring up the FDB sector
AI R6,28 Point to the first cluster
MOV R6,R10 Save the current cluster addr.
CLR R8 Flag = Looking for a Starting Sector
LI R1,2 Start the search at the first sector
CLR R2 No offset at the moment
CREATE MOV R1,R0 Setup the UPDATE with sector number
BL @UPDATE Calc position in the BIT MAP
MOVB *R6,R0 Get the BIT MAP byte for the compar
CZC R7,R0 Is this sector free
JEQ CRE0 Yep, so count it
MOV R8,R8 Nope, so check if looking for SS
JEQ CRE4 Yep, so skip over cluster maker
CLFIN BL @ADDRF Bring up the FDB sector
A R1,R2 Add current sector # to current offset
S R8,R2 Sub starting sector # from current offset
MOV R2,R3 Save current offset into R3
MOV R9,R9 Check if TOTAL AU left is ZERO
JEQ CNXT Yep, so skip over the DEC R3
DEC R3 Nope, so sub 1 from current offset
CNXT BL @BDCLST Build a cluster up in the FDB sector
CLR R8 Start looking for a new SS
MOV R9,R9 Check if TOTAL AU left is ZERO
JEQ CRE4A Yep, then stop building the clusters
S R6,R10 Make R10 cluster offset into the sector
CI R10,255 Check if out of room for another cluster
JGT CRE4B Yep, so exit to diskette full ERROR
A R6,R10 Add starting address into 2K RAM page
CRE4 INC R1 Point to the next sector on the diskette
C R1,@FORSEC Check if at the end of the diskette
JHE CRE4B Yep, so exit to diskette full ERROR [5.7.2017 first cluster is never written if disk fills up]
JMP CREATE Jump back up and check this sector
CRE0 MOV R8,R8 Sector is FREE, check if looking for SS
JNE CRE3 Nope, so don't change R8
MOV R1,R8 Yep, so make R8 the new SS sector number
CRE3 DEC R9 Check if finished building the clusters
JEQ CLFIN Yep, so write the last cluster
JMP CRE4 Nope, so look for another free sector

 

 

 

Edited by InsaneMultitasker
  • Like 2
Link to comment
Share on other sites

  • 1 year later...

Reviewed ROS DSR with respect to recent tipi discussions concerning duplicate device names/subprograms. No concerns identified. Two reminders:

 

1. Figure out if one or both of the prior two allocation bugs were fixed properly so that I can release the next ROS/CFG version

  • [Edit 8/26: unreleased ROS v8.38 contains the fix for post #203]
  • [Edit 8/26: fix for post #205 does not yet exist]

2. Now that the PEB TIPI cards are escaping ;) from the PNW, test/confirm CFG tipi detection is working as expected.

3. Validate ROS CRC per earlier post related to TI controller bug / 8188 vs. 8192 bytes.

Edited by InsaneMultitasker
  • Like 3
Link to comment
Share on other sites

  • 4 weeks later...
  • 4 weeks later...

I still haven't figured out the proper solution to deal with a full disk and the cluster creation that ultimately corrupts the VIB. I need to map out the entire routine when I have a larger chunk of free time. My initial attempts at a fix had unintended 'consequences' for successfully created files. There is also a bitmap wrap-around process for when the end of the bitmap is reached; I am not sure how to ensure the fix doesn't stomp all over this routine.

 

I did come across an explanation for some of the garbage catalogs we all see when accessing an undefined drive. You may recall from past TIPI discussions that if a call to the DSR results in a device not being found, the DSR should gracefully return so that the DSRLNK can continue scanning other peripherals. Fortunately, ROS follows this convention as best I can tell.

 

However, when I inspected a few programs using sector-based catalog routines, I noticed that instead of testing for the condition status (EQ) bit after the DSRLNK routine is called, they only inspect 0x8350 for an error byte. If a level 2 unit/device number is undefined within the DSRs and 0x8350 is set to 0x0000, there is a good chance the program will think a device was found. Address 0x8350 serves a dual purpose for direct sector IO namely, it contains the sector to read (or write). When the program tries to read sector 0 to capture the VIB then fails to check the EQ status bit, it fools itself into thinking there was a valid sector 0. Now, whatever is in memory at the time will be used for the subsequent attempts to find the FDRs for the files and the catalog routine will fail miserably. I don't think there is any way around this specific problem from a DSR perspective.

  • Like 1
Link to comment
Share on other sites

  • 1 year later...

 

On 5/7/2017 at 7:03 PM, InsaneMultitasker said:

While testing the allocation bug fix I ran into another odd scenario.

 

When writing to a physical floppy disk on a Myarc FDC using level 2 IO, if there is insufficient space for the file, a remnant is left behind that consumes the remaining space. The same things happens when saving a program to disk using XB and the level 3 SAVE opcode. I haven't tested the same processes with a TI or CorComp disk controller, but I assume the file stub is written or the remnant is deleted.

 

When writing to a Horizon RAMdisk, the filename and its pointer are written as expected. The filename shows up in a disk catalog. However, the DSR does not write an FDR cluster (chains of 3 bytes which point to the used sectors) for the first group of allocated sectors, leaving the cluster value as >00,>00,>00. This has the unfortunate side effect of redirecting the file to the disk's VIB, sector 0. As many of us well know, a destroyed sector 0 means the disk contents will not be accessible.

 

I need to complete some additional testing and compare controller responses before I (try to?) fix this issue. As far as I can tell, the bug has been lurking in the ROS since the 90s so a while longer won't matter too much. ;) The offending code snip can be found below.

 

Returned to this old 'bone' tonight.  The two most viable solutions at this point: 

 

(1) check the total available AUs during level 2 file creation and inhibit creation of the file/FDR/FDB when too few sectors are available. One challenge here is dealing with the two steps in the file output process, where the FDB is created without allocating any clusters.

(2) write the partial cluster then generate an error condition. The FDB would need to be updated to reflect the partial write and also leaves a remnant behind.  This seems like a recipe for corrupt files, even if it is how other controllers handle it.

 

Next step is to set up my RAMdisk and test the solutions over the holidays.

  • Like 1
  • Thanks 2
Link to comment
Share on other sites

  • 1 month later...

The holidays are approaching quickly!  I transferred the ROS 8.38 files to my PC so that I could assemble and link the ROS via Ralph's XDT99 package.  I also printed the source so that I can mark it up and write notes in the margins - I find that process relaxing and a bit nostalgic ;)   For fun this week I worked through how the Geneve EPROM and MDOS calculate the proper 2k rack CRU address and offset based on the requested sector number; it isn't a linear calculation, instead if hops around a 6k segment which may be an artifact (or workaround) related to how the original ramdisk accessed the 8k SRAM space. Alas, the historical knowledge is challenging to reconstruct after 30+ years.

  • Like 3
Link to comment
Share on other sites

Today I spent some time playing around with file IO and ROS.  When I attempt to copy a 10 sector file to a ramdisk with fewer than 10 sectors free, an FDR is created with no clusters allocated resulting in an 'empty' file.  If you later write to this file in append mode, sectors are allocated for the new records until all available disk space is filled.   Saving a program via XB will also create a similar FDR with an incorrect program image size.  In the cases where only the FDR has been created, editing the file (e.g., file edit via DSKU) points the user to sector 0 however, I haven't come across any programs (so far) that will corrupt sector 0 as a result.   

 

I'm not keen on the Myarc approach of leaving a partial file on disk, though I do understand the DSR reasons for doing so.  I think the better approach would be for the ROS file SAVE and Direct Output (>15) routines to check available space and if not found, immediately return an error condition before creating the FDR and allocating sectors. 

 

More research is required to determine if a fix is feasible and/or necessary.  If anyone has reproducible file corruption scenarios to test while I'm looking into this, please post them in this topic.

  • Like 4
Link to comment
Share on other sites

  • 2 weeks later...
  • 2 weeks later...

Taking advantage of the 'special' ability of the RS232 to receive and transmit without actually enabling the DSR ROM I added a few RS232 output debug routines to ROS.  The primary routine to find a free sector in the bitmap (for FDRs and record IO) seems to be working properly.  However, it seems that one of the calling routines is not handling the "no free sectors" response properly or does not have the right logic to handle edge cases.    Here's a sample output of ramdisk powerup and file copy (the FDR was allocated on sector >8a).  Also of interest is that the expected routine wasn't called to allocate sectors for the file data, so I need to do a bit more digging into the bitmap and cluster  allocation routines.

 

image.thumb.png.885a615baba2cd1032f88b5982abdb31.png

  • Like 2
Link to comment
Share on other sites

There are two main allocation routes where ROS does not handle the edge cases very well, resulting in corrupting the file and/or corrupting the drive AND next adjacent drive; the problem occurs  if/when the file in question consumes the remaining disk space and tries to write beyond the disk's capacity.  I also verified a problem exists with severely fractured file de-allocation.

 

1. Appending to an existing file:

Within the routine ADDSEC (adds a sector to a file) the DSR either adds a sector to an existing cluster or creates a new cluster.  When adding a sector to an existing cluster, ROS did not check for the end of disk, and merrily added sectors from the adjacent disk, which in turn caused the bitmap allocation routine to fail. Adding a single comparison seems to have fixed this problem.  Variable files halt as soon as disk space is exhausted and the last possible record is written to the last sector.  Fixed files in Relative mode halt with an error in the same way; however, when writing to a record that exceeds available space, the file will grow to maximum available disk space before ROS generates the error.  (I will test other cases when time permits).

 

2. Writing a new file or overwriting an existing file via SAVE opcode or level 2 >15 routine:

I have identified where to fix the allocation error but it will require two routines and is a bit more complicated.  For new files, a routine is needed that will scan the bitmap for available sectors then compare the requested sector size to available disk space.  For existing files, a routine is needed that removes the existing file - FDB and sectors - before testing disk space and re-creating the file.  I do not intend to keep the partial file on disk, opting instead to inhibit its creation.

 

3. Severely fractured files

If two files are interleaved until their respective FDR clusters are exhausted, deleting one of the files results in a corrupted or cleared drive.  I'm guessing there is yet another edge case here, whereby the cluster allocation/de-allocation routine fails when the FDR has allocated every cluster  pointer.

 

ROS v8.41:

I've attached ROS841 for anyone who feels a bit adventurous and wishes to play around.  Load with CFG834 or CFG838.  Theoretically, if you were already using ROS834 or ROS838 in your ramdisk, you can save the drive info versus re-creating your drives.  (I've updated from ROS838 without any issues to date)

 

ROS841.zip

  • Like 2
  • Thanks 1
Link to comment
Share on other sites

1 hour ago, InsaneMultitasker said:

 

 

2. Writing a new file or overwriting an existing file via SAVE opcode or level 2 >15 routine:

I have identified where to fix the allocation error but it will require two routines and is a bit more complicated.  For new files, a routine is needed that will scan the bitmap for available sectors then compare the requested sector size to available disk space.  For existing files, a routine is needed that removes the existing file - FDB and sectors - before testing disk space and re-creating the file.  I do not intend to keep the partial file on disk, opting instead to inhibit its creation.

 

 

ROS841.zip 5.31 kB · 2 downloads

Got a thought here.  If a file exists, and you remove it, and then there is no space for the file, then one could conceivably lose everything thinking the original would still be on the device.

 

So, what about identifying the size of the existing file and adding the free space, to confirm there is enough room before deleting and saving.

 

My 2 cents.

  • Like 1
Link to comment
Share on other sites

1 hour ago, BeeryMiller said:

Got a thought here.  If a file exists, and you remove it, and then there is no space for the file, then one could conceivably lose everything thinking the original would still be on the device.

 

So, what about identifying the size of the existing file and adding the free space, to confirm there is enough room before deleting and saving.

 

My 2 cents.

Good thought.  The existing SAVE and level 2 IO opcodes de-allocate the sectors prior to over-writing the file.  If the writing process then fails due to not enough space, remnants remain and/or corruption occurs.  I will go back and revisit how the disk controllers handle this and also assess the ROS source with your idea in mind.  I surmise we would do this by calling the file status routine to obtain the allocated sectors, add the computed free bitmap sectors, then compare the sum to the new requested file size. If the new file size is greater than the sum, ROS would kick out an error while leaving the 'original' file behind.  Your approach might make the fix easier to implement (and safer) since I wouldn't have to muck around with deleting a file.  I've added your idea to my working file.  :)   (This would only work with SAVE and level 2 IO; record-based IO is handled by record and consumes space on a sector-by-sector basis as the file grows).

 

Link to comment
Share on other sites

Hi,

sorry but it´s a about a year ago that I used that, so how do I start the prog ?

I´ve tried all possible options on the E/A-cartridge (I think), but no gain

thx

 

PS The TI-FILEname is ROS840, not ROS841, correct ?

Edited by Schmitzi
Link to comment
Share on other sites

3 minutes ago, InsaneMultitasker said:

Hi Schmitzi,

The filename in the ZIP should be ROS841.   You cannot load it directly; it must be loaded from with the CFG config program.  The CFG program is the one that shows you which devices are in your system, then allows you to load a ROS, format your drives, and set up your options. 

 

ah OK, so where from or which CFG can I use ?

Just add the new file to the ROS814F.dsk ?

BTW, the TI-Filename is shown as "840" in TI99DIR

 

image.png

Edited by Schmitzi
Link to comment
Share on other sites

2 minutes ago, Schmitzi said:

 

ah OK, so where from or which CFG can I use ?

Just add the new file to the ROS814F.dsk ?

BTW, the TI-Filename is shown as "840" in TI99DIR

 

CFG838 is the best option. I believe it is in this thread somewhere.  I am not sure how to post it into the first message of the topic.  If "840" is in TI99DIR, that's fine.  I don't bother looking at that name very often, sorry.  When loaded into CFG it should display 8.41 and 1-12-2020 as the date, if I recall correctly.  (Sorry it is not a release package and thanks for giving it a try)

 

If you load ROS 8.32 or higher into a ramdisk formatted with 8.14F or less, it is recommended to reformat the disks.

 

These newer versions provide disk sizes of up to 3200 sectors.  This was the primary reason for my update years ago; now I am trying to fix bugs that have existed for 30 years.  :)

Link to comment
Share on other sites

1 minute ago, InsaneMultitasker said:

CFG838 is the best option. I believe it is in this thread somewhere.  I am not sure how to post it into the first message of the topic.  If "840" is in TI99DIR, that's fine.  I don't bother looking at that name very often, sorry.  When loaded into CFG it should display 8.41 and 1-12-2020 as the date, if I recall correctly.  (Sorry it is not a release package and thanks for giving it a try)

 

If you load ROS 8.32 or higher into a ramdisk formatted with 8.14F or less, it is recommended to reformat the disks.

 

These newer versions provide disk sizes of up to 3200 sectors.  This was the primary reason for my update years ago; now I am trying to fix bugs that have existed for 30 years.  :)

 

ah Ok, thanks. To edit Post #1 just send an Message to Albert and he makes you "Admin" of this post, so you can edt it infinite

 

Have to search for 8.38 now

 

Is there an RAMDISK-Option in Classic99 to do some testings, once loaded ? I cannot see how to configre it

and my real system is down at the moment

 

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...