Jump to content
IGNORED

Geneve OS development discussion


Recommended Posts

Minor update: I have amended the Horizon Ramdisk Git wiki with references to GENCFG and the MDOS 7.30 release.  Later this year I will upload the source to GENCFG to the repository.

 

Geneve · horizonramdisk/Horizon-Ramdisk-ti994a Wiki (github.com)

 

If someone has the time and willingness to do so, the Ninerpedia article referencing Geneve OS 6.50/FORM needs to be updated to reflect 7.30 and GENCFG.  I suggest that we retain the instructions for <=6.50 , perhaps as a separate page.    Booting MDOS from Horizon Ramdisk - Ninerpedia   

 

(Post #1 has been cleaned up a bit to reflect the 7.30 OS release and tentative next steps)

  • Like 5
Link to comment
Share on other sites

  • 5 weeks later...

It feels like the release was so long ago.   I've been digging into a few cleanup items that have been on my list for a while.

 

@9640News - I added some more RS232 debugging to the SCSI routines and noticed that when a warm restart is initiated, the batch file processor seems to be looking for a file whenever the ampersand "&" is used alone.  (Using the ampersand like this performs a simple reset with no autoexec).  I suspect the batch file processor is trying to process the directory, ie., if you are in "SCS1.FOLDER1." it is looking for "SCSI1.FOLDER1" as a batch.  If you have time to look into this let me know.

 

Secondly, the SCSI low level routines are in need of further rework.  Each device's parameter array needs to be invalidated whenever SCSMAP changes the ID, this will stop the OS from accessing the device with the previous parameters (this was on my radar) and there is at least one sector parameter read that if still active, could under  certain circumstances could randomly read or write a sector in place of the actual sector 0. 

 

I worked through some of the HFDC low level code to track down a lockup condition that mizapf was experiencing in March.  Information has been sent to him for review.

 

Lastly, there are a few enhancements like fixing the SCSI ID activation routine that starts ID 0 during every IO call; they need to be documented and added to the next release. 

 

This week will be busy for me so I don't expect to do any more programming until the weekend at the earliest.

  • Like 3
Link to comment
Share on other sites

For the curious, the spoiler contains a sample low level trace of my SCS2 device with the DSR pulling a DIRectory; there are 19 files and 22 directories in the root.  I restarted the OS with an ampersand (on a non-SCSI device).

 

The OS sends the info to the RS232 at 38.4K where I monitor with hyperterminal.

 

 

 


FDIR - File Descriptor Index Record
DDR - Directory Descriptor Record
DDID - Directory Descriptor Index Record
FDR - File Descriptor Record 
VIB - Volume Information Block (sector 0)
-----------------------------------
DSEL: 0002  READ: 0000 0000  Parameter (direct)  - First selection of SCSI ID 2 after warm restart; read VIB parameters
AU:   READ: 0000 0040    - read the FDIR to determine file count/file pointer list

Direct:   READ: 0000 0001  - Read all 31 sectors of the bitmap to calculate free/available space
Direct:   READ: 0000 0002
Direct:   READ: 0000 0003  (cached)  - internal cache; each SCSI physical sector is 512 bytes; TI is 256 bytes, so we use a rudimentary cache to save time
Direct:   READ: 0000 0004
Direct:   READ: 0000 0005  (cached)
Direct:   READ: 0000 0006
Direct:   READ: 0000 0007  (cached)
Direct:   READ: 0000 0008
Direct:   READ: 0000 0009  (cached)
Direct:   READ: 0000 000A
Direct:   READ: 0000 000B  (cached)
Direct:   READ: 0000 000C
Direct:   READ: 0000 000D  (cached)
Direct:   READ: 0000 000E
Direct:   READ: 0000 000F  (cached)
Direct:   READ: 0000 0010
Direct:   READ: 0000 0011  (cached)
Direct:   READ: 0000 0012
Direct:   READ: 0000 0013  (cached)
Direct:   READ: 0000 0014
Direct:   READ: 0000 0015  (cached)
Direct:   READ: 0000 0016
Direct:   READ: 0000 0017  (cached)
Direct:   READ: 0000 0018
Direct:   READ: 0000 0019  (cached)
Direct:   READ: 0000 001A
Direct:   READ: 0000 001B  (cached)
Direct:   READ: 0000 001C
Direct:   READ: 0000 001D  (cached)
Direct:   READ: 0000 001E  - Final bitmap sector >1F wasn't read?  Max AU boundary error? That's new...

 

Hmmm.. mental note, try cataloging the hard drive without used/free; 31 sectors is a lot of overhead.

 

Direct:   READ: 0000 0000 -  Read sector 0 again
AU:   READ: 0000 0040  - Read FDIR again
AU:   READ: 0000 17D0   - Grab info [FDR] from first file in FDIR 
AU:   READ: 0000 17D0  (cached)  - Not sure why we re-read the first file info, bug?  Inspect
AU:   READ: 0000 4608
AU:   READ: 0000 4610
AU:   READ: 0000 19C8
AU:   READ: 0000 19D0
AU:   READ: 0000 1A10
AU:   READ: 0000 1A18
AU:   READ: 0000 1A20
AU:   READ: 0000 1A28 - still more files
AU:   READ: 0000 1A30
AU:   READ: 0000 17D8
AU:   READ: 0000 17E0
AU:   READ: 0000 47A8
AU:   READ: 0000 1A38
AU:   READ: 0000 1A40
AU:   READ: 0000 1A48
AU:   READ: 0000 1A50
AU:   READ: 0000 4658
AU:   READ: 0000 4650 - Last file pointed to by FDIR sector 40. 
AU:   READ: 0000 2FE8 - First Directory entry, as pointed to by VIB sector 0
AU:   READ: 0000 2FE8  (cached)
AU:   READ: 0000 15A0
AU:   READ: 0000 3EE0
AU:   READ: 0000 0230
AU:   READ: 0000 4618
AU:   READ: 0000 08B0
AU:   READ: 0000 1970
AU:   READ: 0000 1C98
AU:   READ: 0000 1CA8 - more directories
AU:   READ: 0000 0720
AU:   READ: 0000 1980
AU:   READ: 0000 4670
AU:   READ: 0000 17E8
AU:   READ: 0000 1B28
AU:   READ: 0000 1990
AU:   READ: 0000 1950
AU:   READ: 0000 0048
AU:   READ: 0000 1940
AU:   READ: 0000 17A8
AU:   READ: 0000 2320
AU:   READ: 0000 1960
AU:   READ: 0000 0220 - Last directory entry
 

 

  • Like 1
Link to comment
Share on other sites

8 hours ago, InsaneMultitasker said:

It feels like the release was so long ago.   I've been digging into a few cleanup items that have been on my list for a while.

 

@9640News - I added some more RS232 debugging to the SCSI routines and noticed that when a warm restart is initiated, the batch file processor seems to be looking for a file whenever the ampersand "&" is used alone.  (Using the ampersand like this performs a simple reset with no autoexec).  I suspect the batch file processor is trying to process the directory, ie., if you are in "SCS1.FOLDER1." it is looking for "SCSI1.FOLDER1" as a batch.  If you have time to look into this let me know.

 

 

I will look into it.  I should have some time this week as I am on 8 hour days instead of 10, and, it will be rainy weather this week.

  • Like 2
Link to comment
Share on other sites

16 hours ago, 9640News said:

I will look into it.  I should have some time this week as I am on 8 hour days instead of 10, and, it will be rainy weather this week.

Sounds good. 

 

For reference, the TIPI DSR debug log shows this if I type "&" when at the C:\ prompt, which is assigned to "TIP1."

PBin : 0014 0000 8400 0000 0050 0000 0050 0005 TIPI.
PBout: 0014 4000 8400 0000 0050 0000 0050 0005     TIPERR: 4000

 

Same thing happens with subdirectory:

PBin : 0014 0000 8400 0000 0050 0000 0050 0008 TIPI.FF.
PBout: 0014 E000 8400 0000 0050 0000 0050 0008     TIPERR: 0000

 

As noted, the same thing is happening for other devices based on my earlier log review , thus if  "&" is used without a filename, that needs to be captured and ignored.

 

Lastly, when I ran "TEST HDOS" to execute the newly assembled OS, the log unexpectedly showed TIPI access to DSK1.  Took me a few minutes to recall that I updated the OS powerup to map drives 1-4 to TIPI "disk" if TIPI is found with no floppy controller in the system ;)   So this log is a good entry, even if I forgot about the feature. 

PBin : 0014 0000 8400 0000 0050 0000 0050 000D DSK1.AUTOEXEC
PBout: 0014 E000 8400 0000 0050 0000 0050 000D     TIPERR: 0000

 

 

  • Like 1
Link to comment
Share on other sites

On 5/31/2021 at 9:11 PM, InsaneMultitasker said:

and there is at least one sector parameter read that if still active, could under  certain circumstances could randomly read or write a sector in place of the actual sector 0

I had some free time tonight so I commented out the 'bad' sector parameter read code and replaced it with an error trap. So far, I have not found any operations using the code.  I began to untangle more of the C-generated code and have all but confirmed that the old write cache code is inhibited by changes I made in 2009.  This is very good; when I noticed the cache code I was very concerned it could be active and susceptible to the same issue we saw in the read cache.  This month I will continue to research and clean up the DSR code.  The SCSMAP invalidation routine is 'penciled in' and the MAKE file dependency has been updated for when that can be implemented, probably after we confirm whether or not the SCSI device table can be expanded for future use.

  • Like 4
Link to comment
Share on other sites

I believe the fix for the & entry can be fixed in CLI\HCLIS.  There is an error return if the file is not found which would be the case if it was a device/path entry only without filename.  I haven't had time to test, but my thoughts were to read the length byte of the path, and then compare the last character in the PAB to a "." and if it is the period, then bypass the check.  That would avoid the device read.  I need to look over the register usage prior to the XOP call to make sure the test doesn't corrupt a needed register.

 

 

  • Like 1
Link to comment
Share on other sites

I added some debug calls to the write sector routines in preparation for the next round of cleanup and discovered that the original write cache is still active for some IO.  Surprise!  While I think the probability of corruption is very low (the read cache was more susceptible), it still requires review and probably the addition of cache invalidation to mimic the read routine.  I prefer to retain a cache for performance reasons. (The SCSI physical sector is 512 bytes; the TI sector is 256 bytes, thus a read-before-write is required.)

 

AU:   READ: 0000 3598
AU:   WRITE: 0002 B660
AU:   WRITE: 0002 B661  (cached)
*** WARNING! Unexpected WRITE cache!

AU:   WRITE: 0002 B662
AU:   WRITE: 0002 B663  (cached)
*** WARNING! Unexpected WRITE cache!

AU:   WRITE: 0002 B664
AU:   WRITE: 0002 B665  (cached)
*** WARNING! Unexpected WRITE cache!

 

Link to comment
Share on other sites

23 hours ago, 9640News said:

I believe the fix for the & entry can be fixed in CLI\HCLIS.  There is an error return if the file is not found which would be the case if it was a device/path entry only without filename.  I haven't had time to test, but my thoughts were to read the length byte of the path, and then compare the last character in the PAB to a "." and if it is the period, then bypass the check.  That would avoid the device read.  I need to look over the register usage prior to the XOP call to make sure the test doesn't corrupt a needed register.

 

Well, my test code last night did not work.  Need to do some more chasing of the code.

  • Like 1
Link to comment
Share on other sites

For those following this thread, this is part of what I sent the @InsaneMultitasker on a request he had made.

 

Been reviewing the code where a user would type the ampersand (&) at the CLI prompt.

 

Two things happen here.  The CLI code essentially does a CLR @>F110 and does a BLWP @0.  It is during this point, all the XOP's run their powerups again.  In L8\DSRPASS-S, there is this note for the DSR:

 

; AORG >2000:
;      Pointers to parameters (how are they used?)
; XOP->Powerup for DSR. Reset rs232 redirect. Clears SCSI and DSR buffers.
;      Branches to >6000 (HD.FLPPRM) to powerup HFDC/read DIPs/set emulate
;      Configures Floppy controller, clears spooler, init RS232 & buffers
;      Returns via PWRUP7

 

So, regardless, the HFDC light and drive is going to flash and it looks like it checks to see if there is an emulate file if a HFDC exists in the system.  I'm not of the opinion we should go messing with this code.

 

Next, there is a flag set (SETO @NEWAUT) when the ampersand is originally entered at the CLI in the CLI\HCLIS code after the DOSCMD label. Following the detection of the ampersand, the text (if any) is parsed based on the current file path and added to a modifying AUTOEXEC pab.  When the NEWAUT flag setting is hit, it then tries to load a file that was parsed.  So, if it is just an ampersand, then it is a filepath with no filename.  Thus, no DV80 file can be opened.  This is where I have added code that looks at the length of filename, and then looks at the last character of the parsed filename.  If it is a period, then it is just a filepath so I jump around the code that would try to open the DV80 autoexec file.

 

  • Like 2
Link to comment
Share on other sites

@9640News - works great.  The ampersand with a device name only no longer accesses the drive.  I tried the new code with a floppy and various other devices - none of them attempts to read the subdirectory as if it is a batch file! Nice work!!   :)

 

On 6/3/2021 at 6:31 PM, 9640News said:

The CLI code essentially does a CLR @>F110 and does a BLWP @0.  It is during this point, all the XOP's run their powerups again.  In L8\DSRPASS-S, there is this note for the DSR:

 

The powerup is a critical step in the warm restart process - for the ampersand and CTRL-ALT-DEL.  We need to be sure it is executed in the same manner as before and my testing so far shows this is the case.  The comments you pasted from the file accurately reflect what happens at a high level with the DSR, though that will continue to evolve as we proceed with the next pieces of the DSR cleanup work. 

  • Like 2
Link to comment
Share on other sites

The ampersand fix is quite nice to have in place, Beery.  Now when I type "&" to do a warm restart, if I am pointing to a floppy drive, I don't have to wait for the OS to figure out that "DSK1." is an invalid file ;)

 

This weekend I didn't have a lot of free time though I made some headway untangling SCSI code and may have uncovered another legacy routine that we can carve out of the DSR.   I intend to complete the SCSI code rework within the next month, then turn attention back to the powerup and interrupt routines.  This month is a busy one for me - for work and other reasons. 

 

@9640News If you know how to use Genasm to include or exclude code (something like ifdef) I'd like your help to implement it.  I would like to change how the debug code is written so that for a release , Genasm discards the debug code from the OS.  As it stands, every place I've added a debug call forces the OS to execute that call, whether debug is wanted or not. 

  • Thanks 2
Link to comment
Share on other sites

8 hours ago, InsaneMultitasker said:

The ampersand fix is quite nice to have in place, Beery.  Now when I type "&" to do a warm restart, if I am pointing to a floppy drive, I don't have to wait for the OS to figure out that "DSK1." is an invalid file ;)

 

This weekend I didn't have a lot of free time though I made some headway untangling SCSI code and may have uncovered another legacy routine that we can carve out of the DSR.   I intend to complete the SCSI code rework within the next month, then turn attention back to the powerup and interrupt routines.  This month is a busy one for me - for work and other reasons. 

 

@9640News If you know how to use Genasm to include or exclude code (something like ifdef) I'd like your help to implement it.  I would like to change how the debug code is written so that for a release , Genasm discards the debug code from the OS.  As it stands, every place I've added a debug call forces the OS to execute that call, whether debug is wanted or not. 

Well, I do know how, but it would be necessary to add a COPY statement to the top of every file GenASM would use.  The COPY “DEBUG” file would then have a statement with an EQU of 0 or 1 and then the use of the conditional statement. I would use that file as the “master” file with either debug on or off.  I did something very similar in the AFTERHOURS source code, however that was only two  different source files, not the 50+ individual files that must be linked for MDOS.

 

For your work, it would just be needing to edit your current source files adding the COPY “DEBUG” source file with the conditional statement including your source file for the debug routine and then conditional statements when you want to output debug information.

 

Give me a call in the evening sometime when you want to talk about it.

 

  • Like 1
Link to comment
Share on other sites

1 hour ago, 9640News said:

 

Give me a call in the evening sometime when you want to talk about it.

Thank you.   We can leverage the "COPY" files, which will limit the need to edit every source file in the DSR code.  The debug routines live in multiple files/folders, so I would also consolidate those down to one instance. 

 

My thinking is that we would want multiple debug flags segregated by routine type, so that you don't need to turn on SCSI routine debug statements just to see what is happening in a CLI command. 

  • Like 2
Link to comment
Share on other sites

Because I'm never afraid to show my lack of understanding...

It's my understanding the HFDC includes DMA capabilities, further, I understand that to mean, the CPU says - Controller, I want these sectors, loaded in to that memory area, call me when's it done.

   So my questions are:

   1) Is my understanding true?

   2) If so, whats the maximum bandwidth of the different mass media controllers: HFDC, IDE, SCSI - because I think with all the various solid state drives for each of these three controllers, mechanical drives are no longer a factor, unless we want them to be? How about a ram disk, since it essentially doesn't have to go through a controller interface?

 

----

Sorry for the edit, I re-thought my question... If you remove physical drive latency from the equation, you have what ever that particular controller chips maximum sustained output is...

 

Then you get in to --- what is the limitations of the geneve / TI - bus, memory, wait states and cpu speed...

 

Link to comment
Share on other sites

The HDC9234 (HFDC) can only load into its private memory on the card. The private memory is also mapped into the address space so that you copy the contents from there.

 

This is what I actually wanted to ask for myself - how does SCSI PDMA work?

  • Like 1
Link to comment
Share on other sites

Spent some more time playing with the SCSI yarn ball.  One of the error reporting routines uses R4 to save/restore the error code. During a write operation, R4 is used as a loop counter and is set to 0, destroying the previously set value (to the error code label) and subsequently causing the SCSI DSR to write the error value to address >0000.  I am fairly certain the SCSI DSR is paged into the mapper from >0000->1FFF during this operation, so the OS vector at @>0000 is not overwritten.  I'll confirm and then work on removing all of the extraneous error report code from the read and write routines.  

 

The SCSI DSR maintains a list of the device IDs that have been started with the SCSI "START" command since powering on the system; a warm restart does not clear this table.  I've noted this for another day.

 

  • Like 4
Link to comment
Share on other sites

Took a break from the SCSI code this weekend to work on the IDE low level code.  A new powerup routine now tests for device IDE1 and if found, its card's CRU address is saved and the sector IO address is cached.  These two saved values simplify the low level call and eliminate the need to scan the card during each read/write operation (~faster).  This implies that the IDE card will be usable at other CRU addresses in the future.  I don't yet know if a similar  powerup/CRU caching approach is feasible for SCSI and TIPI, though it is certain both cards would benefit from more than is coded within the OS.

  • Like 7
  • Thanks 1
Link to comment
Share on other sites

Just posting a note here for others on some work I am doing.

 

I am working on an ANSI driver for MDOS that would embed itself as an XOP into MDOS.  I don't see it becoming embedded in MDOS at present as it is over 4K in size and MDOS does not presently have that much free memory without growing MDOS by another 8K page.

 

Presently, it would have three opcode calls.  Envision one opcode to reset, second opcode that configures MDOS for either text mode 80 ANSI or ANSI color 80 column, and the third call is for a string display similar to the TTYOUT routine for the Video XOP >0027 call.

 

Would anyone have use for an ANSI driver for anything they may be writing? 

 

I should then be able to incorporate the Telnet command that was added to MDOS to use the driver if installed in either a text or graphics mode environment. 

 

Beery

 

 

 

  • Like 5
Link to comment
Share on other sites

On 6/8/2021 at 6:31 PM, InsaneMultitasker said:

@Jeff White - do you know the ins-and-outs of how the SCSI pDMA works ? See above posts from Michael and Dan.

@dhe, @InsaneMultitasker, @mizapf, here is the short explanation.  The 53C80 is setup in such a way that by continually reading or writing the data register the handshake automatically occurs.  In byte-transfer mode, the 53C80 only responds at the even address.  In word-transfer mode, the 53C80 reponds to both even and odd addresses.  By checking a CRU bit or reading a status register, it can be determined when the transfer can begin.  To speed operation, multiple MOV(B) instructions can be used.  The key here is that the SCSI drive buffers the complete sector contents, be it 256, 512, 1024, or 2048 bytes etc.

 

One caveat with the 99/4A is that word-transfer mode requires swapping bytes because of the sTupId odd-byte before even-byte transfer.

  • Like 4
  • Thanks 1
Link to comment
Share on other sites

Updates to close out the weekend and month of June:

 

Short version:  DSR cleanup continues. Most of the SCSI routine consolidation work is either complete or mapped out.  Powerup/startup code is now written and tested for the IDE and SCSI cards; still need to work out how best to link it all into the Geneve's master DSR. 

  1. Completed review of HFDC and SCSI/IDE/Horizon equate usage.   There have been two separate files since MDOS 2.50S, one for each code base.  After a few adjustments, the SCSI low level driver now references the more extensive HFDC equate file.  The immediate benefit is that there can be no future, accidental divergence - and one less file to track.  The MAKE process has been updated with the proper dependencies.  The OS pagelist/mapping file is now referenced by the scsi/ide/horizon code, for future remediation of hard-coded values.   [HD\EQUATES, HD\WINDS1-1, SCSI2\SCSI_S, HEAD\PAGENUMS]
  2. The SCSMAP command has been updated.  When a SCSx device is mapped, its cached drive parameters are now immediately invalidated.  This protective measure stops the OS from applying the previous drive's parameters to a newly mapped drive. Previously, a warm restart was required to reset the parameters.   Note: this change does not remove the need to perform a warm restart if you change media in a device, such as an EZ135 or SyQuest drive. [CLI\NEWCMDS, other dependencies]
  3. IDE and SCSI card detection has been added to the low level routines. [SCSI2\TIDE9S, SCSI2\POWERS]
  4. --The IDE routine checks for device "IDE1" and if found, locates and caches the sector IO read/write opcode >80.  A call to the IDE card's powerup routine is now written but not yet active.  [TIDE9S]
  5. --The SCSI routine checks for device "SCS1" and if found, checks the SCSI EPROM version to determine the default PDMA setting, then checks for GenMod EPROM.  The SCSI controller's ID is cached for future operations. This will remove all of the 'setup' code from the read/write routines and into a formal powerup.  [SCSI2\POWERS, SECTOR_S]
  6. The SCSI sector IO routine consolidation is about 75% complete.  A few more legacy routines were removed - they were replicated from the HFDC code but are not needed in the SCSI code.  Cleaned up register usage and unused references.  With a but of luck, I can wrap up this deep dive next weekend.  I don't want to tamper with the SCSI write cache until the very end. ;)  [SCSI2\SECTOR_S]
  7. During my debugging, I noticed that Directory Manager reads the VIB and full bitmap [32 sectors] twice during catalog operations.  The second read is used to capture the volume/used/free space. A quick inspection of the C code leads me to believe this can be 'fixed' at a future date.
  8. Prior changes worth referencing:  SCSI device start routine no longer activates ID0;  warm reset with ampersand no longer tries to read the volume as a batch file; pfmcore802 released 4/17, turns off all cards at powerup [review: was this same routine added to the master dsr, gpl, and exec]
  9. TIPI powerup routine, HFDC (dsrpass-s) memory usage validation, and debug enhancements are still on hold.

 

Edit: Tracked down the reason MD (make directory) on a non-existent drive returns an incorrect error code.  The SCSI common sector IO routines do not test for direct sector IO and report internal error code >2 [error 1, shifted] instead of >C [error 6, shifted].  The Horizon and IDE use the same common code and are impacted in the same manner.   Forcing the error routine to return >C confirmed the problem. The solution requires a review of the error reporting routines.  It also requires review of the direct sector IO (DSECIO) flag that is set once and never cleared by the HFDC common routines, forcing certain error traps to report direct sector IO errors. 

  • Like 2
  • Thanks 3
Link to comment
Share on other sites

1 hour ago, Jeff White said:

@InsaneMultitasker, does GenMod Geneve still need to use byte-transfer mode with SCSI cards that have the Becker upgrade?

With a Genmod Geneve, the byte-transfer routines are in effect for both the upgraded and non-upgraded SCSI cards.  I have never "confirmed" that this mode is still needed, though I have had no reason to think that Michael and Harald would have coded it that way if word-transfer mode was possible.  Do you think word transfers should be possible?

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...