Jump to content
Lee Stewart

fbForth—TI Forth with File-based Block I/O [Post #1 UPDATED: 11/16/2019]

Recommended Posts

This thread is a discussion of the evolution of fbForth, my file-based implementation of TI Forth. Though I have replaced the sector-based block I/O of TI Forth with a file-based version for fbForth, I have written utilities to browse TI Forth disks and to copy TI Forth blocks to fbForth blocks files (see latest FBLOCKS file far below).

Several obvious advantages to using file I/O over sector I/O:

  • It is simple to make a system disk of any larger size and density by just copying the system files to the new (usually larger) disk.
  • If there is filespace on the system disk, you can save other useful programs there, such as CorComp's Disk Manager, without fear of corrupting the system disk.
  • There is no danger of corrupting a disk by saving Forth blocks---they are only stored to the current blocks file.
  • One can use disks of different sizes with impunity.

The only significant disadvantage I can think of is not being able to use the new system with the old. The viewer/copier utilities, which I mentioned above and included in the latest FBLOCKS file posted far below, render that merely an inconvenience. There is also a slight performance penalty to reading 8 records one at a time instead of 4 sectors all at once for each block read. This is somewhat mitigated by saving 896 bytes of VRAM.

I am certainly open to any and all suggestions.

...lee

----------------------------------------------------------------------

 

post-29677-0-91560400-1460602652.pngfbForth 2.0: A File-Based Cartridge Implementation of TI Forth is now published as a book available for $14.99 on Amazon.com. It is also on Amazon Europe for various similar prices:

 

post-29677-0-55846500-1503261196.png

 

post-29677-0-91560400-1460602652.png fbForth now has a website of its own: fbforth.stewkitt.com

 

fbForth downloads & cartridge (fbForth 2.0)—

Note: The binary images in 4 menu options will only work with the revision of fbForth with which they were BSAVEd. The current revision is fbForth 2.0:12 and will work with the binary images in FBLOCKS_20191108.zip and later (until the next revision).
  • Like 13

Share this post


Link to post
Share on other sites

Lee,

 

I make this suggestion with my tongue firmly planted in my cheek, but the more I think about it, it would seem appropriate to name your ungraded version of TI FORTH as TI FIFTH.

 

I am a really novice Forth programmer having dabbled a bit with Wycove and TurboForth but I think another advantage to switching to file-based blocks is that it might make it easier to exchange programs with these other file-based Forth programs.

 

Jacques

Share this post


Link to post
Share on other sites

Lee,

 

I make this suggestion with my tongue firmly planted in my cheek, but the more I think about it, it would seem appropriate to name your upgraded version of TI FORTH as TI FIFTH.

 

I very much appreciate your sense of humor and your thoughtful suggestion; but, though I consider it an upgrade, it will still be Forth and still a combination of fig-Forth and Forth-79, which both have been left in the wake of a string of Forth standards (Forth-83, ANS-Forth-94 (ISO-Forth-97) and Forth-2012 (proposal). I also would not want it confused with actual Fifth languages that have been developed (one, definitely tongue-in-cheek), see http://www.articlewo...mming_language.

 

I am a really novice Forth programmer having dabbled a bit with Wycove and TurboForth but I think another advantage to switching to file-based blocks is that it might make it easier to exchange programs with these other file-based Forth programs.

 

Jacques

 

It would certainly make for easier exchanges of programs. However, most of the programs themselves would require a little to, likely, a lot of conversion to run on the other systems. For example, in TI Forth, USE is a user variable pointing to the next (least recently used) block buffer to use; but, in TurboForth, it is a word that expects on the stack the address and character count of a string containing the name of the next blocks file to use. BTW, I fully intend to lift some of @Willsy's routines (in principle if not code) to accomplish this same task (he has already encouraged me to do just that), I will just have to call this one by another name ( USE_FILE ? ).

 

Even with a method of transferring disk blocks to a blocks file, the user will probably need to make changes, especially where explicit block numbers are used and, of course, where s/he may have used the direct-sector access words ( DISK_LO , DISK_HI , DISK_SIZE , OFFSET , DR0 , DR1 , DR2 , DRIVE , RDISK , WDISK , DISK-HEAD , FORMAT-DISK ), which I intend to implement only as necessary and then only as an optional load from the system blocks file, which I intend to call BLOCKS just as @Willsy did for TurboForth.

 

I can see your eyes glazing over, so I'll stop for awhile ... :sleep:

 

...lee

  • Like 1

Share this post


Link to post
Share on other sites

Awesome. I've read through these threads again and I've also been reading through my own TI Forth manual... mine has been highlighted in several places, including chapter 0, the "Dedication". Chapter 0 is interesting as it discusses how TI just straight DROPPED TI Forth and sent the manual and a diskette to each of the officially recognized User Groups.

 

Also, one thing I found interesting is the assembly source code for the words and the last section of the manual which has all the included screens in the disk.

 

The manual is pretty in depth... I look forward to seeing your improvements in the language. :)

Share this post


Link to post
Share on other sites

Another advantage of a file-system-based blocks file is that you can have more than one blocks file per disk, space permitting of course.

 

I wonder if supporting both sector and file based block access might be adding significant 'baggage' to the software? You might want to consider, rather than having both available in TI Forth, just have file-based blocks available, and ship a utility (perhaps an EA/5 program) to do the conversion. This would not be a major inconvenience, since:

  • The files only have to be converted once, at the start of a project, and after that, the sector-based ones can be discarded;
  • The audience (and thus the number of people inconvenienced) is rather small, so you're unlikely to encounter much resistance from the user base (same situation with TurboForth; at this point, I can more or less act with impunity as the number of applications out there is very small, and mostly written by me (and you ;) ))

That will simplify the level-of-effort, and allow you to streamline your code-base.

 

TurboForth has reams of code in there (see the BLOCKS files in the bank0 and bank1 directories of the source) that I consider to be 'proven in use': They were written in 2009, and haven't changed since (only the stack direction got changed after writing them). They are very reliable and don't corrupt etc (at least, it's never happened to me). You're more than welcome to use the source as a resource (that's why I publish the source on the website), or just lift it verbatim and tailor it to your specific needs. Hack away!! :-D

  • Like 1

Share this post


Link to post
Share on other sites

Thanks, Mark. Those are excellent ideas and your magnanimity re the use of your code is much appreciated! :) I should be turning my full attention to it in a day or two---just after I post my updated manual, which is all done except for an appendix on true lowercase and one last proofread.

 

...lee

Share this post


Link to post
Share on other sites

@Willsy...

 

I am looking at how to implement USE (which I must rename :() for TI Forth. It does not appear that you flush dirty blocks before changing to the new blocks file; but, rather, unconditionally zero the block pointers. Did I get that right?

 

...lee

Share this post


Link to post
Share on other sites

@Willsy...

 

I am looking at how to implement USE (which I must rename :() for TI Forth. It does not appear that you flush dirty blocks before changing to the new blocks file; but, rather, unconditionally zero the block pointers. Did I get that right?

 

...lee

 

Not sure, let me 'ave a ganders... (I think I execute EMPTY-BUFFERS)...

Share this post


Link to post
Share on other sites

Yep. You're right:

 

;[ EMPTY-BUFFERS ( -- )
; marks all buffers as unused.
_mtbuf	 li r2,6 ; counter
 li r0,blk0 ; address of first blk
mtbufl clr *r0+ ; zero block number then point to vdp address
 mov *r0,r1 ; get vdp address
 andi r1,>7fff ; set dirty to zero
 mov r1,*r0+ ; write it back, point to next blk
 dec r2 ; decrement counter
 jne mtbufl ; repeat if not finished
 clr @lstblk ; no blocks in memory
 jmp flushx
;]

 

This is the implementation from EMPTY-BUFFERS.

 

This literally zero's the buffers (CLR *R0+) - a zero in the 'assigned block' position means that that buffer has no block assigned to it (that's why block numbers start at 1 :P )

 

The next 16-bit word contains the VDP address of that buffer, with the MSB being the 'dirty' flag (i.e. if set, the block in memory has been changed - FLUSH looks for this).

 

See here for how blocks are arranged in VDP memory (scroll down to the bottom, then scroll up, looking for the comment "set up the pab pointers:") - then read down from there. It should make sense. :? :grin:

 

So, it just zero's them out without doing a flush.

 

IIRC, my reasoning was as follows:

 

If USE *did* execute a flush, a situation could arise, when using floppy disks, where, if a 'dirty' block was in memory, a user could potentially forget, and change floppy disks, and then execute USE. This would be disastrous, as USE would then write the dirty block(s) back - to the wrong disk :-o . So, the onus is on the user. He must execute FLUSH (assuming he wants to do a flush) then change disks, then execute USE. (Of course, if the blocks file on the new disk has the same name, then he doesn't need to execute USE at all!)

 

If the user finds that behaviour inconvenient, he can just change it! That's Forth!

 

: USE FLUSH USE ;

 

and bingo ;)

 

EDIT: Muppetry deleted!

Share this post


Link to post
Share on other sites

...

(Of course, if the blocks file on the new disk has the same name, then he doesn't need to execute USE at all!)

...

 

Oo-oo-oo :-o ... Wouldn't that screw with the disk/file buffering the DSR uses at the top of VDP memory? Unless it's a duplicate disk in every way, it really isn't the same file.

 

...lee

Share this post


Link to post
Share on other sites

Speaking of disk/file buffering, I'm fighting with the TI Forth system trying to figure out how to grab some more space from the top of VDP RAM for more simultaneously open files in bitmap mode. For bitmap mode, the E/A manual says that the Pattern Descriptor Table (PDT) and Color Table (CT) must start at 0000h and 2000h, respectively, or vice versa; but, there's no explanation for the requirement. They can be put other places: The PDT on any 800h boundary and the CT on any 40h boundary. Anybody have any insight?

 

...lee

Share this post


Link to post
Share on other sites

Lee,

 

If you have access to The Cyc look up one of Mike Wright's articles that deals with bit map mode. Attached to Mike's article is a copy of an email from Jeff White that gives an explanation for using 0000h and 2000h. I do not completely understand the reasons but at least I now know that there is a reason for using what initially appears to contradict the general rules.

 

Jacques

Share this post


Link to post
Share on other sites

Oo-oo-oo :-o ... Wouldn't that screw with the disk/file buffering the DSR uses at the top of VDP memory? Unless it's a duplicate disk in every way, it really isn't the same file.

 

...lee

 

Thinking about it, yes, it was a dumb comment. To answer your specific question, in TF's case, I don't think it would hurt the internal (to the OS) disk/file buffering at the top of VDP, because TF always closes the blocks file between operations.

 

For 1 example: 1 LIST does the following:

 

<open blocks file>

<get the block into a buffer>

<close blocks file>

<display buffer>

 

(the open and close file are functions of the word BLOCK)

 

But still, it was a stupid comment (my comment, not yours!); consider the following:

 

<DISK "A"> in DSK1

1 LIST

<change physical floppy in DSK1 to DISK "B">

1 LIST

 

Doh! :dunce:

Share this post


Link to post
Share on other sites

OK...Thierry's site has very good info on bitmap mode and he says the same thing about the locations of the PDT and CT, but also without a reason that I can discern.

 

The reason I am trying to grab more space for files is that I think I need the possibility of at least one more file open for my conversion of TI Forth to a file-based block I/O system. That is, I don't think the current sector-based block I/O system uses the buffering mechanism in high VDP RAM that file I/O requires. I could be wrong, of course.

 

From the VDP Memory Map in Chapter 4 of the TI Forth manual, It looks like I can move the Sprite Attribute table elsewhere, e.g., into part of the Forth Disk Buffer space, because I only need 128 bytes as a record buffer for file I/O of blocks. The only thing that would convince me otherwise would be that maintaining a 1KB buffer is better. The processes to compare would be 8*(get a record into VRAM record buffer; transfer it to a CPU RAM block buffer) and getting 8 records into 8 x 128-byte buffers (1KB all told) followed by transfer of 1KB to a CPU RAM block buffer). It seems to me the only difference is the speed of a single 1KB transfer vs. 8 128-byte transfers from VRAM to CPU RAM. If the former is not significantly faster, the smaller VRAM buffer is simpler and frees up space for other things, particularly in bitmap mode. The slowest process is reading the 8 disk records, anyway. And, that is the same either way.

 

This brings up another point, viz., the VDPMM mentioned above claims there is DSR buffer space for 2 files at the top of VRAM; yet, examination of the TI Forth code on screen #54, line 4 shows there is only enough space reserved for 1 file, even though the very next line claims 2! I can do 2 simultaneous files without moving the SAT, but I cannot get 3 files no matter what I do. It looks as though the only reason to move the SAT would be to get more sprites because, then, the SAT would not overlay the SDT. Sorry for all the rambling, but I gotta figure this out! :P

 

...lee

Share this post


Link to post
Share on other sites

I just had a thought re the TI Forth block buffer!—I could use DSR Subprograms 014h (Access Direct Input File) and 015h (Access Direct Output File) to read/write an entire Forth block (4 sectors = 8 128-byte records) in one operation. The only reason I would not want to do this is if @Tursi's Classic99 FIAD mode does not support such access. From scanning the Classic99 manual, it appears that it does; but, maybe that is only in DOAD mode. @Tursi?

 

...lee

Share this post


Link to post
Share on other sites

I think I will also take this opportunity to hoist the definitions ( VDPMDE , SLIT , WLITERAL , <CLOAD> and CLOAD) from the always-loaded blocks 3, 20 and 21 into the resident dictionary except for the MENU stuff—I'd like to figure out another way to handle the MENU stuff.

 

Also, I'm seriously considering putting the system call definitions from block 33 into the resident dictionary, as well—except, possibly, RNDW , RND , SEED and RANDOMIZE — though, maybe those, too. Any comments?

 

...lee

Share this post


Link to post
Share on other sites

I don't want to make too many changes that don't involve the move from sector I/O to file-record I/O for TI Forth blocks because I would like old TI Forth programs to work with the new system with minimal attention. I am sure I will break programs that load explicit system block numbers because I will be changing most of those block numbers to avoid wasting space, particularly the 15 blocks between 5 and 20 that were occupied by the system files, FORTH and FORTHSAVE.

 

Another conundrum has to do with MESSAGE and ?ERROR . I am going to try to put the error messages into RAM so that every blocks file does not need to store them on blocks 4 and 5. It would probably take about 300 bytes and may not be worth the effort—I just don't know. As it is now, MESSAGE prints a line that is an offset from block 4, line 0 as per fig-Forth design. I could have MESSAGE look first for the error lines in my proposed RAM table and, upon failure, grab the relevant line from the blocks file. I was thinking about changing the offset reference to block 1, line 0; but, maybe I should just keep the old reference point.

 

...lee

Share this post


Link to post
Share on other sites

I just had a thought re the TI Forth block buffer!—I could use DSR Subprograms 014h (Access Direct Input File) and 015h (Access Direct Output File) to read/write an entire Forth block (4 sectors = 8 128-byte records) in one operation.

 

Perk!

 

Ooh! What's these? Never heard of these! Do you have a resource you can point me at?

Share this post


Link to post
Share on other sites

OK...Thierry's site has very good info on bitmap mode and he says the same thing about the locations of the PDT and CT, but also without a reason that I can discern.

 

I think the reason may be to do with GPL, or specifically, the floating-point and/or the transcendental routines. GPL uses VDP memory to do it's stuff (as we know) and unfortunately, the addresses that it uses are hard coded. I'm surmising that the bitmap locations quoted are the best compromise to allow a bitmap system to run with the GPL routines.

 

I think it's probably nothing more than that. Clearly, there's no hardware reason; the VDP chip will happily take it's data from wherever you direct it via it's VDP registers.

 

Mark

Share this post


Link to post
Share on other sites

From the VDP Memory Map in Chapter 4 of the TI Forth manual, It looks like I can move the Sprite Attribute table elsewhere, e.g., into part of the Forth Disk Buffer space, because I only need 128 bytes as a record buffer for file I/O of blocks. The only thing that would convince me otherwise would be that maintaining a 1KB buffer is better. The processes to compare would be 8*(get a record into VRAM record buffer; transfer it to a CPU RAM block buffer) and getting 8 records into 8 x 128-byte buffers (1KB all told) followed by transfer of 1KB to a CPU RAM block buffer).

The difference in terms of speed would be indistinguishable IMO. It's slightly easier for TF in this respect, because the block buffers in TF are in VDP, so (for block operations) no buffer is required; when reading from disk the PAB destination address is changed with each read, so the block is read directly into the correct place in VDP memory. A similar thing is done when writing - no need to transfer individual records to a 'holding area' in VDP.

 

I think that you'll need two buffers in VDP in your case: One for block reads/writes (128 bytes) and a second one (which must already be present) for general purpose file I/O (DV80 files etc). This buffer will have to be 254 (IIRC - or is it 255?) bytes - since that's the max record size of user files.

 

It seems to me the only difference is the speed of a single 1KB transfer vs. 8 128-byte transfers from VRAM to CPU RAM. If the former is not significantly faster, the smaller VRAM buffer is simpler and frees up space for other things, particularly in bitmap mode. The slowest process is reading the 8 disk records, anyway. And, that is the same either way.

Exactly. I wouldn't spend any time thinking about a 1K buffer in VDP - it won't make any noticable difference to IO speed IMO.

 

 

This brings up another point, viz., the VDPMM mentioned above claims there is DSR buffer space for 2 files at the top of VRAM; yet, examination of the TI Forth code on screen #54, line 4 shows there is only enough space reserved for 1 file, even though the very next line claims 2! I can do 2 simultaneous files without moving the SAT, but I cannot get 3 files no matter what I do. It looks as though the only reason to move the SAT would be to get more sprites because, then, the SAT would not overlay the SDT. Sorry for all the rambling, but I gotta figure this out! :P

 

This is where my knowledge of the IO system in the TI begins to break down. All I really know is, tha area from >37D8 to >3FFF is used for 'alchemy' by the TI OS for file i/o. In there be demons. The size of this area is managed by CALL FILES(n) in BASIC. At power up, the VDP RAM is sized for up to 3 simulataneously open files. I didn't have the guts/inclination to muck about in this area, and so I have left it as it is for TF. I have no idea to how execute the equivalent of a CALL FILES(n) from the assembly side of things.

 

If anyone knows how it's done, I'd be interested in knowing the finer details; perhaps it would be useful in the future.

 

Mark

Share this post


Link to post
Share on other sites

...

This is where my knowledge of the IO system in the TI begins to break down. All I really know is, tha area from >37D8 to >3FFF is used for 'alchemy' by the TI OS for file i/o. In there be demons. The size of this area is managed by CALL FILES(n) in BASIC. At power up, the VDP RAM is sized for up to 3 simulataneously open files. I didn't have the guts/inclination to muck about in this area, and so I have left it as it is for TF. I have no idea to how execute the equivalent of a CALL FILES(n) from the assembly side of things.

 

If anyone knows how it's done, I'd be interested in knowing the finer details; perhaps it would be useful in the future.

 

Mark

 

Same references as my last post plus Software Specifications for the 99/4 Disk Peripheral. DSR subprogram 016h is the one that assigns the buffers for the number of simultaneously open files you want. It uses the VDP top of memory stored in 8370h to locate them. I wrote a FILES word for TI Forth based on how TI Forth actually does it:

 

 

HEX

: FILES ( files --- )

1 PABS @ VSBW 16 PABS @ 1+ VSBW

834C C!

PABS @ 8356 !

0A 0E SYSTEM

;

 

PABS is the TIF user variable that holds the VDP address of the beginning of PAB storage. 0A 0E SYSTEM is the equivalent of a DSRLNK link call to 0Ah (subprograms)

 

...lee

Share this post


Link to post
Share on other sites

I just had a thought re the TI Forth block buffer!—I could use DSR Subprograms 014h (Access Direct Input File) and 015h (Access Direct Output File) to read/write an entire Forth block (4 sectors = 8 128-byte records) in one operation.

 

Thanks for the links. Just took a look. The only thing to be careful of when using sector access is the possibility of file fragmentation.

 

This could potentially happen. I seem to remember that it's possible to 'expand' a blocks file in TF just by writing something to a record number that doesn't exist. So, for example, in TF:

 

MKBLK DSK2.TESTBLOCKS 20
S" DSK2.TESTBLOCKS" USE
1 EDIT
< EDIT BLOCK 1 AND TYPE HELLO THEN EXIT EDITOR>
FLUSH
1 BLOCK DROP \ get block one
UPDATE \ flag it for update
1 BUF? DROP \ get the buffer for block 1
75 SETBLK \ associate it with block 75
FLUSH \ write block one to block 75
75 LIST

 

You'll now find that the file has grown magically to 75 blocks, even though it was created as a 20 block file. This is nothing special done in TF, it was the TI DOS that did it.

 

So... Imagine that the file was created, then a DV80 laid down on disk, then the blocks file was expanded - you'd have a fragmented file (shown in catalog listings with an asterisk next to the file name IIRC).

 

It's rare, but it can happen...

 

Just something to bear in mind...!

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...

  • Recently Browsing   0 members

    No registered users viewing this page.

×
×
  • Create New...