fbForth fbForth—TI Forth with File-based Block I/O [Post #1 UPDATED: 06/09/2023]

[Tursi]

Okay, that does make sense.. but why can't you do that?

 

You know the last free address in VDP RAM, therefore you should be able to determine if you have the resources that you need to enter bitmap mode. Technically, you /should/ take the pointer in >8370 at face value and assume that you can not touch any memory above it. It may not exist, it may contain critical data, for whatever reason, some device has requested it be private memory for its own use. By honoring that, you get better compatibility.

 

Invoking bitmap mode doesn't change any of the memory pointers, so you always have that information, and it will only change if anything tries to reserve more memory (which should only happen when the equivalent of CALL FILES is done). So in that case, when the user requests bitmap mode, I would just do the CALL FILES to reduce it down to 1 or 2, then look at >8370 and make sure it's high enough, and then either proceed or notify the user as applicable. (You could even check >8370 BEFORE the unconditional change to test whether you even need to, but I suppose that's overkill ).

[+Lee Stewart]

Okay, that does make sense.. but why can't you do that?

 

You know the last free address in VDP RAM, therefore you should be able to determine if you have the resources that you need to enter bitmap mode. Technically, you /should/ take the pointer in >8370 at face value and assume that you can not touch any memory above it. It may not exist, it may contain critical data, for whatever reason, some device has requested it be private memory for its own use. By honoring that, you get better compatibility.

 

Invoking bitmap mode doesn't change any of the memory pointers, so you always have that information, and it will only change if anything tries to reserve more memory (which should only happen when the equivalent of CALL FILES is done). So in that case, when the user requests bitmap mode, I would just do the CALL FILES to reduce it down to 1 or 2, then look at >8370 and make sure it's high enough, and then either proceed or notify the user as applicable. (You could even check >8370 BEFORE the unconditional change to test whether you even need to, but I suppose that's overkill ).

 

Of course, that does make sense. Even if the user was using more than the default of 3 files max, TI Forth simply disregarded it, so that the return to other-than-bitmap mode, "restores" FILES = 3 (the startup default). I think I have my plan! Thanks, @Tursi!

 

...lee

[+Lee Stewart]

Another minor conundrum—

 

First, a little background: Because fbForth inherits everything from TI Forth that I don't remove or change, I've been looking at the code with an eye to cleaning up what I can tell needs it. Four such words that are used in several other words in the Forth kernel, as well as the optional code from the system blocks file, are B/BUF$ , B/BUF , B/SCR$ and B/SCR . The words with '$' in their names are system user variables and the others are system "constants". B/BUF is bytes/buffer (always 1024) and B/SCR is blocks/screen (always 1). These "constants" are defined to fetch the values stored in the variables—hence, the quotes. TI Forth never used any values other than those above, so I don't see the point in persisting them into fbForth. I plan to simplify B/BUF to be truly the constant 1024 and get rid of the rest, unless someone can make a case for retaining them. These words came from fig-Forth, which predates Forth-79, so compatibility is probably no longer a good retention argument. BTW, we no longer use the term "screen" to refer to the editable unit for storing Forth words (routines). We now use the term "block". When B/SCR was relevant, "block" referred to disk storage and buffer size—particularly when there was a 1:1 correspondence between them and they were less than the size of a Forth screen—and a "screen" was composed of one or more blocks totaling 1024 bytes by convention. Consequently, I plan to simplify words that use B/SCR by eliminating it. In some cases, it dramatically simplifies the word.

 

Now for the conundrum: fbForth (inherited from TI Forth) uses the null character (0) to terminate buffers. There is even a null word in fbForth that is found by the interpreter, which does some checking and, if everything is copacetic, drops the top of the return stack to exit one level higher. It is defined as follows ( {null} represents an actual 0):

*

: {null}
     BLK @
     IF
        1 BLK +! 0 IN !
        BLK @ B/SCR MOD 0=
        IF
           ?EXEC R> DROP
        THEN
     ELSE
        R> DROP
     THEN
;

*

If I remove B/SCR , it simplifies to

*

: {null}
     BLK @
     IF
        1 BLK +! 0 IN ! ?EXEC
     THEN
     R> DROP
;

*

I think I can simplify it further; but, I am not sure. My problem is that I don't know whether the incrementing of BLK and zeroing of IN are necessary except to test that the null is in the last block of a screen (now not necessary). BLK represents the block currently being interpreted, so I don't think it matters after {null} is through because the interpreter will be exited after that. If I don't need those modifications, the definition can now be

*

: {null}
     BLK @
     IF
        ?EXEC
     THEN
     R> DROP
;

*

@Willsy? Anybody?

 

...lee

[Willsy]

 

Now for the conundrum: fbForth (inherited from TI Forth) uses the null character (0) to terminate buffers. There is even a null word in fbForth that is found by the interpreter, which does some checking and, if everything is copacetic, drops the top of the return stack to exit one level higher. It is defined as follows ( {null} represents an actual 0):

 

Yes, I've heard of that before. Sounds like a bonkers way of doing it - though one advantage is you could deliberately insert a null in a block and force the rest of the block to be ignored.

 

In TF, when a block is being interpreted, TIB (the address of the terminal input buffer) is set to the VDP address of the 1024 byte block buffer, >IN is set to 0, and #TIB is set to 1024. When >IN is >= #TIB then you have read past the buffer and you're done. Simples.

 

See:

• TIB
• #TIB
• >IN
• SPAN

[+Lee Stewart]

Yes, I've heard of that before. Sounds like a bonkers way of doing it - though one advantage is you could deliberately insert a null in a block and force the rest of the block to be ignored.

 

You can do that with ;S—and—you can see what you're doing.

 

In TF, when a block is being interpreted, TIB (the address of the terminal input buffer) is set to the VDP address of the 1024 byte block buffer, ...

 

In fbForth/TIForth, when BLK contains 0, input is set to the TIB (strictly for typing from the terminal in fbForth). Otherwise, input is from the block buffer associated with the block number in BLK .

 

But—what about my question re the code cleanup?

 

...lee

[Willsy]

The code looks redundant to me - I wonder if it has something to do with the word --> ? I can't imagine why you would want to increment BLK at the end of a block. Even if THRU is being used, THRU will just call LOAD in a loop, so BLK would be set each time through the loop. So I cannot at the moment see why that code is there. I say try it and see - I presume it's fairly trivial at this point to edit the source and re-assemble?

 

Mark

[+Lee Stewart]

The code looks redundant to me - I wonder if it has something to do with the word --> ? I can't imagine why you would want to increment BLK at the end of a block. Even if THRU is being used, THRU will just call LOAD in a loop, so BLK would be set each time through the loop. So I cannot at the moment see why that code is there. I say try it and see - I presume it's fairly trivial at this point to edit the source and re-assemble?

 

Mark

 

First, it's important to understand that figForth does not require a block to be the same size as an editable Forth screen. TI Forth does; but, the developers chose not to meddle with the variables, constants and words that allowed them to be different. A screen must be an integral multiple of blocks and each block has its own buffer. The TIForth developers could easily have set it up to be 4 blocks/screen because of the disk sector size of 256 bytes. That would have meant 20 256-byte block buffers in RAM for the current 5 1024-byte block buffers—it would have been simple (I think) to implement with the B/SCR and B/BUF constants set to 4 and 256, respectively, and everything else the same. In this context, the reason {null} increments the block # is to do the " B/SCR MOD " to see if the result is 0, meaning that the incremented block # is the first block of the next screen and, of course, that the null occurred on the last block of the previous screen—the point of the test. This should mean that the answer to my question is that I can, indeed, discard the " 1 BLK +! 0 IN ! " part in question.

 

And, yes, it is a trivial matter to edit and re-assemble the source. However, I have yet to modify the code to use level 3 file I/O, so I cannot test it, yet.

 

...lee

[+Lee Stewart]

While composing my last post, it occurred to me that my conversion from direct sector access for fbForth block I/O to level 3 file I/O, could be vastly simplified by making use of the B/SCR and B/BUF constants to force all the Forth words using them to do my bidding. I would only need to convert the system to using 128-byte blocks (easily done) with B/SCR = 8, B/BUF = 128 and rewrite the low-level support code to handle the file I/O of one 128-byte record instead of direct sector I/O.

The only casualty would be that "block" and "screen" would no longer be synonymous. I probably should not do that, however, because the above last had meaning with figForth—even the Forth-79 standard, which followed it, no longer used it; but, rather, forced them to mean the same thing. The only vestige that remained in Forth-79 was B/BUF = 1024, unconditionally.

 

...lee

[Willsy]

Wow cool idea! I like it!

[+Lee Stewart]

Wow cool idea! I like it!

 

What! You think I should do the 8 blocks/screen thing?

 

...lee

[Willsy]

Yes. Why not? If the 'plumbing' is already in there, why not use it? If I have understood correctly then there would be 8 blocks to a screen. So consequtive screens would be numbered 1 8 16 etc... That might go against the grain somewhat for users, so you could have BLOCK multiply the block number by B/SCN in order to provide the logical to physical block number conversion on behalf of the user.

[+Lee Stewart]

Yes. Why not? If the 'plumbing' is already in there, why not use it? If I have understood correctly then there would be 8 blocks to a screen. So consequtive screens would be numbered 1 8 16 etc... That might go against the grain somewhat for users, so you could have BLOCK multiply the block number by B/SCN in order to provide the logical to physical block number conversion on behalf of the user.

 

I'm still hesitant because I would probably have to revert back to calling the editable Forth pages "screens" instead of "blocks". This would certainly maintain consistency with TI Forth, but not with TurboForth and any Forth using Forth-83 and beyond. Of course, there are some inconsistencies anyway, but this one seems major to me.

 

As to screen numbering, 1024-byte screens would be 1, 2, ... and starting 128-byte blocks for those screens would be 8, 16, .... I'm not sure how I would handle blocks 0–7 so as not to disturb "the plumbing". They would probably just be wasted space or, possibly, could be used for information, but would need to have some other mechanism to reach it.

 

...lee

[Willsy]

Yeah - that might be a problem, as I could conceive of people wanting the TF and fbForth disks to be compatible with each other for porting code etc...

 

Mark

[+Lee Stewart]

Yeah - that might be a problem, as I could conceive of people wanting the TF and fbForth disks to be compatible with each other for porting code etc...

 

Mark

 

I could certainly allow Screen #0 to be accessible—TI Forth did. There is a TI Forth variable, DISK_LO , that prevented it because its default value was 1 to preclude accidentally trashing the VIB; but, you could change it to 0 to copy a disk. Again, the problem would be that any word, like INTERPRET , that assumes that Block #0 is actually terminal input, would make it difficult to do much with Screen #0. Words like SCOPY (Screen Copy) worked just fine. I'm not sure the editors cared, either; but, I might incur as much work figuring that all out as I would in just eliminating all references to B/SCR and the difference between "block" and "screen". I think I just talked myself into my original plan. I'll just elaborate the 8 128-byte record reads/writes per block (old "screen") in the low-level support code. I will take a look at how you do it for TurboForth and go from there.

 

...lee

[+Lee Stewart]

Regarding B/SCR , even though I will be eliminating all references to it in fbForth's kernel and optional system code in FBLOCKS, I will probably retain the constant with an unconditional value of 1 for compatibility with old TI Forth code folks may have. It will take only 12 bytes of dictionary space.

 

...lee

[Willsy]

In TF BLK is examined and if >0 then BLOCK is called, otherwise EXPECT is called (to fill up the TIB from the keyboard). That's about it.

 

Internally, within BLOCK, the block number passed in on the stack is decremented by 1. This is so that no space is wasted at the beginning of the blocks file. So "logical" block 1 is "physical" block 0.

Edited September 3, 2013 by Willsy

[+Lee Stewart]

...

 

Internally, within BLOCK, the block number passed in on the stack is decremented by 1. This is so that no space is wasted at the beginning of the blocks file. So "logical" block 1 is "physical" block 0.

 

H-m-m-m...I've been doing that in the low-level code for block I/O. I'll probably continue that motif in v0.92, as well.

 

...lee

[+Lee Stewart]

Regarding block I/O and its relation to user file I/O routines—

 

The user file I/O library in fbForth is optional, i.e., not part of the resident dictionary—it is, after all, 1000 bytes long and not often required by users. However, much of what I need to code for block I/O in the low-level support section of the fbForth kernel is the same. The major difference is the size of the RAM buffer for a given file. The user library assumes it to be the same size as the VRAM buffer (a good thing) and always in the same place, once established for each file. For block I/O, though a given transfer amounts to the same thing, there must be 8 128-byte transfers for each block, with each transfer incrementing the block's RAM buffer address by 128 bytes. It would be nice if I could reuse much of the I/O machinery for blocks to service the user file I/O library, ultimately using less dictionary space when the user file I/O library is loaded. I am thinking that it may not be practical, however, because I do not want to change any words in the user file I/O library in the interest of backward compatibility. If I were to change it to work like @Willsy's TurboForth, any users with TI Forth programs using file I/O would have to update their programs. Once again, I'm torn...

 

...lee

[+Lee Stewart]

One last problem to solve (I think) before I dive into the block I/O code—

 

With USEBFL , I plan to open the blocks file in INPUT mode to discover whether it exists and details, such as number of records, that they are 128 bytes, etc. , and then to close it. The process for each block read will then be to open the file in INPUT mode, read the block and close the file. The process for writing each block will be to open the file in OUTPUT mode, write the block and close the file. That's all pretty straightforward. My problem has to do with copying blocks. For that, I plan to make the code optional, much as @Willsy did for TurboForth. The only way I know to check a file's information, including existence, is to attempt to open it in INPUT mode. For copying blocks from one file to another (even the same file without extra, string-comparison code), I will need to open the destination file in INPUT mode once before opening it in OUTPUT mode to retrieve information to prevent creation or extension of a file. This seems like overkill. I wonder whether I should just trust the user and simply open the destination file in OUTPUT mode and carry out the operation regardless of the consequences. What could happen (intentionally or not) is the extension of an existing file or the creation of a non-existent one with the number of blocks equal to the last destination block written. If the latter, the blocks not written will contain garbage instead of the blanks written to every block in a new file created by MKBFL . Thoughts?

 

...lee

[Willsy]

For copying blocks from one file to another (even the same file without extra, string-comparison code), I will need to open the destination file in INPUT mode once before opening it in OUTPUT mode to retrieve information to prevent creation or extension of a file.

 

Why would you want to prevent creation or extension of a file? It’s a feature of the file system that one can append to a file by writing to it. Why would you want to prevent the user from potentially leveraging that feature to his advantage?

 

I wonder whether I should just trust the user and simply open the destination file in OUTPUT mode and carry out the operation regardless of the consequences.

 

Yep, I would go along with that. In Forth, the user is in control. In BASIC, the computer is in control. Forth aint BASIC

 

What could happen (intentionally or not) is the extension of an existing file or the creation of a non-existent one with the number of blocks equal to the last destination block written. If the latter, the blocks not written will contain garbage instead of the blanks written to every block in a new file created by MKBFL . Thoughts?

 

I personally don’t have a problem with that. If in doubt, document the gotcha’s in the system manual and you’re covered! With Forth, I'm very much a fan of the KISS principle, and try not to second guess the user.

Edited September 6, 2013 by Willsy

[+Lee Stewart]

...

I personally don’t have a problem with that. If in doubt, document the gotcha’s in the system manual and you’re covered! With Forth, I'm very much a fan of the KISS principle, and try not to second guess the user.

 

BTW, when TurboForth's CPYBLK is done, the current blocks file is the source file for the copy. When the source file is the current blocks file, that's just fine; but, is that intended in the situation where neither file is the current blocks file at the start of CPYBLK ? For example, the current blocks file is DSK1.BLOCKS and the user runs CPYBLK 4 8 DSK2.XYZ 10 DSK3.ABC . The current blocks file is now DSK2.XYZ. Intended?

 

...lee

[Willsy]

No - sounds like a bug to me. I'll check it!

[+Lee Stewart]

Regarding CPYBLK vis-à-vis MKBFL —

 

The word CLEAR in fbForth (from TI Forth) takes a block # from the stack, associates a block buffer with it, flushes the buffer, fills the buffer with blanks and marks it as updated. The fact that CPYBLK can conceivably create a blocks file with garbage-filled blocks is easily managed by the user with CLEAR as in the following example, in which a blocks file, DSK2.XYZ, with 24 blocks was created with its first 20 blocks containing whatever was on disk by virtue of the CPYBLK line:

*

CPYBLK 3 6 DSK1.ABC 21 DSK2.XYZ
USEBFL DSK2.XYZ
1 CLEAR     ( writes blanks to block #1)
1 EDIT      ( loads block #1 into the editor)

*

The problem with this is that CLEAR and EDIT would need to be executed for each screen the user wishes to edit. The user could write a word to clear multiple blocks as follows:

*

: CLR_BLKS   ( firstblock# lastblock# --- )
   1+ SWAP DO
      I CLEAR
   LOOP
;

*

Now the user in the first example can execute

*

1 20 CLR_BLKS

*

Perhaps, I should define CLR_BLKS in the fbForth resident dictionary. I could then make use of it in the definition of MKBFL to simplify the low-level code for block I/O. Hm-m-m...

 

...lee

[Willsy]

No - sounds like a bug to me. I'll check it!

 

--BLOCK-00022---------
CR .( CPYBLK - copies a range of blocks to the same, or a differ
ent blocks file.)  .( e.g.)
.( CPYBLK 10 21 DSK1.F1 54 DSK2.F2) CR
.( Will copy blocks 10 to 21 on DSK1.F1 to DSK2.F2 starting at b
lock 54.)
0 CONSTANT strtblk  0 CONSTANT endblk  0 CONSTANT dstblk
VARIABLE srcfn 2 CELLS ALLOT VARIABLE dstfn 2 CELLS ALLOT
: BLW 32 WORD ;
: CPYBLK BLW  NUMBER DROP TO strtblk BLW  NUMBER DROP TO endblk
  BLW  SWAP srcfn ! srcfn 2+ ! BLW  NUMBER DROP TO dstblk
  BLW  SWAP dstfn ! dstfn 2+ ! CR endblk 1+ strtblk DO
  srcfn @ srcfn 2+ @ USE I BLOCK DROP I .
  dstfn @ dstfn 2+ @ USE 0 dstblk SETBLK 0 DIRTY FLUSH
  dstblk 1+ TO dstblk LOOP ." Copied. " CR
  srcfn @ srcfn 2+ @ USE ;

 

Above is the code for release 1.2 - looks okay to me - sets the file in USE to SRCFN (see last line of code). Earlier versions may not have this line.

Edited September 8, 2013 by Willsy

[+Lee Stewart]

--BLOCK-00022---------
CR .( CPYBLK - copies a range of blocks to the same, or a differ
ent blocks file.)  .( e.g.)
.( CPYBLK 10 21 DSK1.F1 54 DSK2.F2) CR
.( Will copy blocks 10 to 21 on DSK1.F1 to DSK2.F2 starting at b
lock 54.)
0 CONSTANT strtblk  0 CONSTANT endblk  0 CONSTANT dstblk
VARIABLE srcfn 2 CELLS ALLOT VARIABLE dstfn 2 CELLS ALLOT
: BLW 32 WORD ;
: CPYBLK BLW  NUMBER DROP TO strtblk BLW  NUMBER DROP TO endblk
  BLW  SWAP srcfn ! srcfn 2+ ! BLW  NUMBER DROP TO dstblk
  BLW  SWAP dstfn ! dstfn 2+ ! CR endblk 1+ strtblk DO
  srcfn @ srcfn 2+ @ USE I BLOCK DROP I .
  dstfn @ dstfn 2+ @ USE 0 dstblk SETBLK 0 DIRTY FLUSH
  dstblk 1+ TO dstblk LOOP ." Copied. " CR
  srcfn @ srcfn 2+ @ USE ;

Above is the code for release 1.2 - looks okay to me - sets the file in USE to SRCFN (see last line of code). Earlier versions may not have this line.

 

 

That's what I was looking at—and, no, it does not restore the original blocks file unless it just happens to be the source file for the copy. I'm sure that this is the usual case; but, it doesn't have to be that way. Perhaps, the directions should say:

 

CPYBLK - copies a range of blocks to the same, or a different blocks file. The source file will be the current blocks file when CPYBLK finishes.

Or—save the current blocks file information before the copy and restore that when the copy is done. It probably could be done with a PAB switch.

The problem with the code as it stands is that there is nothing preventing the user from specifying a second and third file, which will then "restore" the wrong file to current blocks file status. Without the caveat in blue above or additional code to restore the expected file, the user might be confused.

...lee

Edited September 9, 2013 by Lee Stewart