Jump to content

Photo

Camel99 Forth Information goes here

Camel99 Forth Concatentive Programming ANS Forth

190 replies to this topic

#176 TheBF OFFLINE  

TheBF

    Dragonstomper

  • Topic Starter
  • 746 posts
  • Location:The Great White North

Posted Wed Sep 5, 2018 1:41 PM

 

28649 (>6FE9) is, indeed, prime, but >6FE5 (28645) is not.  It is the latter that TI used.  The number (>7AB9) TI used to add to the product of >6FE5 and the value at >83C0 is not prime either.  They do not have any primes in common, however.  I always thought the bit pattern of the two numbers might be important, but I am not sure.

 

It is kind of interesting that the second number is π to 5 places with 1 added to the last place—as though to insure the lowest bit is 1.  In fact the lowest bit of both numbers is 1.

 

...lee

 

 

Good catch.  Moving too fast this morning. I was looking for a close prime number to 6FE5 for the experiment.

 

So the evidence is point to the fact the choosing these internal numbers is important.  


Edited by TheBF, Wed Sep 5, 2018 1:41 PM.


#177 TheBF OFFLINE  

TheBF

    Dragonstomper

  • Topic Starter
  • 746 posts
  • Location:The Great White North

Posted Wed Sep 5, 2018 2:04 PM

Here is a much faster version of your PRNG-testing program that avoids copying the screen from VRAM to CRAM every iteration:

\ Random Screen Fill that avoids using SCAN and whole-screen copying.

VARIABLE BLANK_CNT
VARIABLE ITERATIONS
VARIABLE DUPLICATES 

: WAIT-KEY   BEGIN KEY? UNTIL ;

: UNTILFULL ( -- )
      PAGE
      C/SCR @ BLANK_CNT !     \ initialize to screenfull of blanks
      DUPLICATES OFF
      ITERATIONS OFF
      BEGIN
          C/SCR @ RND DUP VC@
          DUP BL =
          IF
               DROP
               [CHAR] A SWAP VC!
               -1 BLANK_CNT +!      \ decrement blank count
          ELSE
               1+ SWAP VC!
               1 DUPLICATES +!
          THEN
         1 ITERATIONS +!
         BLANK_CNT @ 0=          \ did we hit all the blanks?
      UNTIL
      BEEP
      WAIT-KEY
      PAGE ." Random Screen Fill"
      CR
      CR ITERATIONS @ U. ." iterations"
      CR DUPLICATES @ U. ." duplicates"
;

and here it is converted to fbForth:

Spoiler

 

...lee

 

Not only is it faster, it seems to be accurate.  Back to people management I go.  :_(

Thanks Lee.



#178 TheBF OFFLINE  

TheBF

    Dragonstomper

  • Topic Starter
  • 746 posts
  • Location:The Great White North

Posted Wed Sep 5, 2018 2:30 PM

Here is the little program I use to check the repetition frequency of the PRNG under test.

It uses a 32bit counter because I didn't want to have an overflow.

CREATE DCNT   0 , 0 ,   \ 32bit integer variable, 4 bytes

: DCNT++    ( -- ) DCNT 2@ 1 M+ DCNT 2! ;     \ incr. 32 bit counter
: CLRCNT    ( -- ) 0 0 DCNT 2! ;              \ clear counter
: UD.       ( d -- )  <# #S #>  TYPE SPACE ;  \ print double, unsigned
: .DCNT     ( -- )  DCNT 2@ UD. ;             \ print counter

: NEXTREP    \ find next repetition of Random number
        CLRCNT
        CR ." Searching for:" RNDW DUP  U.
        CR        
         BEGIN
            DCNT++
            RNDW
            OVER =
        UNTIL
        CR  ." Duplicate after "  .DCNT  ." random no.s"  ;

If this is working as I expect, which under Lee's watchful eye am learning is not always the case, :) 

the TI Forth PRNG repeats after 20,000 or so and the GForth method shows 65,536 before a duplicate.

I think using the circular shift limits the TI Forth method, but again, I don't have the math to prove my intuition.

 

Comments welcome.

 

I would never have suspected the intricacies of a PRNG were so great.

I should probably have looked up the current state of the art on testing... oh well, it's a hobby after all.



#179 TheBF OFFLINE  

TheBF

    Dragonstomper

  • Topic Starter
  • 746 posts
  • Location:The Great White North

Posted Fri Sep 7, 2018 3:13 PM

Handle Base File Access Change

 

A while back I discovered a problem with my ANS/ISO File system not selecting properly from one file handle to another file handle.

This is required for example if you wanted to copy data from one file to another file.

 

The files opened correctly, the code was selecting the correct PAB for each open file, but I was not changing the O/S address >8356

(which is called DSRNAM from what I can discover)

 

I did  a kludge fix that involved string processing (yuk!) but the real fix meant adding a new field to the standard PAB.

 

Per the E/A Manual the standard PAB in CAMEL99 Forth is:

 

byte   : opcode

byte   : flag/status

cell    : data buffer address

byte   : record length

byte  : character count (read or written)

cell   : record number

byte  : screen offset (unused)

string: filename  (byte counted string) MAX length in CAMEL99=32 bytes

 

CAMEL99 now has this new field 

cell: DSRNAM   

 

The magic number that the O/S needs in DRSNAM to "select" a file, is the VDP address of the filename string after the  '.' character.

I was doing that correctly  when I created the PAB, but the number was not stored anywhere for fast retrieval.

 

In version 2.0.23 I have added a DSRNAM field to the PAB after the file name buffer. 

When selecting via file handle it is now very simple.

The word SELECT takes the handle and  "selects" the correct VDP PAB address from an array of PABs.

Now it also reaches into the PAB to retrieve the DSR name  and put it into >8356

 

Here is the new code for SELECT with extra comments.

: SELECT  ( hndl -- )
           DUP ?FILES                                   \ test if hndl<= max files
           ]PTAB @ DUP 0= ABORT" Cannot select hndl 0"  \ ]PTAB is array of PAB addresses
           ^PAB !                                       \ store correct PAB in pab pointer 
           [PAB DSRNAM] V@  DSRNAM !  ;                 \ fetch dsrnam from VDP ram, store in CPU ram 

Once I made the decision to alter the PAB structure the rest went quickly.

 

It will get up on GITHUB this weekend.



#180 TheBF OFFLINE  

TheBF

    Dragonstomper

  • Topic Starter
  • 746 posts
  • Location:The Great White North

Posted Wed Sep 12, 2018 2:25 PM

The Ultimate Cure for Your Insomnia

 

The latest version of the manual for Camel99 Forth is up on GitHub.

https://github.com/b...mers Rev1.3.pdf

 

The improvements include

  • Reorganizing sections for clarity
  • Expanded discussion on programming tools and disk utilities
  • Expanded discussion on using Forth Assembler including the use of BLWP from within Forth
  • Expanded discussion on multi-tasking

And oh so much more!

 

It's now 158 pages of fun fun fun!   ;-)

 

That is all.


Edited by TheBF, Wed Sep 12, 2018 2:26 PM.


#181 TheBF OFFLINE  

TheBF

    Dragonstomper

  • Topic Starter
  • 746 posts
  • Location:The Great White North

Posted Mon Sep 17, 2018 4:08 PM

30 Tasks + the Console Running on Camel99 Forth

I have wanted to trying this for a while on this little system.  I always envisioned the multi-tasker to be able to "spawn"  a new task.  That means I need to be able to allocate some memory from somewhere and then setup a new task in that memory block.  In version 2 of the CAMEL99 I wanted a low overhead way to use the low-memory block on the expansion RAM so I did what any Forther would do and turned the memory into a manually controlled stack of memory.  :)

 

Anytime you need some memory you say " n MALLOC" where n is the number of bytes.  Later you can say "n MFREE" to give it back. It's very crude. You can't remove a memory block from the middle of the stack alhough you could re-used it.  Anyway, armed with that I realized I could create as many tasks as the low-memory could hold, up until it smashed into the Forth stacks at the top of low memory.

I can get a maximum of 32.

 

Each task needs 192 bytes and that is called the USER Area.  The constant USIZE returns 192

  • FORK is routine that simply copies the current USER area into a new USER area and sets the 2 stack pointers to the new space.
  • ASSIGN give the task the address of a program to run
  • WAKE sets a flag in the task that it should run.

 

Using the simple MALLOC the routine to SPAWN a new task, with a program passed to it to run, is very simple.

\ create a task in heap, fork it, assign Execution token and run it
: SPAWN  ( xt -- )  
         USIZE MALLOC ( pid) 
         DUP >R FORK
         R@ ASSIGN  
         R> WAKE ;

The spoiler has the demo code which just puts the numbers >DEAD >BEEF on the local stack of each task and increments a global variable called X.

 

The 30tasks video shows the me typing at the console starting everything up and killing it all and doing it again.

You can see how much the console slows down with 30 jobs running.

Each task is set to behave "NICE"ly and not hogging the system too much.  (NICE is a UNIX command that nobody uses. You lookup why) ;)

 

The lowmemtasks video just shows the CLASSIC99 debugger window with all the DEAD BEEF numbers going onto the local stacks of each task as I scroll through a few task blocks.

 

It's a useable thing now.

 

Spoiler

 

Attached Files



#182 TheBF OFFLINE  

TheBF

    Dragonstomper

  • Topic Starter
  • 746 posts
  • Location:The Great White North

Posted Thu Sep 20, 2018 8:29 AM

Everybody Wants to Write to the Screen

 

One of the challenges in a multi-tasking system (or a multi-user system) is how to manage screen control.  In a single task system you need a ROW variable and a COL variable to keep track of the cursor position. That works great until some other process on the machine tries to write to another location than your task is using.  

 

So how do we keep this straight between tasks?

 

In Forth the answer is "USER variables" .  USER variables are in a special memory block called the USER area.

The USER area is unique for each task. In CAMEL99 Forth this includes the VROW and VCOL variables for VDP x,y control as well as the number BASE variable and a bunch of compiler control variables.

 

So USER variables let each task have it's own copy of where it is using the VDP screen. 

That's good, but it does not solve the problem of converting digital numbers into text.

 

Forth typically has a small buffer called the HOLD buffer where numbers are converted. It is placed just past the end of the dictionary typically,  in unused memory. This means if you add new words to the dictionary the HOLD buffer moves up too.  All good until two tasks try to convert a number to text at the same time!

 

The solution in CAMEL99 Forth is to add a USER variable called TPAD (task's pad) that *sets the offset from the end of the dictionary where any tasks places it's hold buffer and also a general purpose buffer call PAD.

(not to be confused with the TI-99 scratch-pad memory) 

 

*Credit where it's due: This idea comes from HsForth written by the late James Kalihan of Springboro Ohio

 

So for each task we must set the offset a little bigger so each task has it's own little space.  This could be automated, but on a little machine like the TI-99 it's best that you know where everything is. ;-)

 

The demo code shows how that is done by creating 3 timers and "PULSE" monitor.  Each timer writes to the screen but in different a radix. (base 10,16 and 2)

 

The PULSE monitor shows how long it is taking for the multi-tasker to get around the entire set of tasks.

 

Part of the reason it's slow is because the KSCAN code in ROM takes about 1.2 mS so that limits how fast we can get around the loop of tasks immediately. If we ran a program in the console things speed up nicely. Or writing a KSCAN that is properly cooperative would be the ideal solution. (remember this tasker is cooperative not preemptive so programs have to be friendly)

 

The Demo video shows you how to control the tasks.

Also notice that when we compile a source code file, the counters stop. This is by design because having multiple tasks compiling into the same dictionary would be fatal!  Adding a  SAMS memory page for each task would fix that but it's just an idea at the moment.

 

Spoiler

Attached Files


Edited by TheBF, Thu Sep 20, 2018 8:36 AM.


#183 TheBF OFFLINE  

TheBF

    Dragonstomper

  • Topic Starter
  • 746 posts
  • Location:The Great White North

Posted Fri Sep 21, 2018 8:50 PM

How to Do Conditional Compilation in a Tiny Machine

 

I was quite happy with the progress on CAMEL99 Forth V2 once I got the file access working.

The original vision for CAMEL99 was to make an 8K kernel that could extend itself with source code files.

I thought is was quite cool to have the source code in text files rather than blocks  

 

So far so good but what about dependencies? 

What if you try to load a file that requires another file's words to be loaded first?

 

Standard Forth has a set of words for this called  [IF]   [ELSE]  [THEN]. You can use these words in your source code like this:

 

( Remember Forth's 'then' means  "then go on with the rest of the program" :-)

TRUE [IF]   INCLUDE SOURCE1.FTH    
    [ELSE]  INCLUDE SOURCE2.FTH    
    [THEN] 

This a very nice solution however to implement them takes quite a bit of code. Here are the official definitions

: [ELSE] ( -- )
    1 BEGIN                                          \ level
       BEGIN BL WORD COUNT DUP WHILE                  \ level adr len
         2DUP S" [IF]" COMPARE 0= IF                  \ level adr len
             2DROP 1+                                 \ level'
          ELSE                                        \ level adr len
            2DUP S" [ELSE]" COMPARE 0= IF             \ level adr len
                2DROP 1- DUP IF 1+ THEN               \ level'
            ELSE                                      \ level adr len
                S" [THEN]" COMPARE 0= IF              \ level
                   1-                                 \ level'
               THEN
             THEN
          THEN ?DUP 0= IF EXIT THEN                   \ level'
       REPEAT 2DROP                                   \ level
   REFILL 0= UNTIL                                   \ level
    DROP
; IMMEDIATE

: [IF]   ( flag -- )  0= IF POSTPONE [ELSE] THEN ; IMMEDIATE
: [THEN] ( -- ) ; IMMEDIATE

This uses string comparisons so I would need to add COMPARE to the system, and it uses REFILL, which I have not implemented per the new Forth 2012 architecture.

altogether it would consume over 100     244 bytes of precious TI-99 memory.

 

So all that to say I am going to use two non-standard words called  NEEDS  and FROM. 

The entire code adds only 36 bytes to the system and it solves my problem nicely. In fact it adds a feature that [IF] [THEN] does not have.

You specify the significant word needed with NEEDS.

: NEED  ( -- ?)  BL WORD FIND NIP  ;

: FROM   ( ? -- ) 
         BL PARSE-WORD ROT ( addr len ? --)
         0= IF  INCLUDED
         ELSE   2DROP
         THEN ; 

With these two words loaded all you do is:   

  

NEEDS HCHAR   FROM DSK1.GRAFIX.F

NEEDS DUMP     FROM  DSK1.TOOLS.F

NEEDS RND        FROM DSK1.RANDOM.F

 

The hard part now is going through the source files and putting the correct conditional information in them. :-)


Edited by TheBF, Sat Sep 22, 2018 5:30 AM.


#184 Willsy OFFLINE  

Willsy

    River Patroller

  • 3,079 posts
  • Location:Uzbekistan (no, really!)

Posted Sat Sep 22, 2018 5:07 AM

That's really nice. You'll need a NEEDS ALL FROM ... too to make it really shine.

Top job!

#185 TheBF OFFLINE  

TheBF

    Dragonstomper

  • Topic Starter
  • 746 posts
  • Location:The Great White North

Posted Sat Sep 22, 2018 5:29 AM

That's really nice. You'll need a NEEDS ALL FROM ... too to make it really shine.

Top job!

 

 

Thanks. I wish it was that smart. :-)

The NEEDS word is just checking the system for one word as a proxy for the file being loaded or not.

I always liked the import directive from MODULA that let you do what you are referring to; pulling in specific words from a library.

That's a bit bigger job for Forth, but it could be done. Probably makes more sense for a cross-compiler.



#186 TheBF OFFLINE  

TheBF

    Dragonstomper

  • Topic Starter
  • 746 posts
  • Location:The Great White North

Posted Mon Sep 24, 2018 12:39 PM

CAMEL99 Forth is Starting to Look like a Shell

 

I have decided that adding the NEEDS/FROM utility is such a game changer that the system needs a revision number.  So I will be releasing V2.1.0 this week with notes on the changes.

 

One big change is the NEEDS/FROM words are loaded first in the START configuration file because it is so handy.  To demonstrate what I mean I wrote a simple file copy utility. In the video you can see that when I load the COPY utility the ANSFILES code is pulled into the system, but when I load the MORE utility, to see file contents, it loads very quickly because the ANSFILES code is already compiled in the system and so only the MORE utility program is compiled.

 

The file COPY code is starting to look very ANS/ISO Forth now.  An interesting thing to note is how we parse the input file names with no temporary buffer. This is done by using "stack strings" that are just the address and length of a string residing on the data stack.  The word FNAME cuts the text input, delimiting on a space character using PARSE-NAME.  Each time it runs, it leaves the data for the string on the data stack. Even though both file name strings are in a single terminal input buffer, it is converted into two separate "stack strings" on the data stack. Neat!

 

Spoiler

Attached Files



#187 TheBF OFFLINE  

TheBF

    Dragonstomper

  • Topic Starter
  • 746 posts
  • Location:The Great White North

Posted Thu Sep 27, 2018 7:17 PM

The watchword of the day is read the manual carefully.

 

I finally got CAMEL99 Forth transferred to floppy disk with the help of Karsul's SAMS card and Magic FM from Arcadeshopper.

It loaded and ran great as long as I did not try to access a file. :(

 

There is a little paragraph in the E/A Manual, on page 404, that states: 

>834A - >836D Used as a stack area by the interpreter, floating point routines,
and DSR routines. Unless console routines are called by your
assembly language program, this area is available for use.

Notice the reference to DSR routines.  I had protected 2 addresses in this area, but used the rest of the space.

 

Well my clever idea of placing an array of user variables in the scratch-pad RAM was probably a bad idea.

I suspect that is why my DSR routine works on Classic99 which does not emulate the DSR code, but fails on real iron.

 

I am going to re-org the user variable list to avoid >834A..>836D and see what happens.

The cross-compiler should make that not too painful. (brave words) :-)



#188 mizapf ONLINE  

mizapf

    River Patroller

  • 3,383 posts
  • Location:Germany

Posted Fri Sep 28, 2018 1:34 AM

You should probably consider a test run on MAME when working with DSRs, since it uses the original ROMs.

#189 Tursi OFFLINE  

Tursi

    Quadrunner

  • 5,283 posts
  • HarmlessLion
  • Location:BUR

Posted Fri Sep 28, 2018 3:05 AM

You should probably consider a test run on MAME when working with DSRs, since it uses the original ROMs.

 

So will Classic99 if you tell it to. It's not in the menu yet, but it's no harder than configuring MAME. Configure a disk image drive on DSK1, DSK2, or DSK3 normally, then exit the emulator. Edit Classic99.ini and change the "Type" for that drive from '2' to '3', and it will access the image using the TI DSR. I don't recommend this for general use yet only because it doesn't support the usual range of Classic99 functions yet - it runs the DSR and nothing more. (So, 180k disk image maximum, DSK1-3 only).

 

In additional when using the default DSR, Classic99 will emit debug in the logs if you stomp on the reserved DSR memory locations, telling you which locations you stomped on and what address the code that did the stomping was at. The debugger is there to help, particularly large amounts of debug are provided for disk access.

 

The following tests are made for every disk access to the Classic99 DSR (and will debug a warning if violated):

-verifies that the workspace is GPLWS at >83E0

-verifies that the CRU base is stored at >83D0

-verifies (loosely) that the DSR entry point was stored at >83D2

-validates the PAB address is greater than >000D (otherwise there isn't room for the PAB when you backup to the start) - this one will refuse to execute the DSR as well

-validates that the DSR name length is 7 or less (this is the "DSK1" part and there are hard-coded limits)

-validates that the PAB opcode is valid

-checks several places if the PAB overruns the top of VRAM

-warns on filenames (including path) longer than 32 characters

-checks the top of VRAM pointer at >8370 for the disk buffers

-verifies the first four bytes of the VRAM buffer header are intact (except LSB of top of RAM pointer)

-verifies the number of files value is less than 9 (maximum supported by Classic99's DSR, more means corruption happened) - there is an option to breakpoint if these values are corrupt and a real TI controller card would crash on the call

 

Then AFTER the DSR call happens, it again checks the pointer at >8370, and validates the first four bytes (including the VRAM top LSB skipped above - that's a bug that won't be an issue in Classic99 yet, I was envisioning CF7 support for that one...) It also checks to see if any user code touched any of the actual buffers.

 

I don't know if that covers the case you ran into, I've been adding cases as I hit them myself. In general though, if Classic99 doesn't throw any warnings in the debug log, you have a pretty good chance of it working. If you test with the TI DSR option, then you're running the actual DSR code. That said, always good to test on multiple platforms, particularly if you are uncertain. The above options are meant to try and help you understand WHY it doesn't run.



#190 TheBF OFFLINE  

TheBF

    Dragonstomper

  • Topic Starter
  • 746 posts
  • Location:The Great White North

Posted Fri Sep 28, 2018 7:08 AM

You should probably consider a test run on MAME when working with DSRs, since it uses the original ROMs.

 

Thanks!



#191 TheBF OFFLINE  

TheBF

    Dragonstomper

  • Topic Starter
  • 746 posts
  • Location:The Great White North

Posted Fri Sep 28, 2018 7:10 AM

 

So will Classic99 if you tell it to. It's not in the menu yet, but it's no harder than configuring MAME. Configure a disk image drive on DSK1, DSK2, or DSK3 normally, then exit the emulator. Edit Classic99.ini and change the "Type" for that drive from '2' to '3', and it will access the image using the TI DSR. I don't recommend this for general use yet only because it doesn't support the usual range of Classic99 functions yet - it runs the DSR and nothing more. (So, 180k disk image maximum, DSK1-3 only).

 

In additional when using the default DSR, Classic99 will emit debug in the logs if you stomp on the reserved DSR memory locations, telling you which locations you stomped on and what address the code that did the stomping was at. The debugger is there to help, particularly large amounts of debug are provided for disk access.

 

The following tests are made for every disk access to the Classic99 DSR (and will debug a warning if violated):

-verifies that the workspace is GPLWS at >83E0

-verifies that the CRU base is stored at >83D0

-verifies (loosely) that the DSR entry point was stored at >83D2

-validates the PAB address is greater than >000D (otherwise there isn't room for the PAB when you backup to the start) - this one will refuse to execute the DSR as well

-validates that the DSR name length is 7 or less (this is the "DSK1" part and there are hard-coded limits)

-validates that the PAB opcode is valid

-checks several places if the PAB overruns the top of VRAM

-warns on filenames (including path) longer than 32 characters

-checks the top of VRAM pointer at >8370 for the disk buffers

-verifies the first four bytes of the VRAM buffer header are intact (except LSB of top of RAM pointer)

-verifies the number of files value is less than 9 (maximum supported by Classic99's DSR, more means corruption happened) - there is an option to breakpoint if these values are corrupt and a real TI controller card would crash on the call

 

Then AFTER the DSR call happens, it again checks the pointer at >8370, and validates the first four bytes (including the VRAM top LSB skipped above - that's a bug that won't be an issue in Classic99 yet, I was envisioning CF7 support for that one...) It also checks to see if any user code touched any of the actual buffers.

 

I don't know if that covers the case you ran into, I've been adding cases as I hit them myself. In general though, if Classic99 doesn't throw any warnings in the debug log, you have a pretty good chance of it working. If you test with the TI DSR option, then you're running the actual DSR code. That said, always good to test on multiple platforms, particularly if you are uncertain. The above options are meant to try and help you understand WHY it doesn't run.

As always this is super help.  Thanks!

 

I will make use of this. It's way simpler than loading an image on a floppy and debugging word by word in Forth, with limited tools... because I have no file support to load them.

 

B







Also tagged with one or more of these keywords: Camel99, Forth, Concatentive Programming, ANS Forth

0 user(s) are browsing this forum

0 members, 0 guests, 0 anonymous users