DZ's IntyBASIC Feature Wish List

[+DZ-Jay]

During the past couple of years, I've been helping some very talented people with performance optimizations in their IntyBASIC programs.  Mostly, this involves off-boarding procedures onto Assembly Language subroutines.  I've also tried to provide "glue code" to integrate some of my existing framework modules (honed down throughout the years with the help of many others), such as the Intellivision Music Tracker and the Pseudo-Random Number Generator libraries.

 

However, in the course of this work, I've come across some limitations in the way that IntyBASIC integrates Assembly Language code, that have made my job much harder than I wish.

 

None of this takes away from the fantastic work that @nanochess has done on the language and the compiler.  Indeed, it is an excellent tool with many, many strengths.  So the issues I've encountered are not so much shortcomings or drawbacks, but rather pain points at the very edges of its capabilities.  I guess you could say that I've been trying to "abuse" instead of "use" the compiler, in ways that I should not.

 

Nonetheless, I present below my humble wish list of features I would like to see added to IntyBASIC in the hope of increasing its versatility and flexibility, and enhancing its integration with lower-level libraries.  I admit that I am not really an IntyBASIC programmer, and I do not know to what extent these features would be used by others; but I figured I may as well ask nicely -- and who knows, if they are simple enough, Óscar may implement them. 

 

My IntyBASIC Feature Wish List
 

• Access to IntyBASIC constants from Assembly Language.
  This could be as simple as munging the names in the same way as is done for variables, and injecting them into the generated assembly code as symbols.
   
• Conditional compilation or logical expression evaluation at compile time.
  The IntyBASIC compiler performs some internal processing during compilation to decide what additional modules or subroutines to add to the final assembly code, but does not give access to this mechanism.  Also, DEF FN user macros perform constant expression evaluation and text substitution at compile time, but logical expressions result in actual generated code (as could be expected).
  
  This prevents such macros from conditionally including one out of possibly many specific statement blocks depending on purely constant values given in the IntyBASIC source; similarly to the way that the compiler itself selects, say, what code to generate depending on the arguments of the PRINT statement.
   
• Assembly code in user macros.
  Right now, DEF FN user macros accept any valid IntyBASIC code -- except the ASM directive.  I'm sure there are valid reasons for this, but I have found that this one limitation prevents a more natural interface between the IntyBASIC and Assembly boundaries.
  
  Specifically, because IntyBASIC does not support a user-accessible pre-processor, one must rely on assembler macros to perform compile/assembly-time processing and conditional code generation.  However, doing so makes them automatically off-limits from the IntyBASIC DEF FN construct, which results in a more convoluted or stilted integration.
  
  Ideally, I would like to define Assembly Language macros, and then wrap them with IntyBASIC user macros, in order to gain access to the parameter passing and results value mechanisms offered by the compiler.
   
• Ability to inject code at the end of the program.
  I know this may be a bit esoteric, and you may even think it's completely unnecessary, but I entreat you to hear me out.  Very rarely, but sometimes, it may be useful to override some functionality of the IntyBASIC framework encoded within the "intybasic_epilogue.asm" file.  Of course, this could be done by hacking the epilogue module itself, but then each project would require its own copy of it, and sharing common code across programmers would require them to be aware of such caveat, etc.
  
  One alternative to this -- which I've employed in the past on assembly language programs -- is to add a patch at the very end of the code, which re-points the program counter and interjects with replacement code.  Obviously such a mechanism is not to be used lightly, but in my experience it has proven to be very useful.
  
  However, because the IntyBASIC compiler always follows the user's program code with the epilogue module, this is not possible.  Perhaps some sort of special marker could be included that states "add the following at the very end."
   
• Global access to fully-qualified local labels.
  This is another advanced feature, but I think it will prove very useful.  Right now you can define local labels with a "." prefix, but these are private to their respective parent label.  This limits their practicality and utility.  It would be better if we could access the local labels by fully qualifying them.  For example:

  ' Table of level parameters
  LevelData:
  .L1:
    Data 1, 1, 1, 1
  
  .L2:
    Data 2, 2, 2, 2
  
  .L3:
    Data 3, 3, 3, 3
  
  ' ...
  
  ' Get a pointer to a specific level table
  #LevelParams = VarPtr LevelData.L2

  This puts the compiler usage of local labels in line with how the assembler treats them.
  
  The way it works right now, you have to specify individual global labels for each sub-table, even though they are all related -- or worse, you will have to use pointer arithmetic to compute individual sub-table pointers by keeping track of the size of each table.  Access to local labels using fully-qualified names offers the best of both worlds:  Ability to have local labels, and the ability to keep the scope of your related labels constrained.
   

• Support for function macros in constant definitions.
  Function macros ("DEF FN") that evaluate to a constant expression should be legal as "CONST" values.  Right now, the compiler rejects anything that is not a literal constant expression, a literal value, or another constant.

 

That's it for now.  I hope that @nanochess at least considers these suggestions.  And if there are better ways to accomplish some of them, then I'd be very happy to learn them. 

 

     -dZ.

Edited July 10, 2022 by DZ-Jay

[Peripheral]

I'll throw my 2-cents in here.

 

Though like @DZ-Jay, I do not use IntyBASIC much, so take these with a grain of salt.

Some of these I've mentioned in other threads or private messages to @nanochess.

 

I'll also reiterate DZ-Jay's sentiment that this is in no way a knock on IntyBASIC.

It's just my desire to make IntyBASIC more like C/C++

The value of IntyBASIC has been clearly demonstrated through all the homebrew releases.

 

1. Wide Bitmaps, so that large pictures can be drawn more naturally.  e.g.

    BITMAP   "...##.....####.."

    BITMAP   "..#..#....#..#.."

    BITMAP   "..#..#....#..#.."

    BITMAP   "..####....#.#..."

    BITMAP   "..#..#....#.#..."

    BITMAP   "..#..#....#..#.."

    BITMAP   "..#..#....#..#.."

    BITMAP   "..........####.."

 

2. Logical && and || for conditionals, complete with short-circuit evaluation, as in C/C++.

    IntyBASIC's current AND and OR are bitwise operations, and to my knowledge,

    conditional expressions are fully evaluated even when the eventual true/false result

    could have been determined.

 

3. Left and right shifts (a << b), (a >> b), (a >>> b).

    ">>" is arithmetic,  ">>>" is logical (i.e., no sign bit propagation)

   I would define these as 16-bit operations, and for performance reasons,

   treat the result as undefined if the shift count is outside the range 0..15.

 

4. Add PUSH and POP.

    This would allow localized saving/restoring of global variables, so they can

    be reused instead of having to declare new variables.  Of course, this puts

    an onus on the programmer to ensure these are safely paired, and may be

    considered too low level for BASIC.  But they're trivial to implement, and

    so might be a pragmatic choice over other alternatives.

    One alternative is to support local variables, but I suspect that would get

    expensive as you would likely have to reserve a register to be the stack frame pointer.

 

5. Nested includes.  In my first attempt at using IntyBASIC I created a header file,

    say "mydefs.bas", which included "constants.bas", and I included "mydefs.bas"

    in my main program.  Quickly discovered that was not supported.  This is not

    a major nuisance given the relatively small size of IntyBASIC programs,

    but was a bit surprising.

 

6. Beef up DEF FN's in constants.bas to put parenthesis around parameters.

    There are a few cases where the parentheses are missing, which could cause

    subtle operator precedence errors.  For example setspritey(aSpriteNo,aYPosition)

    contains (aSpriteNo+8) in its body.  If invoked as setspritey(A OR 9, B), that

    becomes (A OR 9 + 8 ) which is (A OR 17), when it should have been ((A OR 9) + 8 ).

    This is not a compiler issue, just a header file issue.  And if you do fix these (as I

    did in my local copy), you  might notice some instances where the generated code

    is less efficient, as the extra parentheses inhibit some IntyBASIC pattern matching.

    So the optimizer might need some beefing up.

[+cmadruga]

Local variables would be helpful.

 

Plus an easier way to monitor segment/page utilization. Yes, cart.mac and etc exists. Need easier.

Edited July 19, 2021 by cmadruga

[Kiwi]

56 minutes ago, cmadruga said:

Plus an easier way to monitor segment/page utilization. Yes, cart.mac and etc exists. Need easier.

This is true.  Usually I have to resort to use .cfg and use a calculator to measure my segments periodically.  And also refer to cart.mac to determine what segment are open to use. 

The 2 features that I really want is to pass an argument via gosub like DoThis(9,2).

And Inc++;  instead of Inc=Inc+1.  Less typing.

Yeah, it's C, but those are the features I want.  But I'm cool with IntyBASIC the way it is now.

Edited July 19, 2021 by Kiwi

[+DZ-Jay]

Something strange I just noticed:  When compiling conditional expressions, IntyBASIC uses the instruction "BNE" for "Branch on Not-Equal."  However, this instruction is not documented anywhere! (Well, perhaps in datasheets for other CPUs, because it is a common mnemonic.  I also see that @intvnut has employed it sporadically in some of his SDK-1600 examples, especially for loop control.)

 

Not that it matters too much, since it works as expected.

 

Nonetheless, in all the CP-1610 datasheets and in all the as1600 documents, the instruction is referred to as "BNEQ" as a complement to "BEQ" for "Branch on EQual."  The latter itself is an alias to "BZE" for "Branch on ZEro," which itself is complemented by "BNZE" (Branch on Not-ZEro).

 

So, I propose either to add "BNE" to all datasheets, specifications, and other documentation as an official mnemonic equivalent to "BNEQ"; or for @nanochess to change the behaviour of IntyBASIC to use "BNEQ" rather than "BNE" just to be safe and ensure it adheres to the official specifications.

 

It's just a suggestion, in the end, everything works as expected, and "BNE" is very common and clear enough to understand.   It just took me by surprise when I went to check the datasheet to see how many cycles it takes, and didn't find it.

 

     -dZ.

Edited October 16, 2021 by DZ-Jay

[+nanochess]

I was going from 6502 to Intellivision. So probably tried BNE from memory. Glad it work on the first try ?

[+DZ-Jay]

1 minute ago, nanochess said:

I was going from 6502 to Intellivision. So probably tried BNE from memory. Glad it work on the first try ?

Yeah, me too.  That's the thing, it did not look weird at all for me (and I suspect the same for @intvnut, which is why it seems to be sprinkled haphazardly and inconsistently in some of his code).  Still, it seems to be baked into the assembler already, so that's good.  I wonder if "BZ" and "BNZ" are also secretly supported, which is how I remember "BZE" and "BNZE" from other CPUs.

 

     -dZ.

[+DZ-Jay]

@nanochess, while I have your attention, here are a few other potential improvements to the compiler, although I do not know how feasible it would be to implement them.

• Track the state of R5 upon entering a procedure, and if it never changed, avoid the "BEGIN/RETURN" (20 cycles) and replace with a single "JR R5" (7 cycles).  That's a rather big improvement when considering a complex program that may break-up its logic into multiple subroutines.
   
• When an expression in an "IF" clause is evaluated and then used within the conditional block, it is used directly from the target register in the "IF" block, but recomputed in the "ELSEIF" or "ELSE" blocks.  Below is a trivial example to illustrate:

      If (I = 10) Then
          J = I
      Else
          K = I
      End If

  That code results in the following assembly:

      ; If (I = 10) Then
      MVI     var_I,  R0
      CMPI    #10,    R0
      BNE     T7
  
      ; J = I
      MVO     R0,     var_J
      B       T8
  
  T7:
      ; K = I
      MVI     var_I,  R0          ; \_ R0 is guaranteed to still have var_I, 
      MVO     R0,     var_K       ; /     since we know where we came from.
  
  T8:

  
  The same thing happens within an "ELSEIF" expression:  any variables or sub-expressions already resolved, are re-computed.
   

• Consider short-circuit evaluation of Boolean expressions.
   
• Consider treating SIGNED and UNSIGNED as expression modifiers, rather than declarative statements.  Right now, the compiler supports changing the "sign-ness" of a variable at any point, so I can do something like:

     ' ...
  
     ' Treat I as signed
     SIGNED I
  
     IF (I = -127) THEN
       ' do something
     ELSE
       ' do something else
     END IF
  
     ' Resume treating I as unsigned
     UNSIGNED I

  
  However, it could be more expressive to use it like this

     ' ...
  
     ' Temporarily treat I as signed
     IF (SIGNED(I) = -127) THEN
       ' do something
     ELSE
       ' do something else
     END IF

  
   
• Track the use of a "FOR" loop control variable within its block, and if it is not accessed, consider counting down to zero to avoid an "INCR/CMPI/BGT" sequence, and instead use a "DECR/BNZE" sequence.  Bonus points for avoiding updating the control variable in memory and just using a register.

 

I know that these are not trivial problems to solve, and IntyBASIC does a great job at generating good assembly code for the most part.  However, it would be great if the optimizer could maximize the use of registers even more aggressively to reduce the constant access to memory.  I find that these can seriously add up to a huuuuuuge penalty.

 

    Thanks!

    -dZ.

 

Edited October 16, 2021 by DZ-Jay

[+DZ-Jay]

Here's another trivial example of point #2.  The following IntyBASIC code:

    I = J + K
    IF (I = 10) THEN
        I = 0
    END IF

Generates the following assembly code:

    ; I = J + K
    MVI     var_J,  R0       ; \
    ADD     var_K,  R0       ;  > Compute "I" in R0 and store it in var_I
    MVO     R0,     var_I    ; /

    ; IF (I = 10) THEN
    MVI     var_I,  R0       ; But var_I is already in R0!!!
    CMPI    #10,    R0
    BNE     T7

    ; I = 0
    CLRR    R0
    MVO     R0,     var_I

    ; END IF
T7:

 

Notice that right after computing the value of "I" in R0 and storing it in "var_I," it goes ahead and fetches it anyway from memory when evaluating the "IF" conditional expression.

 

Once again, I do not know if the compiler design supports a means to change this behaviour, but it would be a significant improvement if it could.

 

Of course, now that the compiler is mature and stable, we're just nibbling at the edges of its performance.  It's a very good sign. :)

 

   -dZ.

Edited October 17, 2021 by DZ-Jay

[+DZ-Jay]

And another silly suggestion:  support unary positive sign, just for the sake of convention and symmetry with unary negative signs.  In any other programming language "+1" is the same as "1," but IntyBASIC fails with a "bad syntax for expression" error.

 

    -dZ.

[+DZ-Jay]

Here's another small improvement:  Comparisons against zero could use "TSTR" instead of "CMPI #0", which save 2 CPU cycles and yields similar results.

 

Example #1:

  ; If (A < 0) Then
  MVI  var_A, R0
  TSTR R0
  BPL  @@T1
  ; "IF" block

@@T1:
  ; "ELSE" block

 

Example #2:

  ; If (A > 0) Then
  MVI  var_A, R0
  TSTR R0
  BLE  @@T1
  ; "IF" block

@@T1:
  ; "ELSE" block

 

Example #3:

  ; If (A <= 0) Then
  MVI  var_A, R0
  TSTR R0
  BGT  @@T1
  ; "IF" block

@@T1:
  ; "ELSE" block

 

 

[+DZ-Jay]

I've updated the first post with a new entry in the "wish list":  global access to local labels via fully-qualified names.

[+DZ-Jay]

Updated the first post with a new entry for "Support for function macros in constant definitions."

 

   -dZ.

[+DZ-Jay]

@nanochess, here's another minor* optimization that may not be too complicated** for the compiler to do:

 

• Because there is no short-circuit evaluation of Boolean expressions, I find myself nesting my conditional statements rather deeply.
• Most of the time, the conditional statements fully encapsulate the logic, so that there is nothing but IF/ELSE/END IF blocks.
• When that happens, the inner-most blocks must unravel the nest -- one block at a time -- through every single ELSE or END IF with a "goto" ("B" branch operation) to the ELSE or END IF of the next block.

Perhaps it's a bit of a stretch, but it would be ideal if the compiler could detect sequential branches like this and coalesce them into a single one.

 

Here's a trivial example to illustrate.  The following IntyBASIC code:

Dim I, J, K, Foo

If (I = 0) Then
    If (J = 0) Then
        If (K = 0) Then
            Foo = 0
        Else
            Foo = K
        End If
    Else
        Foo = J
    End If
Else
    Foo = I
End If

Print Foo

 

compiles into the following assembly code (formatted and annotated for clarity):

        MVI     var_I,  R0          ; If (I = 0) Then
        TSTR    R0                  ;
        BNE     T7                  ;

        MVI     var_J,  R0          ;     If (J = 0) Then
        TSTR    R0                  ;
        BNE     T8                  ;

        MVI     var_K,  R0          ;         If (K = 0) Then
        TSTR    R0                  ;
        BNE     T9                  ;

        MVO     R0,     var_FOO     ;             Foo = 0
        B       T10                 ;         Else
T9:
        MVI     var_K,  R0          ;             Foo = K
        MVO     R0,     var_FOO     ;

T10:                                ;         End If
        B       T11                 ;     Else
T8:
        MVI     var_J,  R0          ;         Foo = J
        MVO     R0,     var_FOO     ;

T11:                                ;     End If
        B       T12                 ; Else
T7:
        MVI     var_I,  R0          ;     Foo = I
        MVO     R0,var_FOO          ;

T12:                                ; End If
        MVI     var_FOO,R0          ; Print Foo
        MVI     _screen,R4          ;
        MVO@    R0,     R4          ;
        MVO     R4,     _screen     ;

 

Note that the end of the inner block ("If (K = 0) Then") branches directly to T10, which is the END IF of its most immediate outer block; from there it branches directly to T11 , which is the END IF of the next outer block; and there it just branches to T12 to exit the entire thing:

• Foo = 0
•     B T10
•     B T11
•     B T12
• <out>

That's 9 cycles for each branch, a total of 27 cycles, taking the place of a single one.

 

None of this is to say that this behaviour is wrong.  Obviously, the case described above is a special one because there is nothing following the inner most block, and so the END IF of a block coincides with the end of that conditional path.  The IntyBASIC behaviour is correct in order to support the general case when additional code may follow.

 

Nevertheless, I find this special case to be a common pattern (at least in my case), especially in light of a lack of short-circuit evaluation of complex Boolean expressions.

 

Accounting for this sort of pattern would require the compiler to have awareness of the surrounding code, perhaps some look-ahead capability, which may not be easy to accomplish with its current design.  Still, it may be worth exploring adding a second pass that looks at the fully generated assembly and attempts to optimize it by finding redundant or superfluous code like chained branches or contiguous access to the same memory locations, etc.

 

      Cheers!

     -dZ.

 

 

*Not minor at all.

**It will be very complicated.

Edited August 30, 2022 by DZ-Jay

[artrag]

I second this request:

• Access to IntyBASIC constants from Assembly Language.
  This could be as simple as munging the names in the same way as is done for variables, and injecting them into the generated assembly code as symbols

[+DZ-Jay]

On 10/19/2021 at 8:05 AM, DZ-Jay said:

Here's another small improvement:  Comparisons against zero could use "TSTR" instead of "CMPI #0", which save 2 CPU cycles and yields similar results.

 

Example #1:

  ; If (A < 0) Then
  MVI  var_A, R0
  TSTR R0
  BPL  @@T1
  ; "IF" block

@@T1:
  ; "ELSE" block

 

Example #2:

  ; If (A > 0) Then
  MVI  var_A, R0
  TSTR R0
  BLE  @@T1
  ; "IF" block

@@T1:
  ; "ELSE" block

 

Example #3:

  ; If (A <= 0) Then
  MVI  var_A, R0
  TSTR R0
  BGT  @@T1
  ; "IF" block

@@T1:
  ; "ELSE" block

 

 

 

This rule should include not only logical statements, but loop conditions as well.  For instance, consider the following FOR/NEXT loop, which counts down to zero:

For I = 10 To 0 Step -1
  ' ...
Next

 

That code compiles into the following assembly code (formatted and annotated for clarity):

        ; For I = 10 To 0 Step -1
        MVII    #10,    R0          ; \_ Initialize the loop control variable
        MVO     R0,     var_I       ; /
T1:
        ; ...

        ; Next
        MVI     var_I,  R0          ; Get loop control variable
        DECR    R0                  ; Decrement it by one
        MVO     R0,     var_I       ; Save new value
        CMPI    #0,     R0          ; Compare new value to final value
        BGE     T1                  ; Repeat if greater

 

It is clear that IntyBASIC understands that the loop counts down, since it translates "Step -1" into the specialized instruction to decrement by one (DECR).  It should then just be a small step from there to also understand that the end value is zero, and use the specialized instruction to test for zero (TSTR) instead of the more expensive general comparison to a constant (CMPI).

 

That code could look something like this:

        ; Next
        MVI     var_I,  R0          ; Get loop control variable
        DECR    R0                  ; Decrement it by one
        MVO     R0,     var_I       ; Save new value
        TSTR    R0                  ; Compare new value to zero
        BNEQ    T1                  ; Repeat if not zero

 

In fact, armed with that awareness of the loop control values, it can skip the test completely!  You see, the DECR instruction will automatically set the Zero or Sign CPU status flags whenever the value decrements to zero or a negative value, respectively.  That means that the outcome of the TSTR instruction is already provided as a result of decrementing.

 

The code can then be optimized to exploit that:

        ; Next
        MVI     var_I,  R0          ; Get loop control variable
        DECR    R0                  ; Decrement it by one (signaling zero or negative)
        MVO     R0,     var_I       ; Save new value
        BNEQ    T1                  ; Repeat if not zero

 

Avoiding the extra CMPI from the original code saves 8 CPU cycles -- not a bad deal, especially considering that the savings are per iteration.  Those cycles can definitely add up in a long-running loop.

 

      -dZ.

[Lillapojkenpåön]

On 9/25/2022 at 5:41 PM, artrag said:

I second this request:

• Access to IntyBASIC constants from Assembly Language.
  This could be as simple as munging the names in the same way as is done for variables, and injecting them into the generated assembly code as symbols

Maybe there allready is a known workaround? but I would do this in batari Basic

 asm

distance = 66
bytesInColumn = 2
increments = 1
scrollIn = 64
skipBytes = 0

end


 const distance = distance
 const bytesInColumn = bytesInColumn
 const increments = increments
 const scrollIn = scrollIn
 const skipBytes = skipBytes

Would that work in intybasic?

[+DZ-Jay]

6 hours ago, Lillapojkenpåön said:

Maybe there allready is a known workaround? but I would do this in batari Basic

 asm

distance = 66
bytesInColumn = 2
increments = 1
scrollIn = 64
skipBytes = 0

end


 const distance = distance
 const bytesInColumn = bytesInColumn
 const increments = increments
 const scrollIn = scrollIn
 const skipBytes = skipBytes

Would that work in intybasic?

That would not work in IntyBASIC.  The problem is that IntyBASIC constants are merely a compiler construct:  they are evaluated prior to compilation, and their ultimate value is what makes it into the compiled code.

 

In other words, no symbols are generated, and therefore there is no way for the assembler to see them.

 

Likewise, the assembler runs separately on the compiled code, and IntyBASIC just passes assembly code directly to the output stream without processing or context, so it has no awareness of assembler symbols.

 

The ideal solution would be for the IntyBASIC compiler to generate symbols in a similar manner to how it does for variables.  These can then be injected into the output stream (even though IntyBASIC may not have any further use for them), to give the assembler access to them.

   dZ.