jzIntv / SDK-1600 2018-04-21 Release

[intvnut]

All,

I've made a minor but useful update to AS1600 in this release. jzIntv should be unchanged, other than to be freshly compiled. Download the update here:

• http://spatula-city.org/~im14u2c/intv/

Updates:

• The -m (aka --show-map) flag now works again. This will print a memory map summary for your program at the end of assembly.
• The new -e flag (aka. --err-if-overwritten) flag now enables ROM overwrite checks. (More below.)
• The new directives ERR_IF_OVERWRITTEN and FORCE_OVERWRITE now provide the ability to warn about overwriting already-assembled ROM, with the ability to override the warning.

What is ROM overwrite?

 

First, the tl;dr: The most common symptom is that your program has started crashing and you don't know why. Add the '-e' flag to your assemble script, and this will become an assemble time error than a run time error.

 

Longer explanation: Consider the following simple example:

.

        ORG     $5000
        DECLE   1, 2, 3, 4, 5, 6, 7, 8
        ORG     $5004
        DECLE   8, 7, 6, 5, 4, 3, 2, 1

.

This code assembles 8 words at $5000 - $5007, and then assembles 8 more words at $5004 - $500B. The second part overwrites the ROM assembled at $5004 - $500F.

 

AS1600 currently does not warn about this. Usually, when this happens, it is an error, but occasionally it's a feature. For example, I'll often assemble a fixed pattern into memory, and then assemble my game over top of that, so I have a consistent fill value for the portions of ROM I'm not using yet.

 

The latest release of AS1600 adds directives to control this behavior. I'll just paste the documentation from jzintv/doc/utilities/as1600.txt here:

.

------------------------------------------------------------------------------
    ERR_IF_OVERWRITTEN expr     Mark code as "not intended to be overwritten"
    FORCE_OVERWRITE expr        Force code to be overwritten anyway
------------------------------------------------------------------------------

By default, AS1600 lets you assemble new code over addresses you've already
assembled code into.  That allows for some interesting tricks; however, most
often this is really an error.

The ERR_IF_OVERWRITTEN directive controls a flag that indicates whether the
code that follows may be safely overwritten.  0 means "safe to overwrite",
while 1 means "throw an error if overwritten."

>>> Note:  ERR_IF_OVERWRITTEN defaults to 0.  You can change the default at
>>         the command line by adding the flag -e or --err-if-overwritten

For example, if I wanted to fill some ROM with a fixed pattern, and then
overwrite it with final code, I could do something like this:

        ERR_IF_OVERWRITTEN 0    ; About to write some filler data
        ORG $6000
        REPEAT  4096 / 8
        DECLE   -1, -1, -1, -1, -1, -1, -1, -1
        ENDR

        ERR_IF_OVERWRITTEN 1    ; Now overwrite it with real code
        ORG $6000
        ; The following generates no errors or warnings.
fun:    PROC
        MVII    #ISR,   R0
        MVO     R0,     $100
        SWAP    R0
        MVO     R0,     $101
        ;...
        ENDP

        ; This code, however, will trigger an error, because it's overwriting
        ; the code we just assembled at 'fun':
        ORG $6000
        DECLE   12, 34          ; ERROR - ROM overwrite error on $6000 - $6001

The FORCE_OVERWRITE directive gives you the ability to forcibly overwrite
code that was previously assembled with ERR_IF_OVERWRITTEN == 1.  Revisiting
the previous example:

        ERR_IF_OVERWRITTEN 1    ; Now overwrite it with real code
        ORG $6000
fun:    PROC
        MVII    #ISR,   R0
        MVO     R0,     $100
        SWAP    R0
        MVO     R0,     $101
        ;...
        ENDP

        ; With FORCE_OVERWRITE, this code now assembles without errors.
        FORCE_OVERWRITE 1
        ORG $6000
        DECLE   12, 34

The FORCE_OVERWRITE directive is meant for use in specialized macros that
may wish to "back-patch" code that otherwise should have ERR_IF_OVERWRITTEN
turned on.  Use it sparingly.

There is no way to query the current state of ERR_IF_OVERWRITTEN or
FORCE_OVERWRITE.  If you need to track that for some reason, wrap these in
macros.

Truth table:

    ERR_IF_OVERWRITTEN  FORCE_OVERWRITE     Result on an overwrite
            off             off             No error
            off             ON              No error
            ON              off             Report an error
            ON              ON              No error

Note that ERR_IF_OVERWRITTEN tags current code to detect _future_ attempts to
overwrite, while FORCE_OVERWRITE affects the code you're assembling right now.

For example, this still generates an error, because the first DECLE was
assembled with ERR_IF_OVERWRITTEN == 1:

        ERR_IF_OVERWRITTEN 1
        ORG     $6000
        DECLE   1234

        ERR_IF_OVERWRITTEN 0
        ORG     $6000
        DECLE   3456

Conversely, this example does _not_ generate an error:

        ERR_IF_OVERWRITTEN 0
        ORG     $6000
        DECLE   1234

        ERR_IF_OVERWRITTEN 1
        ORG     $6000
        DECLE   3456

[+DZ-Jay]

TL;DR: Do I need to do anything to keep the current behaviour? I employ macros to back-patch parts of ROM, including parts of the EXEC.

 

-dZ.

[intvnut]

TL;DR: Do I need to do anything to keep the current behaviour? I employ macros to back-patch parts of ROM, [...]

 

No. Default behavior is unchanged. You might want to look at what ERR_IF_OVERWRITTEN + FORCE_OVERWRITE do, though, to make your macros more robust.

 

 

[...] including parts of the EXEC.

 

How the heck do you manage that?

[intvnut]

BTW, a quick, quick primer on how to use FORCE_OVERWRITE to implement a back-patch. And by "back-patch", I mean using ORG to rewind the assembly address to overwrite something that was previously assembled.

 

Suppose you had some code like this:

.

MISC:   PROC
        MVI@   R1,  R0
@@bp:   NOP                ; This will get "back-patched" later
        MVO@   R0,  R2
        JR     R5
        ENDP


;.... lots of code ....


; Back patch MISC.bp with a different opcode:

cur     QSET   $           ; Remember current location
        ORG    MISC.bp     ; Rewind assembly pointer to MISC.bp
        DECR   R0          ; This replaces NOP with DECR R0
        ORG    cur         ; Resume assembling at where we left off 

.

This is a purposefully silly example so you don't spend too much time thinking about what the code does. Just focus on the fact we're replacing NOP with DECR R0, by using ORG to temporarily rewind the pointer.

 

The code above works in AS1600 today, both before and after this change. If you add the recommended "-e" flag, though, you'll get an error about overwriting ROM. Here's how to address that:

.

MISC:   PROC
        MVI@   R1,  R0
@@bp:   NOP                ; This will get "back-patched" later
        MVO@   R0,  R2
        JR     R5
        ENDP


;.... lots of code ....


; Back patch MISC.bp with a different opcode:

cur     QSET   $           ; Remember current location
        ORG    MISC.bp     ; Rewind assembly pointer to MISC.bp
        FORCE_OVERWRITE 1  ; "I know what I'm doing!"
        DECR   R0          ; This replaces NOP with DECR R0
        FORCE_OVERWRITE 0  ; "OK, I'm done."
        ORG    $           ; Resume assembling at where we left off 

.

The FORCE_OVERWRITE suppresses the error, because you're telling AS1600 "I know what I'm doing, and I intend to overwrite some previously assembled ROM with new stuff here."

 

Unless you're doing this sort of backpatching, though, the -e flag (or ERR_IF_OVERWRITTEN) will catch errors due to accidentally assembling some code over other, previously assembled code. The current IntyBASIC mechanisms for large memory maps are prone to such errors, especially the crippled set of options available for the IntyBASIC programming contest. Indeed, it's that contest that inspired these enhancements for error reporting.

 

(I still wish we could get cart.mac integrated with IntyBASIC, or some better mechanism for memory map management in IntyBASIC.)

Edited April 23, 2018 by intvnut

[+DZ-Jay]

How the heck do you manage that?

 

Oops! Sorry, was mistaken. That was with bank-switching, not ORG to back-patch. I back-patch my own library code to inject data and pointers which are computed after the entire program is assembled.

 

 

 

No. Default behavior is unchanged. You might want to look at what ERR_IF_OVERWRITTEN + FORCE_OVERWRITE do, though, to make your macros more robust.

 

 

Yeah, seems like a good idea. I was just wondering if I needed to do anything immediately if I were to upgrade today. Thanks.

 

I do agree that this option is a good feature to have, since most of the time such errors are caused by inadvertently overlapping ROM segments.

 

-dZ.

[intvnut]

Oops! Sorry, was mistaken. That was with bank-switching, not ORG to back-patch. I back-patch my own library code to inject data and pointers which are computed after the entire program is assembled.

 

Out of curiosity, why do you need to backpatch data and pointers? DECLE accepts forward references to symbols, including EQU / QEQU symbols. I suppose if you don't have names for those symbols yet, that might be a problem; however, you need a way to refer back to what you're back-patching, so it seems like you have a way to connect those with a symbol.

 

The main use of backpatching in my own code is to assemble different instructions in a stretch of code. For example, ADDI vs. SUBI or some other esoteric things that creep up when you don't know the relative distance between two addresses ahead of time, for example.

Edited April 23, 2018 by intvnut

[+DZ-Jay]

 

Out of curiosity, why do you need to backpatch data and pointers? DECLE accepts forward references to symbols, including EQU / QEQU symbols. I suppose if you don't have names for those symbols yet, that might be a problem; however, you need a way to refer back to what you're back-patching, so it seems like you have a way to connect those with a symbol.

 

The main use of backpatching in my own code is to assemble different instructions in a stretch of code. For example, ADDI vs. SUBI or some other esoteric things that creep up when you don't know the relative distance between two addresses ahead of time, for example.

 

I believe there used to be warnings with EQU at some point, when I made those macros. I believe I excised those from P-Machinery 2.0, but the Christmas Carol code still has a bunch of old legacy code -- including patches to the SDK-1600 library routines which were done this way rather than overwriting the original source in order to maintain compatibility with other code (back then, I used the SDK-1600 modules as a centralized "library" of routines imported into my own code, so it made sense to patch post-assembly rather than to re-write or over-write them. It also made sense not to make a modified local duplicate in case they were updated with bug fixes or enhancements in a future SDK release. Those were the days... )

 

Regardless, I still have patching macros lying around somewhere... I found at least one instance of a patch in P-Machinery 2.0 to optimize the exit of "in-line" procedures (i.e., code that looks like a procedure but is actually "include"'d in-line in the code stream.

 

"In-line Procedures" (IPROCs) in P-Machinery use a special structure macro and are stored "stand-alone" in their own source file, which is later included by using the "IPCALL" macro. They allow you to modularize your code, treating them as if they were separate self-contained routines, while still taking advantage of in-line processing which eliminate the cost of branching in and out of them. For instance, in P-Machinery, the core kernel is one big monolithic process, but in source code, it looks like a bunch of routines calling each other.

 

The pattern of IPROCs is expected to be like this:

MY_INLINE_PROC IPROC

               ; Code ...


               IPRETURN
               ENDIP

The trick is that the ENDIP tracks the distance between IPRETURN and itself, so it knows whether it should use a "B" branch instruction, advance the PC, or just fall-through.

 

By default, IPRETURN injects a branch to the "ENDIP" point and marks the current address counter. If ENDIP is at the very next address, the PC is "rewound" to remove the branch (fall-through); if it is one address after, it is replaced with an "ADDI #2, PC", which is cheaper than the branch; otherwise, the branch remains. The programmer remains blissfully unaware of these machinations, and is able to keep his mental model of calling a sub-routine and returning from it.

 

You call this "inline" procedure with:

IPCALL MY_INLINE_PROC 

Which essentially uses "Include" to inject the "procedure" in-line.

 

-dZ.

Edited April 23, 2018 by DZ-Jay

[intvnut]

Fortunately, the "forward reference to EQU" warning is now a thing of the past. (Yay!)

 

The default behavior of AS1600 isn't changed, so all your code should continue working. If you did decide to make use of the add'l error reporting, bracketing your back-patches with FORCE_OVERWRITE 1 / FORCE_OVERWRITE 0 will allow you to distinguish intentional overwrites from unintentional overwrites.

 

For folks programming in IntyBASIC that aren't using fancy macros like that, just changing the default to "on" with the -e flag still makes the most sense to me. That, combined with the -m flag to keep an eye on your memory map should help folks with their memory map hygiene.

 

At least one person I know spent more than a few hours (probably closer to a day or two) trying to diagnose an error in their program that -e highlights and diagnoses in seconds.

[+DZ-Jay]

Fortunately, the "forward reference to EQU" warning is now a thing of the past. (Yay!)

 

Yeah, sometime last year (when I finally upgraded my ancient as1600), I removed those patches from P-Machinery code and replaced them with forward references. The IPROC ones are about the last ones left. Christmas Carol will get a re-factorning some day...

 

The default behavior of AS1600 isn't changed, so all your code should continue working. If you did decide to make use of the add'l error reporting, bracketing your back-patches with FORCE_OVERWRITE 1 / FORCE_OVERWRITE 0 will allow you to distinguish intentional overwrites from unintentional overwrites.

 

Yeah, it's a good idea. It also protects any code from someone (me?) inadvertently adding the command line instruction to detect the error.

 

All these changes are great and make the tools even better.

 

-dZ.