Action! Source Code Action! Source Code

[Rybags]

The OSS carts are 16K. The problem you'd have though is with the editing interface. Fairly sure Action uses a mix of it's own editor mixed in with the OS one. That might be the biggest hurdle to conversion.

[+slx]

The OSS carts are 16K. The problem you'd have though is with the editing interface. Fairly sure Action uses a mix of it's own editor mixed in with the OS one. That might be the biggest hurdle to conversion.

I supposed that the editing part would have to be done from scratch but the editor could be simpler without making it unuseable. OTOH the BBC has the "monitor" part built in. For the time I lack the time, skills, Atari and BBC knowledge to even think about it but the idea is still tempting

[flashjazzcat]

Jac! said that the editor scatters direct screen writes all over the place. If everything was channelled through calls to a central "put character" routine, it would be much easier to abstract the display device and start supporting different screen geometries (80 columns, etc) and even different platforms. I suppose the intention is - eventually - to do just that.

[pirx]

IMHO with contemporary HDDs, ramdisks and stuff command line compiler would do the trick beautifully.

[2600problems]

any version is welcomed. i just saw this and am going to use it today. you said it was hacked? hacked how exactly?

[2600problems]

great, turbo dos...just great

[+slx]

Jac! said that the editor scatters direct screen writes all over the place. If everything was channelled through calls to a central "put character" routine, it would be much easier to abstract the display device and start supporting different screen geometries (80 columns, etc) and even different platforms. I suppose the intention is - eventually - to do just that.

 

I had a look at the source code and it looks modular enough to separate the compiler from the editor, making into a kind of "command line" compiler as a first step (but then my ML experience is quite limited, especially regarding large projects like this). There seems to be a separate memory allocation part in the compiler which might be a starting point to use more memory on the Ataris.

 

Conversion would also require the library and memory map of the output to be adapted to the target system, so it would be quite a tall order in total, even if using an existing editor on the target system.

[Alfred]

 

I had a look at the source code and it looks modular enough to separate the compiler from the editor, making into a kind of "command line" compiler as a first step (but then my ML experience is quite limited, especially regarding large projects like this). There seems to be a separate memory allocation part in the compiler which might be a starting point to use more memory on the Ataris.

 

Conversion would also require the library and memory map of the output to be adapted to the target system, so it would be quite a tall order in total, even if using an existing editor on the target system.

 

The memory allocator is only used for the editor function. It allocates each line as you type it. The compiler uses fixed size tables for the hash tables and the symbol table simply expands. When compiling from memory, the compiler directly accesses many of the editor zero page locations and line functions to retrieve the source lines. It would be relatively easy to separate the compiler from the editor to make a standalone compiler. I ripped out the editor years ago as a standalone program because I really liked the Action! editor. One modification I made to it was to give it a burst mode to read text in blocks, rather than the slow, line at a time that the cartridge normally uses.

 

[luckybuck]

Hi Alfred!

 

So nice, you are already with us! Please go ahead! :-)

 

Alfred, what about uploading the extracted editor from ACTION!?

 

Thank you very muc in advance. :-)

[+slx]

Thinking about it, there must be code already present to compile from a file to a file as this function is available in the monitor.

 

 

The memory allocator is only used for the editor function. It allocates each line as you type it. The compiler uses fixed size tables for the hash tables and the symbol table simply expands. When compiling from memory, the compiler directly accesses many of the editor zero page locations and line functions to retrieve the source lines. It would be relatively easy to separate the compiler from the editor to make a standalone compiler. I ripped out the editor years ago as a standalone program because I really liked the Action! editor. One modification I made to it was to give it a burst mode to read text in blocks, rather than the slow, line at a time that the cartridge normally uses.

[Alfred]

Thinking about it, there must be code already present to compile from a file to a file as this function is available in the monitor.

 

Not so. The compiler lays out things sequentially in memory. I know there is a file that plays with some of the internal pointers to write code blocks out as they are completed, but having never tried it I dont know how well it works. The compiler is designed to only write the code to ram, and after that it can do one big block save to disk. For example, conditional statements which must jump over the false case are backchained in memory. Only when code gen is complete does the compiler go back and run through all the links and insert the actual target addresses for the JMP instructions. Same for large defined arrays, final address not generated until codegen is complete.

I doubt its possible to make the current compiler do file to file in all cases. If you dont use if,while,for or large arrays (>128 bytes) then it might work.

[Alfred]

Just a comment on modifying the cartridge. The Action! compiler is a marvel of compactness and code efficiency. The guy who wrote it studied compilers and wrote them as a grad student at university. I've attached his project on Micro-SPL which I believe is the precursor to Action!. Have a look at the example code at the end and I think you'll see the similarities.

 

In order to change the compiler you're going to need a very good understanding of how it works. It makes the BasicXE cartridge look like a bicycle in terms of complexity, where Action! is a Veyron. The same zero page location is used in different ways by different parts of the compiler during the same run. The compiler also makes use of a lot of very complicated bit twiddling and masking. I've never seen anything like it. In order to change the compiler, you need to understand ALL of it, because if you make some minor change thinking this isn't important, you'll probably end up blowing up the compiler. For example, you look at the code generator and see arg6 is used as the zp address to store the next code byte. Simple enough, you think. Then you get into the expression evaluator, and after decoding some stack frames you see $C5 and you think, hey isn't that arg6 ? It then turns out arg6 is also the address for the next variable to be added to the previous stack frame as a kind of peephole optimization. Then maybe you start at the front, looking at how the source lines are retrieved from the buffer and you are poking around and you see a jsr to the code generator and see it advancing arg6, and you think WTF? Why is retrieving a source code line in lower memory using arg6 as a reference when arg6 points to the ram after the editor buffer? Stuff like that abounds all over the place. Touch something and you likely impact half a dozen other things that you would never even think of as being connected.

 

I spent months trying to understand it thirty years ago. I finally gave up when I realized it was just too complicated for me. I can do 16 bit hex math in my head, but not if the bytes are in reverse order. If you can multiply hex digits in your head then maybe you can understand what Clinton did. I suspect Clinton himself wouldn't be able to tell you any more in detail how the expression evaluator works, it's just that complex. I even ran Action! under the Mac/65 cartridge using DDT to try and understand parts of it and even watching it in action (heh) what was on paper didn't seem to match what I saw in DDT.

 

Now there are some simple things that could be done. Move the whole program into the 65816 address space and split data and code into separate banks. Improve the editor perhaps using a flash cartridge. What I think is very unlikely to happen is that somebody is going to spend the time to decode the cart to the point where they can add a new data type. For one thing. all 8 bits are used, so you'd have to enlarge the bit mask which means changing a boatload of other variables and procedures. As soon as you do that you've broken the whole data type/expression evaluation which depends on an 8 bit value.

 

Oddly enough the editor and it's memory allocator are junk. Looks like something written by somebody who just learned Basic. I suspect that Clinton came to OSS with just the compiler and Wilkinson or somebody added the editor and monitor to it. The file ALLOCATE.ACT on the Toolkit disk bears an eerie resemblance to the allocator used in the cartridge. Compared the to the trickery of the expression evaluator, the editor is junk, and I find it hard to believe it was written by the same person.

 

Really, the best approach would be to write a new compiler from scratch. I imagine it would take a lot less time than trying to understand what Clinton did. With the larger carts today or the 65816, you wouldn't be under the same tight constraints that he was, and you could write much more elegant code.

 

 

Micro-SPL_Sep79.pdf

[Alfred]

Just looking at my copy of the code, some blasts from the past. Straightforward code parts:

 

;
; FIRST BYTE WAS AN ALPHA SO IS EITHER A RESERVED WORD
; OR AN IDENTIFIER.
;
getnid jsr getnme
bmi getnr1 ; BR FOR MOST NAMES (MIGHT BE UNCOND)

;
; LOCATE AND EXECUTE SPECIAL TOKEN CODE (LIKE GET $4455)
;
getnr0 sta nxttkn
ldx #<lexcmd
ldy #>lexcmd
jmp lookup

;
; COMON EXIT POINT. THE TOKEN TYPE IS STORED IN NXTTOKEN
; AND NXTADDR CONTAINS THE SYMBOL TABLE ADDRESS IF THE TOKEN
; WAS A STRING. OTHERWISE NXTADDR WILL CONTAIN THE ACTUAL
; VALUE, LIKE $4455 CONSTANTS.
; STRING CONSTANTS ARE STORED IN SYMTAB AS WELL, BUT ARE
; PADDED WITH AN EOL.
;
getnr1 sta nxttkn

;GetNr2 LDA $D0
; BEQ GetNr3
; JSR PrintTok

getnr2 ldx addr
ldy addr+1
ismt lda token
rts

 

 

Less than clear:

 

;
; NOT SURE WHAT THIS DOES. NXTADDR IS INVALID, SINCE IT'S
; THE LAST TOKENS VALUE, SO NXTPROP JUST SCREWS UP PROPS.
;
lexp lda #3 ; FETCH THE PARAMTER OPS
jsr nxprop ; FOR DEFINE ?
lda props
ldx props+1
jsr rstp
ldy choff
lexp1 sta delnxt
stx delnxt+1
sty defflg
lda #0
sta choff
jmp getnlp

 

 

This makes sense:

;
; CALL COPYSTR TO COPY THE STRING TO QCODE. IT COPIES THE
; STRING FROM THE SYMBOL TABLE TO QCODE AND RETURNS THE
; ADDRESS OF THE FIRST BYTE OF THE STRING IN Q SPACE
;
mnmstr jsr cpystr ; string ref
jmp mnm040

 

 

This, not so much

 

;
; Make new entry in symbol table
; SYMBOL IS IN SYMTAB. ARG2 & ARG4 ARE THE HASH TABLE
; ADDRESSES AND RY IS THE TABLE OFFSET (COULD BE THE HASH
; VALUE, BUT NOT LIKELY).
;
nwent lda symtab+1 ; STORE THE SYMBOL
sta (arg2),y ; ADDRESS IN THE
lda symtab ; HASH TABLE
sta (arg4),y
lda #<libst ; SCOPE FOR A
ldx #>libst ; LIBRARY RTN NAME
jsr stm ; lookup shadow name
;
; NO BRANCH IF NOT FOUND, SINCE NOT FOUND WILL YIELD
; A ZERO ANYWAY. ODD THAT BUILT-IN ROUTINES ARE DEFINED
; AS UNDECLARED. MAYBE IT'S FIXED LATER ON.
;
lda #undec
ldy arg14 ; LENGTH FROM GETNAME
iny ; PLUS 1 FOR TYPE
sta (symtab),y ; SET AS UNDEC INITIALLY
lda nxtadr ; SAVE SHADOW IF THERE
iny ; WAS ONE ELSE WILL BE ZERO
sta (symtab),y ; save shadow entry

 

You want to upgrade the Action! compiler, you go right ahead and give it a try.

 

 

[+slx]

Not so. The compiler lays out things sequentially in memory. I know there is a file that plays with some of the internal pointers to write code blocks out as they are completed, but having never tried it I dont know how well it works. The compiler is designed to only write the code to ram, and after that it can do one big block save to disk. For example, conditional statements which must jump over the false case are backchained in memory. Only when code gen is complete does the compiler go back and run through all the links and insert the actual target addresses for the JMP instructions. Same for large defined arrays, final address not generated until codegen is complete.

I doubt its possible to make the current compiler do file to file in all cases. If you dont use if,while,for or large arrays (>128 bytes) then it might work.

 

Thanks for your interesting remarks. I always thought it was a one-pass-compiler (or remember it being described as one in reviews). Seems I need to look for something easier as my next project.

[Alfred]

 

Thanks for your interesting remarks. I always thought it was a one-pass-compiler (or remember it being described as one in reviews). Seems I need to look for something easier as my next project.

 

Yeah, it's a terrific one-pass compiler for its size and what it does. Nonsense like the backchains for all the conditionals is just necessary due to the one pass nature yet still needing to resolve internal forward references. I know I come across all gloom and doom about the cartridge, but it's really not something you're going to upgrade in a week. I'm sure it can be done by a good programmer. The difficulty is that because it is so complex, you need the time to sit down and work through the myriad of calls. Besides the very odd bit flipping (parser sets a token to say chart and then inverts with a strange mask to store in the symbol table. why not just save the type?) he's really gone all out with code reuse to save space. For example the same code that parses the parameters for a PROC/FUNC is the same code that parses the global and local variable defines. A few undocumented flags tell it which way to go. Subroutines will call other subroutines up to five or six deep just to use a few instructions. Say you want to upper case a token character because you want to test it against a symbol name. Anybody else would just use a LDA/CMP loop. Not Clinton. No, the parser will call something in the expression evaluator that pushes a zp value onto the expression stack, which then will jump into the codegen to swap two different zp values and compare them and then that exits out via something in the symbol table lookup code that pops something off the expression stack (WTF) and then returns with Carry clear or set if the uppercase value matched. All that BS to save like six bytes. The whole compiler is like that, JMPs all over the place.

 

The key to understanding the compiler is decrypting all of the crazy nested calls where different routines are jumping into the middle or end of something completely unrelated that happens to do the desired operation if some processor flag like Z or C is set. I spent at least six months tracing code paths, commenting what I found, puzzling out what it was doing. I gave up finally for two reasons. One, there is no space in the cart. I forget now, but I think there is something like less than 200 bytes free. So there was no way I was going to be able to make any significant upgrades. The second was, I couldn't remember it all. I had my day job programming S/370 and then S/390 IBM mainframes in assembler plus the stuff I was doing on the Atari. I just couldn't keep the whole compiler structure in my head, so if I put it down for a week, I'd come back and think, now what are the six things GetNextStmt does again ? If you really want to understand it, you'll need the time to really go through it and map everything out. I only got about halfway. I know in detail how most of it works, except for large parts of the expression evaluator and the code generator because they really are the same thing essentially. Today any hand built parsers are recursive descent. Action! is a stack parser. I bought the original dragon book (Aho etc) to try to understand how it works. In the 2nd edition they dropped the chapter on stack parsers, I guess because nobody uses them, too hard to code.

 

So feel free to try it. You don't have to be a genius, although being able to do bit EOR's in your head would help. You need a thick pad of paper and a lot of time with few distractions, so you can work through all the bizarre and amazing ways he squeezed in all that functionality. If you can get to the point where you can mentally conceptualize how it all fits together you're golden and can fix anything. I can only do that for one large project at a time, and right now my brain is filled with the details of a 300K line multi-threaded server that runs a universe like Eve Online. There's no room left in my head for the Action! compiler anymore, so someone else will have to step up. It's certainly doable, but you'll probably have to dedicate maybe a year of your life to do it, and nothing else.

[+slx]

It's certainly doable, but you'll probably have to dedicate maybe a year of your life to do it, and nothing else.

 

I'm out as well then, at least until I'm retired or divorced or both . I'll gather experience with smaller projects until then. The BBC will have to wait.

 

We have the source code now, even if it's only very sparsely commented (Clinton apparently kept the stuff in his head, too) but finding out all that with a debugger is even more impressive!

 

Did you ever share your commented code, BTW? You seem to have it commented more thoroughly than Clinton did. Your remarks might help future Action! code archeologists.

 

Clinton Parker's sources are on sourceforge.

Edited June 5, 2018 by slx

[Alfred]

Yeah, Clinton was pretty sparse with the comments, but then so am I. I mean the guy worked on this for a couple of years at least, assuming he started from the micro-spl so Im sure he knew the code like the back of his hand, so why bother with comments.

 

No, my comments are not in the source I posted, thats the original code. I have five versions of the compiler source based on his. First pass was to convert it to Chucks T816 assembler so I could get a single giant listing. Second pass was to rename a lot of variables to a kind of standard. Each time I would verify that the binary matched the cart so I knew there were no typos. I dont remember now what a couple of the passes were. The fifth one was to remove all the interbank jumps and produce a simple flat runnable binary. Thats the last I worked on it.

[Alfred]

One other note for anybody trying to decode Clinton's code. You must learn the data type bits cold in order to understand the expression evaluator and the code generator:

typet = $70 ; 112

eofid = 127

constt = $80
vart = $88
arrayt = $90
tempt = $a8
funct = $c0
condt = $48

; types
; -----
chart = 1
bytet = 2
intt = 3
cardt = 4
strt = 5
realt = 6

undec = $88

I mean cold. You need to be able to combine them in your head, EOR them with values. I never mastered it myself. You need it because you will see stuff like this:

 

;
; IF IT'S A ONE BYTE ARG, THEN WE SAVE THE ORIGINAL
; TYPE AS IT WAS. TWO BYTE ARGS GET CHANGED. SIMPLES
; BECOME 1-9 AND RECORD PONTER BECOMES $11-9
;
prm030 cmp #vart+intt
bcc prm050 ; one byte arg
; two byte arg
prm040 and #$1f
inc argbytes

prm050 and #$9f
inc argbytes
sta (props),y
rts

all over. Those are my comments. All you had to go with in the source was "cmp #vart+intt" and that's one of the easier ones. I stepped through it with DDT to figure out what it was actually doing. Then of course they get masked to something else. The symbol table stores an EOR'd mask version of the chart, bytet types. So when you are looking at the expression evaluator you need to be able to EOR those bits back in your head to recognize what the mask is doing.

 

; EVEN THOUGH THERE ARE 256 OR LESS ELEMENTS, MAKE SURE
; THE DATA TYPE DOESN'T CAUSE THE ARRAY TO EXCEED 256
; BYTES IN SIZE
;
dcl410 ldy #0 ; get the type
lda (props),y
cmp #arrayt+intt
;
; THIS ISN'T RIGHT. CARD ARRAYS OF LESS THAN 128
; SHOULD ALSO GENERATE SMALL REFERENCES. SOMETHING
; TO FIX.
;
bcs dcl320
; small byte array
sty numargs ; zero numargs
;
; THIS $08 SETS THE CNSTMODE BIT
; FOR THIS ARRAY, SO THAT THE
; CODE GENERATOR WILL USE THE
; LDA ADR,X FORM
;
ora #8 ; SET THE 'SYS' FLAG
sta (props),y ; WHATEVER THAT MEANS

 

Clinton's wife told me that the time in which Action! was developed was a dark period in his life which is painful for him to recall, which is why he is so reticent about about Action!. When you sit down and really look at how godawful complex the whole architecture is, I can hardly imagine the mental discipline to be able to maintain just -knowing- how exactly all of these routines and bit patterns mesh together and to do it for months on end. Small part of the expr/codegen, including peephole optimization:

 

;
; RHS IS A WORD TEMP. LHS IS A CARD/INT
; NOW WE RESET ARG14 TO THE ADDRESS WHERE THE STA TEMP
; WAS CREATED.
;
cga120 ldy #4 ; SET ARG14/15 TO STACK 3/4
jsr loadi
ldy #8
lda arg2
and #$60 ; lhs proc or temp(argument)?
bne cga140 ; yes
jsr stkpz ; GET S.T. VALUE OF VART VIA STK/PROPS
;
; X/Y IS THE VAR ADDRESS OF THE LHS WORD
;
beq cga150 ; page zero
sty arg13
ldy #1 ; GEN STORE TO LHS LSB
lda #$8d ; STA addr16
;
; ADDS Y TO ARG14, STORE IN ARG10
; COPYS ARG14 TO ARG10 FOR (QCODE-ARG14) BYTES
; STORES X & ARG13 IN THE NEXT 2 BYTES
; WE BUMP THE CODE UP ONE AND OVERLAY THE STA TEMP WITH
; THE STORE TO THE TRUE AVAR
;
jsr i30 ; insert STA data16
lda #1
cga130 ldy #5
;
; RESTORE QCODE FROM STACK, ADDING RA TO THE SAVED VALUE
; AFTER THE INSERT THIS PLACES US AFTER EITHER THE LDA #0
; OR THE
;
jsr loadcd
lda #$81 ; STA
jsr op1h ; DO STORE TO THE HIGH BYTE
jmp cga1

cga140 and #$40 ; proc?
bne cga050 ; yes, punt(not the best qcode)
jsr stkaddr ; temp (proc argument)

 

Good luck.

 

 

[Alfred]

Hi Alfred!

 

So nice, you are already with us! Please go ahead! :-)

 

Alfred, what about uploading the extracted editor from ACTION!?

 

Thank you very muc in advance. :-)

 

I think this is the version I normally used. I also have a 65816 version that uses XE banks, but I don't recall if it's finished or if I was still working on it.

AED.txt

[luckybuck]

Thank you so much Alfred, as always! :-)

 

May I upload this and the pdf file from you above to the Wiki with credits to you?

 

 

All the best. :-)

[Alfred]

Thank you so much Alfred, as always! :-)

 

May I upload this and the pdf file from you above to the Wiki with credits to you?

 

 

All the best. :-)

 

Sure, but the pdf isn't mine, I didn't write it.

[luckybuck]

Thank you anyway Alfred, but even more important, please stay with us, we really need you! :-)

 

All the best.

[Alfred]

Jac! has assembled the editor into an xex file. While perusing the source I realized I made one further change. Under SpartaDos you can invoke it from the CLI and pass a filename. Since I'm lazy I renamed the executable to E.COM, which is why it took me a while to find the source which has a different name. So under SpartaDos I could type:

 

 

D1:E MYSRC.ACT

 

and it will autoload the file after the editor initializes. I found it very handy.

Action Editor-MADS.xex

[luckybuck]

In the name of the community, thank you Alfred and thank you JAC! Great work!

 

Now, one of my plans comes true. We have now the best editor for the real A8, even usable for PASCAL, FORTH etc...

 

Incredible! :-)))) Thank you both so much!!!!!!!! :-))))))

[Alfred]

So there's been a little bit of interest in Action! lately, and I've been doing some stuff like upgrading Six Forks to be able to tinker with it. I might have posted about this before, but it's easier to just repeat it here.

 

So for the few people who still code on the Atari, and if you were going to use Action!, what changes do you think would be useful ? They have to be practical, both because the Atari is limited in memory and because nobody really knows how the compiler works well enough to make real architectural changes.

 

First off, the cartridge has to go. I think JAC! said there was something like 147 bytes free. I remember when I looked at it originally when ICD gave me the copy that all my ideas for upgrading it went out the window when I realized there wasn't enough memory to do virtually anything. So today there are a few options as I see it:

 

1. Switch to a supercart like an Atarimax or SIC!. Drawback here is very little doc on how these work, like what bank appears on powerup. From comments elsewhere I get the impression that the AtariMax carts don't even work like real carts, they're more like disk drives.

2. Switch to binary file, using banked ram. Drawback here is limits use to expanded machines or 130XE's.

3. Same as two, except add some wrapper code to allow the compiler to use 65816 memory. Drawback is very few 816's are out there.

4. Split the cart into separate binary files per function. So you'd have the editor, the compiler and the runtime library as all separate functions. Drawback is it makes the system clumsier to use. The worst option of all I think, and won't mention it further.

 

Switching to a big supercart is probably the best, followed by the banked memory option imo. I would probably do the banked ram version first because of the fact that I would need to figure out how these bigger carts work.

 

Fixing some of the bugs, like the one where BYTE memlo=[$02E7] messes up every declare that follows by inserting padding bytes or something. Or that using hex constants fail for array references.

 

Using banked memory (something anyway) to allow for larger binary files. I think it would be reasonable that the compiler should be able to generate code that goes from say $2000 right through to $B000 (36K). So the hash tables and the symbol table have to go somewhere else.

 

Compile to file rather than memory. Would be very disk intensive, I can see US Doubler drives destroying the disk because the double just doesn't handle single random sector writes very well. tends to corrupt the disks a lot. Might only be useful on hard disk systems or Altirra.

 

Expand the editor's memory, allow for banked ram.

 

What other ideas ?