Jump to content
snicklin

MADS (or other Assemblers): Automated Code testing

Recommended Posts

Operating from a Windows PC development environment, I would like to be able to test my 6502 Assembly code.

 

So, for example:

 

I might have this code:

 

org $0600

 

CLC

LDA #5

ADC #3

STA 203

 

<end of code>

 

I would like to be able to execute a command from the PC which will check that the above code puts the value of '8' into memory location 203.

 

It may run Altirra or a 6502 emulator in the background, but I need to be able to extract that memory location and then report it back to the calling program.

 

At the end I want "Test passed" or "Test failed" or something similar displaying.

 

This way I can build up an automated test suite for all of my procedures / sections of code / macros.

 

I have no problem with it bringing up windows whilst testing is going on, but I want all of the windows closed at the end of the tests.

 

Any ideas of the best way of implementing this?

 

I guess I'd need to automatically dump the memory contents and then extract the memory values that I require...

 

 

Share this post


Link to post
Share on other sites

Hello,

 

It sounds like the assert macro of the C language. It doesn't seem to be very hard

 

Hmm, I'm not sure how that can be applied across to 6502 though.

 

Though from reading what you've put and reading between the lines here, perhaps you've put me onto something...

 

I could auto-create 6502 code from Perl which calls the procedures / macros etc and could then fail/break/stop if the wrong result comes out.

 

So I have a spreadsheet with all the procedures (in MADS) to call and the parameters that need to be set.

 

I have some Perl code already for reading spreadsheets.

 

I then run the Perl code to create the 6502 wrapper code which then calls the procedures.

 

That Perl code then gives me "Test passed" or "Test failed" and appropriate exit codes.

 

I may be onto something now, thank you.

Share this post


Link to post
Share on other sites

Pretty sure Jac's integrated environment plugin for Eclipse allows to do the compile+run thing.

 

As an alternate, lower specced editors will often have batch hotkeys. IIRC I used Edit+ which allows Ctrl + <number> to call up a batch file or program and you can pass parameters like module name, directory.

I just used that for the assembly stage - it'd be fairly easy to extend on that to call an emulator that executes the program.

Such programs you generally need to assemble with the binary load Run segment (and optional Inits if needed).

 

Doing further stuff like verifying what goes into a memory location - unknown. Maybe Altirra has something that allows passing some extra parameters to run debug commands or something but you'd need it to happen after the program runs, or probably better idea have a default breakpoint set e.g. for a certain location to be read (like program end address or similar).

Edited by Rybags

Share this post


Link to post
Share on other sites

It does... but the amount of automation is the questionable thing. I've not really done anything with Altirra startup in that area.

Share this post


Link to post
Share on other sites

Altirra can fire commands from a breakpoint, but currently it doesn't have debugger commands to exit the emulator. Also, you have to manually set up these breakpoints as the listing directives (##ASSERT/##TRACE) don't take arbitrary commands. It is possible to queue up breakpoint commands to run when the EXE run address point is hit, though (.onexerun), and there at least .printf/.writemem can be bound as breakpoint triggers.

 

I've thought about exposing an optional 'emulator control' device for development, but hadn't ever gotten around to designing it.

Share this post


Link to post
Share on other sites

Executing the batch program will be no issue. Yes, I can kick off from Eclipse if I wish or I am happy even to link it into my build process. The build process is a .bat file which then calls Perl with a few parameters.

 

I then have a library of procedures which I am working on which I'd like to test. So I'd just call the program/script which does the testing then.

 

If all tests are passed, then the program will be added to a .atr file as it currently is.

 

I envisage testing individual MADS procedures rather than whole swathes of code, with procedures calling procedures etc. Each indvidual procedures doing a small but defined set of work.

 

It's a shame that Altirra doesn't already have the functionality, but to be honest, what I am asking to do is quite unique in that I guess that a lot of homebrew coders don't have test systems in place. The reason that I want one is that I can be amending something slightly and breaking it in the process without realising. If I knew every time that my code is good, I can be satisfied that something further down the line isn't broken because of something in some library somewhere. I am also working on a library of functionality which I would like to be able to use in multiple projects. Yes, it slows down my first development but should speed up my latter development. I don't want changes whilst working on project #2 then breaking functionality in project #1.

 

There's that 6502 emulator also isn't there which I used a while ago. Has anyone seen such functionality there? I need to check it out again....

Share this post


Link to post
Share on other sites

The BA command allows testing a region of memory for accesses - but it works regardless of bank configuration or if Ram, Rom or nothing is present at the specified address/range. But you can limit it to read or write.

Share this post


Link to post
Share on other sites

How about using CC65 and then you can write the assembler test harness in "C" and then run the complete binary in Altirra?

Share this post


Link to post
Share on other sites

How about using CC65 and then you can write the assembler test harness in "C" and then run the complete binary in Altirra?

 

I like your good thinking here and I think that this is getting closer to the final solution.

 

The only thing is, I want to be able to compile just one MADS procedure at a time, rather than compiling all of them together and then calling them all.

 

I then also need Altirra (or similar emulator) to then exit (which isn't currently available).

 

If I all were compiled separately, without being able to exit, I may have a Window pop up with "Test passed" and then I have to close it and then another pops up with "Test passed" and this may go on for however many tests that I have. If I have one per procedure, this could be in the hundreds.

 

If I was to compile all of the MADS procedures in one big bundle, I'll run out of memory quickly on the A8 with all of the test code included with them. And I don't use memory banks as I haven't a clue how to use them, and I want to test to a 65XE's setup.

 

If I can't do this, I won't worry too much, but I think that it would professionalise the work that I am doing. I also fully believe in automation as my last job in the UK was in systems automation. And I'm lazy, I don't like doing things twice! :)

Share this post


Link to post
Share on other sites

Hmm... you've really got me thinking now...

 

Perhaps the test mechanism in 'C' could load in the next test (and compiled code) and the next test and the next test etc until it has finished?

 

I could then maybe automate this to run weekly instead of on each build due to the amount of time to run all of the tests?

 

Then I could find a way of deriving some output out of Altirra/other emulator (can I get it to write to a file without user intervention??) and then after a certain amount of time, issue a kill process command to kill the emulator.

 

Then the build process can check the output in the output file and if it says "All tests passed", it will allow the build of the .atr file at the end.

 

This is getting very complicated though.... and will it be worth it?

Share this post


Link to post
Share on other sites

How about dividing the test harness up into areas of functionality? So you'd end up with several binaries. It should also be possible to cut out the machine specific A8 start-up code in CC65 and insert a minimal 6502 configuration. That would allow you to potentially test on a wider selection of simulators/emulators.

 

How about using the 6502 "emulator" *.DLL used in RMT? Although I'm not sure if there is much (if any) documentation to make use of that.

Share this post


Link to post
Share on other sites

Splitting it up might be a good option. Though I'm starting to think that this level of work might be a bit too much overall.

 

As for the 6502 emulator dll with RMT, I didn't know about this. Does anyone else know anything about it?

Share this post


Link to post
Share on other sites

>This is getting very complicated though.... and will it be worth it?

Whenever I have a case like your's I simple have the code in an include and I have 2 main programs.

One that's the actual program, one that's a test frame.

 

TestFrame.asm

 

ICL "ToBeTested.asm"

 

test

jsr to_be_tested.proc1

 

; Assert post conditions

lda 203

cmp #8

seq

.byte 2; breaks into Altirra's Debugger, so you see the CPU history and what went wrong

 

 

... display some "OK" text on the screen.

RUN test

Share this post


Link to post
Share on other sites

I like the way that you've done this Jac!

 

It seems a much simpler solution which is of course good.

 

There's still the problem of it going into an emulator and not coming out again.

 

I need some time I think, to think it all over and see what suits me best with all of the options available.

 

I'd just like to thank everyone above for all of your suggestions and appreciate your input.

Share this post


Link to post
Share on other sites

The cc65 cross compiler suite uses some sort of 6502 emulator to automate build tests, I believe it's called sim65 (source is in the cc65 source tree)... you may look there?

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...

  • Recently Browsing   0 members

    No registered users viewing this page.

×
×
  • Create New...