When beta testing videos arent generally made that would allow noticing such an odd error.
Fun fact: On at least one game I assisted with, we actually managed to capture an Intellivision hardware bug on video, and worked around it in software. (I think I posted about it on AA, but can't find the thread. This is a different hardware bug than the flip-to-colorstack bug.) The hardware glitch turns out to have some interesting electrical properties that cause it not to appear on all systems, or at all times on the same system. But, for one of our testers, his system showed the glitch reliably, and with a video in hand I was able to meditate on possible causes and eventually reproduce it with a targeted test, and then go tweak the game to eliminate the glitch.
I remember reading somewhere that back in the day some shops would use a VCR to capture gameplay and then go back and review the tape for issues. Still, this stuff takes going over with a fine-toothed comb. dZ mentions building a test plan and then executing to the plan. That is an excellent idea. However, even the best laid test plans can have gaps. I say this after earning my stripes writing some rather creative and brutal chip validation programs that caught test escapes on real, complex systems on a chip produced by my employer. (Ok, that's maybe a bit more hardcore than homebrew video games. But, even when there's millions of $$ on the line, bugs slip through.)
With Space Patrol, I remember using jzIntv's "movie" feature to record stretches of gameplay that I would then pore over frame-by-frame. Especially tough was getting the "apply deferred score on successfully landing" logic, when jumping over craters, rocks and land mines. Oy. Especially tough if you die right near landing. I earned a few grey hairs debugging that logic.
I'll reiterate what many others have said: TESTING IS HARD.
Space Patrol shipped with at least one (harmless but amusing) bug. Sad thing is, I've tried to fix it and I still haven't figured out what's going wrong. Testing is hard, but fixing bugs is sometimes harder!
If you're curious what the bug is, highlight the text below. (I haven't really tried to keep it a secret, FWIW, and find the quirk strangely endearing. I white it out only for those who want to maintain a fiction it's bug free. )
If you hit fire just as the game restarts after dying, the horizontal bullet that the tank shoots comes out of the left border of the screen rather than out of the tank's front cannon.
As for the Elektronite games I've assisted: I can attest to the brutality and thoroughness of the testing. I've put in my own forms of testing, although admittedly, it's been focused on specific elements more than overall gameplay. Elektronite has some truly skilled gamers in their tester roster. I'm quite happy to claim a hardware and technical support credit in lieu of a tester credit. Those testers are good.
For those interested/curious about the hardware bug: It has to do with sequences of uninterruptible instructions. The Intellivision requires code to have interruptible instructions every so often to allow the STIC to fetch a new row of BACKTAB cards from the RA-3-9600 system RAM. It turns out in certain circumstances, you can get a glitch where the leftmost card on a row gets corrupted with garbage for a single display frame if you're on the hairy edge of the maximum amount of non-interruptible time the STIC tolerates. That said, I think there must be some other ingredients, because we had to crank our "max allowed non-interruptible instruction" threshold way down to something like 44 cycles to eradicate the glitch, while the STIC routinely copes with much larger thresholds.
When the glitch happens, it takes a recent value that had been written to system RAM and uses it instead of the value fetched from system RAM for the leftmost card. If you run a test that exposes the glitch in a tight loop, the glitch will show up for several seconds and then eventually fade away. This hints at some weird electrical issue in the RA-3-9600 and the logic that switches its internal buses around. Bleh.
Edited by intvnut, Sat Apr 4, 2015 3:22 AM.