Using overscan to store data?

[deepthaw]

Guy on reddit made this claim:

 

Fun Fact: The Atari 2600 had so little memory that some creative devs used TV scanlines above the viewing area (usually for factory testing) as memory registers, giving them extra space to store bits as black or white pixels.

 

And is using some links that talk about using VBLANK and HBLANK to run code as support.

 

Is this a thing? 2600 has no frame buffer, no VRAM, and hence no way to even know what was drawn to a scanline once its drawn, right?

[+nanochess]

Not possible, the TIA registers are either read only or write only, so no way to save anything.

 

I think there are some bits unused in PIA 6532. Besides that you have only the 128 bytes of RAM to save things.

 

From the very moment the guy said "storing things in scanlines" he is nuts, no way something from TV returns to Atari or any other console.

 

Edit: Point us to the reddit thread so we can fall over the guy

[deepthaw]

https://reddit.com/r/nintendo/comments/8ni3b3/_/dzw7e4k/?context=1

[RevEng]

Nobody did that back in the day, nor does anybody do it today.

 

In theory you could use the display objects to save the state of a few bits heading out of the visible screen, and pull the bits back later using other objects and the collision registers. In practice, it's enough of a pain to not make it worth the trade-off in ROM and cycles.

[deepthaw]

I tried explaining it, but he insists everybody else is wrong and furthermore - everybody who disagrees is all the same person who's been stalking him and he's reporting us all to the mods?

 

I think he has it confused with storing data in the vram of other computers/consoles (which people apparently did) and doesn't realize that the references to scanlines in VBLANK and HBLANK and overscan are being used as shorthand for a number of processor cycles.

[Thomas Jentzsch]

I was tempted to answer. But he behaves so arrogant that he doesn't deserves a correcting answer.

[Thomas Jentzsch]

In theory you could use the display objects to save the state of a few bits heading out of the visible screen, and pull the bits back later using other objects and the collision registers. In practice, it's enough of a pain to not make it worth the trade-off in ROM and cycles.

Not even in theory. As explained above, the TIA registers are either read or write only and there is no screen buffer. So you cannot store anything in the display objects.

 

The only option would be to read set TIA object bits by checking them bit by bit through collision registers. But that's extremely inefficient and AFAIK has never ever been used.

[RevEng]

Not even in theory. As explained above, the TIA registers are either read or write only and there is no screen buffer. So you cannot store anything in the display objects.

 

The only option would be to read set TIA object bits by checking them bit by bit through collision registers. But that's extremely inefficient and AFAIK has never ever been used.

Uh, yeah. That's why I mentioned the collision registers specifically, when I said "in theory".

[Mr SQL]

Nobody did that back in the day, nor does anybody do it today.

 

...pull the bits back later using other objects and the collision registers.

The collision registers indeed read the video RAM, but I'm curious what the other objects are

[ZackAttack]

Obviously the other objects are what you have to collide with to set the collision flags.

Despite this being the dumbest thing you could probably do with the VCS, you could actually sacrifice some objects to gain a few more bits of memory. At least it sounds feasible in theory.

Position the ball to whatever value you'd like to retain (0-159), disable the ball during field rendering and enable it in overscan, use PF and/or P0 to find the current position of the ball. Don't forget collisions are only detected when vblank is disabled. So you'd need to color everything black during the detection phase to avoid visible artifacts.

Of course in practice you will run out of the 5 objects before you run out of the 128 bytes of RAM, that guy clearly has no grasp of reality, and there almost is certainly no game that has ever or will ever attempt to do this. But you could if you wanted to

 

Actual Fun Fact: Coloring everything black and using collision registers to readback the value of a TIA register is used by the bus stuffing driver to detect which data bits can be successfully bus stuffed or not.

[Thomas Jentzsch]

It seems that Darrell has taken care of that idiot.

[Mr SQL]

Obviously the other objects are what you have to collide with to set the collision flags.

 

Despite this being the dumbest thing you could probably do with the VCS, you could actually sacrifice some objects to gain a few more bits of memory. At least it sounds feasible in theory.

 

Position the ball to whatever value you'd like to retain (0-159), disable the ball during field rendering and enable it in overscan, use PF and/or P0 to find the current position of the ball. Don't forget collisions are only detected when vblank is disabled. So you'd need to color everything black during the detection phase to avoid visible artifacts.

 

Of course in practice you will run out of the 5 objects before you run out of the 128 bytes of RAM, that guy clearly has no grasp of reality, and there almost is certainly no game that has ever or will ever attempt to do this. But you could if you wanted to

 

Actual Fun Fact: Coloring everything black and using collision registers to readback the value of a TIA register is used by the bus stuffing driver to detect which data bits can be successfully bus stuffed or not.

 

You've described the referenced display objects which include the partial frame buffer (20 bits) and the 5 sprite buffers.

 

We can read the display objects back with the collision registers as you explained but the "other objects" cannot be considered the display objects without a circular reference.

 

Agree the reddit posters idea was valid and also unlikely, though any game that uses the collision registers (99.9%) is reading the write-only video RAM register buffers (not really write only).

[tschak909]

Oh dear god, reading that Reddit thread basically is telling me that this guy completely misunderstands the concept of the non-visible areas of the screen, and is confusing processing time with "data-storage"

In other words, a fucking idiot.

 

-Thom

[Omegamatrix]

In theory you could use the display objects to save the state of a few bits heading out of the visible screen, and pull the bits back later using other objects and the collision registers. In practice, it's enough of a pain to not make it worth the trade-off in ROM and cycles.

I either did this or was going to do this in one of my hacks. I can't find or remember the rom though. It was a weird situation where I just needed a few bits more of ram. This was long ago...

[RevEng]

I either did this or was going to do this in one of my hacks. I can't find or remember the rom though. It was a weird situation where I just needed a few bits more of ram. This was long ago...

I might have to eat my words that "nobody does it today"

 

This topic has me thinking that we should have a thread somewhere dedicated to weird storage locations. I recall that you cleverly repurposed the 6507 interrupt flag for the star-map display state, with your Starmaster 2-button hack. So long as you stay away from adc/sbc you could do the same with the decimal mode flag. (that would make more sense on the NES, though I doubt you'd ever be scrounging for bits there) There's always the RIOT port bits. I'm pretty sure you could repurpose the capacitor charge-up as a flag, but without a resistor/paddle you wouldn't be able to unset it.

 

I'm sure there are lots of other entertaining ways one might scrounge for bits that I haven't encountered or considered.

[DirtyHairy]

Thinking about it... If you store data in the PFx registers, you get 20bit of storage. If you read it out using a comb consisting of all five objects, spaced 16 pixels, you should be able to read out all 20 bits in five lines (one line positioning + setup, four lines readout with late HMOVEs). However, you lose about 350 bytes of ROM for the readout code, and you cannot use playfield or ball while data is stored

Edited June 2, 2018 by DirtyHairy

[Thomas Jentzsch]

I wonder about the maximum storage in TIA:

• 20 bits for playfield
• 2*2*8 = 32 bits from GRP0/1 (including shadow registers)
• 2*2 = 4 bits bits from ENAM0/1 (including shadow registers)
• 2*1 = 2 bits from ENABL (including shadow registers)
• 5*4 = 20 bits from HM registers
• 5*7.x = ~37 bits from object positions
• 4 bits from CTRLPF (I don't think it is possible to check the score bit)
• 2*5 = 10 bits from NUSIZ0/1
• 2 bits from RESPM0/1
• 2 bits from REFP0/1
• maybe 1 bit from VSYNC
• maybe 3 bits from VBLANK
• ...anything I missed?

That's 137 bits in total. Minus the bits required for the object to check collisions (e.g. BL). So maybe 125 bits or 15.x bytes.

 

Eventually we should start a challenge who can store the most bits and read them most efficient.

Edited June 2, 2018 by Thomas Jentzsch

[deepthaw]

If we're desperate for memory, just ask the player to remember some values for us and input them later.

[Omegamatrix]

I wonder about the maximum storage in TIA:

• 20 bits for playfield
• 2*2*8 = 32 bits from GRP0/1 (including shadow registers)
• 2*2 = 4 bits bits from ENAM0/1 (including shadow registers)
• 2*1 = 2 bits from ENABL (including shadow registers)
• 5*4 = 20 bits from HM registers
• 5*7.x = ~37 bits from object positions
• 4 bits from CTRLPF (I don't think it is possible to check the score bit)
• 2*5 = 10 bits from NUSIZ0/1
• 2 bits from RESPM0/1
• 2 bits from REFP0/1
• maybe 1 bit from VSYNC
• maybe 3 bits from VBLANK
• ...anything I missed?

That's 137 bits in total. Minus the bits required for the object to check collisions (e.g. BL). So maybe 125 bits or 15.x bytes.

 

Eventually we should start a challenge who can store the most bits and read them most efficient.

You should also include what is available outside TIA. Off the top of my head with the proper setting and kernel implementation these should all be available:

 

- Stack Pointer (1 byte)

- INTIM (1 byte)

- TIMINT (1 bit)

- Status Register (4 useful bits - the interupt flag, overflow flag, decimal mode, and carry flag)

- SWCHB (the 3 unused bits - D5, D4, and D2)

- The upper 3 bits of the high address of the Program Counter

 

RevEng's idea of dumping the latches is also a good one that I never considered.

 

I did not include SWCHA as those bits can be pulled down by a control. However thinking about it now I did use the right port for storage in all of my trackball hacks. With SWCHB I have tried it and it works although someone pointed out it is shortening those bits to ground so I went away from it.

 

One of the most useful ways to preserve bits is to use the high three bits of any pointer address as they are all mirrors. I've used that in some of my hacks when ram was limited, but my first goto's are always the stack pointer and timer as they give a whole byte each and are usually easy to implement.

[ZackAttack]

A 2k ROM mirrored to 4k could use bit 12 of PC.

[tschak909]

I smell a mini-book in the works!

 

-Thom

[JeremiahK]

I have actually thought about the possibility of using the collision registers as "RAM".

I once wrote a 6-digit score display that pulled graphics from the stack, and the loop would break when the stack wrapped around to the collision registers. In order for it to work, I had to position objects to collide correctly to set the bits. It was a major pain in the neck, and it was only to save 1 byte of RAM. I am glad to say I have since found a much better solution.

But, yes, you could totally write a complicated routine that would waste many scanline of overscan to use maybe 1 of the collision registers as a 6-bit chunk of "RAM". Although now that I think about it, as soon as the kernel was drawn, it would corrupt your value. So it would only be able to be used during overscan / vblank.

Which is all to say that there is a reason nobody has ever actually done this before.

[+Andrew Davie]

We did use screen memory on some C64 games for storage. Changed the colours so you couldn't see the data. We were really short of RAM.

[Mr SQL]

I have actually thought about the possibility of using the collision registers as "RAM".

I once wrote a 6-digit score display that pulled graphics from the stack, and the loop would break when the stack wrapped around to the collision registers. In order for it to work, I had to position objects to collide correctly to set the bits. It was a major pain in the neck, and it was only to save 1 byte of RAM. I am glad to say I have since found a much better solution.

But, yes, you could totally write a complicated routine that would waste many scanline of overscan to use maybe 1 of the collision registers as a 6-bit chunk of "RAM". Although now that I think about it, as soon as the kernel was drawn, it would corrupt your value. So it would only be able to be used during overscan / vblank.

Which is all to say that there is a reason nobody has ever actually done this before.

 

Definitely awkward for traditional Atari games as you described but not necessarily so for 32-bit ARM games, the bus stuffing driver uses the collision registers to read the bits back for processing.

 

It's innovative because no one has done it before for practical application, which necessarily means folks who are advanced in the field will make fun instead of considering the new idea like you saw initially in this thread and on reddit. That happens a lot with innovation, even in peer review. Part of the phenomena is that many folks learn without thinking for themselves, only copying other peoples ideas.

 

The VCS has incredibly open ended architecture, so new discoveries are still possible including modern discoveries with broad application.

[+Andrew Davie]

 

Eventually we should start a challenge who can store the most bits and read them most efficient.

 

This reminds me of a technique I used on the NES. I used fractions of a bit to store values. For example, I'd store the level (values 0-8 ) in 3.17 bits, and the # lives (0-4) in 2.33 bits. Thus, only 5.5 bits required to store the two. I was pretty pleased with this, and actually wrote to Don Knuth about it. About 6 months later I got a personal message from him, through his secretary's email... telling me it was a known technique but very pleasing.

 

Normally to store a value 0-4 you'd need 3 bits (giving values 0-7, but values 5 and 6 unused), and for 0-8 you'd need 4 bits (giving values 0-15, with 9-15 unused). That's a total of 4+3 = 7 bits needed for the two value ranges. In this example, only 5.5 bits needed... I'm saving one whole bit... but actually 1.5 bits if we're counting fractional bits, which I am For this two-range example it's not much saving but when you start having 10 or so value ranges, you can actually pack a whole lot more into very few bits. Fewer than you'd think! With this very simple example, if I had 7 bits not only can I store the original values 0-4 and 0-8 but I can also put a YES/NO flag in there using 1 of those 1.5 spare fractional bits.

 

I'm sure someone will figure out how I did this. I'm pretty sure I've explained it before.

Edited June 5, 2018 by Andrew Davie