What does the 128byte RAM contains when the game is running? Is there explicit boundary of different parts?

[Dong]

Now I am working on how to represent the state of an Atari game for Reinforcement Learning. I read some papers that using the whole Ram as the state. But the wikipedia of Atari 2600 said that the 128byte could inculde three parts which are scratch space, call stack and the state of the environment. I am wonderring if this is right. And If it is, how to extract the state of the environment?

[+Andrew Davie]

The 6507 uses RAM in the form of a stack, which holds the return address when you call any subroutine (via "JSR" instruction).  The "S" register holds a pointer to the next-free RAM location in the stack, and is usually initialised with $FF. That is, the stack is living at the very top of zero-page memory. It MUST be in zero page on the Atari 2600 (hardwired), though on other machines it lives in other places.  So, as the 6507 goes "deeper" into nested JSR calls, then the stack grows downward.

 

The programmer has just 128 bytes of RAM, starting at location $80 and all the way up to location $FF. Note that the stack also has the same 128 bytes of RAM, starting at $FF and moving all the way down to $80. So, there's a clash - the programmer wants to put variables in that area, and the stack wants to grow to use that area, too.  This is one of the challenges of programming the machine - managing that RAM area efficiently and correctly so that there are no clashes.

Because, if your stack grows and overwrites the variables, probable program crash.  Usually the programmer figures how many bytes, maximum, will be required for the stack... and reserves them.  You figure this out by knowing how "deep" your JSR calls will go. 

The stack can also be used for very temporary, transient, storage of values. You can push a value onto the stack, and pull it back off. But you must leave the stack pointer in the correct state before returning from a subroutine. That is, it MUST point to the location of the return address from the subroutine.  Pushing and pulling from the stack is straightforward, but it's very much up to the programmer - some use it a lot, some try to avoid it.  If you do use the stack for temporary stuff, then you have less space for other variables.

Me, I use an "overlay" system for variables, that lets me define a common "scratch space" in RAM which gets used (and reused) by many different systems. As long as the systems don't try and use the same "scratch space" at the same time, it can appear as though you have heaps more RAM than you actually do.

 

As to "state of the environment" I can only assume this refers to the overall "state" of all the variables in RAM at any given time.  This is tricky and game-dependent - and time-dependent, too.  At some points the stack might be 5 calls deep (i.e., using 10 bytes) and maybe have a few temporary bytes pushed onto it, too. At other points, the stack is empty.

Not to forget that zero page is not necessarily the only RAM available. Many of the modern bankswitching schemes add extra RAM, the size and location of which varies based on the scheme and how the programmer decides to organise things. 

 

 

[DirtyHairy]

In addition, there are aspects of the state that are not represented in RAM at all. In particular, the horizontal positions of the hardware sprites are stored implicitly by the offset of various internal counters of the TIA. Depending on the way the game is implemented, they may also be represented by a number in RAM, but there are games where the position of a sprite is not made explicit as a number anywhere --- the game just performs relative movements after initial positioning.

[JeffJetton]

On 2/10/2020 at 4:22 PM, Andrew Davie said:

That is, the stack is living at the very top of zero-page memory. It MUST be in zero page on the Atari 2600 (hardwired), though on other machines it lives in other places.

As an aside, I've been digging into the memory map lately and found it interesting that, internally, the 6507 is actually "looking" for the stack in page one (from $01FF down to $0100). That's baked into the design of the 6502 (and therefore the 6507 too) and cannot be changed.

 

However address line A8 is not hooked up to the RIOT chip. So any attempt by the 6507 to read or write to anything in page one will have exactly the same effect as reading and writing to page zero (and the other way around). For example, when the 6507 tries to push a value onto the stack at $01FF (binary 0000 0001 1111 1111)  the RIOT sees it as being no different from pushing it onto $00FF (binary 0000 0000 1111 1111), since the underlined bit is irrelevant.

 

Lines A10 and A11 are ignored by the RIOT in the same way, incidentally. So the 6507 could also access that same stack location at $0DFF (binary 0000 1101 1111 1111) or $08FF (binary 0000 1000 1111 1111), and so on.

 

And, of course, lines A13-A15 don't even come out of the chip in the first place, although the 6507 doesn't "know" that. So that same byte will also be accessible at, for example, $E5FF (binary 1110 0101 1111 1111), etc. That's 6 out of 16 bits that flat-out don't matter when you're accessing RIOT RAM!

Edited February 12, 2020 by JeffJetton
Miscounted my bits!

[tokumaru]

The 6502/6507 always accesses the stack at $0100-$01FF, it just so happens that, on the 2600, page 1 is a mirror of page 0. As for the actual memory layout while have are running, there are no rules at all... programmers have complete freedom to use that space however they want. You're not even required to have the stack go from $FF down, the stack can actually start anywhere as long as you're careful and keep it from clashing against the game state variables.

 

Speaking of game state, there's absolutely no standard way to organize that and it varies wildly from game to game. Games programmed in the same framework (e.g. batari Basic) or using the same engine will end up having some RAM map similarities, since they share large portions of code.

 

Each game needs to remember different things, and calculate things in different ways, so they don't really adhere to any kind of standard when organizing that data in the little RAM that's available.

[Dong]

Dear All,

 

you guys are really nice. I am truely sorry to not reply in time. I didn't recieve the notification of this website.

After reading all your replies, I have some new questions. 

 

Even we have 128 bytes, the games still run very well. When we allocate some bytes for the games, this bytes store some really important information, such like in Pong game, the locations of ball and two board .Does those bytes will be moved to other place or temporily reclaimed when the game are running? 

I think it will not do that. For the important bytes in the ram, when it was "born" from the start, it will be reclaimed when the game is over. So if we took the whole ram or we extract the important bytes(by somehow, we suppose we can do it) as the state, it should be right because the important information are stored in the ram during the whole time. 

 

Am I correct?

 

Thanks in advance. 

[JeffJetton]

Depends on the game. As DirtyHairy points out above, not all games store/track the entire game state in RAM. So looking at the RAM will not give you all the important information. (I think Combat might be one of those games... maybe some of Video Olympics too?)

 

That is probably completely unlike just about every other computer system you might be familiar with. It certainly is for me.

 

It's sort of a holdover from the old days of video arcade games, which did not use a microprocessor and had no RAM in the traditional sense. Game state was stored all over the place, in a variety of counters, shift registers, various flip flops, and other hardware components. The Atari 2600 wound up being a bit of a hybrid machine, with the TIA chip (essentially a single-chip version of one of those old-school, discrete-component circuit boards) handling a lot of things all by itself, and the 6507 microprocessor being more of a coworker than a boss.

 

 

[tokumaru]

13 hours ago, Dong said:

Am I correct?

You simply can't assume that all code ever written for a platform will conform to any specific set of standards. Developers are free to use RAM as they please. Like it's been pointed out, some of the game state might not even be stored in RAM... part of it may be implicit from the state of the video hardware, for example. You asked specifically about the lifetime of variables, and that certainly can vary a lot from program to program. In a game with different play styles, the same RAM positions will almost certainly be used for radically different purposes depending on the current level or play style. RAM is also commonly repurposed during screen transitions, when the computations for the next screen may overwrite data that's no longer needed from the previous screen. Another example of radical changes in the meaning of RAM positions is when enemies die and new ones are spawned in their places... depending on the types of enemies/objects occupying these kinds of temporary slots, each position of memory may represent completely different attributes.

 

Quote

like in Pong game, the locations of ball and two board .Does those bytes will be moved to other place or temporily reclaimed when the game are running?

Probably not in a Pong game, which is really simple and will do fine without advanced RAM management, but in games with more complex worlds, 128 bytes is simply not enough to store all of it at all times, so in those cases you can expect more RAM to be shared among different things as gameplay progresses and they're activated and deactivated.