AI Agent57 plays 2600 games at superhuman levels

[RevEng]

 While AI bots that play Atari 2600 games is nothing new, this one is actually pretty special. It plays by just looking at the screen (instead of cheating by looking at memory locations) and excels at complex strategy games like Montezuma's Revenge and Pitfall. It was named Agent57 because of the 57 Atari 2600 games they measured it against.

 

There was good coverage about it at one of my favourite youtube channels, Two Minute Papers...

 

 

There's an overview by the DeepMind team here. If anybody is interested to delve deeper, the work was written up in this paper, which includes the list of 57 games used.

 

Everybody please put down your joysticks and stop playing Atari now. The robots have won.

 

Seriously though, this might be a neat way to test homebrew games, to see that the games are fair and beatable up to a certain level. I was also disappointed to see that they didn't run it against Dragster to see if it could break Todd's record ? nor did they pick Cosmic Ark, so we'll never know how it could fair against @ZeroPage Homebrew and his bid for the badge.

 

[+Muddyfunster]

Well that's my bed time reading for tonight sorted  

 

[+ZeroPage Homebrew]

1 hour ago, RevEng said:

Everybody please put down your joysticks and stop playing Atari now. The robots have won.

 

Seriously though, this might be a neat way to test homebrew games, to see that the games are fair and beatable up to a certain level. I was also disappointed to see that they didn't run it against Dragster to see if it could break Todd's record ? nor did they pick Cosmic Ark, so we'll never know how it could fair against @ZeroPage Homebrew and his bid for the badge.

 

Thanks for the tag! I love watching AI figure out videogames, they usually play them is such unexpected ways. The AI is so jumpy in Pitfall, hilarious. It would have been great to see it blast asteroids at crazy high levels on Cosmic Ark.

 

I'm subscribed to a YouTube channel that applies AI to a series of video games such as Pacman, Asteroids, Snake, etc... called Code Bullet. The videos are really interesting and also entertaining: https://www.youtube.com/channel/UC0e3QhIYukixgh5VVpKHH9Q/videos

 

It looks like he uses publicly available AI code if I'm not mistaken, so in theory somebody could set things up to play some 2600 homebrew?

 

- James

 

 

[RevEng]

I enjoy them too - funny, I already have seen the "A.I learns to run" and "A.I. learns to walk" videos from that channel. I'll check out the rest.

 

I laughed at Agent 57 playing Pitfall too. "You're going the wrong way dummy!", but super impressive anyway.

[christo930]

2 hours ago, RevEng said:

 While AI bots that play Atari 2600 games is nothing new, this one is actually pretty special. It plays by just looking at the screen (instead of cheating by looking at memory locations) and excels at complex strategy games like Montezuma's Revenge and Pitfall. It was named Agent57 because of the 57 Atari 2600 games they measured it against.

 

There was good coverage about it at one of my favourite youtube channels, Two Minute Papers...

 

 

There's an overview by the DeepMind team here. If anybody is interested to delve deeper, the work was written up in this paper, which includes the list of 57 games used.

 

Everybody please put down your joysticks and stop playing Atari now. The robots have won.

 

Seriously though, this might be a neat way to test homebrew games, to see that the games are fair and beatable up to a certain level. I was also disappointed to see that they didn't run it against Dragster to see if it could break Todd's record ? nor did they pick Cosmic Ark, so we'll never know how it could fair against @ZeroPage Homebrew and his bid for the badge.

 

If this is the same one that was going around about 2 years ago, don't believe the hype.

The AI was cheating in several ways, but the biggest way it cheated was running the 2600 emulator at full unthrottled speed and skipping 3 frames (running at like 15fps), only in black and white and at a lower resolution.  These "cheats" allowed it to play games at a speed a real 2600 simply could not play. Also, it wasn't really that generalized.   In the end, it wasn't even that good.  The only game it got 1/2 decent at was breakout.  From what I remember (I am sure) it was "looking at" the screen and not RAM inside  Stella.  Sam Harris had brought it up on Joe Rogan's podcast and was claiming the AI was better than any human player.  The AI had not even reached the level of the average child.

 

I do know that since then someone used an AI on arcade Ms Pac Man and had set some new record.

[RevEng]

37 minutes ago, christo930 said:

If this is the same one that was going around about 2 years ago, don't believe the hype.

The AI was cheating in several ways, but the biggest way it cheated was running the 2600 emulator at full unthrottled speed and skipping 3 frames (running at like 15fps), only in black and white and at a lower resolution.  These "cheats" allowed it to play games at a speed a real 2600 simply could not play. Also, it wasn't really that generalized.   In the end, it wasn't even that good.  The only game it got 1/2 decent at was breakout.  From what I remember (I am sure) it was "looking at" the screen and not RAM inside  Stella.  Sam Harris had brought it up on Joe Rogan's podcast and was claiming the AI was better than any human player.  The AI had not even reached the level of the average child.

 

I do know that since then someone used an AI on arcade Ms Pac Man and had set some new record.

The info is all there. The paper I linked to is from March 2020, and this is the paper the summary video is talking about.

 

The ALE framework they used, which is a bit of a standard now, has the following properties...

 

By default, each observation consists of a single game screen (frame): a 2D array of 7-bit pixels, 160 pixels wide by 210 pixels
high. The action space consists of the 18 discrete actions defined by the joystick controller. The game-handling layer also specifies the minimal set of actions needed to play a particular game, although none of the results in this paper make use of this information. When running in real-time, the simulator generates 60 frames per second, and at full speed emulates up to 6000 frames per second.

 

They run it full-speed so as to getting faster training. If you weren't impressed by previous AI attempts at 2600 games (I wasn't either) it's worth at least watching the summary video, before you write it off.

[christo930]

13 minutes ago, RevEng said:

The info is all there. The paper I linked to is from March 2020, and this is the paper the summary video is talking about.

 

The ALE framework they used, which is a bit of a standard now, has the following properties...

 

By default, each observation consists of a single game screen (frame): a 2D array of 7-bit pixels, 160 pixels wide by 210 pixels
high. The action space consists of the 18 discrete actions defined by the joystick controller. The game-handling layer also specifies the minimal set of actions needed to play a particular game, although none of the results in this paper make use of this information. When running in real-time, the simulator generates 60 frames per second, and at full speed emulates up to 6000 frames per second.

 

They run it full-speed so as to getting faster training. If you weren't impressed by previous AI attempts at 2600 games (I wasn't either) it's worth at least watching the summary video, before you write it off.

Running it at 6k fps is cheating as far as I am concerned because it drastically (by a factor of 100) speeds up the learning process.

 

When I get some extra time I will check it out.  But thanks for posting it.

[RevEng]

5 minutes ago, christo930 said:

Running it at 6k fps is cheating as far as I am concerned because it drastically (by a factor of 100) speeds up the learning process.

 

When I get some extra time I will check it out.  But thanks for posting it.

Nothing stopping you from playing the game at 6k fps and gaining the same advantage.

[RevEng]

Another update in AI-2600 news. This time the AI learned to play the 2600 games by watching youtube play-through videos, and is kept completely ignorant of any internal game states...