AI masters 49 Atari 2600 games without instructions

[+SpiceWare]

http://arstechnica.com/science/2015/02/ai-masters-49-atari-2600-games-without-instructions/

[Thomas Jentzsch]

This is pretty impressive. Though the Space Invaders AI still needs more practice to learn more strategies.

 

Is the complete list somewhere available? I would like to watch more videos and see how really complicated games are played

 

So far I only found:

 

And they are not alone:

 

Also search for HyperNEAT videos.

Edited February 25, 2015 by Thomas Jentzsch

[+SpiceWare]

If you're a subscriber to Ars Technica then you can take the link at the bottom, 10.1038/nature14236, to read the original article in Nature. The original article only has the 2 videos as Ars Technica.

The games, in order of mastery, are:

• Video Pinball (did best at)
• Boxing
• Breakout
• Star gunner
• Robotank
• Atlantis
• Crazy Climber
• Gopher
• Demon Attack
• Name This Game
• Krull
• Assault
• Road Runner
• Kangaroo
• James Bond
• Tennis
• Pong
• Space Invaders
• Beam Rider
• Tutankham
• Kung-Fu Master
• Freeway
• Time Pilot
• Enduro
• Fishing Derby
• Up and Down
• Ice Hocky
• Q*bert
• H.E.R.O (last of at-or-above human level)
• Asterix
• Battle Zone
• Wizard of Wor
• Chopper Command
• Centipede
• Bank Heist
• River Raid
• Zaxxon
• Amidar
• Alien
• Venture
• Seaquest
• Double Dunk
• Bowling
• Ms. Pac-Man
• Asteroids
• Frostbite
• Gravitar
• Private Eye
• Montezuma's Revenge (did worst at)

 

[Thomas Jentzsch]

Thanks for the list. I am not surprised that Montezuma's Revenge is at the end of the list. The AI seems to be best at reaction based games and worst where it comes to real brain work.

 

Looks like we are still (as during the last 30 years ) 10 years away from real artificial intelligence.

[Big Player]

Here is a link to a preview of the paper in Nature. Nice graph there listing out the games.

 

http://www.nature.com/nature/journal/v518/n7540/full/nature14236.html#access

 

 

 

Here is the Breakout video.

 

https://www.youtube.com/watch?feature=player_embedded&v=cjpEIotvwFY

Edited February 25, 2015 by Big Player

[Torr]

That's both really cool, and a little frightening.

[Mr SQL]

Deep Q sounds a lot like Deep Blue, the last computer to pass the turing test - and then immediately dismantled after IBM's stock soared; problem was there was a human ChessMaster hidden inside the machine, it was via ethernet but otherwise no different than the Turk.

 

This has been done many times before, the gag is hundreds of years old! Star Trek did a more convincing job in the 60's and Star Wars too with C3P0 and R2D2 in the 70's.

 

If I had to pick the blurb was right out of Star Trek's M-5 Ulitmate computer manual...

 

Captain Dunsel

 

 

 

[Thomas Jentzsch]

Deep Q sounds a lot like Deep Blue, the last computer to pass the turing test - and then immediately dismantled after IBM's stock soared; problem was there was a human ChessMaster hidden inside the machine, it was via ethernet but otherwise no different than the Turk.

Fact or fiction?

[Mr SQL]

Fact or fiction?

 

It's a fact that a group of ChessMaster's were networked to Deep Blue; they were only supposed to tune the machine but Kasparov's analysis of the game indicated either the machine passed the turing test or they couldn't resist making a few moves for it hence he was playing against an amalgam of man and machine (Transhumanism cyborg opponent and not just a computer). IBM had a lot riding on it and their actions afterwards support his conclusion - hey we've finally got a sentient computer, quick take it apart! Right.

 

Using a bio-inspired system architecture, scientists have created a single algorithm that is actually able to develop problem-solving skills... a neural network learning computer mirrored after Dr. Richard Daystrom's own brain.

Do you believe it?

[fiddlepaddle]

Interesting...

 

It is a fact that, at least going forward, the most capable AI is/will be a combination of machines and people working together to solve problems.

 

As a thought experiment, it is obviously interesting what a machine can do alone, but as a practical matter, the cyborg thing is the thing.

[Thomas Jentzsch]

It's a fact that a group of ChessMaster's were networked to Deep Blue;

If this is a fact, then why is Wikipedia questioning it? You (and Kasparov) may believe it is true, but there are no evidence as far as I can tell. Just speculations.

[Keatah]

It is a fact that, at least going forward, the most capable AI is/will be a combination of machines and people working together to solve problems.

 

Absolutely.

 

I am a little dismayed that I spent thousands of hours playing VCS games only to find a whippersnapper 2 month-old-machine intelligence can do them better. Except of course Missile Command. I can beat anybody any time any place. Including whatever world record is out there should I choose to do so.

[Thomas Jentzsch]

I can beat anybody any time any place. Including whatever world record is out there should I choose to do so.

If the AI should become perfect, than you can maybe match, but not beat it.

[Thomas Jentzsch]

Here is a link to a preview of the paper in Nature.

Can you give some details about the meaning of the three bars?

 

E.g. what does it mean that for Double Dunk the blue bar is very short compared to the gray one? And why the thin line is shorter than the gray one?

[Mr SQL]

If this is a fact, then why is Wikipedia questioning it? You (and Kasparov) may believe it is true, but there are no evidence as far as I can tell. Just speculations.

Kasparov is a GrandMaster; can you find another player of his caliber who agrees with Wikipedia Tom?

 

This program has specific routines (rules engines) that handle pac-man type games, jumping games, etc. No doubt it could not play Kool-aid man unless another rules engine was first added - that is not learning but following instructions which is all computers can do outside of science fiction.

 

There are awesome programs that can write 6502 Assembly (batari BASIC is one of them!) and turn GUI actions into sprawling blocks of high level code, but none of them are writing code on their own anymore than they are mastering games on their own. If the latter were possible the AI could continue learning and start writing some really cool games. It cannot because It is not sentient and creative as claimed.

[Thomas Jentzsch]

No clue how that relates to Deep Blue. AFAIK that never was claimed to be creative or even learning. And regarding cheating, Kasparov is for sure biased.

 

As for playing Atari games, I see a very limited learning here. It seems pretty linear, so Deep-Q fails on anything which requires a bit more complex logic.

 

But I could be wrong, maybe if you give it (much) more time than just a few 100 attempts, the AI would be able to play well at least something rather simple as River Raid. Besides refueling, there is nothing really complex in there.

 

Also it would be interesting to see how an AI which has been successfully trained with e.g. Demon Attack would react when it is facing a similar game like Phoenix. IMO, if it would be able to adapt to different scenarios without having to start from scratch, this comes close to intelligence.

 

If would sure be fun if the required framework would become public and we could do our own experiments.

[Trinity]

I'd have liked to see it fared at Missile Command.

[Keatah]

This is one of the "mysteries and questions" I had when I was a kid. I didn't know what was in IC's at an early age. I thought there was magic in them and went on a spree testing all sorts of AI scenarios. Those random number generators really had me fooled! Games behaved different each time I played them, but stuck true to what they were programmed for.

 

Later on I sadly discovered it all came down to the transistors following elaborate instructions.

[Keatah]

I'd have liked to see it fared at Missile Command.

 

..or a human opponent in a 2 player game.

[Thomas Jentzsch]

 

..or a human opponent in a 2 player game.

Probably the human opponent would be bored to death before the AI would have a chance,

[Emehr]

I'm kinda surprised to see Freeway so far down the list, ranking below the much more complex Robot Tank. The only game with a simpler control scheme is Connect the Dots!

[Mr SQL]

No clue how that relates to Deep Blue. AFAIK that never was claimed to be creative or even learning. And regarding cheating, Kasparov is for sure biased.

 

 

... If would sure be fun if the required framework would become public and we could do our own experiments.

 

That is precisely what Kasparov claimed upon analyzing the moves, that creative human chessmaster intervention occurred multiple times hence the machine otherwise passed the turing test! Seems to me his opinion is the least biased for being a grandmaster and thus the most expert in the subject matter.

 

The required framework and process of adding (programming) specific handlers for each genre ahead of time will doubtless remain forever shrouded like IBM's Deep Blue and Star Trek's M-5.

[Thomas Jentzsch]

Seems to me his opinion is the least biased for being a grandmaster and thus the most expert in the subject matter.

My logic tells me, that he was is the most expert but also the most biased. Because he was the first champion who lost to a computer. Calling out the opponent a cheater in such cases is a normal human behavior.

 

Therefore his claim would have to be verified and confirmed by a majority of other chess grand masters. I cannot see that this happened.

[+FujiSkunk]

I agree with Thomas, but having said that...

 

Therefore his claim would have to be verified and confirmed by a majority of other chess grand masters. I cannot see that this happened.

 

And since IBM dismantled Deep Blue immediately afterward, this claim cannot be verified, one way or the other. Regardless of any bias on Kasperov's part, that's a bit suspicious to me.

[+SpiceWare]

Same here, speculation by the guy who lost doesn't prove anything.