Computer A.I. learns Atari game from scratch

Geoffrey Mohan
Los Angeles Times

Computers have beaten humans at chess and “Jeopardy!,” and now they can master old Atari games such as “Space Invaders” or “Breakout” without knowing anything about their rules or strategies.

Playing Atari 2600 games from the 1980s may seem a bit “Back to the Future,” but researchers with Google’s DeepMind project say they have taken a small but crucial step toward a general learning machine that can mimic the way human brains learn from new experience.

Unlike the Watson and Deep Blue computers that beat “Jeopardy!” and chess champions with intensive programming specific to those games, the Deep-Q Network built its winning strategies from keystrokes up, through trial and error and constant reprocessing of feedback to find winning strategies.

“The ultimate goal is to build smart, general-purpose (learning) machines. We’re many decades off from doing that,” said artificial intelligence researcher Demis Hassabis, coauthor of the study published online Wednesday in the journal Nature. “But I do think this is the first significant rung of the ladder that we’re on.”

The Deep-Q Network computer, developed by the London-based Google DeepMind, played 49 old-school Atari games, scoring “at or better than human level,” on 29 of them, according to the study.

Deep Q “can learn and adapt to unexpected things,” said study author Demis Hassabis of Google DeepMind in London. “These types of systems are more human-like in the way they learn.”

In the submarine game “Seaquest,” Deep Q came up with a strategy that the scientists had never considered.

“It’s definitely fun to see computers discover things that you didn’t figure out yourself,” said study co-author Volodymyr Mnih, also of Google.

Deep Q had trouble with “Ms. Pac Man” and “Montezuma’s Revenge” because they are games that involve more planning ahead, Hassabis said.

Associated Press contributed.