Watch a computer beat one of the world's best 'Super Smash Bros.' players

Pretty soon, there won't be any games left in which the world's best human players can still beat a computer program.

After artificial intelligence managed to best human experts in both Chess and Go, one of the next games that machine-learning researchers have set their sights on is the classic Nintendo game "Super Smash Bros."

Here's video of an computer program - an artificial intelligence "agent" - beating some of the world's top players at "Super Smash Bros. Melee," a Gamecube game that came out in 2001. 

The computer is playing as Player 2, Captain Falcon: 

The AI agent playing as the black Captain Falcon was developed by a team led by Vlad Firoiu, a grad student at MIT studying artificial intelligence. The team published their findings and methods in an unreviewed paper on ArXiv earlier this week. 

What's notable is that the computer is hanging tough with real human players in what is a very complicated game - and it didn't require the resources of a big Google subsidiary to get to that stage, only a few researchers and a technique called deep reinforcement learning. That's because of a "a recent explosion in the capabilities of game-playing artificial intelligence," according to the paper. 

Vlad's AI, nicknamed Philip, uses well-known and well-studied "off-the-shelf" algorithms, including Q Learning and Actor-Critic, to teach a computer to play "Super Smash Bros." from experience. These aren't video game-specific algorithms, but they can learn how to play as Captain Falcon just the same, given enough data. 

In fact, the data came from the computer playing itself over and over again, a technique called reinforcement learning. 

The computer doesn't play exactly like a human, though. It was "hard to discern any real trial or error it was doing, let alone what strategies it was applying," Mafia, also known as James Lauerman, told Business Insider. (He starts playing in the above video at 3:00, and advises that the "situations that take place between 3:00 and 3:40 are nuts.")

"What I think is most interesting is the edge play. He gets me on the ropes on the right edge and I somehow get back. But I remember thinking, how is it so smooth sometimes?" he said. 

Ultimately, "it was weird," the 50th-ranked player in the world said.

Caveats

One reason why Mafia may have thought the agent was "moving so fast he's kinda just chilling in the same spot" is that the computer program has faster reflexes than humans. 

That's one of several simplifications the researchers decided to make to run its experiment. 

Here are a few other ways that the match is not like a real competitive match:

• The agents react faster than humans: "The main criticism of our agents is that they play with unrealistic reaction speed: 2 frames (33ms), compared to over 200ms for humans," the authors write.
• Another issue with the simulation is that it does not understand projectile attacks at all. Nearly every character - except for Captain Falcon - has an attack that sends a projectile. 
• The agent doesn't play Smash Bros. the way a human does, by looking at the pixels on the screen and reacting. Instead, the software reads where the characters are from the game's memory that "consists of things like player positions, velocities, and animation states," Firoiu told Business Insider. 
• There were also odd strategies that could beat the computer. "One particularly clever player found that the simple strategy of crouching at the edge of the stage caused the network to behave very oddly, refusing to attack and eventually KOing itself by falling off the other side of the stage," the researchers wrote. 

What's next? 

Still, the research has a lot of interesting findings that may mean more for AI researchers than Smash pros. 

One interesting finding was that transfer learning, a hot topic in deep learning, applied across characters. This means that an AI agent trained on, say, a character like Fox McCloud, found its skills also applied to characters like Captain Falcon and Peach as well.

"I suspect transfer learning works because many of the fundamentals (how to move, attacking in the direction of the opponent when they are close) are broadly applicable to any character," Firoiu said. 

Another interesting finding: The difficulty of training a computer to play with a given character corresponded with how difficult pros think each character is to play. 

"The data also reveals the overall ease of playing each character - Peach, Fox, and Falco all trained fairly quickly, while Captain Falcon was significantly slower than the rest. This to some extent matches the consensus of the SSBM community," the researchers wrote. 

Firoiu says he plans to continue working on the project. His next step is to play with human-level reaction time. "This would eliminate some of odd strategies the bot uses, and force it into the realm of play a human could relate to," he said. 

Check out the entire paper on ArXiv, and you can look at the researchers' code on Github.