Researchers, including a UT professor and alumnae, developed a new artificial intelligence program for Sony AI America that can outperform the world’s best esports drivers in the PlayStation racing game “Gran Turismo Sport.”

The researchers developed GT Sophy, the AI program, using a technique called deep reinforcement learning, which researchers have previously used to train AI to beat humans in games like chess, Go and Jeopardy. Peter Stone, executive director of Sony AI America, said Sophy is special because like real autonomous cars, it has to make decisions in real time at high speeds. 

“Gran Turismo is a very real simulation,” said Stone, a professor of computer science. “It models the physics of the world very, very accurately, including friction and wind resistance. … There’s really sort of a razor thin control window where the car, if it goes a little bit too fast, it’ll spin out. If it goes a little too slow, it loses the advantage. It’s got to be very, very precise.”

Stone said several of the employees at Sony AI America who worked on the project are former Ph.D. students from his lab at UT. 

“They were the people that I know and trust and who were trained in exactly the kinds of technology that we used to carry out this project,” Stone said.

Peter Wurman, director of Sony AI America, said deep reinforcement learning is a form of trial and error that is similar to teaching a dog new tricks and behaviors. 

“When you train your dog, you get it to do the right thing by giving it rewards and punishments as needed,” Wurman said. “One of the real challenges here was trying to find a way to encode reward signals that allow the agent to learn to be aggressive and fight for its position, but also be a good sport at the same time.”

Stone said Sony AI America is actively considering adding Sophy as a future update to the newest installment of the Gran Turismo series, “Gran Turismo 7,” but researchers are still in the process of improving the AI.

“The goal is not just to have an agent in there that is faster than anybody else can drive. We want to find a way for it to make the game playing experience more enjoyable and more fulfilling for people who play the game,” Stone said.