This AI beat the world’s best Gran Turismo players

A team of researchers at Sony AI have used deep reinforcement learning to teach an artificial intelligence to play Gran Turismo at a world-class level. While previous experiments have taught AI how to drive very fast, this is the first time that one has learned to actually race. And to prove it, the AI beat some of the world’s best GT players in head-to-head competition, as described in a new paper published in Nature this week.

Racing is not easy, and it involves more than just knowing how to drive a car really fast. Car control is obviously important, but so too are tactics, strategy, and the somewhat nebulous concept of etiquette.

Or, as the authors put it, “[a]utomobile racing is a domain that poses exactly these challenges; it requires real-time control of vehicles with complex, non-linear dynamics while operating within inches of opponents.” Some drivers might have limited success through aggression and going for every overtaking opportunity they see. But knowing where to pass and when to wait for a better opportunity—so you don’t get re-passed at the end of the next straight, for instance—is at least as important, as is knowing when to cede to a rival so you don’t end up in the wall or a gravel trap.

Creating realistic AI opponents for racing games is not especially easy, as fans of the genre can tell you. Perhaps the most advanced approach we’ve seen until now has been in the Forza Motorsport franchise, which actually began life as a neural net project at Microsoft Research. Since 2005, Forza games have used data from human gamers to train NPCs, albeit with quite a lot of first-lap incidents along the way.

AI opponents have not been a strength of the Gran Turismo franchise in the past; in general, they have stuck to the racing line and have rarely offered much challenge, even to human players of average ability. So it’s all the more remarkable that Sony AI was able to train an AI called GT Sophy to out-race championship-level e-sports racers.

The AI agents were trained with a number of scenarios on three car and track combinations in Gran Turismo: Sport (Dragon Trail Seaside in a high-performance road car, Lago Maggiore GP in a Gr3 car, and Circuit de la Sarthe in a Redbull X2019 Competition car). Some races had the AI agent on track alone, and others added up to seven normal-game NPC opponents. Track positions, starting speeds, the spacing between cars, and the skill levels of the opponents were all randomized for each run.

The AI agent controlled the steering, as well as braking and accelerating (although not both simultaneously) and did not manually shift gears or adjust traction control or brake bias. And it interacted with the game at 10 Hz, which the authors say should not give it an advantage over professional gamers or athletes.

The AI was rewarded according to how well it progressed along the track since the last observation, and it was given a bonus for passing another car. Penalties were awarded for cutting corners, hitting walls, or slipping tires. It also had to avoid rear-ending NPCs or any other collisions, with all those criteria being weighted differently to select the optimum racing AI.

Learning how to get around the track only required a few hours of training, and within a day or two, GT Sophy was better than about 95 percent of GT: Sport players. After 10 days—45,000 driving hours in total—GT Sophy “achieved superhuman time-trial performance on all three tracks.” In fact, the humans were only close to GT Sophy’s performance on the Maggiore track, where its consistent lap times were about equal to the single best recorded human lap.

But as was already mentioned, other AIs have learned to set very quick lap times on their own, in games or even in real cars on real tracks. To prove GT Sophy’s skills, it went up against some of the world’s best GT: Sport players. The first human-versus-AI test happened in July 2021, when Emily Jones, Valerio Gallo, and Igor Fraga were asked to beat GT Sophy’s lap times in a time trial. Despite being able to see a ghost of the AI’s lap, none was able to beat it.

Interestingly, GT Sophy had identified some racing lines that had eluded even the world’s best human players. Jones was bested by the AI’s lap time at Dragon Trail, 107.964 seconds versus 106.417 seconds. “There were certain corners where I was going out wide and then cutting back in, and the AI was going in all the way around, so I learned a lot about the lines,” she said. “And also knowing what to prioritize. Going into turn 1 for example, I was braking later than the AI, but the AI would get a much better exit than me and beat me to the next corner. I didn’t notice that until I saw the AI and was like, ‘OK, I should do that instead.’”