The medium of video games has continued to see groundbreaking innovations as every year passes. It’s crazy to look at the increase in graphical fidelity, mechanical complexity, and modes of interaction since the early days of the Atari 2600 and the Nintendo Entertainment System. Sure, one of them almost killed the industry while the other saved it, but that’s a discussion for another time.

But of course, we’ve seen various improvements in artificial intelligence for games over the years. While for many players and even developers, when we think of AI for games, we think of non-player characters. But there are so many different ways in which we can apply both symbolic artificial intelligence and the more recent innovations in machine learning. So let’s take a look at some of the most exciting innovations for AI in games and some of the titles that helped make it a reality.

Creatures: The First Foray in Machine Learning in the 1990s

While we’ve seen many headlines of research in deep learning through the use of video games, the idea of machine learning technologies being commonplace in commercial titles is still relatively novel. Game AI for non-player characters has long been the domain of classic (symbolic) artificial intelligence. However, this doesn’t mean the industry has not experimented with the idea over time.

One of the most notable examples is Creatures, a game that dates back to 1996. Players are responsible for breeding and raising little characters known as Norns. These creatures interact with the world, learn by interacting with one another and even formulate their primary language to communicate. This is all powered by artificial neural networks that continue to change as a Norn develops, meaning that its behavior changes.

While this isn’t complicated, it was one of the earliest games that highlighted the ability of in-game characters to learn, evolve, and adapt through the power of artificial intelligence. For a deeper dive into its inner workings, we highly recommend checking out Alan Zucconi’s video which details how it all works.

Player Analytics and Tomb Raider: Underworld

With so many live-service games on the market, developers must stay competitive and relevant to their player base. A great way to achieve that is to be able to understand the behavior of your players so that you can make decisions on how best to improve the experience for everyone. Perhaps a weapon in Fortnite or a piece of armor in Destiny 2 needs a buff, so it proves more effective in certain situations. Or maybe a perk in Dead by Deadlight or a character in League of Legends need nerfed because they’re too powerful, and are disrupting the delicate balance of play, often referred to as the ‘meta’.

The application of data sciences – the idea of algorithmically processing data to learn more about the problems we’re solving – has proven highly influential on the modern games market. The origins of this idea stem from work conducted between Eidos Interactive and ITU Copenhagen on the 2009 release Tomb Raider: Underworld.

The ninth entry starring the ever-popular Lara Croft, was the subject of one of the first data sciences projects in commercial games. In the opening months after its release on the Xbox 360, the game would accumulate data on player performance courtesy of Xbox Live and feed it back to the development team. Further analysis of this data led to some fascinating insights. Allowing the development team to identify specific traits and personality types among the player base and where in the game world players were prone to dying more frequently. Even more impressive was the ability to estimate to a reasonable degree how much of the game a given player might complete based entirely on their performance in the opening level.

Forza’s Drivatar and the Rise of Player Modelling

While you can use player data as a mechanism to gain a more robust understanding of your player base, you can also – provided you have the right data – reproduce the behavior of a player for others to interact with.

There’s no better example of this than Drivatars: a system developed for the Forza racing series. Every Drivatar you go up against isn’t just an AI controller but a trained machine learning agent that seeks to drive like another player on the Xbox Network.

Drivatars started as a collaboration between Forza Motorsport developers Turn ten studios and Microsoft Research, in which players could train their own virtual driver by racing time trials on the Forza Motorsport series in 2005. The agents were bayesian neural networks trained locally on the owner’s own Xbox, given it was the only console at the time with a hard drive that allowed for the processing and storage of data. This feature continued to develop until Forza Motorsport 4 on the Xbox 360 in 2011.

Starting with Forza Motorsport 5 in 2013 on the Xbox One, the system was revamped and expanded to what we have come to know today. Drivatars are no longer processed on your console but the Xbox Network courtesy of Microsoft’s Azure cloud technology. The system can feed on a myriad of gameplay data, recording how players take different turn types, how you handle specific weather conditions, and your overall driving consistency. This is then uploaded to the cloud and begins to update the existing Drivatar to more accurately reflect your current skills. This helps ensure the Drivatar that is later downloaded to your friend’s consoles continues to evolve and improve as you do.

For more information, check out the episode of AI and Games on this topic, which also includes links to many relevant reading materials in the video description.

AI-Driven Testing in The Division 2 & Battlefield V

Let’s face it, there’s no such thing as a game (or any software) without bugs. But reducing how many of them make it into the final product is essential to ensuring a game’s release is successful. However, as games become larger and more complicated, the more difficult it becomes to keep bugs and other broken parts of the game from reaching players.

Here at modl.ai, we’re working hard on our AI-powered testing tools. But this idea has been percolating through game productions for some years now. Massive Entertainment’s series ‘The Division’ employs bots to evaluate server loads and run network tests, but it also has bots designed to run around and play the game like humans do. All in an effort to ensure mission designs work as they should, and that level events trigger as intended. Meanwhile, Electronic Arts introduced test bots to their ever-popular Battlefield franchise starting in 2018. Check out this talk by Jonas Gillberg from GDC 2019 that goes into a lot of detail on how they approached such a massive undertaking.

Improving Art with Texture Upscaling for Mass Effect: Legendary Edition

A recent application emerging from deep learning research is the idea of texture upscaling. This is where we take existing art assets and then use AI to make them larger and crisper than they were before.

This process, an application of generative adversarial networks, is interesting because two distinct elements operate the AI process. There is a generator that is fed the original art assets and is trained to create ‘forgeries’ that are similar but are typically larger and of higher resolution. Meanwhile, the second component is a discriminator that assesses the forgeries and tries to determine whether they’re real or not. The idea being if the discriminator can spot fakes easily, the generator has to work harder to create assets that can avoid scrutiny.

Since the inception of generative adversarial networks only a few years ago, it has become incredibly popular in modding communities for classic games. Titles such as DOOM, Deus Ex, Max Payne, and many more now look dated on modern graphical hardware. Even if you improve the renderer to operate at modern resolutions, the textures will continue to look fuzzy and rough. But texture upscaling it helps give them a fresh coat of paint for fans to revisit once again.

With the release of Bioware’s Mass Effect: Legendary Edition in 2021, we’re now seeing this process applied to AAA titles. As the development team remastered roughly 30,000 textures from the original Mass Effects games in an effort to improve the rendering targets from their original 720p to native 4K.

Deep Learning for More Challenging Racing Games

Given the recent innovations in deep learning and the success of Google DeepMind’s research in training AI to play everything from a myriad of Atari games to compete with the most prolific players of Blizzard’s StarCraft II, it was inevitable we would begin to see this technology appearing in major video game projects.

While many non-player character designs are not best suited for deep learning, given designers want to have the ability to customize contextual responses or tweak them to suit specific gameplay moments. But one area they’re very well suited for is racing games.

Building AI for racing games is often a real challenge, given you need to get drivers to follow race ribbons as efficiently as possible. While this isn’t insurmountable with even a simple rule-based approach providing working results, it becomes all the more complicated once the physics of the vehicle and the racing conditions come into play: an element of design that racing franchises have continued to embellish in the past 15-20 years.

We’ve already talked about the AI racers of Forza, but we’re now seeing the idea of deep learning for racing games continue to gain traction (no pun intended). Starting in 2019, the MotoGP series created their neural network AI codenamed A.N.N.A, while more recently, Gran Turismo 7 was released in March of 2022 with their own AI racer called Gran Turismo Sophy. Unlike Forza’s Drivatar, these are trained without the need to accumulate the data of existing racers. Instead, they are focussed on learning not just how best to race the tracks as effectively as possible but also doing so in a way that respects racing etiquette and ensures they do not become toxic or aggravate opponents in the process.

Closing

This is far from an exhaustive list, but it highlights new ways we can apply AI to games and how innovations in machine learning allow us to tackle new problems or improve on existing ones. It’ll be exciting to see how these innovations continue to permeate the industry in the coming years.