Back to blogs
Generative AI is getting a lot of attention these days, and it’s no wonder, seeing as these programs and apps stand to transform entire industries, but there is another aspect of artificial intelligence that is also gaining speed in the game development world: AI bots for testing. The process of building virtual AI ‘players’ that can sit and play games during development and a rate and scale far beyond what a human can do in a similar amount of time.
These handy little bots can take quality assurance (QA) and quality validation (QV) to new levels early in the development process—which can help bring new games to market with dramatically fewer bugs—but they are also being used to liberate developers, enabling them to unleash their creativity for higher-level tasks.
Fast-forward testing
QA is not just a final touch-up for a product; it starts at the earliest stages of development. Even from the beginning, it’s important to confirm that the product works as intended, and this can often take significant resources and time as games continue to grow during development, while the bug list often grows alongside it! This is why developers are increasingly turning to bots to speed up this process. Helping ensure entire sequences of the game continue to hold up, while also deploying them to find the most persistent of critters hiding deep inside the codebase.
Time is always of the essence in game development. A developer may be reworking parts of a level, and forces either themselves or a QA tester to replay the entire sequence to ensure it still works. The problem is it might take 45 minutes to play from start to finish. But a bot can be programmed to move through the game at lightning speed, which means its journey will not necessarily reflect a human experience, but that is acceptable at this stage. In a shooter game, the bot doesn’t have to find an enemy, line up the shot, confirm the hit, etc.; it can be programmed to take down the enemy with a glance. Therefore, a bot can complete a game in a couple of minutes as opposed to 45. If there’s a bug at the 44-minute mark, the developer will know in seconds. As the development progresses, it gets easier and easier to use bots for validation—and with that, they automatically deliver a wealth of functionality data on a silver platter.
A great example of this is in the 2023 remake of “Dead Space” by EA Motive, which deployed this exact process, by having bots play through each chapter of the game on a daily basis. While many of the smaller bugs might still be in there undetected, it helped re-affirm to the development team that the game was still functional, and playable day-in day-out.
But it’s not just about finding the bugs in the game, it’s also about measuring performance. A bot can cross-reference against an engine-profiler to spot when the memory footprint changes, or the framerate drops and how it relates to their in-game activity. Given the rapid rate at which games can change in shape, and size across a given week – or even a day – of development, the developers can use a bot to play through the day’s efforts once the team has gone home, or during an end-of-week playtest. This can spot new issues, confirm the re-emergence of old ones, and help prioritize tasks for the next round of development.
This was a driving force behind the AI testers deployed in “Tom Clancy’s The Division” franchise by Massive Entertainment. In the original game, Massive deployed bots to help assess the performance of the game servers, ensuring that the systems in place can handle players logging in, dishing out carnage, and logging off again. During development of post-launch content for the first game, they also added a second set of bots that operate more akin to individual players. They run individual missions, identify runtime issues that might occur on different hardware profiles, and report back on whether the game is functionally playable. To find out more about how these bots were deployed in the sequel, check out the video by the YouTube channel ‘AI and Games’ which includes an interview with The Division’s lead AI programmer Philip Dunstan.
When developing multiplayer games, having a basic bot early on can prove beneficial to facilitate the creation of a robust infrastructure. You’re making a 64-player online multiplayer game, but you don’t have the time, or the people, to drop in and test the game with a full server every day. Bots can stress test the game even during the early stages of development. EA’s “Battlefield 2042” deployed AI bots early in development to help ensure that the servers could not only handle the requirements of the overall game design but also play through the game and check that basic mechanics worked as intended.
You may even, over time, find that the bots can become proficient not just as testers, but as possible opponents for end users; resulting in a new type of gameplay experience. Games such as “Titanfall” and “Star Wars: Battlefront 2” deploy bots that serve as cannon fodder. But that said, despite being weaker, a hoard of bots can still pose a threat, which results in engaging gameplay, particularly if the player is having a challenging moment.
Instant Insights
Bots can alleviate the workload for developers. It is especially apparent when addressing validation in specific areas, such as designing puzzle levels for puzzle games. It is important to ascertain whether a puzzle has a solution or if multiple solutions are available – the latter being preferable as it encourages variation and creative problem-solving. If there is only one solution, players may share it among themselves and limit the experience. By employing a bot, it could be possible to instruct them to explore numerous ways to solve a puzzle. If, for instance, they discover 40 different solutions, determining the ideal number of solutions can be subjective and context-dependent, so it is up to the developer to make them strategically decide whether that is too many or too few. Additionally, employing procedural generation and utilizing bots can streamline the creation of new game levels. Bots can swiftly validate the feasibility of levels based on predefined metrics, thereby enabling rapid iteration.
A great example of this in practice is with “Candy Crush” developers ‘King’, who employ AI solvers to playtest levels alongside human players. This allows both for a more exhaustive approach to examining the value of a puzzle from a quantitative perspective, while still having humans bring an equally useful qualitative perspective as to the quality of the experience for end users.
Having a bot that can play through levels is highly advantageous as it provides insights into the complexity and uncovers potential design flaws. These insights can be obtained in a matter of seconds, as opposed to the months of testing required when relying solely on human testers. Nevertheless, it is important to recognize that there are many facets of a game’s quality that a bot cannot effectively evaluate. Human creativity is vital in areas where bots are unable to test, such as identifying inaccurate texture mapping. Our brains can quickly recognize visual elements like a fire hydrant with tree-bark texture, yet training an AI for such tasks remains challenging.
Conclusion:
AI Bots are proving to be an invaluable asset for creative game developers. They can quickly and accurately test the functionality of a game as it evolves while providing instant insights into complex elements. Additionally, they can help create a robust infrastructure for multiplayer games and even become part of the gaming experience themselves. Ultimately, with automated testing and analysis capabilities, AI bots are becoming game-changers for game development.
