How to build affordable ML bots for online multiplayer games?

Flamebait games:
Forge and Fight case study

Forge and Fight is a PvP game specifically designed around a fast and chaotic team-based multiplayer dynamic, developed by Flamebait Games and launched in 2020.

Platforms

PC
Steam

Game

Forge and Fight!

Company

Flamebait Games was founded in 2016 by a team of students at the University of Skövde. It aims to develop games that explore experimental concepts and create fun experiences.

The Challenge

Forge and Fight is based around a simple idea; players battle each other with weapons they create themselves, based on parts collected and scavenged as they play. This means that players can combine parts in thousands of different over-the-top combinations, reflecting their own sense of humour and gameplay style. 

Waiting for players in multiplayer game

As a dedicated PvP game, Forge and Fight uses a matchmaking system to create teams of 4 to 6 in each game session. However, as with any multiplayer game, there is always the risk of there not being enough players available. To avoid this, Flamebait needed AI players to make up for any lack of human players. 

However, Flamebait wanted AI players that could adapt to the randomness of the weapon forging, and play either defensively or offensively, depending on the weapons and tactics of the other members of the team. On top of this, Flamebait also wanted the AI bots to be as close to human players as possible, so differences in behaviours between human players and the AI players was not an issue.

modl.ai worked with the team at Flamebait to build AI bots that could meet this unusual gameplay challenge

If we did the AI alone, I think we would only have been able to create a simple state-machine. That would’ve been a decent but very basic solution. The bots wouldn’t have felt human at all. Given our small team, we chose not to venture into creating bots from scratch and reached out to modl.ai instead. It was a cost-effective solution compared to what we would be able to produce in-house. AI is a sizable chunk of the production, and it seemed easy to detach and outsource.”

Mattias Lindblad – CEO

modl.ai’s Solution

Forge and Fight’s team-play dynamic added extra complexity to the challenge of adding bots to the game since they needed to be both interesting opponents and reliable and useful teammates.

Game play Forge and Fight

We approached the design challenges in two ways. Firstly, we used a ‘classical AI’ approach to implement bots into the game that could react to other players in a reasonably ‘intelligent’ way. There is nothing inherently difficult about this, and the technology and techniques are well understood. However, the result is a bot that works but is not especially interesting to play with or against. Whilst the level of sophistication can be high with bots created in this way, it’s easy to tell the difference between the bot and a human player.
Therefore, the second design challenge was to introduce more variables into the behaviour of the bots to give the impression of unpredictability and risk that you often encounter with human players. We did this by using Imitation Learning to train the bot, using data collected from over 3000 matches that were recorded during the game’s early access period.

With the imitation data adding that crucial ‘human’ element to the AI players, we then added a machine learning component to give the bots the ability to adapt to the playing style of teammates, using the most appropriate moves and tactics from the available options, and turning the bot into a more dependable, dynamic teammate.

Crafting Over-The-Top Weapons to Defeat Your Opponents

When creating the AI players, we also needed to allow for Forge and Fight’s myriad of weapon options thanks to its weapon crafting feature.

Scripted vs modl:play bots

Because we had so much great data from the recorded matches we were able to segment the different weapon configurations into groups, for example, whether they are melee or ranged weapons, are best used defensively or offensively, or the effect they had on the players themselves – such as restricting movement.

We fed this data into the AI players, with the result that their behaviours and play style changed according to its weapon choice – exactly as a human player would. 

Final Result

While the combination of techniques used to create the AI players was a fairly new concept, we didn’t hit too many technical challenges during the project, thanks to the knowledge and experience of the modl.ai team.

The end result of our hybrid AI approach is a dynamic, adaptive and very human-like bot that would have taken many more resources if solved solely with a classical AI or Machine Learning approach.