Back to blogs
As the technology driving games becomes more advanced, teams must be prepared to advance alongside it
Disclaimer: The opinions represented here are those of the individual and do not necessarily represent those of their employer.
The pace of game development isn’t slowing down anytime soon, a reality that People Can Fly’s Alexander Stokes knows all too well. As director of QA, he’s intimately familiar with the tech stack driving modern game development, tools that grow ever more complex. Modl sat down with Stokes to discuss his role, how he views the state of games QA today, and what it takes to lead effective teams in an industry that continues to evolve.
Watch the full interview below, or read on for a selection of key takeaways.
Key Takeaways
- Adopt a risk-based QA approach. Classify bugs by severity so teams can make informed trade-offs and avoid adversarial hand-offs.
- Hire for adaptability and innovation. Seek testers who can build new tools and frameworks to keep pace with evolving game systems.
- Elevate QA as the authority on player impact. Turn technical risks into clear gameplay effects to guide rollout and mitigation plans.
- Embed certification rules from the start. Early compliance checks prevent last-minute failures and costly delays.
- Use AI to automate repetitive tests, not replace QA. Rely on human testers for exploratory, context-driven evaluation that machines can’t replicate.
Approach QA From a Risk Assessment Mindset
QA teams have traditionally been seen as a sort of safety net, a line of defense intended to catch bugs before they slip into production builds. Stokes makes it clear that’s not the case. Bugs are a natural part of a game’s lifecycle, and rather than try to catch them all, QA’s job is to perform risk assessment. Reporting their findings allows other teams to make educated decisions about the project, thereby steering it in the right direction.
“QA comes from this waterfall idea that the product is finished and then you need to get it to [a certain level of] quality…But what I really tried to focus the team on…is QA is here to risk assess. Our job is to determine if something is low, medium, or high risk, because nothing is no risk. If the goal is no risk, QA will always fail…You really want to make sure that you stop your team from going to a combative mindset, which is that QA is the gate that blocks the development team from shipping a game. And instead, what level of risk are we willing to take?”
Risks don’t necessarily always pay off, which makes it all the more important to be aware of them. Fallout 76 released with technical issues largely because live service wasn’t something the studio had attempted before. But Bethesda at least knew there would be “bumps” heading into release, almost certainly thanks to the efforts of QA teams. Eight years of calculated decisions later, and the game is now in a much better and more stable state.
Build Forward-Looking QA Teams
Game development has evolved at an alarming rate, as the size and complexity of triple-A projects increase every year. For Stokes, the rapid pace of change has altered his mentality around hiring new developers. Rather than bringing on folks who can handle the problems facing the industry today, he’s started looking for people who can develop innovative solutions to problems, indicating that they’ll be able to handle whatever gets thrown at them in the future.
“I’ve had to kind of pivot to teaching the whole team to fish instead of giving them fish…I’ve been trying to model our department and my hiring practices, looking for people who are looking for the next big thing instead of just people who can do what I need right now. And so, one of the biggest changes for me in hiring and making a team has been that I need to look for people who are trying to find new solutions and innovative ideas for problems in front of them today.”
In the 1980s, a team of five developers could create a game with a few thousand lines of code. Such was the case with Super Mario Bros. But reality has changed. Game development today is defined more by the sheer scope and interconnectedness of its systems. Cross-platform services, live-ops pipelines, procedural content, and ever-expanding feature sets mean that no single QA checklist can cover everything.
Make QA the Determiner of Impact
Stokes urges developers to break down silos by having Engineering, QA, and Production leaders collaborate in more meaningful ways. Engineering should surface technical risks in any proposed change, QA should assess the potential impact on system quality and user experience, and Production should use both insights to plan rollouts and contingencies. By turning this into a genuine dialogue, QA becomes a strategic partner; its expertise and concerns drive engineering decisions and operational planning.
“It’s necessary to have both [relationships with production and engineering]…Engineering identifies risk. What is the risk in implementing a fix?… But QA has to be the determiner of impact…Production needs this information if they’re going to organize a plan. At a fundamental level… force our leadership at engineering, QA, and production levels to work together… Finding out how we can make it a two-way conversation…because not many people leave room for QA to talk about what’s important to them or give them the credentials that they rightly deserve.”
When a QA team is closely integrated with development, it often becomes the voice of a game’s risk and impact. For example, during Obsidian Entertainment’s development of Tyranny, QA tester David Benefield discovered a subtle but game-breaking bug. A combination of player choices could prevent the story from progressing altogether. He flagged it as a critical issue. Once shown the evidence, the team recognized the high impact on players and implemented a quick fix. The incident prompted Obsidian to elevate its QA process, as afterward, QA analysts were empowered to review content logic before it hit the game, ensuring such blocking bugs would be caught early.
Make Sure Everyone Understands Certification
One of Stokes’ biggest changes was ensuring that his team understood how crucial the certification process is to the end product. Every publisher platform has certain quirks; PlayStation, for example, insists that developers use its suspension menu rather than building their own. Without knowing the final requirements for certification, serious problems can arise.
“The best thing that I ever did…was have a QA tester…sit down and boot the game on their platform…just do a 20 minute recording of talking about all of the things to watch out for…all the restrictions of the network and how that should function…help educate them on what Sony or Microsoft or Nintendo will look for when you’re actually in the hot seat…because once you get to your certification pipeline the last six months, it may be too late.”
A recent case in point was Ubisoft’s arena shooter XDefiant. In mid-2023, just weeks from a hoped-for launch, XDefiant failed an important first-party certification test, forcing the team to delay release to address the compliance issues. According to Executive Producer Mark Rubin, they hadn’t fully anticipated the amount of work needed to meet Sony and Microsoft’s technical requirements. If the game had passed cert in August, it would have shipped, but instead the team spent “3-4 weeks fixing those issues” before resubmitting. Studios that invest in compliance-focused QA from the start can avoid similar costly last-minute delays.
AI Should Supplement QA Teams, Not Replace Them
As conversations about AI’s role in video game development shift, studios must sort through what’s realistic and what’s being overhyped. According to Stokes, the idea that AI could entirely replace human testers has been exaggerated. Today, the tools aren’t robust enough to tackle tasks that need genuine critical thought, and therefore are still just an arrow in the quiver for QA teams.
“The idea that AI can just replace a tester is where the hype is really overblown right now… AI can do specific things. It can test a golden path, it can test in a destructive way right now…But to really replace a tester who’s taking in a lot of complex stimuli potentially and trying to look and think and pivot intelligently is hard to do…when it comes to understanding when a player looks at a scene and decides what to do, that’s something that you still need a QA person.”
Major studios have adopted a similar mentality to what Stokes is pitching. Sega’s Ryu Ga Gotoku Studio, makers of the Yakuza/Like a Dragon series, revealed they’ve built an AI system that runs 24/7 to play-test their games, relentlessly repeating gameplay actions to try and flush out rare or unexpected bugs, while human developers take care of more meaningful work. As Stokes says, the idea isn’t to replace developers, but to take care of inorganic tasks that robots are suited for.
Enhance Your QA Team’s Capabilities
As the industry pushes the boundaries of what games can be, Stokes’ perspective is a reminder that successful QA requires the right mindset. It’s impossible to operate at the forefront of the industry without an openness to collaboration, innovation, and proactive problem-solving skills.
Looking to supercharge your QA efforts? Learn how modl:test, powered by our Behavioral AI Engine for Games, helps automate testing coverage while freeing your team to focus on what really matters. Explore more at modl.ai.
