As an emerging field, the aims and characteristics of usability and player experience testing can often be unknown or unrecognised within game companies, outside of a few key industry leaders, such as Valve and Bungie. This can often lead to comparisons being made with QA testing, or confusing usability testing as an element of QA. As an already established area of game development, it’s a common misconception that they are the similar, or the same fields. This is not the case
There are similar elements, as both involve considering the end user’s experience, and involve getting players to physically play the game. However, they have different goals, and this is what I will be covering today, by looking briefly at the aims of QA, the aims of usability testing and how they differ.
What is QA?
Quality Assurance (or Assessment) is an established field within game development. Often performed at both a developer and publisher level, it typically involves a room full of underpaid gamers endlessly playing a game in every conceivable way. They’re looking for bugs, which will be documented and then passed on to the coding or art department.
Typical bugs would be “the hitbox on that model is wrong”, or “when you shoot the tires on that Jaguar E Type, it makes a metal ricochet sound, not a rubber one”. So the task is largely monotonous, and will involve running into every wall in the game, and testing every dialogue choice, to find one which gives an unexpected result. Moving beyond bugs, QA often includes other areas such as localisation, compliance, and compatibility with a range of devices.
...and taste
Throughout the process, QA is trying to make sure the player can’t get themselves stuck, and no bugs in the game prevent them from completing, or enjoying the game.
What is Usability and Player Experience testing?
Similar to QA, usability and player experience testing involves playing the game. However, this typically wouldn’t be in a similar ‘farm’ setting. Instead, usability tests often attempt to recreate a typical playing environment, to emulate how a player would typically play the game (required: a sofa, and a 2 litre bottle of pepsi .)
Player experience is focused on whether a player enjoys what they do in the game, and whether they understand their goals. This can encompass many sub-categories of player experience, such as how is challenge and interest is maintained, why and where players give up, and how they understand the game. Essentially, the aim of player experience testing is to optimise the game so that people want to play the game.
Meanwhile usability testing blurs slightly more with QA, yet has some fundamental differences. Usability testing is focused on whether the game allows the player to achieve their goals, for example do they notice when something in game changes (such as picking up a new item), or do they understand where they have to go to complete the level. This is different to the ‘bugs’ that QA discovers, since these are game features, not ‘mistakes’.
Why do they differ?
Essentially this is the key difference between the two. QA focuses on the unintentional problems with a game, and aims to make the final product as close to the design features documented. A QA success would be a game with no bugs, and with an implementation that matches the designer’s vision.
My vision is money... lots of money
In contrast, usability and player experience testing aims to influence and guide the designer’s vision. By involving players, they gain an insight into how the game will be received by its audience, and help the designer create the reaction they’re after. As such, the focus is on the game’s intentional aspects, not its unintentional bugs.
This leads to different characteristics in the problems found. QA works largely in the edge cases, and tests every hit box, every wall, every enemy and every dialogue choice. Player experience and usability is much more interested in the average experience, to ensure that players ‘get’ it.
We can see a shift in the gaming industries perceptions of player experience and usability testing, having been championed by leaders such as Microsoft and Valve, more companies are starting to recruit user experience professionals. With the rise of free social games which require the player to ‘get’ games quickly, we can see that the field is becoming a key requirement for success, and is becoming a focal point of how games are made. But it’s not QA!
I find that there is little education of what QA does post 2010 in a lot of companies.
QA can, does and should in all extents provide usability/end user testing. In my company it does and QA is encouraged to do this across the board with respect (and they are not underpaid workers in a ‘farm’ setting). The majority of them are gamers themselves so have a broad resource base for their opinions and thoughts – it would be a waste not to use this to benefit the product, the department and the individuals forming the team.
Feedback on what elements are enjoyable, what makes the game re-playable, mission structure and flow, user options (customisation, faction choice, mission rewards, range of choice for weapons etc) are all integral as part of the in-house QA process.
While having close ties to how the game is “supposed” to be played as per the GDD, this is an even better reason to have those using it in this way to comment on why they have disagreements with it. We have received some of the most valuable (and honest!) end-user input from our QA department and have been able to tweak, fine tune and even rewrite systems based on their feedback throughout the production (and even pre-production) cycle.
Not all QA testers are treated the same, viewed the same, paid the same or behave the same and yes, it is Quality Assurance – the end user is the person that will eventually decide on what the quality of your game is after all.