As games user researchers, we consider usability to be a major factor in the quality of games. However it is not the only aspect that makes games “good”. A game could have fantastic usability, and yet still be boring.
In order to get a snapshot of the importance of usability for game quality, I have looked at some games released last year and listed all the issues found in reviews. For each issues I’ve determined whether the issue is a usability problem, or whether it’s something else – anything from appeal (“the character is ugly”), to the concept (“we’re bored of farming simulators”).
From this I hope to gain a greater understanding of how important usability issues are to games reviewers.
Let’s put a disclaimer up front also – this isn’t an academic study and so I’ve picked some arbitrary games to look at – eurogamer’s reviews for the top 3, mid 3 and bottom 3 metacritic rated games from last year. Where eurogamer didn’t review the metacritic rated game, I moved onto the next game until I found one they had reviewed. I picked eurogamer as it’s my preferred review source – like I said, I’m not being academic!
First we’ll look at the findings, and then consider what this means about games usability and reviews.
The Games:
Grand Theft Auto 5
- Review: http://www.eurogamer.net/articles/2013-09-16-grand-theft-auto-5-review
- Metacritic: 97%
- Usability Issues: 0
- Non-Usability Issues: 4
- % of issues: 0%
- Sample usability issue: N/A
The Last of Us
- Review: http://www.eurogamer.net/articles/2013-06-05-the-last-of-us-review
- Metacritic: 95%
- Usability Issues: 0
- Non-Usability Issues: 2
- % of issues: 0%
- Sample usability issue: N/A
Bioshock Infinite
- Review: http://www.eurogamer.net/articles/2013-03-25-bioshock-infinite-review
- Metacritic: 94%
- Usability Issues: 0
- Non-Usability Issues: 2
- % of issues: 0%
- Sample usability issue: N/A
Killer Instinct
- Review: http://www.eurogamer.net/articles/2013-11-20-killer-instinct-review
- Metacritic: 73%
- Usability Issues: 1
- Non-Usability Issues: 7
- % of issues: 13%
- Sample usability issue: The depth of the combo system is initially hidden: “At first Killer Instinct comes across a little too combo-heavy, almost to the point of feeling mindless”
Amnesia: A Machine for Pigs
- Review: http://www.eurogamer.net/articles/2013-09-09-amnesia-a-machine-for-pigs-review
- Metacritic: 72%
- Usability Issues: 0
- Non-Usability Issues: 6
- % of issues: 0%
- Sample usability issue: N/A
Rain
- Review: http://www.eurogamer.net/articles/2013-10-01-rain-review
- Metacritic: 72%
- Usability Issues: 1
- Non-Usability Issues: 4
- % of issues: 20%
- Sample usability issue: Prompts are too obvious: “they often have to intrude into the action to guide you – occasionally with a flashing object like a switch or a grate – It’s like reading a fairy tale in which someone has helpfully underlined all the important bits for you”
Walking Dead: Survival Instinct
- Review: http://www.eurogamer.net/articles/2013-04-03-the-walking-dead-survival-instinct-review
- Metacritic: 38%
- Usability Issues: 2
- Non-Usability Issues: 8
- % of issues: 20%
- Sample usability issue: Interactive elements are hidden: “poxy doors that don’t open but look the same as the ones that do.”
Fighter Within
- Review: http://www.eurogamer.net/articles/2013-11-29-fighter-within-review
- Metacritic: 23%
- Usability Issues: 2
- Non-Usability Issues: 6
- % of issues: 13%
- Sample usability issue: The game doesn’t recognise its gesture controls: “Kinect struggles to comprehend the most basic of hand movements. Performing simple actions requires gargantuan amounts of perseverance and luck, and frequently leaves you frustrated, furious and exhausted. And that’s just the menu screens.”
Ride to Hell Retribution
- Review: http://www.eurogamer.net/articles/2013-07-10-ride-to-hell-retribution-review
- Metacritic: 13%
- Usability Issues: 8
- Non-Usability Issues: 7
- % of issues: 53%
- Sample usability issue: Unclear where to go: “the level design was so ambiguous about where I should go next that I pressed myself up against every unopenable door, of which there are at least a tantalising 10 in each level, for a chance to press E to open, only to realise that warehouse walls sometimes slide
What does this mean?
Usability issues are mentioned in some reviews, so they are a concern of reviewers and are presumably accounted into the game’s score. Furthermore, we can see that these top games reported no usability issues (and few non-usability issues). Reading the reviews could give the impression that high profile games do not have any usability issues – however this isn’t true. Having just played Bioshock Infinite there were a number of small usability annoyances that persisted through the game (Why do characters insist on talking over the Voxophones!).
So it seems that ‘lesser’ usability issues do not warrant a mention in the text of the review (although this doesn’t mean that they aren’t factored into the game’s score).
With the mid and lower-tier games more problems were found, which covered both non-usability issues (such as the misogyny in Ride to Hell Retribution), as well as the usability issues. In these reviews usability issues were identified and explicitly mentioned in the text of the review, and presumably factored into the score.
The impression I got from looking at these reviews is that while fixing usability issues won’t ensure you make a great game, it will stop you from making a terrible game. You can reach a mid-tier score (around 73%) with usability issues in the game, but the top rated games have spent the time and money to identify and remove usability issues, and this is reflected in eurogamer’s reviews.
With a lot more time, there may be benefit in performing this task over multiple review sources, and all games – which sounds like a dissertation topic to me!
Cool idea. It always makes me a little happy inside when I read a review that has a usability concern listed =)