This week I gave a talk at The Research Thing on how Simulator Sickness can be identified and measured in virtual reality games, and some best practises for avoiding it. It’s based on my experience from PlayStation, and a lot of good research done by others.
I’ve shared the slides below, and my notes. The video of the talk is available here.
So here we go…
I was the lead researcher for two of the launch games for PlayStation VR, PlayStation VR Worlds and Until Dawn: Rush of Blood
I also worked with other researchers from PlayStation’s team on their other launch titles, including Rigs and the installation process for the VR headset itself, in addition to working on many other PlayStation titles.
A few months ago I left PlayStation to lead the research team at Parliament. They are a great team working on some really exciting stuff, but nothing related to VR – so for today my talk is based on my time at PlayStation and as a member of the Games User Research community.
Which brings me to today. I’d like to talk about Video Games. One of the many reasons video games are an exciting medium is because they allow you to have experiences that you couldn’t have in real life.
For example, they let you travel to exciting new worlds. See amazing sights and vistas. Meet new civilisations of alien life forms, completely different to anything you could possibly imagine.
And when you meet them, you can shoot them in the face…
Or you can travel to a city above the clouds – representing a lost American state following Christian Utopian ideals, and explore ideas of community and individualism beyond our every day experiences.
And, when you meet them, you can also shoot them in the face…
Or you could travel back in time and explore Roman or Greek Mythology. The Colliseum, The Fields of Elysium, Mount Olympus…
…and when you are there, you can punch Zeus in the face.
My point being that games lets you have experiences that you would’nt be able to experience in real life. Virtual Reality takes that one step further. Rather than being separated from the experience by the screen, it allows you to actually be *inside* the action, and creates presence – the feeling of being in that environment.
When Virtual Reality started becoming a technical possibility, this lead to the potential of whole new types of experience, perhaps even some that don’t involve shooting someone.
For a while things were good. People experimented with new ideas and scenarios, and explored the breadth of things that could be done in VR. But then reports started coming back that some experiences were wrong, and were making people feel sick. Some so sick that they wouldn’t want to play VR again. This is … simulator sickness!
For the purposes of today, Simulator Sickness is very similar to Motion Sickness, with symptoms such as nausea, stomach awareness and dizziness.
It is triggered by the perception of motion despite being sat still. There are some great talks about the reasons why this occurs, including one by Ben Lewis Evans that I’ve referenced many times, but one theory is that it’s caused by the disconnect between your eyes and your brain.
Your eyes report that you are moving, because they can see things moving past them. Your brain has other ways of telling if you’re moving, and it says that you’re not. There’s a disconnect, and when there’s a disconnect, bad things happen.
So, simulator sickness is a thing. If you are building a VR game or experience, the question is what should we do about it?
Luckily we have a tool to help. In the nineties, the army discovered this issue with sickness when working on flight simulators, and developed the Simulator Sickness Questionnaire. This is a list of 16 symptoms associated with simulator sickness, and each can be assigned a rating from None->Severe based on how the participant is feeling. This gives us something we can use to measure sickness.
The first thing we want to do is work out if there is a problem. A potential way of doing this is by running a small study, playing a representative part of the game, and using the sickness questionnaire before and after. It’s important to take a pre-score as people may come in feeling sick already, and you don’t want that to impact your results.
Each of the symptoms can be converted to a score of between 0 and 3 based on the level of sickness reported, and these can be aggregated and averaged to come up with a ‘sickness score’, somewhere between 0 and 48.
The challenge is deciding what score is ‘ok’. Because you’ve primed people to think about sickness, many will report some level of sickness. Also some people will feel sickness regardless of the contents of the VR experience, so a score of 0 is probably impossible. Perhaps as a rule of thumb a score between 0-10 is fine, a score between 11-30 can be improved, and a score of 30+ is a irredeemably bad experience and you should give up with the whole thing.
So, if your sickness score came back acceptable, that’s great – you can relax and not read on any further.
However we’re still here, so presumably there is a problem with your game or experience. The next thing to do is to ask yourself – do I have a lot of money and VR is very very important to my organisation. If yes, you should A/B test. If no, we’ll look at heuristic review.
Let’s imagine we are the big company, and look at what we could do.
A/B testing is a really powerful tool in this case. VR experiences are large and complicated and have many aspects, and we need to work out what aspect of the experience is causing sickness. A/B testing could allow us to do that.
Once we’ve identified what the problem is, we need to tell if we are fixing it or not. Again, A/B testing can help with that.
It’s science.
Perhaps everyone is familiar with the idea of A/B testing. In short, it’s changing one single aspect of an experience, and measuring it to see if it’s had an effect. In this context, we can alter aspects of the VR experience (the controls, the movement, the textures) and the sicknesss questionnaire gives us the ability to measure the effect. This will generate insights about what is causing sickness in the experience, and what we can do to fix it.
Sounds simple right?
It’s not. I have to thank Cyril for explaining this to me, many times, in simple words.
First of all, sickness is hard to measure.
- It is a subjective measure, and people’s perceptions of sickness are different. What one person may consider mild, another may consider severe. This introduces noise in the data
- Also people have different levels of tolerance to VR, and the same experience may cause different reactions in different people. Again introducing noise.
- Unless you are making completely radical changes, the differences between versions of the game would be very small. With the noise, this makes it hard to notice the effect of your change
- And people build up a tolerance to VR. This means that we need new people each time, otherwise their prior VR experience will influence the results
The impact of all of this is we need a lot of people to come in, to mitigate the impact of noise in the data.
However, not many people own VR headsets. That means they need to come to you. That’s a lot of travel and transport to arrange, which takes a lot of time.
Lastly, because this is science, we need to be rigourous. This means making only one change between each version of the game, and ensuring that their experience is identical otherwise (by duration, what they do in it, etc). This requires a high level of development effort to achieve.
So, there are a lot of reasons why measuring this is difficult.
None of these problems are insurmountable, however they do require a lot of time and money to overcome.
What can we do if we have no time or money?
Luckily, lots of other companies have had time and money and have done work on this already, which researchers can review against. Here are a few that have been highlighted by other talks, but there are many other important factors and there are references to some at the end of this presentation.
First of all, technical performance is key, and if you aren’t able to achieve this it’s not worth looking at anything else.
The frame rate required by VR experiences is extremely high, and needs to be stable, which introduces a high technical load. This means the experience will have to compromise elsewhere, such as with the graphics – but it’s better to have a game that doesn’t look quite as good than a game that no-one can play without throwing up.
As we’ve seen, motion is the major trigger for sickness. Avoiding it is best, but if that’s not possible, it’s important to keep it realistic.
Strafing, as in the picture, is very common in games, but is rarely done in real life. This is the sort of thing you want to avoid encouraging in your game.
Very similarly, rotation is a bad thing. It’s very common in many games for the second thumb-stick to be used for turning. Don’t do this.
Some games have found creative ways of avoiding the need for rotation, for example in London Heist, the player was seated in cars or behind desks to avoid the need for turning. Rush of Blood takes place on a minecart, and brings the action to the player, so they don’t have to rotate.
Another way of minimising motion is through the use of a static HUD, so that parts of the screen don’t move. This can be achieved through a HUD or framing within a car or spaceship so that the eye is exposed to less motion.
It’s also possible to offer options for reducing the FOV, as can be seen during the tutorial for Rigs.
Lastly for now, as we’ve learned it’s possible to build a tolerance to VR. This may take a long time, but this ethos can be applied on a microscale by grading your experience so players have the opportunity to get used to VR before being thrown into the action.
There’s a lot more than this, and other people have done some great work on this. Here are some resources to more on User Research, Simulator Sickness and VR.
- Ben Lewis Evans on Designing to minimise Simulator Sickness
- Laura Glibert on playtesting and cybersickness in VR for Eagle Flight
- Oculus Rift’s simulator sickness guidelines
- My previous presentation on playtesting in VR (link below)
And that’s it! Thanks for reading 🙂
Leave a Reply