4 Challenges with VR Usability Testing

Eran HelftBlockedUnblockFollowFollowing

May 15, 2017

Usability testing is a key aspect in building awesome products. Why? because it works.

You get user feedback -> you understand what works and what doesn’t -> you get an insight into what needs to be improved -> you build a better product -> you make happy users -> you sell more. Easy :)

The thing is, it’s not always easy to conduct usability testing and in VR, it’s even harder.

I’m going to touch upon some of the challenges of conducting usability testing in VR.

Them Testers.

Unless your target audience is gamers or super tech savvy people, it’s safe to assume that they’re not very familiar with VR. In fact, chances are that your test is the very first time they’re even using VR.

This is tricky since often times the initial experience could be quite overwhelming and mess up with the test. A second ago your tester was in your NYC office and suddenly he’s in the Jungle, surrounded by snakes. Any wonder his missing on the “home” button just below the big gorilla?

I’m exaggerating here but you get the point.

Physically. Seriously?

In the pre-digital age, it was much harder to find, reach out and schedule with potential testers. How exactly am I going to get 500 males, ages 22–36 who live in Europe and are graphic designers to bring them to my office to check my new awesome product?

In the digital age, we have Google to help us find them and then a bunch of other tools for conducting remote usability testing- screen capturing, recorded sessions, user testing platforms etc.

But these are all based on the fact that the testers have the needed hardware and we can test remotely. This is hardly the case for VR, how many potential testers have all the needed set up at home or at the office?

You need to bring them into your office, physically, which adds some serious time and money constraints.

Chasing Your Tail.

You’re the boss, but still, in VR it’s hard to control the test. For starters, the testers can’t see or hear you, so you can’t ask questions or guide them.

Often times the testers can feel uncomfortable, get “lost” or distracted and forget the tasks you asked them to complete or simply bump into a wall.

Also, in mobile VR the user is in its (magical) VR world and you’re “outside”. This makes it hard to see what the user is actually doing or at least trying to do. Although there are some tricks to get around this one. For example, cast your GearVR to the TV using Chromecast.

Testing too late.

Testing should be made early on, the faster we start testing the better we are.

When designing a mobile app, you can put together a decent click through prototype in several hours without a single line of code. You can put your concepts to the test in less than a single day. In VR, it’s much much harder.

For a VR application to be tested, it has to feel real enough. It needs to have interactions, animation, transitions and more, all which take time to create. Especially when it’s hard, not to say impossible, to prototype it so you end up coding everything. So much for developing only what’s needed.

This means that user testing in VR happens too late. Probably too late.

Excuses kill companies.

Usability testing is far too important to be set aside, especially in VR where there is so much ground to be covered and the iterative process is needed more than ever.

Every problem has a solution, it’s just a matter of finding the right approach to get around it.

We’re currently experimenting with different ideas and methods for conducting usability testing. I promise to share our outcomes down the line once we have some more concrete solutions.

Until then, happy testing :)