Designing in VR

Designing in VR

Matt Shanks

Matt Shanks

Principal Product Designer & Author/Illustrator for children. I'm out to improve the quality of life for everyone who uses the things that I make.

What happens when you remove the limitations of a screen from a seasoned software product designer, and drop them in to a task of designing for Virtual Reality. Here’s a little peek in to my world.

Virtual Reality (VR) is digital technology’s new love child. A logical progression of the human need to remove the flat glass panel that acts as a barrier between the physical and digital worlds combined with faster and smaller computer processors than ever before. These 2 factors are coming together in a way that means we’re on the precipice of a seismic shift in the way we’re about to begin interacting with machines.

We’re writing the rules of VR now

Having been a software product designer for over 15 years with a background in Graphic Design, tackling the challenges of Web and Mobile experience design seemed fairly straightforward. We’ve got established principles for typography, layout and colour ever since the pioneers of graphic design set the rules over 70 years ago. We’ve evolved some pretty solid digital interaction design principles over recent years too based on these flat, 2D experiences that have come to dominate our lives. These design principles have given us some pretty consistent patterns about how humans interact with screens. But what happens when that screen goes away, when it wraps around your head and sends you in to another world, any world you can imagine possible?

I’ve worked with an incredibly progressive innovation team at one of Australia’s most successful businesses and I experienced a steep learning curve on how to create compelling VR experiences. I’m still learning of course, but here’s a few things that I learnt.

You can’t build what you dream, yet

I’ll cut to the chase on this one, the technology really is in its infancy. Many VR experiences are compelling, but there’s no doubt about it — I can’t help but think that the current state is dangling a carrot in front of us. It’s saying, “Here’s a thing that’s cool, but imagine what it could be.” And that’s the first thing I learned about VR, the technology just isn’t able to make your grand vision come true, yet. It’s the equivalent of trying to imagine that Netflix was possible in 1992 when our screen resolutions were 640 x 480px and our dial up speeds were, well, dial up speeds.

As usual, understanding the limitation of the technology is critical to designing successful experiences when humans interact with it. With VR, we’re not just talking about screen widths and mild inconveniences when we choose not to design for a particular audience anymore. If you ignore the technical boundaries in VR will make people sick or cause them physical injury. Things like frame rates, resolution and processing power of the device will drastically affect a user’s perception in VR. In this way, the design considerations become a blend of those you might expect from an industrial designer or architect with the skills of a software designer. It’s about human perception, ergonomics & spatial awareness but our materials are digital — pixels and photons. Are software designers even the best choice for designing VR environments? It’s an interesting question and one I don’t have the answer for right now. In fact, that’s a whole other article waiting to be written.

Are software designers even the best choice for designing VR environments? It’s an interesting question and one I don’t have the answer for right now. In fact, that’s a whole other article waiting to be written.

Understanding the technical landscape of VR draws on skills that aren’t completely foreign to me. Sure, there’s some new language to use but there’s always been that. Instead of understanding which browsers support certain CSS styles, I’m learning which devices, platforms and manufacturers are better or worse than others at achieving the desired effects we’re wanting to create in our application. Performance has an incredible influence on what’s possible right now for allowing a comfortable experience and so understanding what affects this is also critical. With the technology still not ‘perfect’ by any stretch of the imagination, good VR design needs to be much more simple than you think. We’re not yet at immersive, ‘life-like’ interfaces. At the moment it feels like we’ve got to take a step back in time to make a giant leap forward.

Don’t bother designing in 2D, get in VR as quickly as you can

There’s currently a distinct lack of digital tools that will help a designer design for VR. There’s the old 2D tools like Photoshop or Sketch which have been great for high-fidelity design for web or mobile. Or, you might be familiar with more advanced web prototyping tools like Framer.js and their VR plugin. But, right now, pen and paper has proved our team’s most efficient starting point.

Pen and paper still continues to be the easiest way to communicate ideas within the team. I’ve spent time on my own and iterated through hundreds of interface ideas. I was able to share those quickly with the team and get feedback on what they believe were the pros and cons of each iteration. It gives them ideas and allows them to get started building a prototype directly in VR.

Given the current technology constraints of VR, it’s proved to be a waste of time trying to refine typography and colours or select the ‘right imagery’ and do all that stuff that graphic designers are really good at. We like to call it ‘balancing’ visual elements together so it’s coherent to anyone who might be looking. That all goes out the window when you put on the headset and realise that the white background you thought would make things look nice and clean turned out to be blindingly bright. That the dark text you thought had enough contrast on that white background actually disappears because white ‘background’ pixels bleed in to the dark text and outshine them. That there is no ‘top’ to denote priority UI elements any more. These are just three examples but there’s been many instances of experiencing this sort of difference between a external retina display and an immersive VR display. It’s obvious when you write it down or say it like I’ve just done. It’s not until you see it in VR that you truly understand the effect of any design decision you make.

So, what’s the lesson here? Start with a simple sketch so everyone is on the same page about what you’re trying to achieve in the 3D space, then, as soon as possible, put on a VR headset and begin to experiment and iterate.

Start with a simple sketch so everyone is on the same page about what you’re trying to achieve in the 3D space, then, as soon as possible, put on a VR headset and begin to experiment and iterate.

Experiment & Iterate: Everything is on the table

Despite having interaction design standards in both digital and physical worlds, VR needs to borrow from both worlds to find the best solution.

In the physical world, we’re constrained by physics. We can check if a water kettle is full before we turn it on without looking inside; we just pick it up and feel its weight. The digital world has its own set of constraints: pixels, frame rates and performance but also ‘learned’ metaphors. We can easily recognise a button, a shopping cart, a ‘menu’ now when we’re interacting with a screen. So what does great VR feel like? The truth is we don’t yet know.

In our current team, we’ve developed a process that we think is pretty good given the current state of technology. We’ve found that the easiest way to make those ‘designer’ refinements to a VR experience is to give a designer access to VR. With very little coding knowledge on my part, (but heaps on our engineers part), I’m now able to use our VR development environment to play. I make things bigger, smaller, wrap around your entire head. Move objects further away, closer, in and out of focus until things feel just right, or very wrong. This rapid iteration inside VR means we can explore hundreds of options in a day. We’re quickly finding out what hurts and what doesn’t. What makes us feel a bit queesy or nauseous and conversely, what’s incredibly comfortable.

We can only do this by being OK with trying everything, and we do, we really try everything. If we’ve read on a blog that something makes people feel ill, then we try it and sure enough, make ourselves feel ill. We’ve tried the current ‘best practice’ and we’ve tried the current worst practice and everything in between. This experimentation is never failure. We’re learning every step of the way and it’s only going to help us come to our own conclusions for our own context of use much more quickly.

Where to from here?

We’ve just scratched the surface of VR here in our team and for every question we answer, we ask 24 more but the important thing is we’re getting answers. We know that the technology is in its early stages and so we have to be very mindful of the impact our design ideas have on performance, frame rate and resolution. We also know that there’s no substitute for checking the success or failure of a design in a headset. We know that sketching still works for ideation and communication and we’re inventing ways that designers can easily iterate in VR so we can explore every option, not just the ‘best practice’ ones.

As we head in to our first round of testing the application we’ve built with users who aren’t us, we’ll find a lot more answers and no doubt uncover more questions. It’s an exciting time to be working in VR, at the edge of this transition between how humans and machines interact with each other.

Image via Unsplash
Thanks to Andy Nicholson.

Related posts

Matt Shanks

Design and Development Empathy – Covidence Case Study

It’s been interesting to watch the ways in which software companies are trying to bridge the conceptual and language gaps of designers and developers. There are frameworks, design systems, collaboration tools and so on that are ‘trying to connect designers and developers in a better way’. And, whilst I use these tools regularly, none of them come close to doing the actual work of listening and understanding the needs of ‘the other’.

Read More
Will Rockel

Aligning design tools with design practices – Part 2

In Part 2 of this series, I’ll step through a practical process for assessing the nature and value of the tools you’re currently using, and your organisation’s design maturity, before suggesting some next steps to help you create a roadmap for a better design culture.

Read More