Become a Usability Testing Champion

Become a Usability Testing Champion

James Dowling

James Dowling

Handyman extraordinaire. Code at work and timber at home.

For the enlightened product designers among us, Usability Testing’s immense value is common knowledge. So, why is it that so many of us seem to be failing to sell that fact internally?

I’m a software developer, with a keen interest in product development. That’s caused me to go outside my comfort zone and explore the deeper issues affecting the tools I build every day. I’ve learned a lot in my journey into Usability Testing, and found that organisational buy-in is the hardest part about using modern usability techniques.

This article isn’t about the effectiveness or importance of such usability techniques. Enough has already been said on the subject. Instead, I’m going talk about what needs to be to done to get the necessary traction in your organisation. That’s the traction to effectively demonstrate the value of Usability Testing.

I say “effectively demonstrate”, because half-hearted adoption of usability techniques is worse than no adoption. Why? Because to their critics, a slow, false start will appear to prove these techniques ineffectual, and will actually hamper their inevitable adoption by forward-thinking businesses.

There is enormous, often invisible resistance that needs to be overcome before a business wilfully embraces the benefits provided by focussed, lightweight Usability Testing.

New paradigms in product thinking will have their critics and filibusterers. You would do well to have effective, coherent techniques for dealing with them. The biggest hurdles, which I intend to deal with in this article, are dispelling fears, showing cost effectiveness, and demonstrating results.

The idea is to help you show how regular, simple, cost effective Usability Testing is actually the most powerful kind.

So without further introduction, here’s how to sell Usability Testing within your company, and make it stick.

Dispelling fears that your coworkers, managers and CEO have about Usability Testing

A common, but unspoken fear; “Usability Testing will just showcase our shortcomings

This is a perfectly normal reaction to having the spotlight thrown on your work to date. Unfortunately, it’s also a common initial roadblock to Usability Testing.

An indirect, yet powerful way to showcase the effectiveness of Usability Testing is by running some Usability Testing sessions on your competitors! Focus on the results which demonstrate how simple it would have been for your competitors to identify key usability issues, if they had only done some early Usability Testing. That’s an easy pill to swallow. Be respectful to your competitors, and stakeholders in your business know that their past efforts will be considered respectfully also.

And in the same moment that competitors’ shortcomings are put on show, it’s accepted that Usability Testing is what every company in your field should be adopting.

“Usability Testing will embarrass us with our customer base”

Many clients and companies are (understandably!) very guarded of their reputations. But who says you have to drag in a lot of potential customers to do Usability Testing? Sure, domain experts will have idiosyncrasies worth noting, but Usability Testing is more about finding usability problems than domain ones. Ordinary people can find these usability problems too!

Consider this testing scenario.

Scenario: You’re outside a hospital, visiting a friend.

Task: Walk through the front door.

Result: You pull the door handle… only to find out you had to push the handle instead. Ugh!

Did you have to be a doctor to experience that frustration? Did you need nursing training for that door to slow your roll? Clearly not. The device (the door) didn’t function as intuitively expected. That’s what should be addressed.

Candidates don’t have to be domain experts to have a bad usability experience. So just test with people who are familiar with the primary interface to your product (eg. web browsers), and hand them intelligible scenarios. You’ll have glaring usability issues pouring out of those sessions in no time.

“Usability testing will take us on another, previously unconsidered path, and we don’t have enough runway for that.”

Ok, that may be true, but how did you get here? Where did all that runway go? Maybe you could have saved all that money and time by doing Usability Testing at earlier stages. Doing Usability Testing on designs and prototypes is a lot less emotionally taxing than critically examining a finished product which you have no inclination or ability to modify.

So many projects place Usability Testing at the end of the product lifecycle. But by the time you get there? Scope creep has often enveloped the funds earmarked for Usability Testing, because nobody intended to act on your findings anyway.

Test features early and often, using low-cost methods, and you’ll achieve meaningful outcomes and correct the whole direction of your product for the better. All that, at a time when change is still possible.

“Usability Testing doesn’t tell us anything statistically significant”

Uh-oh! The armchair statisticians are about. They know that Usability Testing is a qualitative thing, and that you’re hardly going to construct an expensive and complicated experiment that definitively proves that a usability issue exists.

But here’s the thing about statistical significance; in layman’s terms, that’s the certainty you have about a theory being true, given an allowable chance of your being wrong. You choose that allowable amount (a “p-value”), before an experiment! Nobody can prove anything absolutely with statistics. Even in pharmaceutical trials, the experimenters always have to say “There is an x% chance that I’m talking total nonsense”. Nobody knows anything for sure. Know that, and find freedom in it!

Fun Fact: No statistician has ever been on an airplane.

For any usability issue, most businesses would act on an 80% chance that you’re making a serious problem go away. You won’t need a formal experiment to find those problems. Here in Melbourne, 80% is about the chance that your train is running on time, but we all still follow our timetables. Consider an example Steve Krug gave recently in his “Do It Yourself Usability Testing” workshop. To paraphrase:

“If someone trips on the carpet outside your office, and the next day, another person trips on that same carpet…. How many people have to trip on that carpet before you acknowledge that it’s highly likely to be a serious issue?”

That’s the kind of issue we’re trying to uncover with Usability Testing. Obvious, identifiable issues with your interfaces. It’s what Usability Testing is best at! And that’s what you should say to nay-sayers who doubt the “absolute truth” of Usability Testing; “We’re looking for the obvious stuff here.”

“It’s very expensive”

True, there is an expensive way to do just about anything, but that’s not the kind of testing I’m talking about here. Eye tracking studios/software are preposterously expensive, and demonstrate little concrete value. Building or renting a testing studio could be a good idea, but only if you’re doing this stuff around the clock!

On the other hand, sensible Usability Testing needs very few candidates, for very little time per month. Free software will be completely sufficient. Let’s see how we can get quite a lot of bang for our buck…

How to do Usability Testing in a cheap and effective way.

All you need to be effective in getting your company to see the benefits of Usability testing is make sure it happens often. Do a minimal amount of Usability Testing, once a month. Make sure you schedule it in, regardless of where your projects are at. If it’s just an afterthought on a project plan, at the whim of other project concerns, it will happen far too late, if at all. So, once a month, plan a morning of Usability Testing, and nothing else infringes on this happening.

You’ll only need about three candidates, for an hour-long session each. Bring in one domain expert, and two regular people. Three hours with these people will bring plenty of important issues to the surface.

Free video capture software is abundant, and the cheaper paid options provide a lot of value. You won’t need to find serious funds to run your Usability Testing sessions.

What to test

As you’ll hopefully be testing early-on, you should start with testing paper designs and prototypes. After that, when you’re building features, test those in isolation. It’s ok if they’re not fully built. Just build a “guard path”, and if the users stray off that path, tell them the truth; that feature isn’t built yet. No big deal.

In any case, you should start with the areas you worry most about. You’re probably right about those, and it would be good to have a video or transcript demonstrating why.

After those areas, choose the areas that are most impactful. Which features do most users use? Which areas could represent bottlenecks in their use of your site?

What you will do with the information you’ve gathered

Usability Testing isn’t about finding out if you have usability issues. They’re there, believe me. There are so many usability issues in the average project, that a morning of Usability Testing will uncover enough work to fill a month of effort.

So, how to prioritise? The pragmatists in your organisation will say “Let’s focus on the low-hanging fruit.” That’s the easiest stuff that gives you nice big wins. Let me save you some time here; That’s not a thing. Glaring issues which are simple to fix and equate to megabucks will have already been considered.

Instead, focus on the most impactful areas. Which workflows affect the most users? Which ones are resulting in poor conversions for your organisation?

That doesn’t mean you have to take the hardest route to solve those impactful issues. In fact, you should only take minimal immediate steps to solve the observed issues. You don’t need to reengineer your whole user flow right away, just because you found some cracks in it. Finding a spot to place some metaphorical duct-tape will do just fine for now. If it fixes the observed issue, it’s fixed. Easy, yes? Cheap? Sure! No need to worry about expenses blowing-out if they don’t need to.

How to get wider organisational buy-in

It’s not enough to just get the go-ahead from some higher-ups or clients. You can’t be the only one to carry the torch for these practices forever. Getting serious organisational buy-in is the only way to make Usability Testing stick as an integral practice for your company.

Nothing sells this harder than having people watch videos of users’ frustrations. Up until their first Usability Testing viewing, people from various departments will view a feature or a problem through their own prism of experience. They’ll have their own opinions on how to solve those problems too. It’s very difficult to get a consensus at that point.

But there’s something about sitting everyone in a room and having them watch a user struggle with their product. It clarifies the situation, and everyone now agrees on the nature and size of the problem! Those “wow” moments happen, and people become very interested in learning more about the usability of their product.

Now, all you need to do is get as many people as possible to watch your usability videos. That won’t happen if people aren’t given dedicated time to view them. Find a time, a day or so after your testing, for people to watch highly condensed versions of your usability sessions. Draw people in with cookies, donuts, coffee, whatever it takes. Your goal is to turn watching and ingesting usability sessions into a regular habit for your company.


Usability Testing is a new concept to many businesses. Uncertainty breeds paralysis. Combat this paralysis by addressing fears upfront, and demonstrating an effective, affordable way to find Usability gold in your product.


This article has been heavily inspired by Steve Krug’s “Do-It-Yourself Usability Testing” workshop.

Many thanks to Andy Nicholson, Matt Shanks, and Scott Rogers for their advice in writing this article.

Related posts

Scott Rogers

Mapping Team Effectiveness

Having worked with hundreds of product teams in organisations of many shapes and sizes, we’re in a unique position to understand many of the challenges teams face, as well as the ways they might improve. One of the tools we use to identify how teams are working and what might be holding them back is the Competing Values Framework.

Read More
Eadaoin Doherty

Product managers and user research

As a Product Manager, I delight in how the collaboration of a product brain and a design brain will always give a better outcome than just one of those brains going solo. However, the reality is, sometimes product peeps have to go it alone when doing user research. I want to reassure you that as a product person, you can do this and get some really valuable insights by following these basic rules.

Read More