Evangelize with Usability
Using Invalid Users to Sell User Centered Design to a large organization

In larger technology companies it can often be difficult to develop an understanding of the advantages of doing good product design early. As Alan Cooper told us, "The word 'design' is toxic in the world of business". More so with Interaction Design and User Centered Design processes which require to be done early, close to the beginning of a project while the product is being defined and the requirements written. It is not unusual to find a number of very skeptical people around, who question, the time, budget and effort which must go into these activities.

So how do you overcome this skepticism? How do you sell Interaction Design to a skeptical audience? The answer is to use Usability Testing as your early option for evangelizing your new principles and approach. By carefully selecting a set of "invalid" test participants, you can sow the seeds for future success.

This strategy is not without it's risks and it could easily backfire if your design is not good. This short paper seeks to advise you how to select the candidate evangelists and how to manage the risks of showing them the product early, so that you get the desired result - an influential band of company evangelists to your User Centered Design cause. People who will go forth and spread the word that your efforts, the budget and the time are not only necessary but essential for the future success of the business.

In an ideal world, you will want to test as soon as possible. In any IT related work, you must be seen to provide frequent, tangible deliverables of real business value. Someone is giving you their budget and their trust, you must repay that with results. Typically, you should never go beyond three months without delivering a result. So the first guideline is that you must run usability tests within the first three months of a project. Every three months is considered frequent for this purpose. Or put another way, you can stall a boss with "Trust me!" only for so long. That 'so long' period seems to be about three months.

The size and scope of the project will determine exactly what you are testing at the end of that three month period. In a smaller project, it may well be the real system, which is probably at the System Integration Test phase or User Acceptance Test phase. On a larger project, it is more likely to be a prototype. It may be a high fidelity prototype or a low fidelity prototype. It is likely to be a prototype featuring a limited set of functionality. It is essential that the prototype you do test, has been developed from a thorough user centered design process and is in itself complete, offering end-to-end functionality. For example, in a home banking web application, it should allow you to perform functions such as account balance check, inter-account funds transfer, and bill payment, completely end-to-end. It must look to the user like a system which has real potential and true business value.

It doesn't matter if the back end is fake. Again with the banking example, there is no need to build the back end integration to the legacy system. Simply fake up content for one single account holder. You can later tell test participants that they must 'role play' that account holder.

So the second guideline is that you must deliver 1 or more User Goals with your prototype. Achieving goals is a means of delivering true value. Goal achievement is tangible and it has true business value.

Some of you may meet resistence to the notion of a prototype which by implication is a 'throw away'. It is worth while remembering the wisdom of Fred Brooks', "Plan to throw one away because you will anyway!" This argument stems from the fact that you can throw away an early prototype and get the design correct or you can wait and take a high risk that you will have to throw away the production system because you didn't get the design correct.

So you have a system or prototype ready for usability testing. You have met the criteria to be frequent, tangible and of true business value. Now it is time to test.

Now it is time to consider how to minimize the risk of showing an early product to skeptical members of the firm. You are trying to achieve a political win for design and design processes. You risk showing a poor design and a badly written piece of software to a skeptical audience. You must consider that bugs in the code will reflect badly on the design and the whole principle of design. To manage this risk, you must take a 3 phase approach.

Test with "friendly" invalid users first

Initially select a few members from your own team or closely related people. For example members of your QA team; technical writers; analysts; people who supplied requirements; marketing people; sales engineers; anyone as long as they are closely related to your project or product and have some skin in the game. They feel that they have a personal input on what has been produced so far.

You need at least 5 and ideally 8 of these friendly, invalid users. Run the usability test in the normal manner. You will always find those 3 or 4 really stupid design errors, or poor choices which need fixed. Have them fixed by the programming team. Ensure that all bugs which crash the system are removed. End of phase 1.

Test with "real" users next

Now it is time to run the proper user testing. You will have selected a number of users, perhaps several sets, based on target market, demographics, known user groups, professions, job specifications etc.. Bring these people in and run the same tests on your new design.

You may choose to refine the design every 5 to 8 user tests. In other words, each time you have enough data, make an improvement to your product, iterate the design quickly. Naturally, you will have to prioritise the changes, some major ones may need to be dropped. You will also need to select your programmers carefully. Not every programmer likes to make rapid changes like this. Select an individual who has "hacker" like mentality and just loves fast lifecycle iterations.

A worthwhile tip is that you might leave 1 day free in your test schedule for every 5 to 8 test participants. This will give your programming team an extra day to make and test any changes that you ask from them.

To summarise, the third guideline: iterate quickly, make several well advised design improvements during testing with real users. And the fourth guideline: select a programmer/programmers who are suited to the nature of prototyping and rapid design changes.

The result of this phase should be some solid usability test results and a much improved design. End of phase 2.

Evangelists must have influence

As an initial strategy for selecting evangelists, you may like to consider the question, "Who do we need to influence?" It might be the third line manager, your bosses boss, or maybe the VP of Finance or the Director of Sales, or maybe the technical support team who are tired of supporting terrible earlier products.

So the first approach might simply be to invite them to the tests. Have a senior official send them an email or a letter, saying that they have been selected to participate in the usability testing of the next generation product and that this is their opportunity to get an early "heads up" on what is coming through the development pipe. That usually works.

If these people don't fit your target demograhic then consider inviting their kids, or parents, or spouses, or golf partners, or whoever will match your demographic but is closely related to the people you need to influence.

Using the Law of the Few

There is a second, more scientific approach, to selecting the evangelists. We can use The Law of the Few described in "The Tipping Point" by Malcolm Gladwell. Gladwell describes three key personality types which we can use to communicate our message. These are Connectors, Mavens and Salesmen. Just a few of these people ought to be enough to tip the skeptics and see that the design message is evangelized throughout the organisation. Connectors, Mavens and Salesmen are the people with influence in an organisation. These are exactly the type of people to whom we need to sell our new design and its benefits.

Connectors are the type of people who know everyone. They are the people you go to, to get the latest gossip in the office. The people at cocktail parties who introduce you to someone that "you really should meet". Everyone knows one or two connectors because the connector makes it his/her business to know you.

Mavens are people who know lots of stuff rather than lots of people. A company maven may well be a product manager. Someone who is employed to know all about all the competitors and their products. Or it might be someone in a development role, who knows lots about technology or lots about the computer network. The kind of guy that you call when your computer is broken. These kinds of people get around the company and they get to know lots of people. The network guy fixes your computer but he also fixes the CEO's computer!

Salesmen are the proverbial bullshit merchants who just don't take "No" for an answer. They could sell sand to arabs and ice to eskimoes. They will tend to latch on to one single thing which they see as the advantage and they will go out and sell that advantage. Finding these people is easy. They probably sold you something recently like a lottery ticket for a charity or a share of a syndicated race horse. Influencing them isn't hard either. Just make sure that they can see that one key advantage and let them go after it.

So now you have selected the final set of test participants. You have arranged for a senior figure to invite them to the tests. You have refined the test and the design that you will be showing them. You are ready for phase 3.

It is important that potential evangelists are tested with goal directed questions. Ask them to solve problems of tangible business value. Something which they can see offers value to the user and either profit potential or improved service, to the business.

Run the test much like any other test. Offer some play time at the beginning, go through suitable introductions and make the test participant feel at ease. Present the test questions as normal.

It is worth remembering that you must keep the test results separate from the phase 2 test results. These phase 3 testers are potentially invalid users and spoil you scientific data from phase 2. However, any negative results obtained from this third group should still be considered valid and you should still act to fix problems uncovered by the evangelist group.

After the questions are complete, ask the participant to provide feedback. Let them talk, let them say what they think. This is your chance to turn them into an evangelist.

It is key that you take the opportunity to sell the user centered design process which led to your product design. Hopefully they will give you an invitation by complimenting the design, or saying something like, "this is much better than KML Corp's competitive offering". It is key that you sell the science of the usability testing. The test participant must not leave with the notion that they just participated in a marketing focus group. Emphasize that the design team derive important data from formal testing and that successive rounds of earlier testing have provided numerous improvements already.

If you have done a good job, then your participants ought to leave the test with a warm feeling. They realize that they achieved a number of goals with the new product and that those goals were achieved as easily as might be expected. Hopefully, they may consider that the design is superior to previous products and better than the competitors products. If this has happened then they will go forth among their colleagues and they will tell them, "the new product - I saw it - excellent. Can't wait to see the sales figures".

With a message like that circulating, you will have no problems in the next budgetary round when you need to ask for renewed funding for Interaction Design and Usability Engineering.