|
|||||||
Learn About Usability Testing
What is usability testing?In a usability test, representative users try to do typical tasks with the product, while observers, including the development staff, watch, listen, and take notes. The product can be a Web site, Web application, or any other product. It does not have to be a finished product. You should be testing prototypes from early paper-based stages through fully functional later stages. What are you looking for in a usability test?In each round of usability testing, you should first identify specific concerns and goals for that round of testing and develop the test to focus on those concerns and goals. For example, at the beginning of a project, you may be testing to set quantitative baselines (such as time, error rates, and satisfaction) for comparison to later tests of your revised site. For another example, if you have Set Measurable Usability Goals, you may be trying to see how well the product is meeting those goals. In a typical usability test, you want to:
How does usability testing fit into user-centered design?Usability testing is a major part of user-centered design. A user-centered design process should include a series of tests developed specifically to evaluate both performance and preference. When should you do usability testing?Test early; test often. Usability testing lets the design and development teams identify problems before they get "set in concrete." The earlier those problems are found and fixed, the less expensive the fixes are. As the project progresses, it becomes more and more difficult and expensive to make major design changes. The more you test and change based on what you learn, the more confident you can be that the site will meet your objectives and your users' needs when it is launched. Iterating—developing a prototype, testing it with users, analyzing the test results, changing the prototype based on the findings, and then repeating the test, analyze, revise cycles—is the best way to produce a successful Web site or Web application. What can you learn through usability testing?In a typical usability test, you may want answers to these questions:
You might also have more specific questions. For example, if one of your concerns for this round of testing is how well the search function works for users, you might focus on these questions:
What should you keep in mind when usability testing?Four important points to keep in mind:
You are testing the site not the users For some users the term "testing" has a negative connotation. We try hard to ensure that participants do not think that we are testing them. We help them understand that they are helping us test the prototype or Web site. In fact, don't use the term "testing" with participants at all. Instead, invite participants to help by "trying out the prototype." When users have difficulty completing a task, we fix the Web site, not the users. Both out loud and in your thoughts, ask "How well does the site allow typical users to meet their goals?" rather than "How well do users do on the site?" Rely more on what you learn about performance than preference We can measure both performance and preference. Performance measures include success, time, errors, etc. Preference measures include users' self-report of their satisfaction and comfort. Some designers believe that if they design to meet users' preferences, the site will enable users to perform well. The evidence does not support that. In fact, people's performance and preference do not always match. One study found that about 70% of users had performance and preference measures that agreed with each other. That is, they either performed well and liked the Web site or performed poorly and disliked the site. However, this leaves a relatively large percentage of people (30%) for whom performance and preference measures did not agree. They either performed well and disliked the site or performed poorly and liked the site. Various reasons have been proposed for why people often rate a site more highly than their performance would lead us to expect. They may blame themselves for the problems that they have, rather than the site. They may think they would be hurting our feelings by giving the site a low score. They may not be aware of the problems they had and think they were successful when they were not. All of these reasons support the recommendation to rely more on what you learn through performance measures than through preference measures. Make use of what you learn Usability testing is not just a milestone to be checked off on the project schedule. A usability test is not finished when the last participant leaves. The team must consider the findings, set priorities, and change the prototype or site based on what happened in the usability test. Try to find the best solution given the reality of your many users Creating any product, including most Web sites and Web applications, requires considering many different users with different ways of working, different experiences, and different questions and needs. Most projects, including designing or revising Web sites, have to deal with constraints of time, budget, and resources. Balancing all those is one of the major challenges of most projects. As you weigh constraints and trade-offs, push for creating the Web site or Web application that will enable the greatest percentage of your users to successfully answer their questions and complete their tasks. Research shows that the cost of supporting unsuccessful users after a product is launched is far greater than the cost of fixing the product to meet users' needs before it is launched. As you consider your personas, their scenarios, and what you learn in usability testing, try to find the ideal solutions to your design problems - given the needs of the different users you have identified. Settling for less than the best means having users be less successful than they otherwise would be. Evidence shows that even after extended use and experience, when they have a less than optimal product to work with, users never achieve the success they would have if they had had the better interface. Do you need a lab to do usability Testing?No. You can do usability testing in either a formal or informal setting. In any type of setting, your methodology can also range from formal to informal. You can do effective usability testing in any of these settings:
You should be doing usability testing even if you do not have a fixed laboratory or access to one. Do not let anyone say, "We can't do usability testing because we do not have a usability lab." Just do it! Do it in any space you can find. How many participants are needed for a usability test?It depends. A typical range is from 8 to 16 (per user group) each test. If each user works with you for an hour, that means one or two days of testing, peruser group. You might need only four to six people to help you find serious problems, if you:
If you have different potential user groups (for example, physicians, patients, researchers), try to include representatives of all these groups. If you are likely to have users with a range of Web or computer experience, try to include both less experienced and more experienced users. If you want to conduct formal quantitative testing on your products or systems, you'll need more people to derive statistical results. For diagnostic usability testing, six to eight users are usually enough to uncover the major problems in a product. If you do iterative (repeated) usability testing over the course of developing the Web site, many users will participate in testing one or another version of the emerging site. Thus, while you may have fewer than 10 participants in each usability test, you may have 15 to 30 people who have tested some version of the site before it is launched. How much does it cost to do usability testing?Cost depends, of course, on the size of the site, how much you need to test, how many different types of users you anticipate having, and how formal you want the testing to be. Having a standard process and reusable materials makes usability testing faster and less expensive. If you or your recruiting firm develops a database of users, recruiting becomes less time consuming and, therefore, cheaper. Consider these elements in budgeting for usability testing:
Remember to budget for more than one usability test. Building usability into a Web site (or any product) is an iterative process. You will find it more valuable to use your budget to do a few small tests throughout development than to do just one large test at the end. Next stepsNow that you understand usability testing, you may want to review the steps in usability testing. The first step is Develop the Test Plan. |
|