What is a heuristic evaluation?

The goal of heuristic evaluation is to find usability problems early in the design of a Web site so that improvements can be made as part of the iterative design process.

Heuristic evaluations usually are conducted by a small set (one to three) of evaluators. The evaluators independently examine a user interface and judge its compliance with a set of usability principles. The result of this analysis is a list of potential usability issues or problems. The usability principles, also referred to as usability heuristics, are taken from published lists. Ideally, each potential usability problem is assigned to one or more heuristics to help facilitate fixing the problem. As more evaluators are involved, more true problems are found.

Some evaluators try to estimate the degree to which each usability issue potentially could impede user performance or acceptance. This is done to help set priorities for making revisions to the system. However, judging the severity of proposed usability issues has been shown to be very difficult to do.

top of page

What is a heuristic?

Molich and Nielsen (1990) wrote a paper that contains a set of heuristics still in popular use:

1. Use simple and natural dialogue.
2. Speak the user's language.
3. Minimize the user's memory load.
4. Be consistent.
5. Provide feedback.
6. Provide clearly marked exits.
7. Provide shortcuts.
8. Provides good error messages.
9. Prevent errors.
10. Provide help and documentation.

Nielsen (1994), after evaluating several sets of heuristics, concluded that a better set of heuristics may be:

1. Ensure good visibility of system status.
2. Have a good match between the system and the real world.
3. Ensure user control and freedom.
4. Use consistency and standards.
5. Design to prevent user errors.
6. Design to facilitate recognition rather than recall memory.
7. Provide for flexibility and efficiency of use.
8. Use aesthetic and minimalist design concepts.
9. Help users recognize, diagnose, and recover from errors.

Gerhardt-Powals (1996) proposed a set of research-based heuristics:

1. Automate unwanted workload:
1. Free cognitive resources for high-level tasks.
2. Eliminate mental calculations, estimations, comparisons, and unnecessary thinking.
2. Reduce uncertainty; display data in a manner that is clear and obvious.
3. Fuse data; reduce cognitive load by bringing together lower level data into a higher-level summation.
4. Present new information with meaningful aids to interpretation:
1. Use a familiar framework, making it easier to absorb.
2. Use everyday terms, metaphors, etc.
5. Use names that are conceptually related to function.
1. Context-dependent.
2. Attempt to improve recall and recognition.
6. Group data in consistently meaningful ways to decrease search time.
7. Limit data-driven tasks:
   1. Reduce the time spent assimilating raw data.
   2. Make appropriate use of color and graphics.
8. Include in the displays only that information needed by the user at a given time.
9. Provide multiple coding of data when appropriate.
10. Practice judicious redundancy.

Theoretically, the heuristics are related to criteria that, if improved, could make a positive difference in the product's usability. Unfortunately, the "usability problems" identified in a heuristic evaluation differ substantially from those obtained in performance testing. Only the Gerhardt-Powals set of heuristics has been validated. If other sets of heuristics are used, designers should first ensure that the evaluation actually would lead to improved Web site performance or acceptance.

top of page

How does it differ from an expert review?

In an expert review, the heuristics are assumed to have been previously learned and internalized by the evaluators. That is to say, evaluators do not use a clear-cut set of heuristics. As a result, the expert review tends to be less formal, and usually there is no requirement to assign a specific heuristic to each potential problem.

top of page

What are the advantages of a heuristic review?

The method can provide some quick and relatively inexpensive feedback to designers. Feedback can be obtained early in the design process. Assigning the correct heuristic can help suggest the best corrective measures to designers.

top of page

What are the disadvantages of a heuristic review?

Using this type of evaluation to identify usability problems is very difficult, research shows that it may result in only a 50% hit rate, and a 20% miss rate.

Results are not based on observations of user behavior, and will result in suggesting changes to the Web site that are not required (about a 50% false positive rate).

If the wrong heuristics are assigned to potential problems, it will mislead designers into applying the wrong solutions to the problems.

top of page