Usability Updates Newsletters

March 2005

Users Are Not Good Designers

by Dr. Bob Bailey

Introduction

Most system designers will tell you that there is a better chance for system success if users are involved in the development process. Years ago, most systems were created by programmers with very little input from users. This caused so many problems that most managers required their designers to rely heavily on potential users while developing new systems. The pendulum swung from one extreme (little user input) to the other (almost total reliance on users).

Today, user involvement is a widely accepted principle in the development of usable systems. The current emphasis on 'user-centered design' can be tracked back to several different sources. One of the most influential was an article written almost 20 years ago by John Gould and Clayton Lewis at IBM (Gould and Lewis, 1985).

A clear definition of user involvement is difficult to find. It seems to be a rather vague concept covering many approaches. In some cases it simply means 'a continued focus on users,' while in others it means 'a regular consulting with users,' and sometimes even 'the actual participation of users.' Thus, the level of user involvement can range from 'users as informants,' to 'users as designers,' to 'users as design managers.'

Most designers seem to believe that having users involved in the design and development of new systems will lead to:

• The improved quality of the system because of more accurate user requirements,
• An improved level of user acceptance, and
• More efficient and effective use.

Related Research

Sari Kujala at the Helsinki University of Technology in Finland conducted a review of the research literature related to user involvement in system design (Kujala, 2003). She wanted to clarify the relationship between user involvement during the design and development of a system, and the ultimate success of the system.

Ms. Kujala reported that the available research had both positive and negative findings. One study (Foster and Franz, 1999) found that designer's perceptions of user participation reliably correlated with four indicators of improved user acceptance. A series of other studies (Barki and Hartwick, 1991; Baroudi, et.al., 1986; McKeen and Guimares, 1997) reported low, but reliable correlations between user participation and:

• Overall system acceptance (r=.42),
• Acceptance of system functions (r=.32),
• Enhanced system usage (r=.28), and
• User satisfaction (r=.18 and r=.42).

She found other studies that produced either neutral or negative findings. One meta-analysis (Ives and Olson, 1984) identified over 30 empirical studies where user involvement was a key variable. These authors concluded that the studies did not provide consistent evidence of positive benefits by having users involved. Another study (Heinbokel, et.al., 1996) reported that greater user involvement actually was related to lower overall success (r= -.47) and fewer innovations (r= -.40).

Conclusions

The best available research suggests that by having users involved in the design and development of a new system will result in a more accurate set of user requirements. In some situations there also may be an improved level of user acceptance. However, the supporting research for the latter conclusion is not as strong. Finally, there is little evidence that systems are either more effective or more efficient when users are closely involved with design decisions.

Users are most useful in helping to define a system, i.e., identifying user requirements. Users are much less valuable in helping to design the system. To make the best use of available users, designers should use them to assist with identifying what the system will do, but not have them involved in determining how best to do it.

Later, during usability testing, potential users can be effectively used as test participants. The focus during this testing should be on assessing users' performance behaviors, and not on collecting their verbal recommendations for design changes. That is, trying to find out where and why users are having problems with a system, and not on ascertaining their ideas for solutions. Keep in mind that the best available research shows that users are good definers, but they are not good designers.

________

Barki, H. and Hartwick, J. (1991), User participation and user involvement in information system development, Proceedings of the 24th Annual Hawaii International Conference on System Sciences, 4, 487-492.

Baroudi, J.J., Olson, M.H. and Ives, B. (1986), An empirical study of the impact of user involvement on system usage and information satisfaction, Communications of the ACM, 29(3), 232-238.

Foster, S.T. and Franz, C.R. (1999), User involvement during information systems development: A comparison of analyst and user perceptions of system acceptance, Journal of Engineering and Technology Management, 16, 329-348.

Gould, J.D. and Lewis, C. (1985), Designing for usability: Key principles and what designers think, Communications of the ACM, 28 (3), March, 300-311.

Heinbokel, T., Sonnentag, S., Frese, M., Stolte, W. and Brodbeck, F.C. (1996), Don't underestimate the problems of user centredness in software development projects: There are many, Behaviour & Information Technology, 15(4), 226-236.

Ives, B. and Olson, M. (1984), User involvement and MIS success: A review of the literature, Management Science, 30(5), 586-603.

Kujala, S. (2003), User involvement: A review of the benefits and challenges, Behaviour & Information Technology, 2003, 22(1), 1-16.

McKeen, J.D. and Guimaraes, T. (1997), Successful strategies for user participation in systems development, Journal of Management Information Systems, 14(2), 133-150.