USE Questionnaire Resource Page

USE stands for Usefulness, Satisfaction, and Ease of Use.  These are the three dimensions that emerged most strongly in the early development of the USE Questionnaire.  For many applications, Usability appears to consist of Usefulness and Ease of Use, which influence one another.  Each in turn drives user Satisfaction and frequency of use.  Users appear to have a good sense of what is usable and what is not, and can apply their internal metrics across domains.  The goals of this site are first to help foster a discussion (ideally leading to consensus) about the attributes that underlie usability from a user's perspective, and second to stimulate the development of a public domain instrument for measuring those attributes. To meet this goal, I would like to offer a library with pointers to resources related to users' perceptions of usability and its importance to them, and I would like to provide a place where researchers and practitioners can share materials and experiences.

This site, therefore, is intrinsically under construction.  To start the discussion, I am providing a list of resources that I have run across that are related to this topic.  I am also providing some information about a USE Qustionnaire that has proven useful in assessing several dimensions of usability across a wide variety of domains.  I would like to add to the site anything in this domain of user perceptions of usability that you would care to share with others.

Do you know of research, Web sites, or other materials that should be included in the resource list?
Do you have a favorite questionnaire that you would like to make available to others?
Have you had experiences with any questionnaires that you would like to share?
Have you tried one of the publically available questionnaires and would like to suggest improvements?
Have you compared one of the publically available questionnaires against one you have purchased?  What have you concluded?
Do you have theory you would like to share?

Send your contributions to this site, questions, or suggestions for improvement to:  alund@acm.org

Current Topic Index

• General Background
• Example Questionnaires
• Draft USE Questionnaire
• One Version
• SUS-type Items
• Items Adapted from Davis' (1989) Scales
• Other Resources

General Background

With such metrics, it should be possible to define a requirement for an acceptable level of usability, and measure progress towards that level with each iteration in a design.  It should be possible to determine the relative importance of various design elements, and as a result it should be possible to prioritize development resources to ensure that improvements have the greatest impact on usability.  It should be possible to use the information to better understand the relative merits of various usability evaluation metrics, and to assess the value of new design techniques emerging from research.  It should even be possible to evaluate alternative solutions to a given user need (e.g., where the need could be satisfied with a software application, a piece of hardware, or perhaps a process solution).

While some tools exist for assessing software usability, they typically are proprietary.  More importantly they do not do a good job of assessing usability across domains.  When re-engineering began at Ameritech, it became important to be able to set benchmarks for product usability and to be able to measure progress against those benchmarks.  It also was critical to ensure resources were being used as efficiently as possible, and so tools to help select the most cost-effective methodology and the ability to prioritize design problems to be fixed by developers became important.  Finally, it became clear that we could eliminate all the design problems and still end up with a product that would fail in the marketplace.  We needed to share ownership of the success of the product, and be able to demonstrate our bottom-line value to the business.  It was with this environment as a background that a series of studies were begun at Ameritech.  The first one was headed by Amy Schwartz, and was a collaboration of human factors, market research in our largest marketing organization, and a researcher from the Univ. of Michigan business school.   I am continuing the studies here at U S WEST Advanced Technologies.

The goal has been to develop a short questionnaire that can be used to measure the most important dimensions of usability for users, and to measure those dimensions across domains.  Ideally it should work for software, hardware, services, and user support materials.  It should allow meaningful comparisons of products in different domains, even though testing of the products happened at different times and perhaps under different circumstances.  In the best of all worlds, the items would have a certain amount of face validity for both users and practitioners, and it would be possible to imagine the aspects of the design that might influence ratings of the items.

The early studies at Ameritech suggested that such a questionnaire was possible.  Interestingly, the results of those early studies were consistent with studies conducted in the MIS and technology diffusion areas which also had identified the importance of and the relationship between Usefulness, Satisfaction, and Ease of Use.  Furthermore, the rich research tradition in these other areas provides theory that may be extended to explain the relationships.  This is an area that clearly provides a link between academic research and practice, and it is informed by several disciplines.  Some work has already been published suggesting that at least one publically available tool drawn from this work can be applied effectively to software interfaces.

It should be noted that one important value of the existing tools is that they provide diagnostic information for correcting usability problems.  The goal of the USE questionnaire as it might evolve over time is to quantify the dependent variables of importance.  The items in a USE questionnaire then might be common across questionnaires designed for specific domains.  If the questionnaire is effective, then domain specific tools such as we have for software can be built to diagnose which aspects of a design require improvement and their relative impact on the usability of the product.  In addition, we'll have the ability to look across domains at patterns in how design elements in various domains impact usability and to find clues that may contribute to a richer theory of design.  From a pragmatic business perspective, we should have the tools to tell whether a product is not just easy to use, but perhaps more importantly to tell whether it has the potential of being successful in the marketplace.  We may even be able to model the impact of usability on product revenue.
 

Draft USE Questionnaire

The questionnaires were constructed as 7-point Likert rating scales.  Users were asked to rate agreement with the statements, raging from strongly disagree to strongly agree.  Various forms of the questionnaires were used to evaluate user attitudes towards a variety of consumer products (mostly hardware products), towards internal software applications (e.g., e-mail and time reporting), and towards voice-mail systems.  Most of the testing occured using products with which the users had had significant experience, but some testing was conducted when the only experience the user had was during the traditional laboratory usability testing. An example of a prototypical questionnaire is available.  The actual questionnaires included additional items begin tested for inclusion, and items intended to measure usage, loyalty, and branding.  One interesting side note is that Marketing found the items worked well as part of telephone interviews to understand customer attitudes about products that had been in the field for some time, and this suggested an approach to linking more traditional market research with usability work in the laboratory.  The items have also been useful in assessing user perceptions of an ISP and of Web sites.

Factor analyses following each study suggested that users were evaluating the products primarily using three dimensions, Usefulness, Satisfaction, and Ease of Use.  Evidence of other dimensions was found, but these three served to most effectively discriminate between interfaces.  Partial correlations calculated using scales derived for these dimensions suggested that Ease of Use and Usefulness influence one another, such that improvements in Ease of Use improve ratings of Usefulness and vice versa.  While both drive Satisfaction, however, Usefulness is relatively less important when the systems are internal systems that users are required to use and users are more variable in their Usefulness ratings when they have had only limited exposure to a product.  As expected from the literature, Satisfaction was strongly related to the usage (actual or predicted).  For internal systems, the items contributing to Ease of Use for other products actually could be separated into two factors, Ease of Learning and Ease of Use (which were obviously highly correlated).

The items that appeared across tests for the 3 factors plus Ease of Learning are listed below.  The items in italics loaded relatively less strongly on the factors.

• Usefulness
1. It helps me be more effective.
2. It helps me be more productive.
3. It is useful.
4. It gives me more control over the activities in my life.
5. It makes the tings I want to accomplish easier to get done.
6. It saves me time when I use it.
7. It meets my needs.
8. It does everything I would expect it to do.
• Ease of Use
1. It is easy to use.
2. It is simple to use.
3. It is user friendly.
4. It requires the fewest steps possible to accomplish what I want to do with it.
5. It is flexible.
6. Using it is effortless.
7. I can use it without written instructions.
8. I don't notice any inconsistencies as I use it.
9. Both occasional and regular users would like it.
10. I can recover from mistakes quickly and easily.
11. I can use it successfully every time.
• Ease of Learning
1. I learned to use it quickly.
2. I easily remember how to use it.
3. It is easy to learn to use it.
4. I quickly became skillful with it.
• Satisfaction
1. I am satisfied with it.
2. I would recommend it to a friend.
3. It is fun to use.
4. It works the way I want it to work.
5. It is wonderful.
6. I feel I need to have it.
7. It is pleasant to use.

If you would like to collaborate to evolve an instrument that will be available in the public domain, please contact me.  If you try some of these items out as part of your testing, I would be very interested in whatever results you can share.