CS 635 Usability Test Report
A Web-based Visualization Tool For Analyzing Multi-dimensional Data
HCI Systems, Inc. is a medium-sized electronics manufacturing firm building circuit boards for use in modems, home security and general electronics products. HCI uses a direct sales force to sell directly to manufacturers. Salesmen and managers both spend a great deal of time on the road working with customers. Because most manufacturers now use just-in-time inventory methods, there is always concern on the part of potential customers over the ability of HCI to deliver product in quantity and on-time.
Current systems at HCI require more than a few minutes to retrieve needed information over 80 percent of the time. As such they are not suitable for answering customer questions and are not responsive to company management in general.
The Project Team was appointed to develop an information retrieval system that would be accessible from multiple locations, provide needed information quickly, and simplify complex data. An initial iteration prototype of such an interface consists of an ActiveX control imbedded in a web page which is accessible from the company's main web page.
A user profile was developed using a questionnaire distributed to a sample of potential users. The complete report is at Appendix C.
The typical user is either a professional or a member of management at some level and is college educated—with at least a bachelors degree. He or she is comfortable using web browsers and expects to use the system in a variety of locations (home office, customer sites and on the road). The preferred way of viewing complex data is in the form of charts, tables and graphs.
The project team identified eight tasks that users would need to perform in conjunction with using the system:
Usability Objectives and Requirements
Criteria were developed to evaluate
The principle requirements were that a user would need to be able to access the interface from a customer's office and retrieve the necessary data before the customer loses interest—a matter of minutes.
Usability was measured in a two phase process, beginning with a subjective questionnaire presented to a focus group following a presentation and demonstration of the prototype followed by usability testing against criteria which measured ease of use by the ability of users to perform tasks with in a stated time constraint or number of mouse clicks.
Method
Usability was tested along two lines:
a. The prototype was presented to a focus group including a brief demonstration. After which the focus group members were asked to fill out a brief survey. Questions related to the appearance of the system and to the understandability of the interface and the date provided by the interference. This information is subjective and does not track directly to the stated criteria, but provides some useful clues to problems the system might face.
b. A small group of users was given a brief overview of the system and then were asked to perform a series of tasks related to the usability criteria. These responses were timed or counted in accordance with the stated criteria and the data recorded in a matrix for further analysis. This group was also given the survey.
Results and conclusions
Survey: Although the favorable response rate was, in all cases, greater than 75 percent, a significant number of users had difficulty with the initial explanation, understanding the controls and options, or understanding the information presented. There were at least 10 recommendations relating to adding labels or instructions to the interface. Design decisions relating to the affordance of the controls will have to be revisited. In addition it is clear that the planned training video may be required early in the process to properly evaluate the understandability of controls, parameters and interface.
Usability Test.: At least 60% of test subjects, completed all four criteria based tests successfully. However, 2 of 5 had trouble choosing the appropriate set of data and selecting data dimensions. Three had trouble remembering to click ShowData. This result confirms the conclusions drawn from the survey data about the affordance of the interface: it is not as clear as we had thought. In addition further attention must be given to the design and placement of the Submit ('ShowData') button. This button seems to be required by the technology that we used, but alternatives should be explored. If the button cannot be eliminated, it should be made more prominent.
Recommendations
Result of our testing were reasonably positive and supportive of the needs established by company management and our user profile. Therefore, the project should be continued. The next phase should attack three problems:
Additional usability tests should be conducted in thirty days, once the above has been accomplished.
Appendices