In order to compare the strategies consumers use to make predictions about the ease-of-use of products, the authors conducted a card sorting exercise with subject-matter experts (SMEs) to refine a coding scheme from current, widely-used usability definitions and measurement tools. Nine SMEs sorted 83 cards containing aspects found in usability definitions, guidelines, measurement tools and descriptions. The resulting fifteen clusters define a coding scheme that is both comprehensive and flexible. The refined taxonomy does not seek to redefine usability, but to provide a mechanism for researchers wanting to use a coding scheme that includes methods for classifying strategies such as usability attribution to system features, metacognitive behaviors (planning and evaluating), and exemplar-based strategies (memory for performing similar tasks).
Towards a Taxonomy for Classifying Intuitive Usability Prediction Strategies in Non-Experts
Proceedings of the Human Factors and Ergonomics Society Annual Meeting, vol. 60, 1: pp. 1264-1268. , First Published September 8, 2016.
Managing home networks is fraught with usability challenges due, in part, to the adoption of conventions, tools, and terminology meant for professional network administration. Eliciting user needs is challenging because of the gulf between the language used by network professionals and the less precise, unstandardized terms used by home administrators. Here we modify a traditional Structured Brainstorming (SB) technique to help circumvent these thorny communication problems. We describe our specific implementation and provide a sample of the information obtained from our SB sessions.
Proceedings of the Human Factors and Ergonomics Society Annual Meeting, vol. 60, 1: pp. 1284-1288. , First Published September 8, 2016.
Consumers’ expectations about the usability of products may be an important contributor to their overall satisfaction with a design. The purpose of this study was to explore how incongruencies between predicted usability and actual usability of a design affect a user’s post-use satisfaction ratings for a design. Participants predicted the usability of two stove designs and then used the designs to perform a task. Interactive stoves had varying degrees of burner-knob s-r compatibility. In general, the relative dissatisfaction associated with the “hard” design compared to the “easy” design was dependent on the initial usability prediction of the participant. Participants who initially predicted the design would be difficult-to-use had lower satisfaction ratings for easy-to-use designs than participants who initially predicted the same designs would be easy-to-use. These finding suggests that making a product look easy-to-use may be as important as actually making it easy-to-use, therefore designers should consider designing for both initial impressions of usability as well as actual usability.
Proceedings of the Human Factors and Ergonomics Society Annual Meeting, vol. 58, 1: pp. 1314-1317. , First Published October 17, 2014.
As more designers allow users to customize the look and feel of interfaces, users will be required to recognize the implications of their choices on their future performance, comfort, and enjoyment. Understanding the limits of people’s predictive capabilities may be an important component in identifying why people choose one product over another based on ease of use or why people have difficulties identifying tasks that can be performed together. The purpose of this study is to further explore users’ biases for utilizing the amount of white space in the stimulus as a predictor of task difficulty, to validate discrepancies between predicted task difficulty and performance outcomes found in previous research and the human factors literature, and to identify task-specific strategies that are used to anticipate task difficulty. The study uses the NASA Task Load Index (NASA-TLX) in prospective difficulty judgments for these three types of tasks: 1) a stimulus-response compatibility task, 2) a target acquisition task and 3) a perceptual search task. In general, participants predicted lower task demand for designs with more intervening white space. For the visual search task, these estimates of demand were consistent with participants’ actual performance reaction times. However, for the stove design and Fitts’ tasks participants rated tasks that were likely to result in more errors as less challenging suggesting that the type of task is an important factor in participants’ abilities to predict relative task difficulty.
Proceedings of the Human Factors and Ergonomics Society Annual Meeting, vol. 57, 1: pp. 1600-1604. , First Published September 30, 2013.
As shopping from online retailers continues to increase, designers need to be aware of the strategies consumers are using to predict the ease-of-use of products based on appearance. The following study investigates the influence of task type (stimulus-response compatibility task, target-acquisition task, perceptual-search task) on these strategies. The results suggest that for abstract tasks (motor and search), participants are relying on elements in or interactions with the actual stimulus such as the predicted time to complete the task and the number of items in the stimulus. However for tasks that are more concrete (using a stove), participants rely on their past experiences and familiarity with the task to predict task difficulty.
Proceedings of the Human Factors and Ergonomics Society Annual Meeting, vol. 56, 1: pp. 1867-1871. , First Published December 20, 2016.
Although the use of prospective workload judgments (i.e., judgments obtained from users prior to any actual interaction with a product) may be appealing for a variety of logistical reasons, a growing literature highlights the biases and metacognitive misconceptions that sometimes lead such judgments to be far from what is found in post-performance evaluations. The current study uses the NASA Task Load Index (NASA-TLX) in a prospective workload judgment task that employs two familiar stimulus sets from the human factors literature as to-be-rated designs: 1) control-burner arrangements on cooktops, and 2) control layouts for pointing tasks that vary in terms of Fitts’ Law parameters. Participants made reliable errors (compared to known performance outcomes) when judging both stimulus sets. In general, lower workload judgments were associated with designs that had greater intervening white space between controls and displays/targets.
Proceedings of the Human Factors and Ergonomics Society Annual Meeting, vol. 55, 1: pp. 1313-1317. , First Published September 1, 2011.
Multi-display surgical environments have the potential to increase performance and efficiency while decreasing errors and workload. However as more and more information is required for complex task execution and decision making, we must continually assess how the information is presented and whether we are helping or hindering surgeons by providing more content. Most laparoscopic surgeries are performed utilizing a single, two-dimensional (2-D) display. In the current experiment, we compared display usage, subjective workload, and workload measured via eye-tracking data to determine the effectiveness of an additional three-dimensional (3-D) display for a simulated surgical search task. We found that while participants did use the additional display in less demanding conditions (e.g., with fewer search targets), they did not use the supplemental display in conditions with greater demands, and they did not receive a substantial benefit from the presence of the supplemental display in either condition. Both increased saccades per target and increased perceived workload via the NASA-TLX provided support that more workload was experienced in conditions with more targets. And while participants did perceive decreased workload for more targets when the 3-D display was available, eye-tracking metrics were not consistent with participants’ subjective workload estimates. Since subjective workload ratings may be influenced by expectancies for benefits for the additional display, future research should attempt to understand this workload dissociation as well as breakdowns in the usage of supplemental displays as a function of task difficulty.
Proceedings of the Human Factors and Ergonomics Society Annual Meeting, vol. 55, 1: pp. 1626-1630. , First Published September 1, 2011.
Technological innovations are at the forefront of advances in minimally invasive surgery. Reduced visual and haptic cues, along with frame-of-reference problems with location and scale can cause surgeons to become disoriented. While most laparoscopic surgeries are performed via the use of a limited, singlescope, two-dimensional (2-D) view presented on a monitor in the operating room, there is demand for the availability of three-dimensional (3-D), global views. We compared workload, task-completion time, and the ability to recreate spatial mental representations between study participants who used the current scope-view display and those who used a dual-view display that included both the scope view and a computationally generated global view. We found no statistically reliable improvements for the dual-view display over the single-view display for any of our criterion measures, although trends were toward a dual-view advantage for workload in all tasks and accuracy in the reconstruction task, despite participants' claims that they did not utilize the global view during the experiment. Future research is needed to better understand the information available on global views that can enhance performance during surgical tasks and participants' decisions regarding when to use different views to support their performance.
Proceedings of the Human Factors and Ergonomics Society Annual Meeting, vol. 54, 19: pp. 1581-1585. , First Published September 1, 2010.
Prospective workload measures are used to assess individuals' expectations about tasks they are facing, how difficult they think the tasks will be and how well they expect to perform. In this study, 43 participants used the NASA-TLX subjective workload scale to predict the difficulty of surgical training tasks. The goal of the study was to determine the accuracy of their predictions and whether the act of assessing tasks before performing them affected their judgments post-performance. Regarding initial performance, results showed that participants formed prospective judgments that were consistent with their retrospective judgments, but they underestimated physical demands. After only minimal practice, however, their retrospective judgments deviated from both the experimental group's initial predictions and the control group's initial retrospective assessments. Anticipating mental demand was particularly challenging. No significant differences were found between the control and experimental conditions for post-performance assessments, suggesting that pre-performance assessment of workload has no effect of post-performance judgment of task difficulty.
Proceedings of the Human Factors and Ergonomics Society Annual Meeting, vol. 54, 19: pp. 1704-1708. , First Published September 1, 2010.
The NASA-TLX subjective workload assessment is typically used immediately following a participant's performance in an experimental task to assess the workload experienced. However, it is sometimes necessary to assess the anticipated workload of a task before the task is actually performed. This study compares the workload assessments of participants who performed two minimally invasive surgical training tasks to participants who only saw descriptions of the two tasks. Results showed that participants who were asked to rate the anticipated workload underestimated the overall workload required for the “cannulation” task, while overestimating the overall workload required for the rope task. Interactions between task type and condition also were found in three of the NASA-TLX subscales (mental demand, effort, and frustration). Overall, the reliability of participants' prediction of the difficulty depended on the task being evaluated and the particular measure of difficulty that was being assessed. In general, physical facets of workload appeared to be more accurately assessed than cognitive facets.
Proceedings of the Human Factors and Ergonomics Society Annual Meeting, vol. 53, 18: pp. 1383-1387. , First Published October 1, 2009.
Kent, T. M., Fu, B., Walls, B. D., Seidelman, W., Sublette, M. A., Lee, M., . . . Yang, R. (2016). Does an abstract weld pool visualization help novice welders assess the performance of a weldbot? Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 60(1), 1309-1313.
Popham, J., Lee, M., Sublette, M., Kent, T., & Carswell, C. M. (2016). Flashy or functional: The impact of graphical content on the effectiveness of resumés. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 60(1), 1329-1333.
Lee, M., Carswell, C. M., Miller-Spillman, K., & Sublette, M. (2015). Clothing & hf/e: A hedonomic and eudaimonic look at the original wearables. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 59(1), 981-985.
Lee, M., Kent, T., Carswell, C. M., Seidelman, W., & Sublette, M. (2014). Zebra-striping: Visual flow in grid-based graphic design. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 58(1), 1318-1322.
Lee, M. P., Carswell, C. M., Seidelman, W., & Sublette, M. (2014). Environmental control issues in a new energy-efficient building. Ergonomics in Design, 22(4), 8-14.
Seidelman, W., Carswell, C. M., Lio, C. H., Grant, R. C., Sublette, M., Field, M., . . . Clarke, D. (2010). Potential performance costs associated with large-format tiled displays for surgical visualization. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 54(19), 1430-1434.
Crouch, J., Lee, M., Carswell, C. M., Patrick, T., Seidelman, W., & Sublette, M. (2013). The impact of aesthetic design on bus shelter usability. In HFES (Ed.), Proceedings of the human factors and ergonomics society annual meeting (Vol. 57, pp. 1490-1494).
Lee, M., Carswell, C. M., Seidelman, W., & Sublette, M. (2013). Green expectations: The story of a customizable lighting control panel designed to reduce energy use. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 57(1), 1353-1357.
Lee, M., Carswell, C. M., Seidelman, W., & Sublette, M. (2012). The design of product comparison tables and its effects on decision making. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 56(1), 1654-1658.
Popham, J., Lee, M., Sublette, M., Kent, T., & Carswell, C. M. (2017, September). Graphic vs. Text-Only Résumés: Effects of Design Elements on Simulated Employment Decisions. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting (Vol. 61, No. 1, pp. 1242-1246). Sage CA: Los Angeles, CA: SAGE Publications.
Kent, T. M., Carswell, C. M., Lee, M., & Sublette, M. A. (2017, September). Do Aesthetic Design Principles Predict Visual Appeal of a Simple Control Panel?. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting (Vol. 61, No. 1, pp. 1414-1418). Sage CA: Los Angeles, CA: SAGE Publications.