TY - JOUR
T1 - Conceptualizing usability for the ehealth context: Content analysis of usability problems of ehealth applications
AU - Broekhuis, Marijke
AU - van Velsen, Lex
AU - Peute, Linda
AU - Halim, Meilani
AU - Hermens, Hermie
N1 - Publisher Copyright: © Marijke Broekhuis, Lex van Velsen, Linda Peute, Meilani Halim, Hermie Hermens. Originally published in JMIR Formative Research (https://formative.jmir.org), 27.07.2021. This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on https://formative.jmir.org, as well as this copyright and license information must be included.
PY - 2021/7/1
Y1 - 2021/7/1
N2 - Background: Usability tests can be either formative (where the aim is to detect usability problems) or summative (where the aim is to benchmark usability). There are ample formative methods that consider user characteristics and contexts (ie, cognitive walkthroughs, interviews, and verbal protocols). This is especially valuable for eHealth applications, as health conditions can influence user-system interactions. However, most summative usability tests do not consider eHealth-specific factors that could potentially affect the usability of a system. One of the reasons for this is the lack of fine-grained frameworks or models of usability factors that are unique to the eHealth domain. Objective: In this study, we aim to develop an ontology of usability problems, specifically for eHealth applications, with patients as primary end users. Methods: We analyzed 8 data sets containing the results of 8 formative usability tests for eHealth applications. These data sets contained 400 usability problems that could be used for analysis. Both inductive and deductive coding were used to create an ontology from 6 data sets, and 2 data sets were used to validate the framework by assessing the intercoder agreement. Results: We identified 8 main categories of usability factors, including basic system performance, task-technology fit, accessibility, interface design, navigation and structure, information and terminology, guidance and support, and satisfaction. These 8 categories contained a total of 21 factors: 14 general usability factors and 7 eHealth-specific factors. Cohen κ was calculated for 2 data sets on both the category and factor levels, and all Cohen κ values were between 0.62 and 0.67, which is acceptable. Descriptive analysis revealed that approximately 69.5% (278/400) of the usability problems can be considered as general usability factors and 30.5% (122/400) as eHealth-specific usability factors. Conclusions: Our ontology provides a detailed overview of the usability factors for eHealth applications. Current usability benchmarking instruments include only a subset of the factors that emerged from our study and are therefore not fully suited for summative evaluations of eHealth applications. Our findings support the development of new usability benchmarking tools for the eHealth domain.
AB - Background: Usability tests can be either formative (where the aim is to detect usability problems) or summative (where the aim is to benchmark usability). There are ample formative methods that consider user characteristics and contexts (ie, cognitive walkthroughs, interviews, and verbal protocols). This is especially valuable for eHealth applications, as health conditions can influence user-system interactions. However, most summative usability tests do not consider eHealth-specific factors that could potentially affect the usability of a system. One of the reasons for this is the lack of fine-grained frameworks or models of usability factors that are unique to the eHealth domain. Objective: In this study, we aim to develop an ontology of usability problems, specifically for eHealth applications, with patients as primary end users. Methods: We analyzed 8 data sets containing the results of 8 formative usability tests for eHealth applications. These data sets contained 400 usability problems that could be used for analysis. Both inductive and deductive coding were used to create an ontology from 6 data sets, and 2 data sets were used to validate the framework by assessing the intercoder agreement. Results: We identified 8 main categories of usability factors, including basic system performance, task-technology fit, accessibility, interface design, navigation and structure, information and terminology, guidance and support, and satisfaction. These 8 categories contained a total of 21 factors: 14 general usability factors and 7 eHealth-specific factors. Cohen κ was calculated for 2 data sets on both the category and factor levels, and all Cohen κ values were between 0.62 and 0.67, which is acceptable. Descriptive analysis revealed that approximately 69.5% (278/400) of the usability problems can be considered as general usability factors and 30.5% (122/400) as eHealth-specific usability factors. Conclusions: Our ontology provides a detailed overview of the usability factors for eHealth applications. Current usability benchmarking instruments include only a subset of the factors that emerged from our study and are therefore not fully suited for summative evaluations of eHealth applications. Our findings support the development of new usability benchmarking tools for the eHealth domain.
KW - Content analysis
KW - EHealth systems
KW - Mobile phone
KW - Summative evaluation
KW - Usability benchmarking
KW - Usability framework
UR - http://www.scopus.com/inward/record.url?scp=85111641250&partnerID=8YFLogxK
U2 - https://doi.org/10.2196/18198
DO - https://doi.org/10.2196/18198
M3 - Article
C2 - 34313594
SN - 2561-326X
VL - 5
JO - JMIR formative research
JF - JMIR formative research
IS - 7
M1 - e18198
ER -