Usability Testing Finds Problems for Novice Users of Pediatric Portals

Maria T. Britto, Holly B. Jimison, Jennifer Knopf Munafo, Jennifer Wissman, Michelle L. Rogers, William (Bill) Hersh

Research output: Contribution to journalArticle

56 Citations (Scopus)

Abstract

Objective: Patient portals may improve pediatric chronic disease outcomes, but few have been rigorously evaluated for usability by parents. Using scenario-based testing with think-aloud protocols, we evaluated the usability of portals for parents of children with cystic fibrosis, diabetes or arthritis. Design: Sixteen parents used a prototype and test data to complete 14 tasks followed by a validated satisfaction questionnaire. Three iterations of the prototype were used. Measurements: During the usability testing, we measured the time it took participants to complete or give up on each task. Sessions were videotaped and content-analyzed for common themes. Following testing, participants completed the Computer Usability Satisfaction Questionnaire which measured their opinions on the efficiency of the system, its ease of use, and the likability of the system interface. A 7-point Likert scale was used, with seven indicating the highest possible satisfaction. Results: Mean task completion times ranged from 73 (± 61) seconds to locate a document to 431 (± 286) seconds to graph laboratory results. Tasks such as graphing, location of data, requesting access, and data interpretation were challenging. Satisfaction was greatest for interface pleasantness (5.9 ± 0.7) and likeability (5.8 ± 0.6) and lowest for error messages (2.3 ± 1.2) and clarity of information (4.2 ± 1.4). Overall mean satisfaction scores improved between iteration one and three. Conclusions: Despite parental involvement and prior heuristic testing, scenario-based testing demonstrated difficulties in navigation, medical language complexity, error recovery, and provider-based organizational schema. While such usability testing can be expensive, the current study demonstrates that it can assist in making healthcare system interfaces for laypersons more user-friendly and potentially more functional for patients and their families.

Original languageEnglish (US)
Pages (from-to)660-669
Number of pages10
JournalJournal of the American Medical Informatics Association
Volume16
Issue number5
DOIs
StatePublished - Sep 2009

Fingerprint

Pediatrics
Parents
Cystic Fibrosis
Arthritis
Chronic Disease
Language
Delivery of Health Care
Efficiency
Surveys and Questionnaires

ASJC Scopus subject areas

  • Health Informatics

Cite this

Usability Testing Finds Problems for Novice Users of Pediatric Portals. / Britto, Maria T.; Jimison, Holly B.; Munafo, Jennifer Knopf; Wissman, Jennifer; Rogers, Michelle L.; Hersh, William (Bill).

In: Journal of the American Medical Informatics Association, Vol. 16, No. 5, 09.2009, p. 660-669.

Research output: Contribution to journalArticle

Britto, Maria T. ; Jimison, Holly B. ; Munafo, Jennifer Knopf ; Wissman, Jennifer ; Rogers, Michelle L. ; Hersh, William (Bill). / Usability Testing Finds Problems for Novice Users of Pediatric Portals. In: Journal of the American Medical Informatics Association. 2009 ; Vol. 16, No. 5. pp. 660-669.
@article{d07c4784f48f418aaf12e9a4d392b02f,
title = "Usability Testing Finds Problems for Novice Users of Pediatric Portals",
abstract = "Objective: Patient portals may improve pediatric chronic disease outcomes, but few have been rigorously evaluated for usability by parents. Using scenario-based testing with think-aloud protocols, we evaluated the usability of portals for parents of children with cystic fibrosis, diabetes or arthritis. Design: Sixteen parents used a prototype and test data to complete 14 tasks followed by a validated satisfaction questionnaire. Three iterations of the prototype were used. Measurements: During the usability testing, we measured the time it took participants to complete or give up on each task. Sessions were videotaped and content-analyzed for common themes. Following testing, participants completed the Computer Usability Satisfaction Questionnaire which measured their opinions on the efficiency of the system, its ease of use, and the likability of the system interface. A 7-point Likert scale was used, with seven indicating the highest possible satisfaction. Results: Mean task completion times ranged from 73 (± 61) seconds to locate a document to 431 (± 286) seconds to graph laboratory results. Tasks such as graphing, location of data, requesting access, and data interpretation were challenging. Satisfaction was greatest for interface pleasantness (5.9 ± 0.7) and likeability (5.8 ± 0.6) and lowest for error messages (2.3 ± 1.2) and clarity of information (4.2 ± 1.4). Overall mean satisfaction scores improved between iteration one and three. Conclusions: Despite parental involvement and prior heuristic testing, scenario-based testing demonstrated difficulties in navigation, medical language complexity, error recovery, and provider-based organizational schema. While such usability testing can be expensive, the current study demonstrates that it can assist in making healthcare system interfaces for laypersons more user-friendly and potentially more functional for patients and their families.",
author = "Britto, {Maria T.} and Jimison, {Holly B.} and Munafo, {Jennifer Knopf} and Jennifer Wissman and Rogers, {Michelle L.} and Hersh, {William (Bill)}",
year = "2009",
month = "9",
doi = "10.1197/jamia.M3154",
language = "English (US)",
volume = "16",
pages = "660--669",
journal = "Journal of the American Medical Informatics Association",
issn = "1067-5027",
publisher = "Oxford University Press",
number = "5",

}

TY - JOUR

T1 - Usability Testing Finds Problems for Novice Users of Pediatric Portals

AU - Britto, Maria T.

AU - Jimison, Holly B.

AU - Munafo, Jennifer Knopf

AU - Wissman, Jennifer

AU - Rogers, Michelle L.

AU - Hersh, William (Bill)

PY - 2009/9

Y1 - 2009/9

N2 - Objective: Patient portals may improve pediatric chronic disease outcomes, but few have been rigorously evaluated for usability by parents. Using scenario-based testing with think-aloud protocols, we evaluated the usability of portals for parents of children with cystic fibrosis, diabetes or arthritis. Design: Sixteen parents used a prototype and test data to complete 14 tasks followed by a validated satisfaction questionnaire. Three iterations of the prototype were used. Measurements: During the usability testing, we measured the time it took participants to complete or give up on each task. Sessions were videotaped and content-analyzed for common themes. Following testing, participants completed the Computer Usability Satisfaction Questionnaire which measured their opinions on the efficiency of the system, its ease of use, and the likability of the system interface. A 7-point Likert scale was used, with seven indicating the highest possible satisfaction. Results: Mean task completion times ranged from 73 (± 61) seconds to locate a document to 431 (± 286) seconds to graph laboratory results. Tasks such as graphing, location of data, requesting access, and data interpretation were challenging. Satisfaction was greatest for interface pleasantness (5.9 ± 0.7) and likeability (5.8 ± 0.6) and lowest for error messages (2.3 ± 1.2) and clarity of information (4.2 ± 1.4). Overall mean satisfaction scores improved between iteration one and three. Conclusions: Despite parental involvement and prior heuristic testing, scenario-based testing demonstrated difficulties in navigation, medical language complexity, error recovery, and provider-based organizational schema. While such usability testing can be expensive, the current study demonstrates that it can assist in making healthcare system interfaces for laypersons more user-friendly and potentially more functional for patients and their families.

AB - Objective: Patient portals may improve pediatric chronic disease outcomes, but few have been rigorously evaluated for usability by parents. Using scenario-based testing with think-aloud protocols, we evaluated the usability of portals for parents of children with cystic fibrosis, diabetes or arthritis. Design: Sixteen parents used a prototype and test data to complete 14 tasks followed by a validated satisfaction questionnaire. Three iterations of the prototype were used. Measurements: During the usability testing, we measured the time it took participants to complete or give up on each task. Sessions were videotaped and content-analyzed for common themes. Following testing, participants completed the Computer Usability Satisfaction Questionnaire which measured their opinions on the efficiency of the system, its ease of use, and the likability of the system interface. A 7-point Likert scale was used, with seven indicating the highest possible satisfaction. Results: Mean task completion times ranged from 73 (± 61) seconds to locate a document to 431 (± 286) seconds to graph laboratory results. Tasks such as graphing, location of data, requesting access, and data interpretation were challenging. Satisfaction was greatest for interface pleasantness (5.9 ± 0.7) and likeability (5.8 ± 0.6) and lowest for error messages (2.3 ± 1.2) and clarity of information (4.2 ± 1.4). Overall mean satisfaction scores improved between iteration one and three. Conclusions: Despite parental involvement and prior heuristic testing, scenario-based testing demonstrated difficulties in navigation, medical language complexity, error recovery, and provider-based organizational schema. While such usability testing can be expensive, the current study demonstrates that it can assist in making healthcare system interfaces for laypersons more user-friendly and potentially more functional for patients and their families.

UR - http://www.scopus.com/inward/record.url?scp=69549129523&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=69549129523&partnerID=8YFLogxK

U2 - 10.1197/jamia.M3154

DO - 10.1197/jamia.M3154

M3 - Article

C2 - 19567793

AN - SCOPUS:69549129523

VL - 16

SP - 660

EP - 669

JO - Journal of the American Medical Informatics Association

JF - Journal of the American Medical Informatics Association

SN - 1067-5027

IS - 5

ER -