Assessing the difficulty and time cost of de-identification in clinical narratives

David Dorr, W. F. Phillips, S. Phansalkar, S. A. Sims, J. F. Hurdle

Research output: Contribution to journalArticle

41 Citations (Scopus)

Abstract

Objective: To characterize the difficulty confronting investigators in removing protected health information (PHI) from cross-discipline, free-text clinical notes, an important challenge to clinical informatics research as recalibrated by the introduction of the US Health Insurance Portability and Accountability Act (HIPAA) and similar regulations. Methods: Randomized selection of clinical narratives from complete admissions written by diverse providers, reviewed using a two-tiered rater system and simple automated regular expression tools. For manual review, two independent reviewers used simple search and replace algorithms and visual scanning to find PHI as defined by HIPAA, followed by an independent second review to detect any missed PHI. Simple automated review was also performed for the "easy" PHI that are number- or date-based. Results: From 262 notes, 2074 PHI, or 7.9 ± 6.1 per note, were found. The average recall (or sensitivity) was 95.9% while precision was 99.6% for single reviewers. Agreement between individual reviewers was strong (ICC = 0.99), although some asymmetry in errors was seen between reviewers (p = 0.001). The automated technique had better recall (98.5%) but worse precision (88.4%) for its subset of identifiers. Manually de-identifying a note took 87.3 ± 61 seconds on average. Conclusions: Manual de-identification of free-text notes is tedious and time-consuming, but even simple PHI is difficult to automatically identify with the exactitude required under HIPAA.

Original languageEnglish (US)
Pages (from-to)246-252
Number of pages7
JournalMethods of Information in Medicine
Volume45
Issue number3
StatePublished - 2006

Fingerprint

Health Insurance Portability and Accountability Act
Costs and Cost Analysis
Health
Medical Informatics
Research Personnel
Research

Keywords

  • Computerized medical records systems
  • De-identification
  • Health Insurance Portability and Accountability Act
  • Medical informatics computing
  • natural language processing

ASJC Scopus subject areas

  • Health Informatics
  • Health Information Management
  • Nursing(all)

Cite this

Dorr, D., Phillips, W. F., Phansalkar, S., Sims, S. A., & Hurdle, J. F. (2006). Assessing the difficulty and time cost of de-identification in clinical narratives. Methods of Information in Medicine, 45(3), 246-252.

Assessing the difficulty and time cost of de-identification in clinical narratives. / Dorr, David; Phillips, W. F.; Phansalkar, S.; Sims, S. A.; Hurdle, J. F.

In: Methods of Information in Medicine, Vol. 45, No. 3, 2006, p. 246-252.

Research output: Contribution to journalArticle

Dorr, D, Phillips, WF, Phansalkar, S, Sims, SA & Hurdle, JF 2006, 'Assessing the difficulty and time cost of de-identification in clinical narratives', Methods of Information in Medicine, vol. 45, no. 3, pp. 246-252.
Dorr, David ; Phillips, W. F. ; Phansalkar, S. ; Sims, S. A. ; Hurdle, J. F. / Assessing the difficulty and time cost of de-identification in clinical narratives. In: Methods of Information in Medicine. 2006 ; Vol. 45, No. 3. pp. 246-252.
@article{6355723d942949998609bb62ab34016e,
title = "Assessing the difficulty and time cost of de-identification in clinical narratives",
abstract = "Objective: To characterize the difficulty confronting investigators in removing protected health information (PHI) from cross-discipline, free-text clinical notes, an important challenge to clinical informatics research as recalibrated by the introduction of the US Health Insurance Portability and Accountability Act (HIPAA) and similar regulations. Methods: Randomized selection of clinical narratives from complete admissions written by diverse providers, reviewed using a two-tiered rater system and simple automated regular expression tools. For manual review, two independent reviewers used simple search and replace algorithms and visual scanning to find PHI as defined by HIPAA, followed by an independent second review to detect any missed PHI. Simple automated review was also performed for the {"}easy{"} PHI that are number- or date-based. Results: From 262 notes, 2074 PHI, or 7.9 ± 6.1 per note, were found. The average recall (or sensitivity) was 95.9{\%} while precision was 99.6{\%} for single reviewers. Agreement between individual reviewers was strong (ICC = 0.99), although some asymmetry in errors was seen between reviewers (p = 0.001). The automated technique had better recall (98.5{\%}) but worse precision (88.4{\%}) for its subset of identifiers. Manually de-identifying a note took 87.3 ± 61 seconds on average. Conclusions: Manual de-identification of free-text notes is tedious and time-consuming, but even simple PHI is difficult to automatically identify with the exactitude required under HIPAA.",
keywords = "Computerized medical records systems, De-identification, Health Insurance Portability and Accountability Act, Medical informatics computing, natural language processing",
author = "David Dorr and Phillips, {W. F.} and S. Phansalkar and Sims, {S. A.} and Hurdle, {J. F.}",
year = "2006",
language = "English (US)",
volume = "45",
pages = "246--252",
journal = "Methods of Information in Medicine",
issn = "0026-1270",
publisher = "Schattauer GmbH",
number = "3",

}

TY - JOUR

T1 - Assessing the difficulty and time cost of de-identification in clinical narratives

AU - Dorr, David

AU - Phillips, W. F.

AU - Phansalkar, S.

AU - Sims, S. A.

AU - Hurdle, J. F.

PY - 2006

Y1 - 2006

N2 - Objective: To characterize the difficulty confronting investigators in removing protected health information (PHI) from cross-discipline, free-text clinical notes, an important challenge to clinical informatics research as recalibrated by the introduction of the US Health Insurance Portability and Accountability Act (HIPAA) and similar regulations. Methods: Randomized selection of clinical narratives from complete admissions written by diverse providers, reviewed using a two-tiered rater system and simple automated regular expression tools. For manual review, two independent reviewers used simple search and replace algorithms and visual scanning to find PHI as defined by HIPAA, followed by an independent second review to detect any missed PHI. Simple automated review was also performed for the "easy" PHI that are number- or date-based. Results: From 262 notes, 2074 PHI, or 7.9 ± 6.1 per note, were found. The average recall (or sensitivity) was 95.9% while precision was 99.6% for single reviewers. Agreement between individual reviewers was strong (ICC = 0.99), although some asymmetry in errors was seen between reviewers (p = 0.001). The automated technique had better recall (98.5%) but worse precision (88.4%) for its subset of identifiers. Manually de-identifying a note took 87.3 ± 61 seconds on average. Conclusions: Manual de-identification of free-text notes is tedious and time-consuming, but even simple PHI is difficult to automatically identify with the exactitude required under HIPAA.

AB - Objective: To characterize the difficulty confronting investigators in removing protected health information (PHI) from cross-discipline, free-text clinical notes, an important challenge to clinical informatics research as recalibrated by the introduction of the US Health Insurance Portability and Accountability Act (HIPAA) and similar regulations. Methods: Randomized selection of clinical narratives from complete admissions written by diverse providers, reviewed using a two-tiered rater system and simple automated regular expression tools. For manual review, two independent reviewers used simple search and replace algorithms and visual scanning to find PHI as defined by HIPAA, followed by an independent second review to detect any missed PHI. Simple automated review was also performed for the "easy" PHI that are number- or date-based. Results: From 262 notes, 2074 PHI, or 7.9 ± 6.1 per note, were found. The average recall (or sensitivity) was 95.9% while precision was 99.6% for single reviewers. Agreement between individual reviewers was strong (ICC = 0.99), although some asymmetry in errors was seen between reviewers (p = 0.001). The automated technique had better recall (98.5%) but worse precision (88.4%) for its subset of identifiers. Manually de-identifying a note took 87.3 ± 61 seconds on average. Conclusions: Manual de-identification of free-text notes is tedious and time-consuming, but even simple PHI is difficult to automatically identify with the exactitude required under HIPAA.

KW - Computerized medical records systems

KW - De-identification

KW - Health Insurance Portability and Accountability Act

KW - Medical informatics computing

KW - natural language processing

UR - http://www.scopus.com/inward/record.url?scp=33745562886&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=33745562886&partnerID=8YFLogxK

M3 - Article

C2 - 16685332

AN - SCOPUS:33745562886

VL - 45

SP - 246

EP - 252

JO - Methods of Information in Medicine

JF - Methods of Information in Medicine

SN - 0026-1270

IS - 3

ER -