Development of a quality assessment scale for retrospective clinical studies in pediatric surgery

Shawn J. Rangel, Jennifer Kelsey, Christopher E. Colby, JoDee Anderson, R. Lawrence Moss, D. Dudgeon, S. Rangel

Research output: Contribution to journalArticle

60 Citations (Scopus)

Abstract

Background/Purpose: More than 97% of the clinical research in pediatric surgery consists of retrospective data. Although these studies have significant limitations, they fundamentally shape clinical practice within the field. In this report, the authors describe the development and potential applications of a standardized quality assessment scale designed for retrospective studies in pediatric surgery. Methods: The authors developed a comprehensive quality assessment instrument incorporating 30 items within 3 subscales. These subscales were designed to assess 3 integral components of study design: clinical relevance, reporting methodology, and the strength of conclusions. Global quality ratings (poor, fair, or good) were derived by combining scores from each subscale. To examine inter-rater reliability, 10 retrospective studies from the pediatric surgery literature were assessed with the instrument by 6 independent reviewers. Results: Inter-rater reliability was excellent as assessed by the level of agreement for all items within the instrument (84.6% concordance, n = 1,573 total item-choices) and for individual subscales (range, 73.3% to 85.8%, n = 60 to 1,258). The extent of agreement among reviewers was 82% for classifying reports into qualitative categories by global quality scores (n = 60). The ranking of papers by individual reviewers was highly predictive of overall ranking by mean quality scores (n = 60 rank pairs, r = 0.83; P <.01). Conclusions: The authors have developed a standardized and reliable quality assessment scale for the analysis of retrospective data in pediatric surgery. Potential applications include: (1) providing the practicing surgeon with a knowledge base to critically evaluate published retrospective data; (2) providing a standardized methodology for the systemati(review of existing retrospective data; and (3) developing standardized reporting guidelines for use in peer-reviewed journals.

Original languageEnglish (US)
Pages (from-to)390-396
Number of pages7
JournalJournal of Pediatric Surgery
Volume38
Issue number3
DOIs
StatePublished - Mar 1 2003
Externally publishedYes

Fingerprint

Retrospective Studies
Pediatrics
Knowledge Bases
Guidelines
Clinical Studies
Research

Keywords

  • Quality assessment
  • Retrospective clinical studies

ASJC Scopus subject areas

  • Surgery

Cite this

Rangel, S. J., Kelsey, J., Colby, C. E., Anderson, J., Moss, R. L., Dudgeon, D., & Rangel, S. (2003). Development of a quality assessment scale for retrospective clinical studies in pediatric surgery. Journal of Pediatric Surgery, 38(3), 390-396. https://doi.org/10.1053/jpsu.2003.50114

Development of a quality assessment scale for retrospective clinical studies in pediatric surgery. / Rangel, Shawn J.; Kelsey, Jennifer; Colby, Christopher E.; Anderson, JoDee; Moss, R. Lawrence; Dudgeon, D.; Rangel, S.

In: Journal of Pediatric Surgery, Vol. 38, No. 3, 01.03.2003, p. 390-396.

Research output: Contribution to journalArticle

Rangel, SJ, Kelsey, J, Colby, CE, Anderson, J, Moss, RL, Dudgeon, D & Rangel, S 2003, 'Development of a quality assessment scale for retrospective clinical studies in pediatric surgery', Journal of Pediatric Surgery, vol. 38, no. 3, pp. 390-396. https://doi.org/10.1053/jpsu.2003.50114
Rangel, Shawn J. ; Kelsey, Jennifer ; Colby, Christopher E. ; Anderson, JoDee ; Moss, R. Lawrence ; Dudgeon, D. ; Rangel, S. / Development of a quality assessment scale for retrospective clinical studies in pediatric surgery. In: Journal of Pediatric Surgery. 2003 ; Vol. 38, No. 3. pp. 390-396.
@article{207b55f6d0364fd5af5f30a32be59665,
title = "Development of a quality assessment scale for retrospective clinical studies in pediatric surgery",
abstract = "Background/Purpose: More than 97{\%} of the clinical research in pediatric surgery consists of retrospective data. Although these studies have significant limitations, they fundamentally shape clinical practice within the field. In this report, the authors describe the development and potential applications of a standardized quality assessment scale designed for retrospective studies in pediatric surgery. Methods: The authors developed a comprehensive quality assessment instrument incorporating 30 items within 3 subscales. These subscales were designed to assess 3 integral components of study design: clinical relevance, reporting methodology, and the strength of conclusions. Global quality ratings (poor, fair, or good) were derived by combining scores from each subscale. To examine inter-rater reliability, 10 retrospective studies from the pediatric surgery literature were assessed with the instrument by 6 independent reviewers. Results: Inter-rater reliability was excellent as assessed by the level of agreement for all items within the instrument (84.6{\%} concordance, n = 1,573 total item-choices) and for individual subscales (range, 73.3{\%} to 85.8{\%}, n = 60 to 1,258). The extent of agreement among reviewers was 82{\%} for classifying reports into qualitative categories by global quality scores (n = 60). The ranking of papers by individual reviewers was highly predictive of overall ranking by mean quality scores (n = 60 rank pairs, r = 0.83; P <.01). Conclusions: The authors have developed a standardized and reliable quality assessment scale for the analysis of retrospective data in pediatric surgery. Potential applications include: (1) providing the practicing surgeon with a knowledge base to critically evaluate published retrospective data; (2) providing a standardized methodology for the systemati(review of existing retrospective data; and (3) developing standardized reporting guidelines for use in peer-reviewed journals.",
keywords = "Quality assessment, Retrospective clinical studies",
author = "Rangel, {Shawn J.} and Jennifer Kelsey and Colby, {Christopher E.} and JoDee Anderson and Moss, {R. Lawrence} and D. Dudgeon and S. Rangel",
year = "2003",
month = "3",
day = "1",
doi = "10.1053/jpsu.2003.50114",
language = "English (US)",
volume = "38",
pages = "390--396",
journal = "Journal of Pediatric Surgery",
issn = "0022-3468",
publisher = "W.B. Saunders Ltd",
number = "3",

}

TY - JOUR

T1 - Development of a quality assessment scale for retrospective clinical studies in pediatric surgery

AU - Rangel, Shawn J.

AU - Kelsey, Jennifer

AU - Colby, Christopher E.

AU - Anderson, JoDee

AU - Moss, R. Lawrence

AU - Dudgeon, D.

AU - Rangel, S.

PY - 2003/3/1

Y1 - 2003/3/1

N2 - Background/Purpose: More than 97% of the clinical research in pediatric surgery consists of retrospective data. Although these studies have significant limitations, they fundamentally shape clinical practice within the field. In this report, the authors describe the development and potential applications of a standardized quality assessment scale designed for retrospective studies in pediatric surgery. Methods: The authors developed a comprehensive quality assessment instrument incorporating 30 items within 3 subscales. These subscales were designed to assess 3 integral components of study design: clinical relevance, reporting methodology, and the strength of conclusions. Global quality ratings (poor, fair, or good) were derived by combining scores from each subscale. To examine inter-rater reliability, 10 retrospective studies from the pediatric surgery literature were assessed with the instrument by 6 independent reviewers. Results: Inter-rater reliability was excellent as assessed by the level of agreement for all items within the instrument (84.6% concordance, n = 1,573 total item-choices) and for individual subscales (range, 73.3% to 85.8%, n = 60 to 1,258). The extent of agreement among reviewers was 82% for classifying reports into qualitative categories by global quality scores (n = 60). The ranking of papers by individual reviewers was highly predictive of overall ranking by mean quality scores (n = 60 rank pairs, r = 0.83; P <.01). Conclusions: The authors have developed a standardized and reliable quality assessment scale for the analysis of retrospective data in pediatric surgery. Potential applications include: (1) providing the practicing surgeon with a knowledge base to critically evaluate published retrospective data; (2) providing a standardized methodology for the systemati(review of existing retrospective data; and (3) developing standardized reporting guidelines for use in peer-reviewed journals.

AB - Background/Purpose: More than 97% of the clinical research in pediatric surgery consists of retrospective data. Although these studies have significant limitations, they fundamentally shape clinical practice within the field. In this report, the authors describe the development and potential applications of a standardized quality assessment scale designed for retrospective studies in pediatric surgery. Methods: The authors developed a comprehensive quality assessment instrument incorporating 30 items within 3 subscales. These subscales were designed to assess 3 integral components of study design: clinical relevance, reporting methodology, and the strength of conclusions. Global quality ratings (poor, fair, or good) were derived by combining scores from each subscale. To examine inter-rater reliability, 10 retrospective studies from the pediatric surgery literature were assessed with the instrument by 6 independent reviewers. Results: Inter-rater reliability was excellent as assessed by the level of agreement for all items within the instrument (84.6% concordance, n = 1,573 total item-choices) and for individual subscales (range, 73.3% to 85.8%, n = 60 to 1,258). The extent of agreement among reviewers was 82% for classifying reports into qualitative categories by global quality scores (n = 60). The ranking of papers by individual reviewers was highly predictive of overall ranking by mean quality scores (n = 60 rank pairs, r = 0.83; P <.01). Conclusions: The authors have developed a standardized and reliable quality assessment scale for the analysis of retrospective data in pediatric surgery. Potential applications include: (1) providing the practicing surgeon with a knowledge base to critically evaluate published retrospective data; (2) providing a standardized methodology for the systemati(review of existing retrospective data; and (3) developing standardized reporting guidelines for use in peer-reviewed journals.

KW - Quality assessment

KW - Retrospective clinical studies

UR - http://www.scopus.com/inward/record.url?scp=0037370545&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0037370545&partnerID=8YFLogxK

U2 - 10.1053/jpsu.2003.50114

DO - 10.1053/jpsu.2003.50114

M3 - Article

C2 - 12632355

AN - SCOPUS:0037370545

VL - 38

SP - 390

EP - 396

JO - Journal of Pediatric Surgery

JF - Journal of Pediatric Surgery

SN - 0022-3468

IS - 3

ER -