Prediction of outcome in intensive care unit trauma patients: A multicenter study of acute physiology and chronic health evaluation (APACHE), trauma and injury severity score (TRISS), and a 24-hour intensive care unit (ICU) point system

Mary J. Vassar, Frank R. Lewis, Jody A. Chambers, Richard Mullins, Paul E. O'Brien, John A. Weigelt, Minh Tuan R Hoang, James W. Holcroft

    Research output: Contribution to journalArticle

    63 Citations (Scopus)

    Abstract

    Objective: To conduct a multicenter study to validate the accuracy of the Acute Physiology and Chronic Health Evaluation (APACHE) II system, APACHE III system, Trauma and Injury Severity Score (TRISS) methodology, and a 24- hour intensive care unit (ICU) point system for prediction of mortality in ICU trauma patient admissions. Methods: The study population consisted of retrospectively identified, consecutive ICU trauma admissions (n = 2,414) from six Level I trauma centers. Probabilities of death were calculated by using logistic regression analysis. The predictive power of each system was evaluated by using decision matrix analysis to compare observed and predicted outcomes with a decision criterion of 0.50 for risk of hospital death. The Youden Index (YI) was used to compare the proportion of patients correctly classified by each system. Measures of model calibration were based on goodness-of-fit testing(Hosmer-Lemeshow statistic less than 15.5) and model discrimination were based on the area under the receiver operating characteristic curve (AUC). Results: Overall, APACHE II (sensitivity, 38%; specificity, 99%; YI, 37%; H-L statistic, 92,6; AUC, 0.87) and TRISS (sensitivity, 52%; specificity, 94%; YI, 46%; H-L statistic, 228.1; AUC, 0.82) were poor predictors of aggregate mortality, because they did not meet the acceptable thresholds for both model calibration and discrimination. APACHE III (sensitivity, 60%; specificity, 98%; YI, 58%; H-L statistic, 7.0; AUC, 0.89) was comparable to the 24-hour ICU point system (sensitivity, 51%; specificity, 98%; YI, 50%; H-L statistic, 14.7; AUC, 0.89) with both systems showing strong agreement between the observed and predicted outcomes based on acceptable thresholds for both model calibration and discrimination. The APACHE III system significantly improved upon APACHE II for estimating risk of death in ICU trauma patients (p <0.001). Compared with the overall performance, for the subset of patients with nonoperative head trauma, the percentage correctly classified was decreased to 46% for APACHE II; increased to 71% for APACHE III (p <0.001 vs. APACHE II); increased to 59% for TRISS; and increased to 62% for 24-hour ICU points. For operative head trauma, the percentage correctly classified was increased to 60% for APACHE II; increased to 61% for APACHE III; decreased to 43% for TRISS (p <0.004 vs. APACHE III); and increased to 54% for 24-hour ICU points. For patients without head injuries, all of the systems were unreliable and considerably underestimated the risk of death. The percentage of nonoperative and operative patients without head trauma who were correctly classified was decreased, respectively, to 26% and 30% for APACHE II; 33% and 29% for APACHE III; 33% and 19% for TRISS; 20% and 23% for 24-hour ICU points. Conclusion: For the overall estimation of aggregate ICU mortality, the APACHE III system was the most reliable; however, performance was most accurate for subsets of patients with head trauma. The 24-hour ICU point system also demonstrated acceptable overall performance with improved performance for patients with head trauma. Overall, APACHE II and TRISS did not meet acceptable thresholds of performance. When estimating ICU mortality for subsets of patients without head trauma, none of these systems had an acceptable level of performance. Further multicenter studies aimed at developing better outcome prediction models for patients without head injuries are warranted, which would allow trauma care providers to set uniform standards for judging institutional performance.

    Original languageEnglish (US)
    Pages (from-to)324-329
    Number of pages6
    JournalJournal of Trauma - Injury, Infection and Critical Care
    Volume47
    Issue number2
    DOIs
    StatePublished - Aug 1999

    Fingerprint

    Injury Severity Score
    APACHE
    Multicenter Studies
    Intensive Care Units
    Wounds and Injuries
    Craniocerebral Trauma
    Area Under Curve
    Calibration
    Mortality
    Decision Support Techniques
    Trauma Centers
    Patient Admission

    Keywords

    • APACHE system
    • Illness severity scoring
    • Outcome prediction
    • Traumatic injury
    • TRISS methodology

    ASJC Scopus subject areas

    • Surgery

    Cite this

    Prediction of outcome in intensive care unit trauma patients : A multicenter study of acute physiology and chronic health evaluation (APACHE), trauma and injury severity score (TRISS), and a 24-hour intensive care unit (ICU) point system. / Vassar, Mary J.; Lewis, Frank R.; Chambers, Jody A.; Mullins, Richard; O'Brien, Paul E.; Weigelt, John A.; Hoang, Minh Tuan R; Holcroft, James W.

    In: Journal of Trauma - Injury, Infection and Critical Care, Vol. 47, No. 2, 08.1999, p. 324-329.

    Research output: Contribution to journalArticle

    @article{1f1261205d71494e807e8b5dc18904f6,
    title = "Prediction of outcome in intensive care unit trauma patients: A multicenter study of acute physiology and chronic health evaluation (APACHE), trauma and injury severity score (TRISS), and a 24-hour intensive care unit (ICU) point system",
    abstract = "Objective: To conduct a multicenter study to validate the accuracy of the Acute Physiology and Chronic Health Evaluation (APACHE) II system, APACHE III system, Trauma and Injury Severity Score (TRISS) methodology, and a 24- hour intensive care unit (ICU) point system for prediction of mortality in ICU trauma patient admissions. Methods: The study population consisted of retrospectively identified, consecutive ICU trauma admissions (n = 2,414) from six Level I trauma centers. Probabilities of death were calculated by using logistic regression analysis. The predictive power of each system was evaluated by using decision matrix analysis to compare observed and predicted outcomes with a decision criterion of 0.50 for risk of hospital death. The Youden Index (YI) was used to compare the proportion of patients correctly classified by each system. Measures of model calibration were based on goodness-of-fit testing(Hosmer-Lemeshow statistic less than 15.5) and model discrimination were based on the area under the receiver operating characteristic curve (AUC). Results: Overall, APACHE II (sensitivity, 38{\%}; specificity, 99{\%}; YI, 37{\%}; H-L statistic, 92,6; AUC, 0.87) and TRISS (sensitivity, 52{\%}; specificity, 94{\%}; YI, 46{\%}; H-L statistic, 228.1; AUC, 0.82) were poor predictors of aggregate mortality, because they did not meet the acceptable thresholds for both model calibration and discrimination. APACHE III (sensitivity, 60{\%}; specificity, 98{\%}; YI, 58{\%}; H-L statistic, 7.0; AUC, 0.89) was comparable to the 24-hour ICU point system (sensitivity, 51{\%}; specificity, 98{\%}; YI, 50{\%}; H-L statistic, 14.7; AUC, 0.89) with both systems showing strong agreement between the observed and predicted outcomes based on acceptable thresholds for both model calibration and discrimination. The APACHE III system significantly improved upon APACHE II for estimating risk of death in ICU trauma patients (p <0.001). Compared with the overall performance, for the subset of patients with nonoperative head trauma, the percentage correctly classified was decreased to 46{\%} for APACHE II; increased to 71{\%} for APACHE III (p <0.001 vs. APACHE II); increased to 59{\%} for TRISS; and increased to 62{\%} for 24-hour ICU points. For operative head trauma, the percentage correctly classified was increased to 60{\%} for APACHE II; increased to 61{\%} for APACHE III; decreased to 43{\%} for TRISS (p <0.004 vs. APACHE III); and increased to 54{\%} for 24-hour ICU points. For patients without head injuries, all of the systems were unreliable and considerably underestimated the risk of death. The percentage of nonoperative and operative patients without head trauma who were correctly classified was decreased, respectively, to 26{\%} and 30{\%} for APACHE II; 33{\%} and 29{\%} for APACHE III; 33{\%} and 19{\%} for TRISS; 20{\%} and 23{\%} for 24-hour ICU points. Conclusion: For the overall estimation of aggregate ICU mortality, the APACHE III system was the most reliable; however, performance was most accurate for subsets of patients with head trauma. The 24-hour ICU point system also demonstrated acceptable overall performance with improved performance for patients with head trauma. Overall, APACHE II and TRISS did not meet acceptable thresholds of performance. When estimating ICU mortality for subsets of patients without head trauma, none of these systems had an acceptable level of performance. Further multicenter studies aimed at developing better outcome prediction models for patients without head injuries are warranted, which would allow trauma care providers to set uniform standards for judging institutional performance.",
    keywords = "APACHE system, Illness severity scoring, Outcome prediction, Traumatic injury, TRISS methodology",
    author = "Vassar, {Mary J.} and Lewis, {Frank R.} and Chambers, {Jody A.} and Richard Mullins and O'Brien, {Paul E.} and Weigelt, {John A.} and Hoang, {Minh Tuan R} and Holcroft, {James W.}",
    year = "1999",
    month = "8",
    doi = "10.1097/00005373-199908000-00017",
    language = "English (US)",
    volume = "47",
    pages = "324--329",
    journal = "Journal of Trauma and Acute Care Surgery",
    issn = "2163-0755",
    publisher = "Lippincott Williams and Wilkins",
    number = "2",

    }

    TY - JOUR

    T1 - Prediction of outcome in intensive care unit trauma patients

    T2 - A multicenter study of acute physiology and chronic health evaluation (APACHE), trauma and injury severity score (TRISS), and a 24-hour intensive care unit (ICU) point system

    AU - Vassar, Mary J.

    AU - Lewis, Frank R.

    AU - Chambers, Jody A.

    AU - Mullins, Richard

    AU - O'Brien, Paul E.

    AU - Weigelt, John A.

    AU - Hoang, Minh Tuan R

    AU - Holcroft, James W.

    PY - 1999/8

    Y1 - 1999/8

    N2 - Objective: To conduct a multicenter study to validate the accuracy of the Acute Physiology and Chronic Health Evaluation (APACHE) II system, APACHE III system, Trauma and Injury Severity Score (TRISS) methodology, and a 24- hour intensive care unit (ICU) point system for prediction of mortality in ICU trauma patient admissions. Methods: The study population consisted of retrospectively identified, consecutive ICU trauma admissions (n = 2,414) from six Level I trauma centers. Probabilities of death were calculated by using logistic regression analysis. The predictive power of each system was evaluated by using decision matrix analysis to compare observed and predicted outcomes with a decision criterion of 0.50 for risk of hospital death. The Youden Index (YI) was used to compare the proportion of patients correctly classified by each system. Measures of model calibration were based on goodness-of-fit testing(Hosmer-Lemeshow statistic less than 15.5) and model discrimination were based on the area under the receiver operating characteristic curve (AUC). Results: Overall, APACHE II (sensitivity, 38%; specificity, 99%; YI, 37%; H-L statistic, 92,6; AUC, 0.87) and TRISS (sensitivity, 52%; specificity, 94%; YI, 46%; H-L statistic, 228.1; AUC, 0.82) were poor predictors of aggregate mortality, because they did not meet the acceptable thresholds for both model calibration and discrimination. APACHE III (sensitivity, 60%; specificity, 98%; YI, 58%; H-L statistic, 7.0; AUC, 0.89) was comparable to the 24-hour ICU point system (sensitivity, 51%; specificity, 98%; YI, 50%; H-L statistic, 14.7; AUC, 0.89) with both systems showing strong agreement between the observed and predicted outcomes based on acceptable thresholds for both model calibration and discrimination. The APACHE III system significantly improved upon APACHE II for estimating risk of death in ICU trauma patients (p <0.001). Compared with the overall performance, for the subset of patients with nonoperative head trauma, the percentage correctly classified was decreased to 46% for APACHE II; increased to 71% for APACHE III (p <0.001 vs. APACHE II); increased to 59% for TRISS; and increased to 62% for 24-hour ICU points. For operative head trauma, the percentage correctly classified was increased to 60% for APACHE II; increased to 61% for APACHE III; decreased to 43% for TRISS (p <0.004 vs. APACHE III); and increased to 54% for 24-hour ICU points. For patients without head injuries, all of the systems were unreliable and considerably underestimated the risk of death. The percentage of nonoperative and operative patients without head trauma who were correctly classified was decreased, respectively, to 26% and 30% for APACHE II; 33% and 29% for APACHE III; 33% and 19% for TRISS; 20% and 23% for 24-hour ICU points. Conclusion: For the overall estimation of aggregate ICU mortality, the APACHE III system was the most reliable; however, performance was most accurate for subsets of patients with head trauma. The 24-hour ICU point system also demonstrated acceptable overall performance with improved performance for patients with head trauma. Overall, APACHE II and TRISS did not meet acceptable thresholds of performance. When estimating ICU mortality for subsets of patients without head trauma, none of these systems had an acceptable level of performance. Further multicenter studies aimed at developing better outcome prediction models for patients without head injuries are warranted, which would allow trauma care providers to set uniform standards for judging institutional performance.

    AB - Objective: To conduct a multicenter study to validate the accuracy of the Acute Physiology and Chronic Health Evaluation (APACHE) II system, APACHE III system, Trauma and Injury Severity Score (TRISS) methodology, and a 24- hour intensive care unit (ICU) point system for prediction of mortality in ICU trauma patient admissions. Methods: The study population consisted of retrospectively identified, consecutive ICU trauma admissions (n = 2,414) from six Level I trauma centers. Probabilities of death were calculated by using logistic regression analysis. The predictive power of each system was evaluated by using decision matrix analysis to compare observed and predicted outcomes with a decision criterion of 0.50 for risk of hospital death. The Youden Index (YI) was used to compare the proportion of patients correctly classified by each system. Measures of model calibration were based on goodness-of-fit testing(Hosmer-Lemeshow statistic less than 15.5) and model discrimination were based on the area under the receiver operating characteristic curve (AUC). Results: Overall, APACHE II (sensitivity, 38%; specificity, 99%; YI, 37%; H-L statistic, 92,6; AUC, 0.87) and TRISS (sensitivity, 52%; specificity, 94%; YI, 46%; H-L statistic, 228.1; AUC, 0.82) were poor predictors of aggregate mortality, because they did not meet the acceptable thresholds for both model calibration and discrimination. APACHE III (sensitivity, 60%; specificity, 98%; YI, 58%; H-L statistic, 7.0; AUC, 0.89) was comparable to the 24-hour ICU point system (sensitivity, 51%; specificity, 98%; YI, 50%; H-L statistic, 14.7; AUC, 0.89) with both systems showing strong agreement between the observed and predicted outcomes based on acceptable thresholds for both model calibration and discrimination. The APACHE III system significantly improved upon APACHE II for estimating risk of death in ICU trauma patients (p <0.001). Compared with the overall performance, for the subset of patients with nonoperative head trauma, the percentage correctly classified was decreased to 46% for APACHE II; increased to 71% for APACHE III (p <0.001 vs. APACHE II); increased to 59% for TRISS; and increased to 62% for 24-hour ICU points. For operative head trauma, the percentage correctly classified was increased to 60% for APACHE II; increased to 61% for APACHE III; decreased to 43% for TRISS (p <0.004 vs. APACHE III); and increased to 54% for 24-hour ICU points. For patients without head injuries, all of the systems were unreliable and considerably underestimated the risk of death. The percentage of nonoperative and operative patients without head trauma who were correctly classified was decreased, respectively, to 26% and 30% for APACHE II; 33% and 29% for APACHE III; 33% and 19% for TRISS; 20% and 23% for 24-hour ICU points. Conclusion: For the overall estimation of aggregate ICU mortality, the APACHE III system was the most reliable; however, performance was most accurate for subsets of patients with head trauma. The 24-hour ICU point system also demonstrated acceptable overall performance with improved performance for patients with head trauma. Overall, APACHE II and TRISS did not meet acceptable thresholds of performance. When estimating ICU mortality for subsets of patients without head trauma, none of these systems had an acceptable level of performance. Further multicenter studies aimed at developing better outcome prediction models for patients without head injuries are warranted, which would allow trauma care providers to set uniform standards for judging institutional performance.

    KW - APACHE system

    KW - Illness severity scoring

    KW - Outcome prediction

    KW - Traumatic injury

    KW - TRISS methodology

    UR - http://www.scopus.com/inward/record.url?scp=0032791088&partnerID=8YFLogxK

    UR - http://www.scopus.com/inward/citedby.url?scp=0032791088&partnerID=8YFLogxK

    U2 - 10.1097/00005373-199908000-00017

    DO - 10.1097/00005373-199908000-00017

    M3 - Article

    C2 - 10452468

    AN - SCOPUS:0032791088

    VL - 47

    SP - 324

    EP - 329

    JO - Journal of Trauma and Acute Care Surgery

    JF - Journal of Trauma and Acute Care Surgery

    SN - 2163-0755

    IS - 2

    ER -