L'APPLICATION DE L'ANALYSE ROC EN PSYCHIATRIE

The fundamental principles of ROC analysis are described. This method provides a means to assess the overall discriminant power of psychiatric rating scales for the full range of their scores. For each cut-off, an instrument has a sensitivity (true positive rate) and a specificity (true negative rate). High values of these coefficients are desirable although they are inversely related. ROC curves can be obtained by plotting the false positive rate and the true positive rate for different thresholds of the rating scale. The curves which would be obtained with a perfect, a worthless and a typical instrument are drawn to illustrate various situations found in ROC analysis. Among the several indices proposed, the area under the curve (AUC) is the most commonly used index to assess the overall discriminant power of an instrument. To calculate this area, the non parametric trapezoidal method is advocated. The area under the curve varies between 0.50 which corresponds to the chance line up to 1.0, a value associated with perfect accuracy. This parameter can be interpreted as the probability of classifying correctly the subjects of a pair where one is normal and one is diseased. Then, the appropriate statistic to compare several ROC indices is provided for the general case of independent observations. When the observations are paired, the standard error of the difference between two areas needs to be corrected. Taking into account the amount of variance due to non independence allows the test to maintain statistical power. A practical demonstration of the use of ROC analysis in applied psychiatric research is then used to illustrate this method. Data were collected with the Child Behavior Checklist on a sample of 237 children attending psychiatric services. Data for the control group, matched by age, sex and SES, were drawn from a large community sample. ROC analysis is used to assess the discriminant ability of various scores derived from the CBCL, using attendance to psychiatric services as a criterion. The social competence scales of the CBCL are shown to be, on average, less accurate than behavior problem scales in predicting psychopathology. The accuracy is identical in each sex. The statistical comparisons between ROC indices are applied in this example, and two detailed calculations are shown in the case of independant and paired comparisons. The advantages of ROC indices over the more classical t statistics are emphasized. As with any assessment of validity, the attention is drawn on the reliance of the results of ROC analysis upon the quality of an external criterion. Other applications of ROC analysis are suggested such as the determination of an optimal cut-off on a rating scale, the evaluation of different scoring systems, or cross-cultural psychiatric research.