The current study shows (a) the conclusion of Regehr and Colliver is based on assumptions that can be challenged, and (b) ROC analysis indeed can be used in sequential testing, but only if the procedure is modified according to the results of a corresponding loss function analysis. Recently, Regehr and Colliver (2003) argued that under certain theoretically derived conditions the use of the loss formula in sequential testing is functionally identical to using classic ROC analysis. Earlier, we doubted the validity of the procedure and proposed to set the cutpoint by minimizing the loss, defined as the weighted sum of the screen negatives and the false positives. Several authors ignored this difference and applied classic ROC analysis to a sequential test. However, in the sequential procedure there are no false negatives, because the result of the complete test is considered the final outcome. In a diagnostic test an optimum cutpoint is obtained by minimizing the weighted sum of false negatives and false positives using Receiver Operator Characteristic (ROC) analysis. The procedure may result in a reduction of testing resources, but at the cost of false positives (candidates who pass the screen but would fail the complete test). Candidates who fail the screen sit the complete test, whereas those who pass the screen are qualified as a pass of the complete test. Initially, all candidates take a screening test consisting of a part of the OSCE. Occurrence Handle10.Sequential testing is applied to reduce costs in SP-based tests (OSCEs). Toronto: University of Toronto Bookstore Custom Publishing.ĪrticleTitleA comparative analysis of the costs of administration of an OSCE Cohen (eds.), Proceedings of the sixth Ottawa Conference on Medical Education, pp. Sequential testing with a standardized-patient examination: an ROC analysis of the effects of case-total correlations and difficulty levels of screening test cases. Occurrence Handle10.1097/00001888-199710000-00032ĪrticleTitleScreening test length for sequential testing with a standardized-patient examination: a Receiver Operator Characteristic (ROC) AnalysisĬolliver, J.A., Markwell, S.J., Travis, T.A., Schrage, J.P. Occurrence Handle10.1097/00001888-199509000-00022ĪrticleTitleSequential testing in the objective stuctured clinical examination: selecting items for the screen The current study shows (a) the conclusion of Regehr and Colliver is based on assumptions that can be challenged, and (b) ROC analysis indeed can be used in sequential testing, but only if the procedure is modified according to the results of a corresponding loss function analysis.ĪrticleTitleCost analysis of objective structured clinical examinations Sequential testing is applied to reduce costs in SP-based tests (OSCEs). Sequential testing is applied to reduce costs in SP-based tests (OSCEs).
0 Comments
Leave a Reply. |