Close
SciStat
 

Sample size: Comparison of two ROC curves

Description

Calculates the required sample size for the comparison of the areas under two ROC curves (derived from the same cases). The sample size takes into account the required significance level and power of the test.

Required input

  • Type I error - alpha: the probability of making a Type I error (α-level, two-sided), i.e. the probability of rejecting the null hypothesis when in fact it is true.
  • Type II error - beta: the probability of making a Type II error (β-level), i.e. the probability of accepting the null hypothesis when in fact it is false.
  • Area under ROC curve 1: hypothesized area for the first ROC curve.
  • Area under ROC curve 2: hypothesized area for the second ROC curve.
  • Correlation in positive group: the hypothesized rank correlation coefficient in the positive group (abnormal cases)
  • Correlation in negative group: the hypothesized rank correlation coefficient in the negative group (normal cases)
  • Ratio of sample sizes in negative / positive groups: enter the desired ratio of negative and positive cases. If you desire both groups to have an equal number of cases you enter 1; when you desire twice as many cases in the negative than in the positive group, enter 2.

Example

You are interested to show that the discriminating power of two assays (performed on the same cases), with an area under the ROC curve of 0.825 and 0.9, is significantly different. From previous studies you know that the rank correlation between the two assays is 0.4 in both positive and negative cases.

You enter the values 0.825 and 0.9 for Area under ROC curve 1 and Area under ROC curve 2. Next you enter 0.4 for Correlation in positive group and Correlation in negative group.

You expect to include twice as many negative cases than positive cases, so for the Ratio of sample sizes in negative / positive groups you enter 2.

For α-level you select 0.05 and for β-level you select 0.20 (power is 80%).

Results

After you click the OK button the program will display the required sample size. In the example 133 cases are required in the positive group and 266 in the negative group, giving a total of 399 cases.

A table shows the required sample size for different Type I and Type II Error levels.

References

  • Hanley JA, McNeil BJ (1982) The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 143:29-36.
  • Hanley JA, McNeil BJ (1983) A method of comparing the areas under receiver operating characteristic curves derived from the same cases. Radiology 148:839-843.

Link

Go to Sample size: Comparison of two ROC curves.