Meta-analysis: area under ROC curve
A meta-analysis integrates the quantitative findings from separate but similar studies and provides a numerical estimate of the overall effect of interest (Petrie et al., 2003).
Under the fixed effects model, it is assumed that all studies come from a common population, and that the Area under the ROC curve is not significantly different among the different trials. This assumption is tested by the "Heterogeneity test". If this test yields a low P-value (P<0.05), then the fixed effects model may be invalid. In this case, the random effects model may be more appropriate, in which both the random variation within the studies and the variation between the different studies is incorporated.
SciStat.com uses the methods described by Zhou et al. (2002) for calculating the weighted summary Area under the ROC curve under the fixed effects model and random effects model.
Studies: a variable containing an identification of the different studies.
- Area under ROC curve (AUC): a variable containing the Area under the ROC curve reported in the different studies.
- Standard error of AUC: a variable containing the Standard error of the Area under the ROC curve reported in the different studies.
Filter: a filter to include only a selected subgroup of studies in the meta-analysis.
- Forest plot: creates a forest plot.
- Marker size relative to study weight: option to have the size of the markers that represent the effects of the studies vary in size according to the weights assigned to the different studies. You can choose the fixed effect model weights or random effect model weights.
- Plot pooled effect - fixed effects model: option to include the pooled effect under the fixed effects model in the forest plot.
- Plot pooled effect - random effect model: option to include the pooled effect under the random effects model in the forest plot.
- Diamonds for pooled effects: option to represent the pooled effects using a diamond (the location of the diamond represents the estimated effect size and the width of the diamond reflects the precision of the estimate).
- Funnel plot: creates a funnel plot to check for the existence of publication bias. See Funnel plot.
The program lists the results of the individual studies included in the meta-analysis: the area under the ROC curve, its standard error and 95% confidence interval.
The pooled Area under the ROC curve with 95% CI is given both for the Fixed effects model and the Random effects model (Zhou et al., 2002).
Fixed and random effects model
Under the fixed effects model, it is assumed that the studies share a common true effect, and the summary effect is an estimate of the common effect size.
Under the random effects model the true effects in the studies are assumed to vary between studies and the summary effect is the weighted average of the effects reported in the different studies (Borenstein et al., 2009).
The random effects model will tend to give a more conservative estimate (i.e. with wider confidence interval), but the results from the two models usually agree when there is no heterogeneity. When heterogeneity is present (see below) the random effects model should be the preferred model.
Q is the weighted sum of squares on a standardized scale. It is reported with a P value with low P-values indicating presence of heterogeneity. This test however is known to have low power to detect heterogeneity and it is suggested to use a value of 0.10 as a cut-off for significance (Higgins et al., 2003). Conversely, Q has too much power as a test of heterogeneity if the number of studies is large.
I2 is the percentage of observed total variation across studies that is due to real heterogeneity rather than chance. It is calculated as I2 = 100% x (Q - df)/Q, where Q is Cochran's heterogeneity statistic and df the degrees of freedom. Negative values of I2 are put equal to zero so that I2 lies between 0% and 100%. A value of 0% indicates no observed heterogeneity, and larger values show increasing heterogeneity (Higgins et al., 2003).
All MedCalc's meta-analysis reports include two tests to detect possible publication bias: Egger's test (Egger et al., 1997) and Begg's rank test (Begg and Mazumdar, 1994).
Egger's test is a test for the Y intercept = 0 from a linear regression of normalized effect estimate (estimate divided by its standard error) against precision (reciprocal of the standard error of the estimate).
Begg's test assesses if there is a significant correlation between the ranks of the standardized effect sizes and the ranks of their variances.
The Forest plot shows the area under the ROC curve (with 95% CI) found in the different studies included in the meta-analysis, and the overall effect with 95% CI.
The random effects model will tend to give a more conservative estimate (i.e. with wider confidence interval), but the results from the two models usually agree where there is no heterogeneity. If the test of heterogeneity is statistically significant (P<0.05), then more emphasis should be placed on the random effects model.
- When the option Marker size relative to study weight was selected, then the size of the markers that represent the effects of the studies vary in size according to the weights assigned to the different studies.
- When the option Diamonds for pooled effects was selected then the pooled effects are represented using a diamond. The location of the diamond represents the estimated effect size and the width of the diamond reflects the precision of the estimate.
A funnel plot is a graphical tool for detecting bias in meta-analysis. See Funnel plot.
- Begg CB, Mazumdar M (1994) Operating characteristics of a rank correlation test for publication bias. Biometrics 50:1088-1101.
- Borenstein M, Hedges LV, Higgins JPT, Rothstein HR (2009) Introduction to meta-analysis. Chichester, UK: Wiley.
- Egger M, Smith GD, Schneider M, Minder C (1997) Bias in meta-analysis detected by a simple, graphical test. BMJ 315: 629–634.
- Higgins JP, Thompson SG, Deeks JJ, Altman DG (2003) Measuring inconsistency in meta-analyses. BMJ 327:557-560.
- Petrie A, Bulman JS, Osborn JF (2003) Further statistics in dentistry. Part 8: systematic reviews and meta-analyses. British Dental Journal 194:73-78.
- Zhou XH, NA Obuchowski, DK McClish (2002) Statistical methods in diagnostic medicine. New York: Wiley.