You are viewing the site in preview mode

Skip to main content

Table 3 Meta-analyses of the proportions of associations for each criterion and level of evidence (random effects)

From: Validity of observational evidence on putative risk and protective factors: appraisal of 3744 meta-analyses on 57 topics

  n/N associations (crude proportion) Proportions
[95% CI] from meta-analysis of 57 topicsa
I2 Range of proportions across topics
Level of evidence:
 Convincing 99/3744 (2.6%) 1.3% [1.0%; 2.2%] 73.9% 0–16.7%
 Highly suggestive 253/3744 (6.7%) 4.6% [2.9%; 6.6%] 85.7% 0–33.3%
 Suggestive 440/3744 (11.8%) 11.0% [8.5%; 13.8%] 83.9% 0–50%
 Weak 1497/3744 (40.0%) 39.1% [34.8%; 43.5%] 86.2% 0–71.4%
 Non-significant 1455/3744 (38.9%) 34.7% [29.2%; 40.3%] 90.8% 0–80.7%
Criteria:
 Statistical significance
  P < 10-6 762/2289 (33.3%) 29.0% [24.9%; 33.3%] 74.8% 0–66.7%
  P < 0.001 1377/2289 (60.2%) 58.6% [54.1%; 63.0%] 73.3% 0–100%
 Cases > 1000 (or > 20,000 participants for continuous outcomes) 1182/2107 (56.1%) 65.3% [56.9%; 73.2%] 94.9% 1.7–100%
 Largest study with P < 0.05 1343/1781 (75.4%) 74.9% [71.2%; 78.4%] 63.6% 28.6–100%
 95% prediction interval that excluded the null 642/2136 (30.1%) 30.3% [26.5%; 34.2%] 71.0% 9.0–100%
 Absence of large heterogeneity (I2<50%) 1050/2277 (46.1%) 46.6% [41.8%; 51.3%] 79.5% 0–88.2%
 No evidence of small study effects (P > 0.10) 1628/2164 (75.2%) 75.3% [72.2%; 78.3%] 63.2% 40–100%
 No evidence of excess significance (P > 0.10) 1599/2052 (77.9%) 77.7% [72.6%; 82.5%] 83.0% 33.3–100%
  1. aMeta-analyses for the individual criteria excluded associations with missing data. Meta-analyses for the levels of evidence were conducted across all 3744 associations regardless of their statistical significance status. The meta-analyses for the individual criteria were conducted across the 2289 statistically significant associations. Out of 2289, statistically significant associations, 182 associations did not report on the number of cases,508 on whether the largest study had P < 0.05, 153 on the 95% prediction interval, 12 on the I2 for heterogeneity, 125 on the small study effect test, and 237 on the excess of significance bias. Data on all 7 criteria were available for 1457 statistically significant meta-analyses