We appraised 432 sex-difference claims in 77 eligible articles. Authors stated that sex comparisons were decided a priori for 286 claims (66.2%), while the entire sample size was used in 210 (48.6%) claims. Appropriate documentation of gene-sex interaction was recorded in 55 claims (12.7%); documentation was insufficient for 303 claims and spurious for the other 74. Data for reanalysis of claims were available for 188 comparisons. Of these, 83 (44.1%) were nominally statistically significant at a P = .05 threshold, and more than half of them (n = 44) had modest P values between .01 and .05. Of 60 claims with seemingly the best internal validity, only 1 was consistently replicated in at least 2 other studies. (Patsopolous et al. Journal of the American Medical Association 298:880-893; 2007 – abstract)
Nature (“Let down by the statistics”) and Science (“Epidemiologist sees flaws in papers on genes and gender”) report on the study. David Balding sums it up this way in Nature: “This paper reveals an entire industry of prominently reported results that are largely unjustified and probably mostly false.”
Leave a comment