An Empirical Assessment of Meta-Analytic Practice

In the three decades after the publication of the first meta-analyses in the behavioral sciences, hundreds of articles and a number of technical guides have emerged concerning meta-analytic practice and reporting standards. The purpose of the present study is to review the practice and reporting standards of a random sample of published meta-analyses (n = 100) in psychology and related disciplines in the decade from 1994 through 2004. We focus on practice and reporting at each stage of the meta-analytic process and explore differences between psychological subdisciplines. These findings suggest that the practice of meta-analysis in the last decade has not yet converged on a set of common standards, though some expert recommendations are beginning to be heeded. Authors should be attentive to proper procedure and reporting in light of the numerous threats to the validity of a meta-analysis. Ironically, even though meta-analysts often struggle with incomplete or inconsistent reporting in primary research they are themselves not entirely consistent in reporting their methods and results.