Mario Gaudino 1 , N Bryce Robinson 1 , Antonino Di Franco 1 , Irbaz Hameed 1 , Ajita Naik 1 , Michelle Demetres 2 , Leonard N Girardi 1 , Giacomo Frati 3 4 , Stephen E Fremes 5 , Giuseppe Biondi-Zoccai 3 6
1 Department of Cardiothoracic Surgery Weill Cornell Medicine New York NY. 2 Samuel J. Wood Library and C.V. Starr Biomedical Information Centre Weill Cornell Medicine New York NY. 3 Department of Medical-Surgical Sciences and Biotechnologies Sapienza University of Rome Latina Italy. 4 IRCCS NEUROMED Pozzilli Isernia Italy. 5 Schulich Heart Centre Sunnybrook Health Science University of Toronto Toronto Canada.6 Mediterranea Cardiocentro Napoli Italy.
J Am Heart Assoc. 2021 Aug 3;10(15):e019903.
Background Quality of the peer-review process has been tested only in small studies. We describe and summarize the randomized trials that investigated interventions aimed at improving peer-review process of biomedical manuscripts. Methods and Results All randomized trials comparing different peer-review interventions at author-, reviewer-, and/or editor-level were included. Differences between traditional and intervention-modified peer-review processes were pooled as standardized mean difference (SMD) in quality based on the definitions used in the individual studies. Main outcomes assessed were quality and duration of the peer-review process. Five-hundred and seventy-five studies were retrieved, eventually yielding 24 randomized trials. Eight studies evaluated the effect of interventions at author-level, 16 at reviewer-level, and 3 at editor-level. Three studies investigated interventions at multiple levels. The effects of the interventions were reported as mean change in review quality, duration of the peer-review process, acceptance/rejection rate, manuscript quality, and number of errors detected in 13, 11, 5, 4, and 3 studies, respectively. At network meta-analysis, reviewer-level interventions were associated with a significant improvement in review quality (SMD, 0.20 [0.06 to 0.33]), at the cost of increased duration of the review process (SMD, 0.15 [0.01 to 0.29]), except for reviewer blinding. Author- and editor-level interventions did not significantly impact peer-review quality and duration (respectively, SMD, 0.17 [-0.16 to 0.51] and SMD, 0.19 [-0.40 to 0.79] for quality, and SMD, 0.17 [-0.16 to 0.51] and SMD, 0.19 [-0.40 to 0.79] for duration). Conclusions Modifications of the traditional peer-review process at reviewer-level are associated with improved quality, at the price of longer duration. Further studies are needed.