How Enhancing Access to Replication Studies Can Improve Scientific Self-Correction

Replicating scientific experiments is crucial for establishing reliable research. However, finding replication studies can be a challenge. A dedicated team of researchers aims to change this trend by posting their replication findings, along with links to the original studies, on the post-publication peer review platform, PubPeer.

Over the last decade, numerous attempts to reproduce scientific experiments have been made, yet many have failed to validate original results. The inability to locate these replication studies often leads to wasted research time and resources. If scientists remain unaware of replication attempts, especially concerning unsuccessful studies, they may incorrectly cite the original research without acknowledging any underlying issues.

Don Moore, a social psychologist at the University of California, Berkeley, stated, “If PubPeer helps increase awareness of both successful and unsuccessful replications, it will promote scientific self-correction and expedite scientific progress.” Moore’s 2012 study on overconfidence was successfully replicated this year as well.

The team behind this initiative plans to create discussion threads on PubPeer for approximately 2,400 replication studies in batches of around 100. These replication studies will be indexed in the FORRT Library of Reproduction and Replication Attempts (FLoRA), part of the Open and Reproducible Research Training Framework (FORRT), which aims to educate researchers on open science and reproducibility.

FORRT launched a comprehensive database in 2024 specifically designed to catalog replication studies. However, database creators at Germany’s Münster Open Science Center highlight that locating these studies remains challenging, as they often have no direct links to the original research in many databases.

Josefina Vajnerova, a psychologist at Birkbeck, University of London and co-leader of the PubPeer project, noted, “Replications tend to be cited less frequently and are often less visible than their original counterparts.”

Out of the roughly 2,400 studies currently cataloged, around 1,000 were successfully replicated independently, while 865 replication attempts did not succeed. The remaining studies had partial results reproduced.

Winerowa and her team intend to inform all authors of papers that have generated threads on PubPeer. While the initial contacts elicited few responses, none of the authors who did reply requested the removal of comments.

Although PubPeer is commonly used to highlight potential issues or errors in research papers, some researchers express frustration when their work is discussed, even in the absence of found errors. Winerowa argues that researchers should welcome the platform highlighting studies that confirm their findings. “It helps other scholars understand that replication was successful,” she asserts. “While some comments on PubPeer may seem biased, many do not indicate serious concerns with the original paper.”