Replication and the Limits of a Single Study
Separate a new experiment from a new analysis
Running an existing dataset through the same code checks whether the computational result can be obtained again. Collecting new data to test the same scientific question asks whether the finding recurs in another investigation. Both activities matter, but they address different uncertainties.
The National Academies: Understanding Reproducibility and Replicability explains its distinction between computational reproducibility and replicability, while noting that terminology varies across disciplines. When reading a peptide paper, describe what the follow-up actually did rather than relying entirely on the label replication. A second chart drawn from the same data is not an independent biological experiment.
Look for independence and comparability
A follow-up can differ in investigators, laboratory, material batch, model or analysis. These differences may reveal whether a result is robust to ordinary variation. They can also make a disagreement harder to interpret if several important features change at once.
For a hypothetical peptide assay, ask whether the new work used the same sequence, chemical form, endpoint and observation time. A different reagent or modified assay may test a related question. Recording those changes helps determine whether the evidence challenges the original result directly or explores the boundary of its applicability.
Compare estimates rather than labels
One study may cross a statistical threshold while another does not, even when their effect estimates are compatible. Read magnitude and uncertainty before declaring a contradiction. A smaller follow-up might simply provide less precision, while a well-powered comparison may materially narrow the plausible range.
The Cochrane Handbook, Chapter 15: Interpreting results and drawing conclusions supports interpretation that considers effects and uncertainty rather than threshold labels alone. Agreement also requires care: several studies using the same flawed measurement can repeat a systematic error. Independent methods that address the same question may add information that repeated use of one assay cannot.
Build a cumulative evidence record
A useful evidence table groups genuinely independent studies and records the main differences between them. Include null and mixed findings alongside favourable ones. Identify whether several papers reuse the same participants, samples or underlying dataset before treating the publication count as the number of replications.
Material records can help readers interpret laboratory differences. Our certificate library concerns documentation, while our research overview concerns scientific context. Neither replaces independent experiments. Together with clear study records, they help formulate a specific next question: which feature of the reported effect should another group be able to observe under comparable conditions?
Related research reading
- Preprints and Peer-Reviewed Papers: Recording Publication Status
- Multiple Endpoints: Which Question Was the Study Designed to Answer?
Explore the research blog and original batch-document library for further context.