Open File Drawer

Like all scientists, members of our lab sometimes design and conduct experiments that we don’t ultimately consider “successful.” What we mean by this is not that the results weren’t “statistically significant” but rather that the evidence didn’t ultimately help us to adjudicate between the alternative hypotheses we were trying to test. 

Often after an experiment fails in this sense, we design a better one and try again. But sometimes we simply decide to move on from the project of testing the hypothesis in question, either temporarily or permanently. Ambiguous data are hard to publish, and also probably not of interest to a very broad audience, so it may not make sense to invest a lot of time writing the experiment up and getting it through peer review. At the same time, information about the experiment and results may be of interest to some, particularly other researchers investigating similar hypotheses. Due in part to routinely preregistering experiments, for each experiment we thought carefully about our design choices and collected a sample with sufficient power to detect anticipated effects.

So, we are trying out an experiment on the practice of science itself. We are creating brief write-ups of the completed experiments that we don’t intend to immediately iterate on or try to publish–the kinds of outcomes that go in labs’ figurative "file drawers"–and posting them here where they are accessible to all. This will often involve lightly editing preregistration text and then appending results; we also plan to routinely include links to open materials, data, and code. 

The hope is that this will be useful to other researchers considering similar topics or those conducting meta-analyses. Perhaps some of these reports will also spark collaborations in which another team will want to follow up where we left off; please get in touch if this is ever the case! It may also facilitate our own lab reviving projects down the road, if interest rekindles and we need a good reminder of what we’ve done before.

In sum, the reports below are not preprints or in prep manuscripts, nor are they a collection of all the lab’s null results (which may deserve publication!). They are reports of results that, despite our best efforts, we feel are inconclusive, and that we don’t have the time or resources to follow up on in the immediate future. We hope they are useful!