This is a short one today, it’s been a busy week. I just wanted to share an anecdote from my work:

We’ve been operating under a certain hypothesis for as long as I’ve worked here. We think if we do a certain experiment a certain way we’ll get certain results. We haven’t managed to get those results yet but we are tweeking and revising the experiments in an attempt to do so. Yesterday I randomly ran into a professor who shared with me a paper he had just published, a paper which seemed to indicate that the results we were searching for my not be possible, or at least might not be possible using the experiment we were doing. Now why had we believed our experiment would work? Well we read a different paper that seemed to indicate it would.

So now I have a conundrum, I have this old paper that says what we’re doing will definitely work, and this new paper saying maybe it won’t. What do I do? I start by re-reading both papers to make sure I’m not misunderstanding them, and I come upon something I never realized: the old paper may not have proven what it thought it proved. Maybe the results from the old paper are actually closer to the results from the new paper, but were just interpreted wrong. If that’s the case then the new paper is correct and our experiment won’t work. We read the old paper and believed it’s interpretation, but we didn’t put enough effort into validating that it’s interpretation was correct based on its data, we assumed the paper had done that well enough. But with the benefit of the new paper we can see that maybe its interpretation was wrong.

This is a very heavy conclusion: the paper we have been basing our research on might have a wrong conclusion. It’s a harsh accusation but in science it’s sometimes necessary to speak out and make these accusations. You can’t keep going down the wrong path or you’ll never go anywhere.