There’s a joke from “That Mitchel and Webb Look” that I want to dissect like a frog for a moment. The video is just one minute long, but if you don’t want to watch it I can summarize it here:

• “So for the sketches we’re filming, I’m thinking we’ll make them “hit, hit, miss, hit, miss, miss”
• “Do we have to film all the misses as well as the hits? Why not only film the hits and use those for the show?”
• “Well it’s a sketch comedy show, it has to be hit and miss.”

The joke doesn’t need to be explained, but I will anyway: why does a sketch comedy show have a lot of sketches that miss the mark, as well as ones that are laugh out loud funny? Isn’t it easier to just film the hits? Well obviously the writers didn’t think those misses would miss the mark, they thought those misses might be hits as well, that’s why they wrote them and filmed them. You don’t know for sure what will be a hit and what will be a miss before you release the show.

A similar pattern is discussed with venture capital investing. Venture capitalists invest in hundreds of startups on the assumption that around 90% of them will fail and make no money at all. The 10% that succeed are expected to pay for all the failures. Well then why don’t venture capitalists *only* invest in the successes and not waste money investing in the failures? Again: they don’t know for sure what will succeed or fail before investing. A huge amount of time and money goes into predicting the success or failure of startups so these VCs can try to invest wisely, but it isn’t a solved problem by any means.

And if you think this investing problem has an obvious solution, take out a personal loan and invest 50,000$ in a single startup that *you know for sure* is guaranteed to be successful. You’ll 1000x your money and be able to pay off the loan and interest easily.

But this pattern of “why not only go for the hits?” holds true in science as well. But here many people don’t seem to understand or believe it.

Governments, corporations, and charities invest billions into potentially lifesaving treatments every year. 90% of those scientific ventures will come to nothing, only a few will be successful. But you don’t know for sure which will succeed and which won’t before you try.

I think of this because I all too often see people complain about “why did we invest X number of dollars into researching such and such, when Y was invented with so much less?” A World War 2 version of this is the infamous refrain about how the project to develop a better bomb-sight for American planes costed more than the Manhattan Project which made nuclear bombs. A modern version of this complaint might be complaining that the Amyloid hypothesis in Alzheimer’s disease has received so much funding despite never curing Alzheimer’s.

In both cases though, our best foreknowledge seemed to indicate that this was the right path. Nuclear fission was completely unproven tech, the scientists themselves were pessimistic about their abilities to make a bomb out of it. When the first test of a real nuclear bomb took place, the scientists involved had a bet going for how much power the bomb would produce (with some predicting it would be a dud). *EVERY SINGLE ONE OF THEM* drastically underestimated the power of the bomb they had created, the most wildly optimistic predictions underestimated the bomb’s power by half.

By contrast air-power was a proven war winner when the USA started spending billions on bomb-sights. Germany’s blitzkrieg had used massive air power to help them overwhelm, surround, and destroy, other nations all across Europe. Air power could destroy the railroads and bridges that let troops move across modern battlefields, it could destroy the factories where the troop’s guns and tanks were made, and domination of the air allowed an army a far better picture of the battlefield then their enemies had. In this scenario, the allies looked at the success of German air power and believed that upping their own air power might similarly prove dividends. They never got the total success of the German blitzkrieg, but overwhelming air power was at least part of how the USA held on in the Korean war, so it wasn’t a complete waste.

Similarly, the evidence for Alzheimer’s disease has always seemed to point toward Amyloid Beta playing a key role. The evident failure of drugs targeting Amyloid Beta means there’s a lot more we have to learn, but just because the Amyloid Hypothesis is flawed doesn’t mean a competing hypothesis is automatically right. Putting billions towards the Tau or neurotransmitter hypotheses is not guaranteed to have brought success, in fact these hypotheses were studied even during the dominance of the Amyloid Hypothesis, and neither of them produced working drugs either.

People have a video-game understanding of research, as I’ve complained about before. They think that if we just put enough money towards the correct hypothesis, we’ll find what we’re looking for. But we don’t know what’s correct before we commit our money, and if our hypothesis fails, we don’t even know if we just haven’t thrown *enough* money at the problem, or if we’re chucking good money after bad. Which answer you lean to likely says more about your politics than about the quality of the research itself. Should we throw more and more money towards commercial nuclear fusion, even though that industry has never once succeed in even the most modest goals set for itself? Should we cut off the Amyloid Hypothesis, even though a century of research shows that Amyloid Beta does play a key role in Alzheimer’s disease? Everyone seems to think they already know the answer, but few are willing to prove it with evidence.