Hardly any innovation board gets by without a scoring model these days, and criteria and weights give the prioritization a coat of objectivity. That coat is the problem: underneath the scores sits the same gut feeling as before, it's just harder to see. You end up with a number with two decimal places that everyone trusts because it comes out of a spreadsheet.
Scoring models mix math and intuition. Those are exactly the two things you should keep strictly apart.
Why point scales don't reflect reality
People are bad with any number larger than five. On a scale from 1 to 5, a project with little benefit gets a 1 and one with a lot of benefit gets a 5, but the scale doesn't show you that the second one might deliver a hundred or a thousand times as much. We simply tend to rate things too similarly. Weighting the criteria doesn't fix this either, because hardly anyone weights one criterion a hundred times higher than another, even when it would deserve it. For orders of magnitude we can no longer picture, though, we've had a solution for thousands of years: math. And math only works if you don't stir it together with intuition. A scoring model does exactly that, it squeezes gut feeling into cells and makes the result look like a calculation. The imprecision alone would be forgivable, the unearned trust isn't.
One metric instead of twenty criteria
The alternative is to drop the point scoring entirely and pick a single metric instead (normally the net present value), onto which the contribution of every aspect is properly quantified. What honestly can't be quantified, strategic relevance for example or strengthening the brand, still belongs in the evaluation, but as text and clearly on the intuition side, not weighted pseudo-mathematically. In the end a person makes the decision anyway, and that person takes both into account, the numbers and the soft factors.
“But NPV kills innovation”
The objection is well known, prominently from Clayton Christensen: discounted cash flow punishes distant, uncertain returns, so the incremental project always wins. The point is valid, but neither extreme is of any use. Many idea owners massively overestimate the probability of distant benefits, and innovation then turns into daydreaming. Discounting distant returns is certainly not bad practice, and in a properly set-up model (with annual price increases, for example) some moonshot projects come out looking attractive too. In most moonshot cases, however, the uncertainty isn't quantified at all, which is why nobody trusts them, and often the cases are simply badly built. In nine out of ten cases the short-term project honestly should win. The moonshot project should only win if it really holds good cards.
How the business case actually gets built
You start with profit and split it into revenue and costs, and those in turn into their drivers. For each driver you set range assumptions instead of point values, and a Monte Carlo simulation turns them into the probability distribution of the NPV. Every effect that can drive success belongs in the model, and the assumptions that bring a lot of uncertainty into the NPV get broken down further or validated. It's important to apply the same criteria everywhere (how many years you model, which price increases apply), otherwise you're comparing apples and oranges. At the end each project comes down to a few numbers: the expected NPV, the risk (that is, the standard deviation of the NPV), and the investment or value at risk. Plus a description that addresses the soft factors.
And with 80 ideas in the pipeline?
For a long time the usual objection to this approach was the effort, and with AI that objection is basically settled. Also, it isn't the innovation manager who builds eighty models, it's each idea owner building exactly one, for their own idea. With Bayescase, our innovation management software, this works via an interview: you describe your idea and the rough case comes together in the conversation. The thinking behind it is to start with a rough case and only validate and refine the projects that look promising.
That way you can prioritize based on value, risk and investment size, and use your intuition where it belongs, on the factors that can't be calculated.
Frequently asked questions
- How do you evaluate innovation projects?
- With a single metric instead of many weighted criteria, normally the net present value. To get there, you split profit into its drivers (revenue, costs, and those further down), set range assumptions instead of point values for each driver, and a Monte Carlo simulation produces the probability distribution of the NPV. Factors that can't be quantified (strategic relevance, for example) go into the evaluation as text, clearly separate from the calculation. Important for comparability: the same modeling criteria for all projects, such as the same time horizon.
- What's wrong with scoring models and weighted scoring (Nutzwertanalyse)?
- Point scales hide orders of magnitude: between a project rated 1 and one rated 5 there can be a factor of 100 or 1,000 in actual benefit, and weightings rarely capture that either. Scoring models also mix math and intuition, so the result looks like a calculation but is gut feeling at its core. You then trust the number for the wrong reason.
- What metrics do I need to prioritize innovation projects?
- Three values per project are enough: the expected NPV (value), the standard deviation of the NPV (risk, or uncertainty) and the investment or value at risk. Together with a short description of the factors that can't be quantified, that lets you prioritize a portfolio in a way others can follow.
- How do I know whether our evaluation of innovation projects is good enough?
- A good test is whether any two projects can be compared directly in euros and risk, and whether the selection would hold up in front of finance. Where your evaluation stands within the whole innovation process is what the innovation process check shows in about three minutes.
