Optimisation of decision making

Optimisation strategies are important for a huge range of industries. From optimising manufacturing, to improving corporate business processes, whatever your domain, optimisation is simply about doing things better. We were recently introduced to an excellent and beautifully simple explanation of an optimisation problem by Lê Nguyên Hoang from MIT. It has applications in a whole host of industries, but Hoang uses a brilliant everyday experience that most people have been through to introduce it.

Processes occur throughout organisations, whether service processes in the field, manufacturing in a factory facility, or in business processes at the corporate heart of the organisation. Processes, unfortunately happen over time, and at different points in the process, we usually have incomplete information, but still have to make the best possible decision, which will probably be irrevocable - we can't go back in time and change it later. This is the challenge Hoang explores: how to make the best irrevocable decisions as more information is accumulated. Choosing a clever strategy can make it surprisingly easy to make good decisions.

For example suppose you have a concept development team as part of your R&D department. Their job is to come up with potential new product ideas, and your task is to select the best products to take forward into full development. Of course, they don't come up with hundreds of new products all at once, they produce a steady flow of ideas - say one per week - but you've only got the resources to take forward one product each year. You could wait until the end of the year, and evaluate all the product ideas, but that will cause potential lost time, with the team down the hall waiting to take it into full development having to wait a whole year before they can get started.

Trash CanEach week you have an irrevocable decision, take this weeks idea forward, or bin it and wait for something better.

But if you take a decision before the end of the year, what's the probability of getting the best idea of the whole year? Well intuitively its one in 52, because each week you're making the same choice, and you can only take one forward each year. However, there are strategies that can actually do much better than this. In fact, there is a strategy that picks the best idea at least 36% of the time, without waiting till the end of the year.

The strategy that achieves this is simple. The first few ideas we automatically reject, and we use them as a learning phase. We then select the idea that is better than all the ones we've considered so far. This in fact yields a probability of at least 36% of choosing the best idea.

This example shows how a relatively simple strategy can improve the probably above that we might intuitively expect. In reality we can devise much more complex strategies. For example, most of us would be happy with a product that has lots of potential, but not necessarily the best.

Dollar SignIn the commercial world most things have a cost, so we have to build into our strategy that different choices have different costs attached. In our scenario, there'll be a cost to the full product development, and the cost will vary depending on the idea that goes through to full development. Moreover, we only have a limited amount of cash to pay for the development - an R&D budget.

In addition, we may not have to make just one choice. Perhaps we can take forward the 5 best ideas of the year.

Of course, with the introduction of multiple choices, costs, budget and concepts with variable potential, the problem becomes more complex, but one of the better strategies is very similar to the simple strategy outlined above. Namely that we automatically reject the first ideas we consider, but use them to assess how cost of development is related to potential of the product, subject to the constraint of our budget. We then use this information to identify a good opportunity when we carry on checking ideas beyond our initial learning phase.

There is actually an even better strategy. Essentially its an adaptation of the strategy from the previous paragraph, but is a more continuous learning process. In the paragraph above, we made a fixed number of learning trials, then threw all our budget at the first ideas that looked good for the money they cost. Instead we can continuously accumulate information about what makes a concept idea good value for money, and as our knowledge improves we can become more willing to part with a greater proportion of the budget we have available.

In fact, its recently been proven that this is strategy is the best you can hope for - you'll not be able to do better. Its nice to know you can give yourself the best chance of success within the constraints of your budget.

There are many other wide ranging applications of these methods. For example it has been applied to organ donor matching, where a near, but not precise match is required. Do you hold out for a better match, or go with the near enough one? Clearly its ideal to give the most people the best possible matches. Another example is online dating websites, where we need to present the best possible matches, but not keep our singleton waiting forever. Stock market investments are another application, or lending money to a new business venture, where we may only be able to examine one investment opportunity at a time, but we don't want our money sitting around for too long.

Yes or NoIn fact any process where you have to make 'go' or 'no go' decisions repeatedly, but can't go back and change past decisions, will lend themselves to this type of analysis, and its great to know you can devise a strategy to make the best possible decisions.


So go ahead an optimise your decisions today!

If you want further reading, then we thoroughly recommend Lê Nguyên Hoang's really simple exposition of this technique using a wonderful everyday example. It does get a bit mathematical, but he does shield you from the really complex stuff!