Incentives, Solutions, and Machine Learning

Traditional economic wisdom about solving problems and incentives says:

We get what we pay for.

As a logical corollary, we don’t get what we don’t pay for.

This is known as opportunity cost. We focus on what we want, and leave behind the things we judge less important.

At a small level, this is an entirely sensible philosophy that we use to guide our everyday decision-making. If we’re hungry and we pay for Thai food for lunch, by extension we are not paying for a burger and fries. This is an acceptable trade-off; at the conclusion of the meal, we shouldn’t be hungry.

However, when we apply this philosophy to bigger decisions, we often create unintended consequences. For example, if we decide feeding the hungry and poor is a focus, we might invest in food programs. However, in doing so with such a single minded focus, we might underinvest in jobs programs. In this scenario, it’s easy to see that cutting investment in jobs programs will likely have the unintended consequence of creating more poor, hungry people.

The reality is that complex issues rarely have a single solution. Most often, complex issues have extremely complex solutions, ones which we humans struggle to understand and solve. The good news is that artificial intelligence and machine learning may be able to solve these problems better than we ever could.

A simplistic example of this is the decision tree, a type of statistical analysis very common in machine learning. Decision trees help us to understand what choices lead to the outcome we seek; in virtually every case, using a decision tree system will yield multiple ways to achieve an outcome:

In the simplistic SEO example above, I built a decision tree to evaluate what makes a link authoritative and trustworthy using Moz data and IBM Watson Analytics. No one variable – no one easy solution or incentive – in the model governs trustworthiness of a link; rather, a combination of variables likely drives trust – and several models exist. I’d turn this analysis into a usable outcome by building an SEO marketing program around the data and testing to see which model generates the best results.

For our largest societal problems, we’d use similar analysis but at a much larger scale, and with machine learning to take outcomes and feed them back into the model to improve it continually. While we can solve for SEO problems in just a few iterations, solving something like poverty will likely require massive computational resources and years to do the analysis – but it’s possible, within our reach.

If we do this, we will better understand that no one incentive or solution will fit a complex problem. Building and sharing this understanding will help reduce “quick fix” solutions everywhere, from marketing to business to politics. If we build well, the rise of the machines may well lead to the rise of a new, better civilization.