We Got It Wrong: Data Isn’t About Decision-Making

We data people have got it all wrong for decades. We think we do this data stuff to support decision-making. We build data lakes and platforms to enable business to “make informed decisions.” We develop highly visual and interactive dashboards to allow people to analyze data and “make data-driven decisions.” We train machine learning models to “predict outcomes and recommend decisions.” It’s all about decisions. Decisions run the world. We humans run the world through decision-making. Except that we don’t. It does not work that way.

First, some context. We’ve known for decades that it is very hard to get data and analytics initiatives to deliver expected value. This was true in the era of BI and is still true in the era of data science, machine learning, artificial intelligence, cloud computing and you-name-it. A great deal of it has to do with our passion for tools (the solution), which makes us neglect the business, human, or environmental problem we should be obsessed with. We also don’t see the holistic picture of the organizational change required to implement something that sustains. Well, some of us do, but still surprisingly few.

Those that do claim that “technology isn’t the obstacle, culture is,” among other simplistic but yet vague explanations. Obviously, culture seems hard when you approach the problem as a technology problem, with technology approaches and with technology skills and resources. If your frame of mind is technology, data and analytics, then yes, culture will seem hard. But you know what, if you approach the problem as a cultural problem, with cultural approaches and cultural skills and resources, then culture won’t seem so difficult. In current business parlance, culture is often defined by negation, relegated to a residual category—the “other stuff” left when everything else is defined. It’s the weird stuff that doesn’t catch the empiricist’s eye, the “general vibe, you know…” Or wait! You must have your company’s “core values” somewhere, right?

Effective cultural work is a systematic process involving analysis, definition, and operationalization across various levels—from shared practices to representations in material surroundings; from dominant to emergent forms, from hegemonic to contesting forms. There are hundreds of years of well-designed practice for this. It’s called social science.

We have misunderstood how humans work

So, culture isn’t the obstacle. Typical approaches to the problem of change are the obstacle. So why is this then? It’s not because our data and analytic programs don’t have a change management workstream focusing on communicating progress via emails and roadshows. It’s not because we didn’t create a solid business case or well-crafted benefit statements that everyone buys into. We have to add one more root cause to the list of “why it’s so hard to implement data stuff in organizations.” This is a big one: We have completely misunderstood how humans work.

As a species and as individuals, humans are first and foremost doers and actors, who skillfully make their way in social worlds. Only secondarily are we deliberate decision-makers. This doesn’t make the latter feat any less an important part of human conduct, but the order of things is important to understand: we act based mainly on intuition, and sometimes, when it’s desperately needed, we might give deliberate pre- or after-thought to how we justify our actions.

We construct reasons as justifications for social interaction. We don’t construct reasons to guide action. This is the main point of numerous studies by some of the leading cognitive and social scientists like Dan Sperber (2017), Hugo Mercier, and Jonathan Haidt (2012).

“Humans are an uber-social species, constantly evaluating each other to figure out who would make the best cooperation partners: who is competent, who is nice, who is reliable. As a result, we’re keen to look our best, at least to people whose opinions we value. […] We not only spontaneously justify ourselves when our behavior is questioned but also learn to anticipate when justifications might be needed, before we actually offer them. This creates a market for justifications.” (Mercier 2020, 206)

Pouring vast amounts of data over things doesn’t change this. Actually, it might make things worse.

More data ≠ better decisions

Data as such has zero value. If it has value, it’s negative. Data operations eat resources and are at worst a tool for rationalization of bad decisions. “When we evaluate justifications for our own views, or views we agree with, our standards are low,” as Mercier’s studies point out (2020, 209). Excessive data abstractions not grounded in tangible contextual insight are an easy tool for just such justifications. As Gerd Gigerenzer, a psychologist of bounded rationality has shown, in circumstances of uncertainty, actually less data (and more reliance on heuristic thinking) is often good for smart decisions.

An ocean of data can lead to problems especially when data is big but lacks grounding in qualitative understanding of human behavior, phenomena and meaning, as well as contextual interpretation of relevance (otherwise known as thick data). This, luckily, is starting to be old news for most organizations we work with. But there’s more. Two other points we want to stress relate to what is actually done with all that data and how. Again, pouring in loads of data doesn’t do much good, and can do much bad when:

1. Data is used as part of only solitary (instead of social) decision-making, and when interpretation, deliberation and decision-making is isolated from the social contexts of contest and debate. We know that reasoning excels as a social system, not as an individual act.
2. Data by itself is assumed to lead to impactful behavior change in the organization. Behavior change is the last mile that makes everything else worthwhile. Or makes everything plain useless, if it doesn’t happen.

This is why quantitative data always needs to be supported by qualitative, cultural and behavioral insights about (a) humans whom the data tells something about and (b) humans (in the organization) whose actions it should affect.

Facilitate social deliberation and change behaviors

One of the most common and typical cases for informing decision-making with data is to produce new dashboards for sales roles, especially in a B2B context. So, we blend CRM data with financial data, throw in some digital engagements if available, and produce fancy dashboards with every possible metric available within a click or two. We then expect our salespeople to magically change how they’ve gone about their work up to that moment. We expect their customer targeting to improve and sales to increase. But too often nothing changes, and we end up with the conclusion that we did everything right technically, but culture was the obstacle. We think we failed with change management and explaining the reasoning behind the change – but we’re wrong.

One of the authors of this post (Lasse) has worked with data and analytics for practically his whole career. Only recently, after joining a design thinking community, has his eyes opened to appreciate the value of design thinking as a philosophy and the importance of applied social science, which is what Antti has been working with his whole career. We have not made an impact unless we have changed behaviors. That’s essentially how culture should be perceived—as the coordination of practices, habits and behaviors. Assumptions, theories, simulations and calculations are all great, but they always need to be supplemented by critical scrutiny on how they affect actual praxis. And the way to impact behaviors is to be obsessed with understanding how we humans are and act, as individuals and as members of our communities.

Another crucial lesson from design thinking pertains to the value of social deliberation—working in multidisciplinary teams to interpret, understand, learn, and iterate. Mercier and Sperber stress especially this in their important work on reasoning: reasoning excels as a social system. Each individual is inevitably bound by what they call the “myside bias” (confirmation bias), but the social setting introduces a need to produce good justifications which can actually alter others’ opinions. This social contest and evolution of ideas is the mechanism of incorporating varied enough insights to actually produce good decisions. Individual reasoning alone, with or without data, is not enough. Good reasoning, by nature, is social. A well-working setting for such social behavior usually needs some organizational design.

We need to get into the real world

Let’s go back to our example of the sales rep and the failed dashboards. At no point did we for example stop to think and appreciate that a day in the life of a sales representative is very different to that of an office worker. Sales reps work very flexible hours, are highly mobile, and the nature of their work is extremely social; it’s all about people. So maybe we should not design dashboards for them in our 9-to-5 office desk bubble where all we see is laptops, data, and metrics. No wonder we think culture and change management are hard. We did not approach the problem as a behavioral problem, and we did not put effort into understanding the overall context of the targets of the change. It’s not that we didn’t communicate and ask for feedback, but we did not nudge the sales reps to change their habits. We just assumed that when presented with facts and information, they will automatically change how they go about their day.

Further, we also failed to appreciate the social setting of the sales rep and how they plan and decide on their actions. Our mission should probably have been to design the practices and behavioral patterns in the sales rep’s social setting. How they interact with their manager and peers, how they work with the analyst or data scientist behind the analysis, and how they engage with other HQ staff. How do they use the data and information in social context to deliberate on the right conclusions to be drawn and decisions to be made? And then change – or not change—their behaviors.

In short, the simple things we could have done differently here:

• Get out of our office and do this with the sales teams in their real world, following and understanding their practices and behavioral patterns.
• Approach the problem as a behavioral problem, aim to fully understand these behaviors in their social context (use thick and thin data), and aim to design behaviors, not just information delivery.
• If we then think we will change behaviors by delivering information in a dashboard-like solution, we should validate this hypothesis by observing how the sales teams use and behave with the dashboards. Pay attention to the social dynamics, as that is how we best make informed decisions. The dashboard is likely to be just a subplot to how the problem is best addressed.

Less information, more change

In a way, this whole blog post is borderline oxymoronic. We are making rational and evidence-based arguments about how facts and information don’t automatically mean people make decisions and then change behaviors based on the information presented. Essentially, if our argument is true, most of you will not change your behaviors based on what we have presented here—even if we are making a strong case for why you should. However, with this blog post we are contributing to the social evolution of ideas. And we know that we have a fair share of readers who already recognize problems with current intellectualistic approaches to changing behaviors. While this blog post by itself may have only limited impact to changing behaviors, the ideas we communicate—if they resonate socially—do build ground for it.

To wrap up, let’s try nudging you toward a better success rate by a few simple recommendations:

• Stop buying the “culture is an obstacle” excuse: Data and analytic initiatives still fail to deliver their promised full value. We now know technology alone isn’t the solution and are led to think that culture is the biggest obstacle. This conclusion does not help us and only offers a sloppy excuse.
• Use the right tools for the job at hand: Resource design thinkers and social scientists in your projects, to work alongside your usual data wizards. Thinking of culture as an obstacle shows that we are approaching change with the wrong approaches, methods and skills (just like you wouldn’t hire a humanist to build your data pipelines, would you?). Once we start embracing cultural tools to implement change, culture won’t seem so difficult anymore.
• Stop enabling decision making only and start designing behaviors: We humans are not solitary decision-makers; we are doers who use data to rationalize decisions our intuition has already made. In our interactive dashboards we will always find and cherry pick the variable that supports the decision our intuition already made, and we will use this to justify ourselves to others.
• Design for behaviors in their social context: Where we excel as fact-based decision-makers is when our ideas are tested in social settings and when participants are willing to argue for, listen to, and adopt their thinking within a diverse group. If we aim to improve decision-making, we should design behaviors for these types of settings.
• Include measuring behavioral changes in your success criteria. Only tracking user feedback and other opinions will too often fool you into thinking your work is done—when it has only just started. Always measure behaviors.

As data leaders, it is our responsibility to appreciate this difference between decision-making and behaviors in human and cultural contexts. We must start being obsessed with behaviors because then our efforts will finally start having their full promised impact. The world does not need more reports and dashboards; the world needs smarter behaviors. Don’t accept the fluffy excuse of culture being an obstacle anymore. The real obstacle is our narrow approaches to changing behaviors, and this obstacle can be overcome by blending technical and program skills with design thinking and social science. The latter has been around for much, much longer than big data, cloud, and AI combined.

Originally published in Solita’s blog.

References

Gigerenzer, Gerd (2013). Risk Savvy: How To Make Good Decisions.

Haidt, Jonathan (2012). The Righteous Mind: Why Good People Are Divided by Politics and Religion.

Mercier, Hugo (2020). Not Born Yesterday: The Science of Who We Trust and What We Believe.

Mercier, Hugo & Sperber, Dan (2017). The Enigma of Reason.