Expert Exchange: Focus on the Actual Value of Predictive Models

Client Inquiry:

We've been challenged to identify a monetary value for our predictive models. Do we stick to talking about their potential value or is there another way to approach leaderships concerns about ROI?

Expert Takeaways: 

1. Realized Value

Identifying the value of predictive models is a fair request; otherwise, an analytics organization becomes a cost center.

• The good news is that it's easier to measure the value of predictive models than almost anything else because you can measure the delta between having a particular model and not having the model.
• Use the potential value to get business buy-in, but move beyond that to the realized value. Give your team credit for what the business actually achieves rather than what it could potentially achieve.
• It's frustrating when the business doesn't make the changes necessary to achieve realized value, especially when a manager mistakenly thinks that the model itself creates the value. It's what they do with the model that counts.

A model only produces value if data flows through it. You provide the model, but the business must execute the models. De-prioritize projects that you suspect will sit on the shelf. Your job in analytics is always to build models that will improve business outcomes.

2. Change Management

Educate the business about where analytics value comes from, so they'll start making decisions to realize actual value. They'll need to employ change management practices.

• Suppose you build an algorithm that's worth six million dollars for a call center. If the leader doesn't know they have to write new scripts, create new call routes, and so on, to realize value, then the six million dollars of potential value doesn't matter.
• Get buy-in from the business that it will take action with the help of change management. If it does, you can claim the realized value of the model.
• Start small and only with leaders who are ready to act.
• Use top-down leverage as necessary. Share potential-value findings with executives.
• Have frank conversations and change your intake process so that it includes questions for the business: How will you build the solution into your business process? How will you hold people accountable? Do you have somebody who can lead the change management effort?
• Look for early indications from business partners that they will follow through. If they don't have someone in place to take responsibility for execution, nothing will happen.
• You may need to add a few change management experts to your team.

3. Getting Started

Start by tracing value where it's easy to track and it makes sense.

• Factor in opportunity cost too. To calculate value, some models may require dedicated time in the weeds; the opportunity cost may be too high. You can ask the leader, “Do you really want me working on the valuation of this model rather than developing that new model?”
• Start the education process with the analytics organization first. Get your team members comfortable with the idea that the models they build won't count if they don't produce value for the business.
• Focusing on realized value may put extra pressure on your team to become change agents. Highlight that they're not losing their jobs. You're engaged in a transformation to help the business put models into action.
• Escalate the challenge so that leaders know that you're producing good work, but you don't have the partnership on the business side to realize the value.
• Let business leaders know what you're doing and why a transformation benefits them. Ask them how you can work with their teams. Leaders may be surprised to learn that their teams aren't actually incorporating the models into their processes.