What is your definition of success? Sometimes the question feels like a trap. Sometimes it’s an opening, a cathartic release. Sometimes it signals maturity (or immaturity) or vision or lack thereof. Sometimes it clarifies. Sometimes it obscures. If we were to pose this question in a freshman composition class, we’d get a million different answers, and the difference between the As and Bs would be the restraint in the clichés. For data and analytics, this question sheds light on how far the field has come. In the era of Analytics 1.0, say, circa 2005, all but a few analytics professionals would have framed their success through the lens of fulfilling requests and supporting internal decisions. Now that we’re embarking on the era of cognitive technologies (or Analytics 4.0), CDAOs are pushing their enterprises toward advanced analytics capabilities or pushing the boundaries of what’s possible – depending on the industry and analytics maturity. In either case, it’s an opportunistic time to define your enterprise, team, and personal KPIs as a chief data or analytics officer.
At last, it appears that CEOs and boards at companies of all stripes have finally gotten around to reading Competing on Analytics: The New Science of Winning, the invaluable book by Tom Davenport and Jeanne G. Harris. First published in 2007, Competing on Analytics makes the case for using analytics to build a distinctive capability. While digital native companies such as Netflix are served up to the reader as golden-boy case studies, the book, at its core, is really a clarion call for legacy organizations to get serious about the power of analytics as a key business driver. Davenport and Harris profess that it is no longer viable to simply treat data and analytics as a back-office afterthought. Instead, modern competition means competing on analytics (as the title of book makes abundantly clear). As we explored in an earlier blog in this series, this transition requires a new breed of leader at large digital immigrant enterprises. In the old world, analytical leadership was the province of any manager or individual contributor seeking to make an impact. Data-driven, analytical thinking was everyone’s job but no one’s sole responsibility. Now, market surveys reliably report at least half of F500 organizations have a chief data and/or analytics officer. Although it’s an emerging, oftentimes ambiguous (and unrealistically scoped) role, the point here is that for companies to compete on analytics they must position the D&A function in proximity to power and sponsor D&A leaders who bring orchestration, collaboration, and a strong sense of humility to the job. In our conversations with podcast guests, striking a vision and setting a strategy that aligns with business goals and objectives proved to be the most discussed leadership quality (we call this an enterprise mindset and consider it the most important quality for data and analytics leadership). Associated behavior traits include strong business acumen and savviness, a talent for scalable and “practical innovations,” and the ability to influence key stakeholders. These leadership traits and qualities will clearly influence what the D&A function prioritizes and why – and how the team measures success.
In our final blog exploring insights from seasons one and two of the Leading Analytics Podcast, we will unpack key themes in our conversations related to success metrics. First, we asked our guests to define “success” as an analytics leader and then lay out five key metrics they use to measure the success of their data and analytics organization. Below is a summary of what we will call ultimate measures of progress and key metrics that analytics leaders use to define value. Of the variety of answers, four broad categories emerged: Metrics demonstrating success for the business, customer, analytic operations, and analytics team.
Data and Analytics Success: The Ultimate Measure(s) of Progress
For high-performing D&A leaders, the first-order definition of success is whether or not their teams are solving problems that have a material impact on the business. As Gina Papush, Global Chief Data and Analytics Officer at Cigna put it: “If there’s not a number next to it, we should question why we are doing it.” For Papush, you start and end with business needs and impact and you must find a way to quantify your team’s initiatives. This is a pragmatic, value-oriented approach that ties everything to measurable outcomes and ROI. An analytics leader must ask (and to steal a phrase from Cameron Davies at Yum! Brands): What kind of value are we driving to the organization in a “finance bought-in” way? How is my team translating data into profit? How is our work helping the business meet its goals? How well are we doing at improving decisions? Are we making a significant, positive impact on the performance of the enterprise? A customer-centric approach is critical to the definition of D&A success. Equally important is the ability to show sustainability and analytics adoption. But success for CDAOs should not be solely tied up in ROI. Engagement at the level of the analytics team is paramount to accomplish “business-first” goals.
Key Metric 1: Delivering Business Value
“Business outcomes. Period.” That was John Pyhtila’s answer when asked his definition of success. Pyhtila’s punchy response would resonate with all of the data and analytics leaders we’ve interviewed, as the enterprise of delivering business value is front-and-center for their teams. A 2021 survey from Wakefield Research validates this theme: 71% of data executives say their leadership expects revenue growth from their investment in data science. To achieve this goal, CDOs and CAOs should first align their metrics with key business partners. The efforts of the D&A team must be in sync with enterprise ambitions. Albert Marinez, Chief Analytics Officer at Intermountain Healthcare, takes a systematic approach to this alignment and calculates the percentage of work that is aligned directly to a strategic pillar of the organization. From here, it’s critical that analytics teams have a clear ROI goal and create tangible results. That said, tracking analytics initiatives purely in dollar terms can invite its challenges. Focusing on just the number obscures the full story and breadth of impact successful analytics projects can have on the business. For example, when at Dow, Erika McBride began using senior leader testimonials to tell a more holistic story about the impact of her team’s work. For McBride and her team, this evolution in storytelling didn’t deemphasize their focus on translating data into profit, but rather amplified the overall value of data and analytics at Dow.
Which brings us to the importance of asking the right questions when unpacking the value of your data and analytics projects. Globally, where is the value being created? Is it top-line growth revenue? Are you reducing costs? Improving efficiencies? Going into new markets? What capability is the analytics initiative adding to the business? How many groups are impacted by the analytics? What is the number of decisions and actions taken based on the analytics? Where is the biggest impact (i.e. not just measuring but optimizing)? To take another example, Tarun Kataria, Vice President of Demand Analytics at Mars, filters analytics impact through the lens of speed and agility. To demonstrate value, Kataria looks at the time it takes to answer a question or solve a problem, and the dollar value it took the team to solve for it (distilled in the equations Q/T and Q/$). In this way, Mars “relentlessly and ruthlessly” brings down the cost to answer questions, solve problems, and inject speed into what they do.
Key Metric 2: Solving for Customer Problems
As Tom Davenport has vividly outlined the benefits of a product orientation for enterprise analytics and AI here and here, high-performing CDAOs obsessively solve for customer problems. Davenport’s blogs focus on using a product thinking approach for internal analytics (the stuff consumers may never see), but the same principles apply wherever you’re applying product thinking principles. Assigning a value and responding to internal/external stakeholder satisfaction is a key consideration for the leaders we’ve interviewed. What is the customer experience of the analytics output or product? What is our customer engagement as measured through net promoter scores? Are we a trusted source of advice for customers? Are we producing the same or better outcomes at a lower cost to the customer? What is the satisfaction rate of teams served? How well did we do? How can we improve? What number of analytics ideas are coming from end-users? As mentioned above, aligning goals and objectives across the business is crucial for this “customer-centricity” metric. John Kahan, former CDAO at Microsoft, infused the idea of customer success into his definition of analytics success. It was Kahan’s mission for all of Microsoft’s leaders, not just those in analytics, to use data and analytics to drive customer success.
Key Metric 3: Analytic Operations
Another key metric for high-performing analytics leaders revolved around the explainability and sustainability of their analytics products. Being able to show high adoption rates and reusability are key indicators of success. What percentage of recommendations are being used? How quickly? Is the team getting to its backlog? Do we have business partners who see our roadmaps and are excited about what we do? Demonstrating replication of measurable outcomes builds breadth and depth. To broaden the scope of “operations,” how many people across the enterprise are engaging in training? Change management is almost always the biggest hurdle for analytics initiatives, so it is important to track and share success in behavior change. And at the core of any analytics project should be thoughtful consideration of its ethical underpinnings. As Bill Franks points out, the most important step toward ethical analytics is intentionality. Weaving ethics into analytics and AI is critical to the fabric of responsible operations.
Key Metric 4: Analytics Talent and Team Engagement
In Q1 of 2022, IIA’s theme in the Analytics Leadership Consortium was “talent defense” strategies. Cohorts explored strategies and tactics to hold onto highly skilled data scientists in a talent-starved market. Any leader worth her salt will be keen on attracting and retaining the best talent in the world, but it’s another thing to do it well – especially if you’re competing against the Googles of the world. And yet, the focus on team engagement among our guests revealed just how impactful you are as a leader if you possess a people mindset. The CDOs and CAOs we’ve interviewed place an emphasis on fostering a team and analytics culture that values recognition, connection, and engagement. Top CDAOs build out strong, curious teams that challenge one another and strive to get better for themselves and the firm. What practices and habits have you instilled into your teams to stoke curiosity, to challenge them, to create connections (in remote or hybrid environments no less)? On a broader scale, how connected is your team to the business? After all, being viewed (and used) as a trusted business partner is a key indicator of engagement.
Conclusion:
It is no surprise the most revealing success metrics are intimately tied to the six core leadership mindsets we explored a few blogs back in this series. Elements of all of the mindsets – from enterprise to change agent to people – are sprinkled throughout. A new breed of leader calls for new measurements. What is your definition of success?
To listen to the complete library of IIA’s Leading Analytics Podcast, check out our website or download on your favorite podcast platform.