Vendors of software and software-based online services have long employed a product focus with their offerings. They seek to discover what customers want in the products before they build them. They have product managers to oversee the process of product design, creation and market introduction. They solicit ideas from customers and customer-facing support employees about how the product can be improved after it is introduced. They measure the development cycle time, bugs, revenues, and profitability of products. Usually, heads of product oversee these activities. In short, product orientation has long been an important component of success in the software industry. As one perspective puts it, software product managers must balance the different perspectives of the technology, the business, and the user experience—all without having direct control of any of these domains.
Over the past decade or so, as software increasingly incorporated data and analytics features, the “data product” has become popular as well. These are products for which the primary purpose is doing something with data—collecting, managing, analyzing, or facilitating consumption of it. Some organizations used the data product term to refer to offerings for internal employees, but the most common usage was for data-oriented products intended for use by external customers. Of course, usually, these data products also involve software.
Now, many organizations are beginning to adopt a product focus for analytics and AI applications intended for use by their employees. They are adopting many of the activities and behaviors used with software and data products, and they are applying them to analytics and AI that may never be seen by or sold to customers. In this post I will describe the value of a product orientation for data, analytics, and AI initiatives that are intended for internal consumption.
Why Is a Product Orientation Necessary?
There is one primary reason why organizations need to adopt a product focus for analytics and AI: models simply don’t get implemented often enough. There are multiple survey results suggesting that model deployment rates are low. The most recent one, a small poll of data scientists from KDNuggets, found that the majority of respondents said that between 0 and 20% of models are deployed. An older VentureBeat article suggested (from an IBM executive’s remarks) that 87% of models aren’t deployed. Gartner predicted in 2019 that only 20% of data science projects would yield business value. Other surveys suggest that companies have had problems in getting economic value from their AI investments, in large part because they don’t have enough production deployments.
There is some evidence that this problem is beginning to be addressed in companies. However, it seems likely that there are still deployment issues in many companies. Any improvement in the problem could be a result of companies recognizing the deployment issue and establishing a product orientation to address it.
What Needs to Be Managed?
Even when analytics and AI models are deployed in companies, a product orientation can make them more successful. Analytics and AI models typically require technical implementation and integration with existing technology architectures. Before implementation, they may also require both technical tuning to reach the desired performance level, and user interface improvements to make the system that embeds the models sufficiently understandable and easy-to-use by nontechnical employees.
New models often also require substantial organizational and behavioral changes. These changes may include process redesign or improvement, job definition changes, technical training, or other types of re-skilling. These changes and the new technical capabilities may require substantial time and communications with the business stakeholders for the new capability in order to ensure their trust and support.
A product manager and product management process can coordinate and monitor all of these different types of changes. Different types of skills are necessary to successfully achieve each type of change, so specialized roles are required to participate in the process of model deployment. Only a product manager, however, can oversee the entire process and ensure that the specialists are collaborating and engaged at the right time. He or she may not possess any of the necessary analytical or technical skills in depth, but needs to understand and appreciate them all at a basic level.
Large analytics and AI projects increasingly have a level of complexity that is common in enterprise software, where a product orientation has thrived. In addition, they also add the challenges of algorithm development and analytics or machine learning engineering. Some companies that include analytics and AI in their external customer offerings have found that the product orientation is very helpful, and that traditional software or digital product managers need to add analytics and AI to the portfolio of issues they must address and coordinate.
An Example of the Benefits Product Orientation Brings
Manav Misra is the Chief Data and Analytics Officer of Regions Bank, a financial institution with $163 billion in assets and one of the nation’s largest full-service providers of consumer and commercial banking, wealth management, and mortgage products and services. Misra was originally a computer science professor, but has held several jobs in consulting and enterprise software. From that background, he learned the importance of product management disciplines: understanding what the end user needs; putting together development cycles starting with a minimum viable product; building complete solutions; and monitoring their use and effectiveness over time.
“Enterprise software companies,” he noted in an interview, “can’t hand something half-baked to an enterprise customer. In consulting I saw that most analytics teams only build a model, and they don’t generally focus on its deployment. At Regions, I didn’t want to fall into that trap.”
When he accepted the leadership of data and analytics at the bank, he began to develop an approach to successfully building and deploying analytics and AI products for the various internal business partners at the bank. He concluded that he needed to bring in new people in new roles who would be responsible for deploying a complete solution.
The bank’s “data product partners” are each aligned with one of their business or support unit leaders. They initially serve as bidirectional translators, connecting the opportunities of analytics and AI with business needs. After use cases are identified and prioritized, they ensure that complete solutions are developed and implemented successfully within their units, and that they provide substantial business value. Their job is then to focus on how the product is adopted and used, how well the user interface is working, how many people use it, and ensuring that the product delivers or exceeds the value promised in the original business case.
Misra said that there are a few important components involved in the success of such a role and the discipline of product management for analytics and AI initiatives:
- The goal must be to address a critical business priority for partners, not to develop a cool algorithm. When completed, the project should deliver substantial incremental value to the business.
- Teams should work with agile methods and include data scientists, data managers, data visualization experts, user interface designers, and platform and infrastructure developers to build robust solutions.
- Teams should employ all the software engineering disciplines, applying them to data science.
- All team members, and especially the data product partners, must always think about the end user, and must implement solutions that are engaging and likely to be adopted by them.
- All stakeholders should understand that a product requires continued care and feeding. Unlike a project, it isn’t a “once and done” effort. Resources need to be dedicated over the life of the product to ensure improved functionality and continued usability are built into future versions of the product.
- Measure everything, including baseline performance, the impact of the data product, and any results and/or revenue generation or internal savings.
- Remember that you’re also in the promotion business. It’s not simply enough to do the work of data products. The results of what we do may speak for themselves, but only to those that hear the impact of the uses of data. Misra’s organization publishes an internal quarterly newsletter that is circulated across the bank—to help build awareness and drive demand for their partnership.
- The most difficult component is ensuring that “we have the right people on the bus” in the product partner role. They have to have the right mindset and background for the job. Many product partners at Regions came from the business side. They need to be able to sit in staff meetings with their business partners and understand their priorities.
Regions has been employing the product orientation for three and a half years. It has been quite successful, with over ten revenue-generating/cost-saving products (with incremental impact in eight figures) and several more for internal support functions. Even though the new product focus challenged the mindset that deployment is not the job of data scientists, they are thrilled by the attention they get from the value they are delivering.
At a recent senior leaders’ meeting for the bank, for example, Misra’s group was invited to set up a booth featuring a variety of successful data products. Not only did it provide a forum to show the results of the group, but it also provided sparks of ideas and future use cases for the company’s senior executives. Now the culture is one in which everybody—data scientists included—has to think about how the solution gets deployed, and how it performs in terms of the model, the technology, and the deployment environment.
Misra has found that the focus on value achievement has been highly beneficial to his group’s operations. It avoids the danger of becoming a service bureau that works on projects of low value. They prioritize projects with the highest value, and try to eliminate or pass on low-value work. The value delivery focus also helps the Data and Analytics function when asking for budgets.
“We have a culture at Regions that is really very consistent with best practices in data, service or application development and delivery processes,” Misra said. “I think a large part of the success we’ve had comes from the work we do before we ever deliver a product, to really understand what our business partners want and need and to develop solutions that help them reach their goals.”
Misra’s team has a perfect batting average for deployment of prioritized products. That success comes from pre-work devoted to building collaborative and inclusive business cases; recognition of the product partner role; and most of all, the culture of deploying well-constructed and valuable analytics and AI solutions to business partners. In short, the product orientation has been a great success at Regions.