IIA’s Research and Advisory Network (RAN) clients leverage battle-tested frameworks and an exclusive network of over 150 active practitioners and unbiased experts to plan, prioritize, and execute strategic enterprise data and analytics initiatives. We regularly check the pulse of trending topics for the RAN community and facilitate critical conversations in virtual roundtable format for peer-to-peer exchange.

In a recent roundtable discussion with IIA RAN clients, data and analytics leaders from diverse industries gathered to share their challenges and successes in the field of customer analytics, particularly strategies around personalization and customization. The discussion covered a wide range of topics, from ROI potential to how the human role is evolving.

This discussion was moderated by JP Snow, IIA Expert and Principal and Founder of Customer Catalytics. Here is Snow’s recap:

1. Personalization offers huge and growing ROI potential

Participants described using personalization for targeting, messaging, cross-selling, self-service, field support, problem diagnosis (e.g., medical diagnosis) and journey navigation. The ROI potential is strongest when personalization can be scaled through analytics and automation. Personalization projects tend to gain support where the impact is direct and measurable. For this reason, the field of marketing tends to be at the forefront of applying personalization. Innovations in AI language capabilities are enabling better chatbot and customer service capabilities. Because generative AI and LLMs produce relatively unstructured outputs, they are closing gaps that models with discrete outputs couldn't address.

2. We're just beginning to leverage LLMs and generative AI for personalization

Tenured industry professionals remind that AI as an umbrella term can include logistic regression and other machine learning techniques that have been used for analytics and personalization for decades. Participants in this roundtable focused on opportunities within the rapidly advancing LLM space. Use cases tended to involve enabling personalization where it wasn't previously possible. Chatbots are a natural fit because they involve comparatively unstructured inputs and outputs. Companies are also using language analytics to explore, categorize and quality check large volumes of customer verbiage gathered through service interactions, complaints and surveys. The analytics field is seeking ways to leverage generative AI more for established personalization applications, but doing so will require more progress with data "feature engineering" and in the business and regulatory structures.

3. Data capture and structuring is a key challenge for leveraging AI for personalization

If having the data is ubiquitous across analytics initiatives, the relative newness of LLMs implies an even bigger challenge for any AI applications that use them. Getting the right data in the right structure was a common problem across participants. Moreover, because LLMs work with language, personalization efforts are limited until the models can understand the terms and language patterns unique to a given industry. Participants remarked that off-the-shelf language packages typically need to be augmented with industry-specific categorization or sentiment mapping. Several participants had experience with formal taxonomy projects to create the right data structure for training or deploying personalization models. Data privacy was also a common concern. Participants mentioned several strategies for anonymization, field masking and clean room environment.

4. The human touch matters more than ever, and the human role is changing

Based on this roundtable discussion, most personalization still involves a "human in the loop" at some point in the process. Companies aren't willing to turn personalized guidance completely over to an AI model, especially where health, safety or finances are involved. Among cases mentioned in this session, the most common pattern involved a model generating solutions that are then validated, filtered or translated by a person at the point of delivery. This group also described cases where AI takes a secondary role, serving to monitor or cross-check large volumes of human decisions. Regulated industries face additional requirements for human validation, sometimes requiring multiple levels of sign-off. Though necessary, such requirements for human approval can create bottlenecks that limit the ROI potential for AI-generated decisions. It's also challenging to get business and compliance leaders comfortable with the complex and changing field of AI. Some industries are impeded in their use of generative AI because industry policies require a company to "show the math" upon which an individual recommendation is based. While most personalization use cases leveraging AI involved a human role at some point, there are cases where full automation prevents any human involvement. Embedding personalized decisioning into a device is one such example of full automation. In these cases, the models must be rigorously tested and stable enough to work without in-the-moment human support. The human role in model development and product design is crucial for ensuring the right human experience for the end user.

IIA virtual roundtables are exclusive, invite-only discussions designed to promote peer-to-peer exchange on pressing challenges in the data and analytics community. Seats are limited and reserved for C-suite data and analytics leaders or equivalent at mid- to large-sized enterprises. Conversations are geared toward non-digital native companies. If you meet these criteria, contact us for more information.