Navigating AI Governance: Breakthrough Conversation with Lindsay Mico

In this blog series, Jason Larson, head of content at IIA, interviews IIA Experts who help clients navigate top data and analytics challenges within their enterprises. With over 150 active practitioners and unbiased industry experts, IIA’s expert community provides tailored support, plan validation, and ongoing guidance to drive key analytics outcomes.

In this Breakthrough Conversation, Lindsay Mico, IIA Expert and head of data science at Providence, shares his insights on navigating the complexities of building robust, adaptable AI frameworks within organizations. Drawing from real-world experience in the healthcare sector, Mico discusses the critical elements of AI governance, including the role of data science, the importance of cross-functional collaboration, and how to ensure AI solutions meet both business needs and regulatory standards. The conversation touches on the challenges of aligning AI models with evolving regulations, offering a practical, first-principles approach to governance. Throughout the discussion, Mico emphasizes the importance of performance validation, risk assessment, and ensuring AI systems work ethically and effectively in real-world applications. This interview has been edited and condensed, and while his responses draw on day-to-day experience, they represent his personal views and not those of his employer.

It might be useful to start with a 30,000-foot view. How do you define AI governance? What’s your overall stance on this?

Lindsay Mico: I’ll preface this by saying that, while this draws on my day-to-day work life, it’s my personal professional opinion, not a policy statement from my current employer.

I think of governance in two main buckets. One is determining what’s the right thing to do. How do we choose the right use case? How do we prioritize it? How do we resource it? The second is once we’ve decided to move forward, how do we execute it in a way that’s resource-efficient, solves the problem, and does so safely, securely, and equitably? Essentially, how do we ensure the solution works as it should, for the good of society?

I’m mission-driven, and I work in a mission-driven industry, so this is particularly important for me. From a regulatory standpoint, it’s key too. We’ve seen regulations, though sometimes erratic, that focus on minimizing bias in AI solutions and reducing harm—particularly in healthcare, though you can apply this to other industries as well.

The first bucket is something we’ve got a good handle on in the industry. It’s how good IT shops decide which new technologies to deploy. AI isn’t really different from any other technology; it’s just code with math inside it. So, if you have a solid governance process for deciding when to deploy new technology, you should have one for AI as well. If you don’t, that’s a good “canary in the coal mine” for cleaning up your processes. It could be anything from dashboards to curated data sets, cloud platforms, or any vendor solution. But the point is, IT already knows how to make these decisions—it’s just about applying that process to AI.

The other category, or bucket, is where I think the real work lies today. I see this as a particular challenge, especially as I talk to my peers, read about industry trends, and see what vendors are doing. I’m noticing a real lack of analysis and validation of AI solutions. There’s this idea that, with AI, let’s say GPT for example, we simply plug it into a software application, and it generates a result. Data goes in, something comes out, and it runs as expected. And often, that’s accepted as sufficient.

As a data scientist, that kind of thinking stresses me out, but as an engineer, I get it. When you’re testing a data engineering pipeline, the first question is, “Does it run?” Does it compile, execute, or throw an error? APIs have become so mature, and so much of the complexity is hidden beneath the surface that we get results that seem reasonable. This is both a strength and a weakness. One issue, for example, is the "hallucination" problem. As we all know by now, AI can generate results that appear rational but are actually off base.

What I’m getting at is this: testing and validating AI or machine learning solutions is the bread and butter of data science. That’s what being a data scientist is all about. It’s not just about fitting a model; it’s about understanding the messiness of data and the inherent unpredictability of output. You develop this skill through experience and training. How do you test models with out-of-sample data? That’s where the real work starts.

So, when you release an AI solution into the wild, you want to have confidence that it will work according to some statistically sound benchmark or criteria. The problem is, the engineers using today’s AI solutions often don’t have the necessary skills to validate them properly. It’s not that they’re ill-intentioned. It’s just that these are real, specialized skills. That’s why data science is an actual discipline.

Today, we see a lot of engineers building AI, and that’s great. We want to expand AI’s reach and impact across industries. We’re not going to do that by saying you need a data science degree—that’s a narrow perspective. What we do need, though, as leaders, is to build thoughtful governance structures on top of engineering and development pipelines. These structures can help educate teams so they can develop their own skills and have a better sense of how to test AI-driven solutions. Engineers already know how to run unit tests, but they need to grow their understanding of how to test AI solutions in particular.

Additionally, there needs to be a “backstop” in place—a group of experts who can review the work and say, “You think you’ve tested this enough, but we don’t think you have. Here’s why.” It’s not about being confrontational. It’s more consultative—acknowledging that the application seems promising, but there might be things we haven’t considered yet. For instance, we might not be convinced that it will behave equitably across different demographics, like between men and women. You might skip this test because it requires extra data or statistical analysis, but it’s an important check to make. While parity isn’t a perfect metric for equity, it’s definitely a good place to start.

Before we move on, I just want to mention a third aspect, a third bucket: good engineering practices, particularly in cybersecurity, data protection, and privacy. Groups are generally pretty good at this—or should be—but it’s still something to be mindful of.

So, have you memorialized an AI governance framework for your team? And if so, how would you suggest a fellow D&A leader get started with building or refreshing their AI governance, especially through the lens of this messier bucket you mentioned?

Lindsay Mico: I’ve been part of developing a governance framework. What I can say is that governance really takes a village. It requires a diversity of skill sets and backgrounds within an organization.

It involves teams like legal, ethics, compliance, privacy, human resources, and potentially even finance. Many, though not all, AI applications require input from these various groups.

A key point, though, is that subject matter experts—say, people in legal who work with regulations every day—will need data to make informed, risk-based decisions. That’s where the model risk management or data science teams come in. These teams generate the mathematical estimates of performance, particularly when it comes to out-of-sample performance. How will the AI solution perform with millions of daily examples in the system, compared to the few hundred or thousand that we tested during the engineering process?

There’s an order of operations to this. First, you need to get solid math behind the AI solution. Only then can you present it to teams like legal or compliance. They’ll review it and determine whether the risk is tolerable given the reward. Rarely in business do we encounter zero-risk situations, so you’ll have to make decisions based on a risk-versus-reward assessment.

The key takeaway here is that the starting point for building a governance process is a conversation—and building relationships—with people across the business. If you’re in a large, especially legacy, organization, you’ll need to speak with a lot of different teams. That’s where I recommend leaders begin: someone who works with AI regularly should proactively engage with various groups around the company to identify stakeholders who can support the governance process.

This will involve many different pillars, especially the shared services that big corporations typically have. These groups will need to weigh in as AI applications are deployed. The model you’re aiming for is a blend of management—making decisions based on data science and mathematical assessments. For example, you might assess that an AI solution is 94% accurate and then weigh the impact of being wrong.

Have you had experiences where there hasn’t been total buy-in on the need for analytics? Have you encountered situations where you needed to gain support for what you’re doing before you could even talk about governance?

Lindsay Mico: I don’t think any technology solution, including AI, can succeed without leadership buy-in. In fact, it’s pretty hard to make it work without buy-in from the employees who are going to use the solution. In some industries, you might be able to force people to adopt a solution, but that’s definitely not true in healthcare. You have to bring people along, and the solution needs to solve a problem they care about.

The idea of a centralized group forcing AI adoption for a problem that business leaders don’t agree is a problem, and that frontline employees don’t see as a problem, is at best an inefficient use of resources.

That’s the main point: I truly believe the business line should identify the problem. There is a role for focused teams that analyze the business, look for inefficiencies, and figure out where the sweet spot is—making life better for workers while improving the bottom line. The dream is better work experiences, better financial outcomes, and better results for the customer, right? There’s definitely a place for business analysis, but I wouldn’t recommend simply dictating AI usage.

How would you summarize the critical elements of a governance framework, especially one with a solid foundation?

Lindsay Mico: You need a strong intake pipeline for identifying the problems, challenges, or opportunities that you want to address. It's important to ensure you're solving the right problems. This principle should apply to any technology, not just AI. We only have so many resources—both human and financial—so you need to think rationally about how you're using them. Don’t solve a problem just because a vendor presents a cool solution. Instead, take a step back and think through it from first principles. What problems does your business really need to solve? This is the first bucket, and it’s something that good IT shops should already be doing.

Once you’ve addressed that, the next step is to make sure you have a robust program structure in place to thoughtfully validate the quality of the solution. Critically, this isn’t about analyzing the quality of the model itself—whether it’s GPT, Llama, or any other tool. It’s about evaluating the output of the model. For example, if you're using Llama or GPT to write customer-facing communications, you need to analyze the quality of that communication.

You’re not just assessing whether the model works, but how the output will fit into the broader workflow. If a human is going to review and edit the communication, then you need to evaluate the quality after that editing process, because that’s what customers will see, and that’s what will impact real-world outcomes. You can apply this process across various use cases in AI.

You need the analysis done with the appropriate rigor, using data science methods at corporate scale—think about both in-sample and out-of-sample testing, the diversity of populations, and different demographics. This way, when you release the solution into the wild, you have a rational understanding of where it will work and where risks might lie.

Next, you want to involve the groups within your organization that can make risk-based decisions. Whether it’s business leaders, legal, compliance, or human resources, these teams need to assess the solution. For example, you might say, "This will work 98% of the time, but here’s the 2% where we want you to weigh in." This is similar to what happens in contract negotiations—there are compromises, and someone needs to make a risk-based determination. A contract specialist or lawyer helps the business make that decision.

Going back to having a robust team, what are the gaps you most often see in building out that team, and what creative solutions have you brought to the table to help fill those gaps?

Lindsay Mico: I’ll admit, I’m biased. I’m a data scientist, so I’m advocating for more data science here. But I’ve noticed, both in public and private conversations with peers, that many organizations aren’t hiring large numbers of data scientists to build and manage these applications. Instead, they’re letting engineering teams handle the development.

And again, that’s fine. I’m not trying to gatekeep the process. But in order to properly govern AI, you absolutely need scientists—people who understand experimental design. That’s really the core of what we’re doing. Analyzing whether something works is essentially an experiment. You’re collecting data, stratifying it, sampling it, and using statistical methods to estimate performance, with uncertainty. That’s just experimental design.

I’m originally a research scientist. Not every data scientist comes from that background; many are engineers who’ve learned the math along the way. But I truly believe that having a scientific background is essential for properly analyzing data. It’s not much different from conducting clinical trials. We’re not reinventing the wheel here. It’s the same process we use when testing if a vaccine or a new drug works. We just need to bring that approach to AI and generate numbers that allow for informed decisions about risk and reward.

To close out the conversation, let’s touch on adaptability, particularly when it comes to regulations and compliance. From your perspective, how do you build out and implement a governance framework in a way that’s agile and adaptable to an ever-changing regulatory landscape?

Lindsay Mico: For a bit of context, I’m a contributor to CHAI, the Coalition for Health AI. CHAI has been navigating this issue while building an AI model card—a standardized documentation format for AI applications in healthcare. The goal was to align with best practices and federal regulations. As we launched into January, we had draft guidance from various federal agencies. That guidance may or may not be adopted, and it’s uncertain. So, we’ve had to, in very practical terms, navigate this process.

For me, a good way to think about this is that as AI professionals and data scientists, we already know how to document performance. We don’t need new processes for that. Just as IT professionals know how to document installation and data flows, we can approach the problem from first principles. We can do what should be done, and then, when regulations emerge, it’s mostly about additional refinement or reformatting.

But if you’ve already done the work—documenting performance, stratifying data by factors like education, demographics, gender, age, race, and income—you’re already most of the way there. You might not have everything, but you’ll have done the scientifically rational and appropriate work.