We have discussed many aspects of the analytics and AI journey in healthcare: the rapidly changing business landscape and the need to accelerate our data innovation journey, the operating model and entrepreneurial mindset that provides the framework for a scalable, agile ecosystem, the skills needed to lead the expedition, the process for getting started, the dashboard framework used to streamline and unify decision making (i.e., Pulse), the potential of self-service analytics to democratize data innovation, the step change that comes from intelligence automation and AI, the importance of data quality and trust to fuel the journey, the anatomy of the platform that powers ecosystem, and voyaging to the cloud. See links in sidebar to explore this content starting with Part 1.

In this piece, we will explore how a pediatric healthcare system is leveraging these practices, frameworks and tools to accelerate their data innovation journey, and creating a data-driven culture along the way. Let’s get started.

Background

Children's Minnesota is one of the largest freestanding pediatric health systems in the United States. Founded in 1924, their mission is to champion the health needs of children and their families. They are committed to improving children’s health by providing the highest-quality, family-centered care, advanced through research and education.

This independent, not-for-profit health system provides a comprehensive range of services, from routine check-ups to specialized care for the most complex conditions (e.g., neonatology, cardiology, cancer and blood disorders, and neuroscience). They are “the kid experts.”

This system has two hospitals with 462 staffed beds located in Minneapolis and St. Paul, nine primary care clinics, seven rehabilitation centers, nine specialty care sites and one ambulatory surgical center. In 2023, the system served 162,000 patients. This translates to 85,000 ED visits, 389,000 clinic visits and 24,000 surgical procedures.

Children’s Minnesota is the only Level I Trauma Center in Minnesota that provides exclusive care to children from birth to young adulthood. The system is also the first and only hospital in Minnesota, and the thirteenth in the nation, to be verified as a Level I Surgery Center by the American College of Surgeons.

Children’s Minnesota is part of a clinical integrated network, the Children’s Health Network. They also have a growing academic research center that is advancing translational research.

The Journey

The Children’s Minnesota analytics journey started like most. Recognizing the need, they implemented a traditional data warehouse environment. Over time this effort could not keep up with demands—business, clinical, and research. In response, service areas began to build department solutions, leveraging the data warehouse where possible and sourcing other data directly from operational systems (e.g., clinical, financial, human resource and supply chain, etc.). They also began to leverage external reporting. In time, this resulted in a patchwork of solutions that were increasingly redundant, manual, fragile and prone to error – becoming prohibitively expensive to maintain and scale.

More importantly, these solutions failed to meet the business's needs and made very little progress toward building a data-driven culture. This is an all-too-common story within and outside healthcare—albeit a crucial step on the journey to becoming a data-driven organization. All that said, something needed to change.

When Children’s Minnesota decided to refresh its business strategy, executive leadership included analytics as one of three digital transformation initiatives. The analytics initiative provided seed funding to improve the core technology. Equally importantly, it provided funding for an experienced analytics director, a core group of data analysts and an interim CDAO. The CDAO functioned as part of the senior leadership team.

The analytics director (solid line) and data platform manager (dotted line) reported to the CDAO. The CDAO reported to the CIO (chief information officer) and head of strategy. Over time, this expanded to include the CFO (chief financial officer). There was also a very close partnership with the chief value officer (CVO) because of the role as an executive sponsor. We will discuss that later when we review the operating model.

To frame this leg in the analytics journey, the team decided to assess the analytics and AI ecosystem, leveraging an established industry methodology. This proved effective in aligning the organization on the current state of the ecosystem, and establishing and communicating near-term plans for improving maturity and measuring progress.

The assessment rated the ecosystem maturity at the lowest level possible. Over the following two years, the level of maturity significantly improved, trending to mid-level maturity by the end of 2024 and toward the highest level by 2027. More importantly, the organization demonstrated significant growth in its ability to leverage analytics to streamline operations, reduce cost, improve outcomes and enrich patient and workforce experience—on the path to becoming a data-driven organization. Let’s discuss how this success was achieved.

How Success Was Achieved

To accelerate its journey of leveraging analytics and AI, and becoming a data-driven organization, Children’s Minnesota needed to move beyond traditional practices to deliver solutions to an agile operating model with an entrepreneurial mindset. To give identity to this movement and unify the analytics community, this initiative was branded Enterprise Analytics (EA).

A first step toward success was to align with strategic and operational priorities informed by the strategic plan and guidance from senior leadership. The next step was to build credibility with leaders by demonstrating value quickly through incremental releases of capabilities called data products, and reducing sustainment costs through deprecation of unused reports and automation—this also had the added benefit of improved quality and trust with reporting.

EA leverages a federated operating model where analytics resources are organized into small teams focused on the business's strategic and operational priorities. These multidisciplinary teams are called delivery teams, as a reminder of why they exist—to deliver value. They operate like startups and are deeply integrated into the business they support.

A key component of success was delivery speed, which was achieved by avoiding the many traps that can consume large amounts of time without contributing to immediate success. Examples of these time traps include avoiding organizational realignments and data-sourcing disputes. For instance, delivery team members were assembled without moving individuals around on an organizational chart, since battles about reporting lines can be lengthy and yield little value. Instead, the focus was on how to organize most effectively to get the work done.

Time-consuming efforts like inventorying all of the data in the organization were not tackled as discrete projects; rather, they were worked just-in-time in small pieces only as they directly intersected with the success of a delivery team in demonstrating value to the business. Disagreements over data sources were largely avoided by adhering to the basic tenet: “There will never be perfect data or a perfect model; a good enough model is often helpful in getting the ball rolling—performance over perfection.”

Embedding delivery teams into the areas they support (e.g. quality improvement, patient access, patient flow, etc.) helped achieve success, and yielded deliverables incrementally and often that solved difficult, high-value problems that aligned with strategic and operational priorities. In partnership with the delivery team leaders, service line leadership (a.k.a. delivery team sponsors) shared the responsibility for the team's success—setting priorities and budgets, and guiding overall governance of their analytics endeavors.

The delivery team is chartered, funded and governed by the business area in which they are aligned. These teams are motivated to create value for their business and integrate into business operations. As long as they continue to deliver increased value, the business continues to fund them. If they do not increase value, or greater value can be delivered elsewhere, the team is shut down and resources are reallocated. There are currently eight delivery teams, as presented in Figure 1.

Delivery Teams are co-led by a product and dev owner with support from the delivery sponsors and an advisory team. Delivery teams use agile delivery methods (e.g., scrum, extreme programming, etc.) that involve continuous 2-week data product releases (sprints) and regular backlog and planning sessions to guide priorities.

The team comprises three to six members, with a mix of resources depending on the team charter and focus. The team generally includes data analysts, data engineers, and data architects (fractional), and as the community matures, it can include data scientists and software engineers. The team doesn’t have project managers because it is small and can self-manage, allowing more capacity to build data products.

Another aspect of the team’s dynamics is no hard lines between roles. On the contrary, team members are encouraged to explore other roles and wear multiple hats, and sometimes, it is necessary to deliver on commitments to the business.

Most delivery teams are aligned vertically within the area of the business they support. There are two exceptions—the platform services and community teams. These two teams work horizontally to increase efficiency and alignment with the vertical delivery teams.

The platform services delivery team is responsible for the health and evolution of the data platform. This is particularly challenging because data engineering sits within each vertical, and very independent, delivery teams. The platform services team works closely across these vertical teams to ensure that they are building a platform that looks like it was built by a single team and not a Frankenstein. This is the most critical role of platform services, and over time, it is primarily facilitated through self-service tools, standard practices and automation, working in partnership across delivery teams.

Platform services is also responsible for shared tools used across vertical delivery teams to innovate with data. This team also provides development and operations (DataOps) support, including release management, incident management, data management, daily data refresh, tool upgrade and maintenance, etc.

The community services delivery team is a smaller team with a big responsibility. They form and nurture the fabric that binds the analytics community. This involves stewardship and evolution of the operating model, formation of the EA strategy, quarterly planning workshops, training and education to develop data fluency, promoting community events (e.g., all hands, demo days, town halls, etc.), organizing executive sponsor huddles, etc.

To promote alignment and community across delivery teams, there is a senior leadership team (S-Team) comprised of leadership from each team. Leadership is often represented by the dev owner rather than the product owner. The only requirement is that one of the leaders participate. The CDAO is also part of the S-Team.

The S-Team meets quarterly to review the previous quarter's success and align on plans for the upcoming quarter. Rather than focusing on every effort, the S-Team focuses on the big rocks. The objectives and key results (OKRs) framework is used to define these rocks.

The big rocks comprise 60-70% of the workload and represent the things that the S-Team has committed to complete by the end of the quarter. Though most big rocks align with a single delivery team, the S-Team is responsible for ensuring all the big rocks get done. With the remaining capacity, delivery teams focus on pebbles and sand at their discretion—often this is where innovation happens.

The S-Team meets weekly to track the progress of big rocks and handle escalations from delivery teams, which are rarely escalated further. The meeting also provides a forum for leadership to collaborate on special topics related to the community's evolution, such as tools, practices, and governance.

An executive sponsor team (E-Team), comprised of the CIO, CFO, CVO, CMO, and VP of strategy, provides support and guidance for the S-Team. The E-Team meets with the S-Team monthly to review accomplishments, plans and challenges. Generally, the E-Team is not the decision-making body—that is the responsibility of S-Team. The E-Team has the critically important role of providing context that helps the S-Team make informed decisions. There are only a couple of exceptions. The E-Team will step in if they believe the S-Team is going off a cliff (this has yet to happen), and they approve all funding, and the formation and sunset of delivery teams.

Community Evolution

The journey began with the formation of the E-Team and the development of an action plan. This plan aligned stakeholders on the current state of the environment, vision going forward, and game plan for getting there. The goal was to plan just enough to get started and evolve the plan—central to the community's ethos, learn by doing.

Rather than taking a revolutionary approach to implementing the action plan, the E-Team and CDO took an evolutionary approach by implementing two delivery teams—platform services and quality improvement.

The platform services delivery team was responsible for the data and analytics platform all other delivery teams leverage. This team had to learn to work differently. As all steps to create a data product moved to the vertical delivery team, the role of platform services shifted to teaching these teams how to fish rather than doing the fishing.

The first vertical delivery team was the quality improvement delivery team. This team was selected for several reasons—strong alignment with strategic and operational priorities, strong support from clinical senior leadership, and willing and invested data consumers. Although several other business and clinical areas were considered, this area was best prepared to hit the ground running, and offered an excellent opportunity to demonstrate material and visible impact.

The focus for the next five months was launching these two teams, learning the new operating model, getting into a rhythm of demonstrating value, and growing data fluency. About five months later, the patient flow and patient access delivery teams were launched, followed by the clinical value, clinical research and clinical ops delivery teams. With each new delivery team, analytics competency, data content, data governance, and data platform evolved and matured. The figure below illustrates this evolution.

Success Stories

Delivery Team Productivity

The delivery teams' goal is to quickly establish a rhythm in which they demonstrate value incrementally and often. For the first two delivery teams, this rhythm took about four months to achieve and about twelve months for velocity to peak and stabilize—roughly doubling year over year. Subsequent delivery teams ramped up quicker because they benefited from previous delivery teams' experience, tools and practices.

In addition to improving productivity over the legacy approach, which was centralized and transactional, the new approach did significantly more to promote self-service analytics, growth in data fluency and report automation. The less expected and valuable benefit was having a better understanding of pent-up demand. Because each delivery team is embedded in the business they support and manage a backlog of all requests, they were better able to articulate the need and justify future investments.

Self-Service Analytics

One of the early successes was the implementation of self-service analytics for clinical data consumers to support continuous improvement across the six domains of healthcare quality—initially focused on the strategic initiative to optimize perioperative services.

The product selected was AdaptX. To minimize upfront investment and risk, the quality improvement delivery team implemented the solution in waves—with a demonstration of ROI before moving on to the next wave. The initial launch took about two months, with the full rollout to perioperative services over six months.

Several successes were realized, including improvements in start times and turnaround times in the OR and patient flow in the PACU. Another benefit was that data analysis that previously took weeks to complete with support from data engineers and analysts now took minutes. This significantly reduced the cost of supporting this analysis. More importantly, this empowered clinicians with autonomy that promoted innovation with data and employee enrichment.

Dashboard Framework

With the implementation of the patient access delivery team, the business had the people and tools needed to acquire its data, build data marts, and produce dashboards and reports. This significantly accelerated the pace of development and improved the quality of the data and reports.

Part way through this journey, the team decided to pilot a dashboard framework that could be used by patient access leadership to guide, streamline, and unify decision-making, leveraging the Pulse Framework. Power BI was leveraged as part of this framework because of its ability to share objects across dashboards.

Patient access had envisioned this for awhile, but delivery had been prolonged, given the transactional relationship with IT data analysts and engineering resources. Now that these resources were part of “their” delivery team with a shared charter, there was an urgency and ability to collaborate and innovate that had not existed before. In a month, they accomplished what they had been unable to complete in the prior year. Equally important, with each data product release, data quality and taxonomy, report automation and data fluency improved.

They have since successfully implemented the Pulse Framework, which has contributed to the success of the patient access initiative. This framework is now being formalized and leveraged by other delivery teams across the analytics community.

Data Marts

To accelerate delivery and promote a single source of truth, the delivery team leadership (i.e., S-Team) decided to implement data marts shared across delivery teams. To ensure the quality of these data marts, each was assigned a delivery owner responsible for their health. This didn’t mean the owner needed to make all the changes. However, these owners needed to review changes before they were deployed to production. This has proven very effective in improving data quality and trust.

To date, there are more than a dozen data marts. These include encounters, orders, appointments, referrals, experience, patient, provider, clinic taxonomy, and medication administration. Star schema design is leveraged, and many of these early data marts are transactional facts that can be summarized as needed to support future use cases. As critical mass was achieved, the delivery cost decreased, and the speed increased thanks to reuse.

Data Platform

One of the significant efforts in the first year was upgrading the data platform database. The previous version was built on massively parallel technology. However, the database was significantly undersized for the demand. As a result, the daily refresh was finishing later and later in the day, and developing new capabilities was painfully slow, causing people to wait a long time for queries to complete.

To improve data availability and developer productivity, the team migrated to YellowBrick. This technology was compatible with previous database technology, simplifying the migration. Also, it is massively parallel, allowing data and workloads to be distributed across many servers. This was a game-changer.

As a result of this implementation, the daily refresh was completing five hours earlier, and developer productivity significantly improved. In the past six months, the delivery teams implemented significantly more data marts than they did in the prior twelve months, in part because of the improved performance.

Lessons Learned

Though Children’s Minnesota is at the beginning of their journey to accelerate the growth of a data-driven community that drives equitable actions to improve the lives of children and their families, much has been learned in two short years. Some of these have been discussed above. Here are a few more to consider:

Executive Engagement

Executive engagement is essential to the success of an analytics and AI journey. It is possible to make the journey without being part of the strategic plan and related funding support—at least initially. However, without engaged leadership, the journey is nearly impossible. It is equally essential to have executive leadership representation from business, clinical, technology, and research if that function exists. This creates diversity and shared ownership.

Forging a Community

A strong community is at the core of a healthy and resilient analytics ecosystem. Nothing is more important. This doesn’t require moving resources around to align reporting to a single executive leader. Actually, this will hamper progress and make it nearly impossible to create a shared ownership structure between business, clinical, and technology.

The operating model has several inherent features that promote forging a community. For example, organizing around a brand (e.g., enterprise analytics) owned by the community and NOT a particular area or department. The action plan is an essential tool for energizing and aligning the community with the brand, mission, and charter before starting the journey.

In addition, the operating model leverages the S-Team to align delivery locally and the E-Team to align delivery globally. This alone goes a long way toward creating an analytics community. In addition, shared events, such as town halls, demo days, and year-in-review, played a vital role.

Partnerships Rock!

Integrating business, clinical and technical resources on a delivery team creates a partnership that promotes teamwork and shared accountability—difficult, if not impossible, to achieve with centralized or hub-and-spoke operating models where the relationship is more transactional. This improves the ability to align capacity to strategic and operational priorities and ensure the most critical work is getting done. Additonally, this allows everyone on the team the ability to see how their contributions impact the mission, encourages team members to step outside their comfort zone to learn new skills so they can better contribute to the team’s success, and allows teams to adapt to support shifts in demand quickly. In short, teams get more done and have fun doing it.

Pilots Over RFPs

Sometimes, it is essential to do an RFP, especially if there is a significant upfront investment. However, they are costly, time-consuming and don’t adequately validate the vendor’s claims. Fortunately, with most analytics and AI projects, there is a better way to get to value quickly, reduce upfront costs, and manage risk by leveraging pilots.

A pilot can quickly vet a product with a small investment in the real world. This provides the ability to quickly learn by doing and validating the business case before doubling down. This also provides the means to try many things rather than just a few. With all the changes taking place in the market around analytics and AI, this can be a considerable advantage and reduce the risk of selecting a suboptimal product.

Perfect Data Isn’t Perfect

Data is rarely perfect. Trust in data is achieved by aligning with the data consumers on the accuracy, completeness, and timeliness of data needed to support the use case. Because delivery teams are shared by business, clinical, and technology, they can more easily partner to understand data and find the right level of data accuracy to meet their needs.

Automation Pays Dividends

As delivery teams are formed, it is essential to quickly identify inherited legacy reports, and decide whether to deprecate or automate. Make this a priority. Often, an invisible, strong force will try to convince you to defer because you don’t have time.

DON’T GIVE IN. Every hour you save through automation at the start of the journey liberates team members from mundane work allowing them to build data products. The time saved will pay significant dividends over the first twelve months of your journey and beyond.

Rightsize the Data Architecture

Most mature data platforms have several layers of data, each more distilled for use than the other. The data lake is used to store transactional data in its raw form. The data refinery stores cleansed and integrated data using a third-normal form (3NF) or data vault design. The final layer is the data marts, where the data is organized to be easily understood and used to support many use cases, often leveraging the star schema design.

One of the significant learnings is that the integration layer is optional. No doubt some of you are thinking, “You're nuts.” Please bear with me. Building data marts directly from the data lake significantly reduces cost and time-to-market in delivering data products—assuming transactional facts are leveraged and dimensions conform. This isn’t to say there are no use cases for leveraging the data refinery. However, it should only be used if the business case justifies the investment.

The other learning is to only build the dimensions and measures into the data marts that are needed. Avoid building features or data that don’t have an immediate use. For example, slowly changing dimensions add cost with no immediate return. If something can be added now or later, and the cost is roughly the same, defer the investment until the feature is needed.

Looking Forward

Children’s Minnesota has gone from the lowest rating on an industry benchmark to close to the middle point of achieving the highest rating in less than two years. Here are some of the big rocks on the horizon to further accelerate its growth to the highest rating by 2027.

Epic Implementation

With Epic's implementation, their analytics suite (Cogito, Cognitive Computing, and Cosmos) will be integrated into the analytics ecosystem to accelerate the journey further. This will also involve retrofitting the data platform to source data from Epic.

Predictive Analytics

Three things will transform the healthcare landscape: improve health equity, automate manual processes, and shift from being reactive to proactive. As delivery teams mature, so does data fluency for them and the data consumers they serve. Several areas are prepared to begin to leverage predictive analytics. Over the next two years, the focus will be on improving business and clinical operations. For example, leveraging predictions of census to align staffing with demand better.

Data Ethics Framework

With the adoption of predictive analytics, legacy standards and practices are insufficient to ensure data is used responsibly. So, a top priority is implementing a data ethics framework like that discussed in the piece “Powering Your Journey with AI.”

Intelligent Automation

As the adoption of the dashboard framework (Pulse) and self-service analytics grows, the data coverage expands and becomes more trusted, the data and analytics platform evolves, and the community becomes more adept at leveraging predictive algorithms and AI, data fluency will grow, and the community will begin to look beyond leveraging algorithms to help people be more effective to how to leverage it to automate what people do, and provide advanced capabilities to assist physicians.

This is the era of intelligent automation and represents a significant step change in the evolution of leveraging analytics and AI. This occurred in the late 90s in retail (e.g., Amazon) and is now happening in healthcare. Several books cover this topic from a healthcare and non-healthcare perspective: Hacking Healthcare by Tom Lawry and Competing in the Age of AI by Marco Iansiti and Karim Lakhani.

Voyaging to the Cloud

Migrating to the cloud will reduce operating costs, improve productivity, enhance security and controls, provide access to advanced analytics and AI capabilities, and scale to support demand. This will happen in stages with a pilot to vet vendor offerings and gain practical experience before migrating the data and analytics platform to the cloud. This is being considered as part of the Epic migration effort, given that much of the data pipeline needs to be rewritten to source data from Epic.

In Closing

Children's Minnesota's journey toward becoming a data-driven organization demonstrates the transformative power of a well-executed analytics strategy in healthcare. By implementing an agile, federated operating model with cross-functional delivery teams, they've achieved remarkable progress in just two years—evolving from the lowest industry benchmark to nearly the median of the highest rating.

Their success story underscores the importance of forging a strong analytics community, leveraging partnerships, and adopting a pragmatic data management and architecture approach. As Children's Minnesota looks ahead to 2027, its focus on leveraging Epic, predictive analytics and implementing a robust data ethics framework positions it to reach the highest industry rating and fundamentally transform healthcare delivery.

Their journey is an inspiring blueprint for other healthcare organizations striving to harness the power of data to improve patient care, streamline operations, and drive innovation in an increasingly complex healthcare landscape.