Expert Exchange: Data Governance and Data Quality in Our Transition to Cloud

Client Inquiry:

In our data-driven transformation, we’re currently focusing on data governance and data quality, and transitioning from an on-prem data warehouse environment to the Azure cloud. We’re early on our journey. What should we be aware of at this point? What don’t we know?

Expert Takeaways:

Governance Council

Create a governance council that can provide feedback and subject matter expertise. Use the council to help you arrive at a standardized data architecture.

• Keep the topics engaging to keep participants coming. Reference data is a good topic because everyone has an opinion on what should be the source of truth.
• Good topics include the domains to start with and the attributes that you believe are critical across the organization. Get direction on the right domains to tackle first.
• After reaching agreement about the reference data, you can use that information in your modeling for a common, canonical, model for the enterprise.
• You don’t have to start from scratch. You can propose starting with established standards, such as ISO standards. Present various value sets and code sets that you’ll use to represent how you’ll standardize and publish and make data available.
• The folks that consume data often also need a voice because they set priorities around what needs to be worked on first.
• Reach agreement around the source and the standards for the code sets. You can publish the code sets to a tool like Alation so that folks can pick up the code sets and use them for their subsequent mappings.
• In your discussions, always save space for reference data as its own domain. Reference data needs its own targeted space at the table.

Canonical Data Model

Generate a canonical data model that represents the migration from your current enterprise data warehouse (EDW) to Azure.

• From a strategy standpoint, you can use the model for the build-out of your core master data management (MDM) domains. The domains will feed into the model.
• The model is valuable when bringing in new sources. People can map data to the common model instead of creating random hubs and data marts.
• The model will also benefit your dashboard folks from the consumption layer.
• Add the model to the data catalog so that people better understand how data should be looked at across the enterprise from a high-level standpoint.
• The model is a helpful tool when interacting with the governance council. When thinking about topics to bring to the council, you can show them what you need their help working on within the bigger context.

Inventory of Applications

• An easy win to bring to the council is an inventory of applications.
• Inventory all of your current applications: on prem, cloud, and subscriptions.
• You’ll probably find that you've got multiple teams subscribing to the same types of data subscriptions. You can: a) save costs by getting rid of duplication and b) share the subscriptions so that more folks have access to the data.
• With higher maturity, you can move the subscriptions from the business units into a centralized space within Azure so that the subscriptions start to serve as sources of truth. You can use them as standardizing data and push them out as part of your data across the organization.
• The inventory can help your Azure and data governance teams quickly discover who the stewards, custodians, and owners are from an application-level standpoint.
• One of the first things you want to do for your catalog is load your inventory of applications so that people across the enterprise can see what’s available.
• When bringing your data into Azure, include application identifiers, names, and other metadata so that you can assess your cost savings for storage.