Agile Data Governance Tools and Approach

Determining if Agile Data Governance is Right for You

This is simple, have you ever heard one of the below statements come from one of your business users?

• “Endless emails to find the right person who has or knows about the data that I need”
• “I get multiple answers for the same business question”
• “I need to combine these datasets and have to wait for IT to help”

Or maybe you have heard one of these statements from your IT team.

• “Everyone comes to us asking for data”
• “We don’t know who has access to what data”
• "We don't have a way to share the data we have"
• “If we make a change to the data, we don’t know what it’s going to impact upstream”
• "The data in our lake (or anywhere) is a big mess” We like to call this a data swamp.  

If you can say “yes” to any of the above, then you are a great fit!

Building the Foundation

So now, assuming you are a fit, let’s look at what an effective data governance program does.

• Provides the right resources to deliver the right business value at the right time
• Determines what to govern and establishes business value driven by governance
• Ensures that the right people are represented and creates accountabilities
• Articulates decision-making and communication processes for the roles, responsibilities, accountabilities, ownership, and decision rights needed to govern
• Adapts to meet the organization’s changing needs
• Represents the governance framework that isn’t a hierarchical

Those are the why for building a data governance program. But the how of building your data governance program, is where methodologies and frameworks come in. Data governance frameworks can be close-knit or loosely structured, and both can work. The suitability of the different approaches to data governance depends on the culture, maturity, objectives, structure, size, complexity and technology of the organization in question.

To get it right from the get-go, the foundational phase of a data governance initiative must include multiple discovery sessions to identify opportunities, quantify business value, identify stakeholders, assess sentiment, prioritize focus areas, develop goals, and define a data governance model roadmap. To implement this foundation, we help customers define strategic objectives of data governance, educate stakeholders, obtain senior management buy-in, and designate data stewards.

Once these initial steps are complete, the organization’s data governance council can make an informed decision as to whether to go with a highly or loosely structured framework. A less structured approach would have a loosely knit team of individuals who systematically work the roadmap to accomplish the data governance goals. The highly structured team would be supported by dedicated roles as well as clearly defined tools, templates, and processes. We recommend the latter for more enterprise-wide efforts and the SME Team can help support open roles.

The Stakeholders

The establishment of the overarching enterprise-wide data governance best practices framework is the rock upon which your entire data governance program rests upon. It identifies a data governor, the data governance council (headed by the data governor), and a steering committee including people from multiple lines of business. This is key to obtaining buy-in and to ensuring the adoption of standards, processes, and guidelines at the tactical level of your organization.

One common misconception is that an agile governance model eliminates or downplays the need for traditional roles such as data governance lead, data owner, and data steward. Instead, it encourages you to re-imagine those roles and to embrace new ones.

Agile governance thinking may also contribute to more granularity in certain roles. For example, the traditional data steward may be subdivided into business data steward and a technical data steward (traditionally known as a data custodian). These steward roles work hand in hand to ensure business processes are supported by technology infrastructure, with the technology data steward recommending improvements in processes based on better and more creative technology usage.

We recommend identifying explicit stakeholder roles, such as data consumers, data producers, and other participants, even when they may not be formally engaged in governance activities. Their participation is truly important and should be recognized.

Teams should establish a meeting cadence where people regularly come together to identify issues and create and implement solutions, particularly those connected to new and emerging programs. This helps governance remain connected to the real-time needs of the business and isn’t seen as a hurdle.

Applying Agile Methodology to Data Governance

This is where the fun starts, identifying a business problem and gathering stakeholders who know about the problem and are trying (or have tried) to solve it. Go for a quick win, stakeholders should think about questions they’d like to answer with data that work towards solving the business problem. From there, choose a question.

Once the group establishes the question to answer, data producers gather data for data consumers to use. That curation has to happen in a durable place so its fruits can be preserved for others in the future. New knowledge and reusable assets will be created and captured with each iteration.

Within these cycles, data producers will quickly learn what’s working and what’s not about the data sources they’re curating, and they can make improvements in real-time. Doing analytics with a living, evolving data asset focuses stakeholders and provides valuable insights at high frequency. That’s why Agile Data Governance practitioners see ROI in days instead of months. By cataloging the work as it happens, and not only the “finished” analysis, teams continuously learn from each other and elevate their data literacy. That’s because people learn data skills and domain knowledge faster by doing the work and seeing their peers solve real problems.

Transparency and iteration lead to progressively higher quality as teams refine analysis and data sources one step at a time. The completed reproducible output now gives people a jumping-off point. When people document their analysis as part of the workflow—not as an afterthought, which is today’s unfortunate norm—their coworkers can find, understand, reuse, and adapt it. As with software development (and everything else in life), it’s easier to start a data project if you’ve got something to build on.

As teams build data assets together and watch each other solve real business problems, the community of data producers, data consumers, and domain experts within the organization grows. Useful, creative, once-rare data practices will spread from team to team and become true, widespread best practices. Anyone who wants to make data-driven decisions will finally find what they need without friction or fear.

Agile Data Governance Tools

data.world stands out as a trailblazer in the realm of agile data governance tools by promoting a culture of collaboration and inclusivity within organizations. This platform goes beyond the traditional notion of data management, emphasizing not only the importance of data security but also the value of fostering open communication and sharing of insights. By creating an environment where data governance practices are openly discussed and exchanged, agile data governance tools enable teams to work together seamlessly, breaking down silos and encouraging a holistic approach to data utilization.

The ethos of data.world lies in the belief that data is not just a resource to be controlled, but a powerful asset that can drive transformative outcomes when leveraged collectively. Through its innovative features and user-friendly interface, data.world empowers users to not only access and analyze data, but also to collaborate, learn, and grow together. By democratizing data access and promoting a deeper understanding of its strategic importance, agile data governance tools cultivate a community of data enthusiasts who are passionate about harnessing the power of data for positive change.

Agile Data Governance in Action

To give you a more practical feel of what agile governance looks like, here are some of the key activities that would be involved.

• Knowing your Stakeholders, Keeping them Engaged
   
  • Include data owners in the program’s steering committee. Data owners are usually representatives of business units who are charged with ensuring the accuracy of data.
    
    
  • Data owners, data stewards, and users receive training that raises their understanding of the need for governance, how the business will benefit, and what specific role they are expected to play in the entire process. This equips participants with the knowledge needed, communicates expectations, and delivers a consistent message.
  
  
• Creating a Single Pane of Glass
   
  • Make the agile work stories visible and readily accessible to the steering committee so they can conveniently keep tabs on what is being tackled at any point in time. The key is to make sure everyone understands what needs to be done and can readily refer back to the work plan if they want clarification.
    
    
  • You could do this via a simple non-tech technique such as Post-It notes, an Intranet dashboard, a large TV screen, an agile project management software or any combination of these.
  
  
• Adopting a Centralized Knowledge Base
   
  • Information is meant to be shared. Consolidating business terminology, data definitions and structures, and governance practices into a data catalog, like data.world, enables trusted access and collaboration throughout the enterprise.
    
    
  • A data catalog keeps both the data and the business connected and in-context so that you actually understand the data. No matter if you have one database or hundreds of applications, your data catalog will democratize your data ecosystem and streamline data access and self-service.
  
  
• Starting Small, Growing Fast
   
  • Data governance is not a one-and-done initiative, it is an ongoing effort. By setting realistic expectations and implementing agile methodology your program is more likely to see greater adoption and will avoid the “Icarus” situation.
    
    
  • For example, plan to complete and approve work in some format on a weekly or monthly cadence. This could run iteratively until the work’s outcome conforms to those realistic expectations.
  
  
• Reflecting on Wins (but don’t stop there)
   
  • Just because the work is considered complete and signed off by the data governance council, does not mean that the learning should stop. There’s always room for improvement. Ergo, once work is complete, team members must regularly come together to identify areas of improvement. This retrospection is integrated into the framework, so it’s triggered automatically.

The Payoff for Agile Data Governance

In today’s big-data-fast reality, many organizations have hundreds of thousands of data sources, potentially millions of different data sets, and a growing number of self-service users consuming that information broadly. Traditional top-down, workflow-driven data governance just can’t keep up. What’s needed is advanced data solutions paired with a modern approach to governance that’s agile, scalable, and takes advantage of machine learning to meet the needs of data-driven businesses.

In short, the payoff for agile data governance is substantial. Data users will be much happier to participate in the program and will develop a strong conviction that indeed the strategic end goals of the data governance best practices framework are achievable. As the data governance program matures and the focus starts to shift to strengthening the pillars of an agile data governance framework such as data management, data security, and data privacy, then new standards, policies, procedures, and controls will be required as well as updates to existing ones.

Agile data governance doesn’t mean there isn’t a need for rules and decisions about data such as those provided by a centralized, decentralized, or hybrid approach. Taking an agile approach incorporates the best parts of traditional models but shifts the focus to providing support that empowers individuals to collaborate and get more value from data.