Agile methodology has transformed the process of software development to be iterative, where requirements and solutions evolve through collaboration between self-organizing cross-functional teams. The ultimate value in Agile development is that it enables teams to deliver value faster, with greater quality and predictability, and greater aptitude to respond to change. We call these “quick wins”.
This same methodology can be applied to data governance or, really, whenever an activity can be clearly identified, defined, prioritized, broken down into releases and each release further dissected into smaller tasks known as sprints.
|
Provides the framework behind delivering business value to the right resources at the right time. |
Enables an organization's ability to adapt to changing needs and perform cost benefit analysis on their data governance efforts. |
Establishes quantitative and qualitative value of data governance, driven by accessible and trusted data. |
But let me ask you this, is agile governance the right approach for you? Before I can help you answer that, we need to look at the conventional or legacy approach to data governance.
We have found that many organizations follow a linear path that begins with creating a data dictionary, defining data domains across the enterprise, and appointing a data steward to run the entire process. Those are all important to data governance, but the approach creates bottlenecks with the data governor, data governance council, data steward and stakeholders upfront.
In the short term, top management’s decision to initiate the set-up of data governance offers no added value to field teams and may even be seen more as a disturbance or an additional constraint than an urgent need. With this asymmetry of input versus output, it’s not hard to see how the initiative can quickly run out of steam before it has taken off the ground.
What is agile data governance? We think our friends over at data.world said it best.
Agile Governance is the process of creating and improving data assets by iteratively capturing knowledge as data producers and consumers work together so that everyone can benefit. It adapts the deeply proven best practices of Agile and Open software development to data and analytics.
We start by having the customer identify small, bite-sized initiatives that hold strategic importance for them. From there, we put these through multiple iterations to perfection then build on the successes to embark on progressively more complex initiatives. For example, start with deploying a data catalog to stop your data lakes from turning into data swamps. That way, the data governance council can keep all stakeholders abreast of decisions, milestones and wins while keeping work inputs focused and limited.
The business benefit is realized much faster and thus generates even more data governance enthusiasm among stakeholders, getting this buy-in is key to perpetuating the maturity of your Business Intelligence Program.
I would be remiss to not point out that a common pitfall of agile data governance is that due to the small scale of focus initiatives, there’s a risk that the decision-making process will be inadvertently confined in a silo. However, this can be overcome if the entire program is infused with a ‘big picture’ mindset from the onset.
This is simple, have you ever heard one of the below statements come from one of your business users?
Or maybe you have heard one of these statements from your IT team.
If you can say “yes” to any of the above, then you are a great fit!
So now, assuming you are a fit, let’s look at what an effective data governance program does.
Those are the why for building a data governance program. But the how of building your data governance program, is where methodologies and frameworks come in. Data governance frameworks can be close-knit or loosely structured, and both can work. The suitability of the different approaches to data governance depends on the culture, maturity, objectives, structure, size, complexity and technology of the organization in question.
To get it right from the get-go, the foundational phase of a data governance initiative must include multiple discovery sessions to identify opportunities, quantify business value, identify stakeholders, assess sentiment, prioritize focus areas, develop goals, and define a data governance model roadmap. To implement this foundation, we help customers define strategic objectives of data governance, educate stakeholders, obtain senior management buy-in, and designate data stewards.
Once these initial steps are complete, the organization’s data governance council can make an informed decision as to whether to go with a highly or loosely structured framework. A less structured approach would have a loosely knit team of individuals who systematically work the roadmap to accomplish the data governance goals. The highly structured team would be supported by dedicated roles as well as clearly defined tools, templates, and processes. We recommend the latter for more enterprise-wide efforts and the SME Team can help support open roles.
The establishment of the overarching enterprise-wide data governance best practices framework is the rock upon which your entire data governance program rests upon. It identifies a data governor, the data governance council (headed by the data governor), and a steering committee including people from multiple lines of business. This is key to obtaining buy-in and to ensuring the adoption of standards, processes, and guidelines at the tactical level of your organization.
One common misconception is that an agile governance model eliminates or downplays the need for traditional roles such as data governance lead, data owner, and data steward. Instead, it encourages you to re-imagine those roles and to embrace new ones.
Agile governance thinking may also contribute to more granularity in certain roles. For example, the traditional data steward may be subdivided into business data steward and a technical data steward (traditionally known as a data custodian). These steward roles work hand in hand to ensure business processes are supported by technology infrastructure, with the technology data steward recommending improvements in processes based on better and more creative technology usage.
We recommend identifying explicit stakeholder roles, such as data consumers, data producers, and other participants, even when they may not be formally engaged in governance activities. Their participation is truly important and should be recognized.
Teams should establish a meeting cadence where people regularly come together to identify issues and create and implement solutions, particularly those connected to new and emerging programs. This helps governance remain connected to the real-time needs of the business and isn’t seen as a hurdle.
This is where the fun starts, identifying a business problem and gathering stakeholders who know about the problem and are trying (or have tried) to solve it. Go for a quick win, stakeholders should think about questions they’d like to answer with data that work towards solving the business problem. From there, choose a question.
Once the group establishes the question to answer, data producers gather data for data consumers to use. That curation has to happen in a durable place so its fruits can be preserved for others in the future. New knowledge and reusable assets will be created and captured with each iteration.
Within these cycles, data producers will quickly learn what’s working and what’s not about the data sources they’re curating, and they can make improvements in real-time. Doing analytics with a living, evolving data asset focuses stakeholders and provides valuable insights at high frequency. That’s why Agile Data Governance practitioners see ROI in days instead of months. By cataloging the work as it happens, and not only the “finished” analysis, teams continuously learn from each other and elevate their data literacy. That’s because people learn data skills and domain knowledge faster by doing the work and seeing their peers solve real problems.
Transparency and iteration lead to progressively higher quality as teams refine analysis and data sources one step at a time. The completed reproducible output now gives people a jumping-off point. When people document their analysis as part of the workflow—not as an afterthought, which is today’s unfortunate norm—their coworkers can find, understand, reuse, and adapt it. As with software development (and everything else in life), it’s easier to start a data project if you’ve got something to build on.
As teams build data assets together and watch each other solve real business problems, the community of data producers, data consumers, and domain experts within the organization grows. Useful, creative, once-rare data practices will spread from team to team and become true, widespread best practices. Anyone who wants to make data-driven decisions will finally find what they need without friction or fear.
data.world stands out as a trailblazer in the realm of agile data governance tools by promoting a culture of collaboration and inclusivity within organizations. This platform goes beyond the traditional notion of data management, emphasizing not only the importance of data security but also the value of fostering open communication and sharing of insights. By creating an environment where data governance practices are openly discussed and exchanged, agile data governance tools enable teams to work together seamlessly, breaking down silos and encouraging a holistic approach to data utilization.
The ethos of data.world lies in the belief that data is not just a resource to be controlled, but a powerful asset that can drive transformative outcomes when leveraged collectively. Through its innovative features and user-friendly interface, data.world empowers users to not only access and analyze data, but also to collaborate, learn, and grow together. By democratizing data access and promoting a deeper understanding of its strategic importance, agile data governance tools cultivate a community of data enthusiasts who are passionate about harnessing the power of data for positive change.
To give you a more practical feel of what agile governance looks like, here are some of the key activities that would be involved.
In today’s big-data-fast reality, many organizations have hundreds of thousands of data sources, potentially millions of different data sets, and a growing number of self-service users consuming that information broadly. Traditional top-down, workflow-driven data governance just can’t keep up. What’s needed is advanced data solutions paired with a modern approach to governance that’s agile, scalable, and takes advantage of machine learning to meet the needs of data-driven businesses.
In short, the payoff for agile data governance is substantial. Data users will be much happier to participate in the program and will develop a strong conviction that indeed the strategic end goals of the data governance best practices framework are achievable. As the data governance program matures and the focus starts to shift to strengthening the pillars of an agile data governance framework such as data management, data security, and data privacy, then new standards, policies, procedures, and controls will be required as well as updates to existing ones.
Agile data governance doesn’t mean there isn’t a need for rules and decisions about data such as those provided by a centralized, decentralized, or hybrid approach. Taking an agile approach incorporates the best parts of traditional models but shifts the focus to providing support that empowers individuals to collaborate and get more value from data.