Search
While the data industry in general has trended toward a centralized data management architecture, this approach can cause numerous challenges. These challenges are often at odds with what many organizations want - increased scalability, greater data agility, better data quality, enhanced data observability, faster access, and improved data interoperability.
Using data mesh and fabric architectural approaches, as defined in Table 1 and depicted in Figure 2, can help an organization meet its data management goals faster.
Table 1
Figure 2
This distributed domain-centered approach to data management provides many benefits, including:
Figure 3
Many data issues caused by using a centralized data management architecture can be addressed by implementing data mesh or fabric. These broad issues include a lack of data source knowledge, limitations in adapting to evolving business information needs, difficulty in scaling to meet increased data volumes and types of data, and resistance from data stewards in providing access to data assets for ingestion, resulting in organizational fiefdoms with little enterprise alignment.
Emily Robertson-Yeingst, product marketing leader at TIBCO, describes common pain points that push organizations toward data fabric, such as, “...needing access to the right data at the right time, needing to trust data, and needing to know I can find the data I need.” She notes that data mesh needs are a little different than a centralized data architecture and include, “...getting the right data to the right end user, consistent and clean information, and obtaining an infrastructure that is agile and scalable.”
Marcus Gilg, the Chief Information Officer at Deutsche Bahn Regio Bus in Germany, recently transformed his organization’s data management architecture by implementing a data fabric and mesh concept, recognizing, “...there was a need to develop synergy and increase data usage,” to best utilize data for decision-making.
Both the experts note that data fabric and mesh address challenges organizations experience with their current data management architectures. Data leaders should also ensure there are well-defined information needs that a data fabric and mesh approach aims to enable. This will result in more tangible benefits to data consumers and executives investing in these efforts.
As you cultivate your strategy and consider prospective timelines to implement data mesh and fabric, understand that this will be a multi-year plan. As Gilg observes, data mesh and fabric are ‘long-term journeys with long-term benefits.’
Below are some questions to consider:
Once your strategy is developed, the foundation of data fabric and mesh can be implemented. The key pillars will vary based on the maturity of data management in your organization. Some of them are:
Unified Data Catalog and Metadata Management — This provides a centralized view of all data assets, enabling teams to discover and understand data, improve data quality, and promote collaboration.
Data Governance — It delivers structure via decentralized data governance. Governance is pushed out to the domain teams and these teams are responsible for creating and enforcing data governance policies — including data privacy and security policies, data quality policies, and data access policies — within their domain.
Data Domains — Logically organize and prioritize data assets and identify data sources associated with each data domain to inform the rollout of the data fabric. Data domains are important to data mesh because they align data ownership with business domains, promote an independent evolution of data products, and break down data silos.
Data Orchestration — This provides a means to manage the flow and usage of data, ensuring that data is consistent and accurate.
Data Quality Program — This supplies insights into the health of data and raises awareness of quality issues. Such programs are instrumental in allowing domain owners to provide access to trusted data products.
As opposed to implementing data mesh and fabric across all your data domains, consider starting with one or two cases to understand the process and impact. Demonstrating quick wins and establishing credibility are critical to sustaining momentum and increasing the velocity of your strategy.
According to Gilg, in implementing data fabric and mesh at Deutsche Bahn, he “...started with small use cases with the business-critical data, and ultimately improved/added …different use cases.”
Once you have demonstrated wins in your data fabric and mesh approach, you will need leadership support for broader implementation. As Gilg explains, “...before training people, it is important to create the data culture. Obtain support from top management first to get buy-in for a data-driven organization.”
With leadership support in hand, it is time to create a socialization and enablement plan that includes personnel training. Below are some questions your plan should consider:
Like any emerging technology or approach, data mesh and fabric are still being evaluated in real-world contexts. However, there are many reasons to believe that data mesh and fabric approaches have the potential to offer significant benefits to organizations that adopt them. If data mesh and fabric are a fit for your organization, you should follow the five steps outlined above to understand your organization’s data challenges and how data mesh and fabric can help systematically address them.
Co-authored by Bob Audet, Kyle Armstrong, Selene Bainum, and Rich Hernandez with TechTarget.
Guidehouse is a global consultancy providing advisory, digital, and managed services to the commercial and public sectors. Purpose-built to serve the national security, financial services, healthcare, energy, and infrastructure industries, the firm collaborates with leaders to outwit complexity and achieve transformational changes that meaningfully shape the future.