⛏️Data Management: Data Fabric (Definitions)

"Enterprise data fabric (EDF) is a data layer that separates data sources from applications, providing the means to solve the gridlock prevalent in distributed environments such as grid computing, service-oriented architecture (SOA) and event-driven architecture (EDA)." (Information Management, 2010)

"A data fabric is an emerging data management and data integration design concept for attaining flexible, reusable and augmented data integration pipelines, services and semantics, in support of various operational and analytics use cases delivered across multiple deployment and orchestration platforms." (Jacob O Lund, "Demystifying the Data Fabric", 2020)

"A data fabric is a data management architecture that can optimize access to distributed data and intelligently curate and orchestrate it for self-service delivery to data consumers." (IBM, "Data Fabric", 2021) [source]

"A data fabric is a modern, distributed data architecture that includes shared data assets and optimized data management and integration processes that you can use to address today’s data challenges in a unified way." (Alice LaPlante, "Data Fabric as Modern Data Architecture", 2021)

"A data fabric is an emerging data management design for attaining flexible and reusable data integration pipelines, services and semantics. A data fabric supports various operational and analytics use cases delivered across multiple deployment and orchestration platforms. Data fabrics support a combination of different data integration styles and leverage active metadata, knowledge graphs, semantics and ML to automate and enhance data integration design and delivery." (Ehtisham Zaidi, "Data Fabric", Gartner's Hype Cycle for Data Management, 2021)

"Is a distributed Data Management platform whose objective is to combine various types of data storage, access, preparation, analytics, and security tools in a fully compliant manner to support seamless Data Management." (Michelle Knight, "What Is a Data Fabric?", 2021)

"Data Fabric is an integrated layer of data sources and connection processes based on active metadata." (Eberhard Hechler et al, "Data Fabric and Data Mesh Approaches with AI", 2023)

"A data fabric is a customized combination of architecture and technology. It uses dynamic data integration and orchestration to connect different locations, sources, and types of data. With the right structures and flows as defined within the data fabric platform, companies can quickly access and share data regardless of where it is or how it was generated." (SAP)

"A data fabric is a distributed, memory-based data management platform that uses cluster-wide resources - memory, CPU, network bandwidth, and optionally local disk – to manage application data and application logic (behavior). The data fabric uses dynamic replication and data partitioning techniques to offer continuous availability, very high performance, and linear scalability for data intensive applications, all without compromising on data consistency even when exposed to failure conditions." (VMware)

"A Data Fabric is a technology utilization and implementation design capable of multiple outputs and applied uses." (Gartner)

"A data fabric is an architecture and set of data services that provide consistent capabilities across a choice of endpoints spanning hybrid multicloud environments." (NetApp) [source]

"Data fabric is an end-to-end data integration and management solution, consisting of architecture, data management and integration software, and shared data that helps organizations manage their data. A data fabric provides a unified, consistent user experience and access to data for any member of an organization worldwide and in real-time." (Tibco) [source]

🪙Business Intelligence: Data Fabric (Just the Quotes)

"Data architects often turn to graphs because they are flexible enough to accommodate multiple heterogeneous representations of the same entities as described by each of the source systems. With a graph, it is possible to associate underlying records incrementally as data is discovered. There is no need for big, up-front design, which serves only to hamper business agility. This is important because data fabric integration is not a one-off effort and a graph model remains flexible over the lifetime of the data domains." (Jesús Barrasa et al, "Knowledge Graphs: Data in Context for Responsive Businesses", 2021)

"Data fabrics are general-purpose, organization-wide data access interfaces that offer a connected view of the integrated domains by combining data stored in a local graph with data retrieved on demand from third-party systems. Their job is to provide a sophisticated index and integration points so that they can curate data across silos, offering consistent capabilities regardless of the underlying store (which might or might not be graph based) […]." (Jesús Barrasa et al, "Knowledge Graphs: Data in Context for Responsive Businesses", 2021)

"A Data Fabric has its focus more on the architectural underpinning, technical capabilities, and intelligent analysis to produce active metadata supporting a smarter, AI-infused system to orchestrate various data integration styles, enabling trusted and reusable data in a hybrid cloud landscape to be consumed by humans, applications, or other downstream systems." (Eberhard Hechler et al, "Data Fabric and Data Mesh Approaches with AI", 2023)

"A data fabric is a pattern that is very similar to a data mesh in that both provide solutions encompassing data governance and self-service: discovery, access, security, integration, transformation, and lineage. [...] In simple terms, a data fabric is a metadriven means of connecting disparate sets of data and related tools to provide a cohesive data experience and to deliver data in a self-service manner." (Hubert Dulay & Stephen Mooney, "Streaming Data Mesh", 2023)

"A data fabric is an architectural approach to provide data access across multiple technologies and platforms, and is based on a technology solution. One key contrast is that a data mesh is much more than just technology: it is a pattern that involves people and processes. Instead of taking ownership of an entire data platform, as in a data fabric, the data mesh allows data producers to focus on data production, allows data consumers to focus on consumption, and allows hybrid teams to consume other data products, blend other data to create even more interesting data products, and publish these data products - with some data governance considerations in place." (Hubert Dulay & Stephen Mooney, "Streaming Data Mesh", 2023)

"Data Fabric architecture utilizes active metadata, knowledge graphs, and semantic enrichment, combining intelligent information integration and transformation technologies to intelligently support data consumers, for example, business users."  (Eberhard Hechler et al, "Data Fabric and Data Mesh Approaches with AI", 2023)

"Data Fabric’s building blocks represent groupings of different components and characteristics. They are high-level blocks that describe a package of capabilities that address specific business needs. The building blocks are Data Governance and its knowledge layer, Data Integration, and Self-Service." (Sonia Mezzetta, "Principles of Data Fabric: Become a data-driven organization by implementing Data Fabric solutions efficiently", 2023)

"Data Fabric is a composable architecture made up of different tools, technologies, and systems. It has an active metadata and event-driven design that automates Data Integration while achieving interoperability. Data Governance, Data Privacy, Data Protection, and Data Security are paramount to its design and to enable Self-Service data sharing. The following figure summarizes the different characteristics that constitute a Data Fabric design." (Sonia Mezzetta, "Principles of Data Fabric: Become a data-driven organization by implementing Data Fabric solutions efficiently", 2023)

"Data Fabric is a distributed data architecture that connects scattered data across tools and systems with the objective of providing governed access to fit-for-purpose data at speed. Data Fabric focuses on Data Governance, Data Integration, and Self-Service data sharing. It leverages a sophisticated active metadata layer that captures knowledge derived from data and its operations, data relationships, and business context. Data Fabric continuously analyzes data management activities to recommend value-driven improvements. Data Fabric works with both centralized and decentralized data systems and supports diverse operational models." (Sonia Mezzetta, "Principles of Data Fabric: Become a data-driven organization by implementing Data Fabric solutions efficiently", 2023)

"Enterprises have difficulties in interpreting new concepts like the data mesh and data fabric, because pragmatic guidance and experiences from the field are missing. In addition to that, the data mesh fully embraces a decentralized approach, which is a transformational change not only for the data architecture and technology, but even more so for organization and processes. This means the transformation cannot only be led by IT; it’s a business transformation as well." (Piethein Strengholt, "Data Management at Scale: Modern Data Architecture with Data Mesh and Data Fabric" 2nd Ed., 2023)

"Gaining more insight into data, simplifying data access, enabling shopping-for-data, augmenting traditional data governance, generating active metadata, and accelerating development of products and services are enabled by infusing AI into the Data Fabric architecture. An AI-infused Data Fabric is not only leveraging AI but also likewise an architecture to manage and deal with AI artefacts, including AI models, pipelines, etc." (Eberhard Hechler et al, "Data Fabric and Data Mesh Approaches with AI", 2023)

"In Exploiting semantic knowledge graphs can support interpretability and explainability of nearly all AI model types (including DL models) by discovering and depicting semantic and non-obvious relationships or depicting an ML model in a simplified and more readable, explainable way., a Data Mesh solution organizes data around business domain owners and transforms relevant data assets (data sources) to data products that can be consumed by distributed business users from various business domains or functions. These data products are created, governed, and used in an autonomous, decentralized, and self-service manner. Self-service capabilities, which we have already referenced as a Data Fabric capability, enable business organizations to entertain a data marketplace with shopping-for-data characteristics." (Eberhard Hechler et al, "Data Fabric and Data Mesh Approaches with AI", 2023)

"[...] it is the Data Fabric architecture that enables the Data Mesh. In other words, the Data Fabric is the architectural underpinning to implement a Data Mesh solution." (Eberhard Hechler et al, "Data Fabric and Data Mesh Approaches with AI", 2023)

"The Data Fabric architecture can help enterprises address the challenges of data and AI governance effectively, including the orchestration and exchange of metadata across organizational implementations. First, Data Fabric pulls data from disparate data sources and orchestrates metadata exchange across organizational systems, thus providing a holistic view of data and AI at the enterprise level, which lays a solid technology foundation for a consistent and unified enterprise-level data and AI governance. Likewise, a Data Fabric architecture serves as a foundation for a Data Mesh solution, which is supporting organizational or departmental data and AI governance initiatives. Second, the advanced automation and AI technologies employed by a Data Fabric architecture can greatly simplify the implementation of data and AI governance at the enterprise or organizational level, enabling organizational federated Data Mesh initiatives, where orchestration and exchange of metadata across organizations need to be implemented as well." (Eberhard Hechler et al, "Data Fabric and Data Mesh Approaches with AI", 2023)

"The data fabric is an approach that addresses today’s data management and scalability challenges by adding intelligence and simplifying data access using self-service. In contrast to the data mesh, it focuses more on the technology layer. It’s an architectural vision using unified metadata with an end-to-end integrated layer (fabric) for easily accessing, integrating, provisioning, and using data."  (Piethein Strengholt, "Data Management at Scale: Modern Data Architecture with Data Mesh and Data Fabric" 2nd Ed., 2023)

"While a Data Fabric is an architecture that facilitates the end-to-end integration of various data and AI pipelines across hybrid cloud environments through the use of intelligent and automated systems and applications, a Data Mesh should be seen as a solution, which is geared toward delivering data-as-a-product in an organizational federated approach." (Eberhard Hechler et al, "Data Fabric and Data Mesh Approaches with AI", 2023)

"While a data mesh seeks to solve many of the same problems that a data fabric addresses - namely, the ability to address data in a single, composite data environment—the approach is different. While a data fabric enables users to create a single, virtual layer on top of distributed data, a data mesh further empowers distributed groups of data producers to manage and publish data as they see fit. Data fabrics allow for a low-to-no-code data virtualization experience by applying data integration within APIs that reside within the data fabric. The data mesh, however, allows for data engineers to write code for APIs with which to interface further. Without clearly defined boundaries, domains appear to be too interconnected, and ownership becomes either political or subject to interpretation. For instance, a large retailer most likely has multiple domains. [...]" (Hubert Dulay & Stephen Mooney, "Streaming Data Mesh", 2023)

"At its core, a data fabric is an architectural framework, designed to be employed within one or more domains inside a data mesh. The data mesh, however, is a holistic concept, encompassing technology, strategies, and methodologies." (James Serra, "Deciphering Data Architectures", 2024)

"It is very important to understand that data mesh is a concept, not a technology. It is all about an organizational and cultural shift within companies. The technology used to build a data mesh could follow the modern data warehouse, data fabric, or data lakehouse architecture - or domains could even follow different architectures. (James Serra, "Deciphering Data Architectures", 2024)

🎯Piethein Strengholt - Collected Quotes

"For advanced analytics, a well-designed data pipeline is a prerequisite, so a large part of your focus should be on automation. This is also the most difficult work. To be successful, you need to stitch everything together." (Piethein Strengholt, "Data Management at Scale: Best Practices for Enterprise Architecture", 2020)

"One of the patterns from domain-driven design is called bounded context. Bounded contexts are used to set the logical boundaries of a domain’s solution space for better managing complexity. It’s important that teams understand which aspects, including data, they can change on their own and which are shared dependencies for which they need to coordinate with other teams to avoid breaking things. Setting boundaries helps teams and developers manage the dependencies more efficiently." (Piethein Strengholt, "Data Management at Scale: Best Practices for Enterprise Architecture", 2020)

"The logical boundaries are typically explicit and enforced on areas with clear and higher cohesion. These domain dependencies can sit on different levels, such as specific parts of the application, processes, associated database designs, etc. The bounded context, we can conclude, is polymorphic and can be applied to many different viewpoints. Polymorphic means that the bounded context size and shape can vary based on viewpoint and surroundings. This also means you need to be explicit when using a bounded context; otherwise it remains pretty vague." (Piethein Strengholt, "Data Management at Scale: Best Practices for Enterprise Architecture", 2020)

"The transformation of a monolithic application into a distributed application creates many challenges for data management." (Piethein Strengholt, "Data Management at Scale: Best Practices for Enterprise Architecture", 2020)

"A domain aggregate is a cluster of domain objects that can be treated as a single unit. When you have a collection of objects of the same format and type that are used together, you can model them as a single object, simplifying their usage for other domains." (Piethein Strengholt, "Data Management at Scale: Modern Data Architecture with Data Mesh and Data Fabric" 2nd Ed., 2023)

"Data products should remain stable and be decoupled from the operational/transactional applications. This requires a mechanism for detecting schema drift, and avoiding disruptive changes. It also requires versioning and, in some cases, independent pipelines to run in parallel, giving your data consumers time to migrate from one version to another." (Piethein Strengholt, "Data Management at Scale: Modern Data Architecture with Data Mesh and Data Fabric" 2nd Ed., 2023)

"Decentralization involves risks, because the more you spread out activities across the organization, the harder it gets to harmonize strategy and align and orchestrate planning, let alone foster the culture and recruit the talent needed to properly manage your data." (Piethein Strengholt, "Data Management at Scale: Modern Data Architecture with Data Mesh and Data Fabric" 2nd Ed., 2023)

"Enterprises have difficulties in interpreting new concepts like the data mesh and data fabric, because pragmatic guidance and experiences from the field are missing. In addition to that, the data mesh fully embraces a decentralized approach, which is a transformational change not only for the data architecture and technology, but even more so for organization and processes. This means the transformation cannot only be led by IT; it’s a business transformation as well." (Piethein Strengholt, "Data Management at Scale: Modern Data Architecture with Data Mesh and Data Fabric" 2nd Ed., 2023)

"The data fabric is an approach that addresses today’s data management and scalability challenges by adding intelligence and simplifying data access using self-service. In contrast to the data mesh, it focuses more on the technology layer. It’s an architectural vision using unified metadata with an end-to-end integrated layer (fabric) for easily accessing, integrating, provisioning, and using data."  (Piethein Strengholt, "Data Management at Scale: Modern Data Architecture with Data Mesh and Data Fabric" 2nd Ed., 2023)

"The data mesh is an exciting new methodology for managing data at large. The concept foresees an architecture in which data is highly distributed and a future in which scalability is achieved by federating responsibilities. It puts an emphasis on the human factor and addressing the challenges of managing the increasing complexity of data architectures." (Piethein Strengholt, "Data Management at Scale: Modern Data Architecture with Data Mesh and Data Fabric" 2nd Ed., 2023)

"When performing experiments, the first step is to determine what compute infrastructure and environment you need.16 A general best practice is to start fresh, using a clean development environment. Keep track of everything you do in each experiment, versioning and capturing all your inputs and outputs to ensure reproducibility. Pay close attention to all data engineering activities. Some of these may be generic steps and will also apply for other use cases. Finally, you’ll need to determine the implementation integration pattern to use for your project in the production environment." (Piethein Strengholt, "Data Management at Scale: Modern Data Architecture with Data Mesh and Data Fabric" 2nd Ed., 2023)

🎯🏭Eberhard Hechler - Collected Quotes

"A data architecture defines data standards in an organization, including how data is accessed and consumed. It furthermore describes the data structures used by the business units. Data integration also depends on the defined data architecture standards since data integration requires interaction between data." (Eberhard Hechler et al, "Data Fabric and Data Mesh Approaches with AI", 2023)

"A Data Fabric has its focus more on the architectural underpinning, technical capabilities, and intelligent analysis to produce active metadata supporting a smarter, AI-infused system to orchestrate various data integration styles, enabling trusted and reusable data in a hybrid cloud landscape to be consumed by humans, applications, or other downstream systems. Data cataloging to generate and leverage active metadata is seen as a vital component of any Data Fabric." (Eberhard Hechler et al, "Data Fabric and Data Mesh Approaches with AI", 2023)

"A Data Fabric needs to serve analytical and transactional data consumption patterns to, for instance, address MLOps, trustworthy AI, MDM, inferencing, IoT, edge, and 5G." (Eberhard Hechler et al, "Data Fabric and Data Mesh Approaches with AI", 2023)

"A Data Mesh views data primarily as organized around domain owners who create business-focused data products, which can be aggregated and consumed across distributed consumers, organizations, and Line of Business (LoBs) in a self-service and shopping-for-data fashion. Transforming data from disparate data sources to be consumed as data-as-a-product is an essential paradigm of any Data Mesh." (Eberhard Hechler et al, "Data Fabric and Data Mesh Approaches with AI", 2023)

"A data product is based on semantically related raw data that is transformed into a meaningful business context and easily discoverable and consumable by business users." (Eberhard Hechler et al, "Data Fabric and Data Mesh Approaches with AI", 2023)

"An enterprise data warehouse is a central repository of integrated and transformed, structured data from disparate sources and used for reporting and data analysis." (Eberhard Hechler et al, "Data Fabric and Data Mesh Approaches with AI", 2023)

"Any project execution would be very difficult without implementation and usage of the right product capabilities. The selected products should support the data sources and platforms in your organization and provide AI-augmented functionality to ingest and automatically enrich metadata, allowing business users to easily understand, collaborate, enrich, and access the right data, to quickly establish an environment for highly automated and consistent governance and automatically secure data across the organization."(Eberhard Hechler et al, "Data Fabric and Data Mesh Approaches with AI", 2023)

"Building a data product is enabled by the data domain owner; however, building a data product itself is primarily driven by the data product owner, which can be a marketing or a customer care organization, an after-sales team, or even an individual business user. The data product owner is collaborating with data engineers, data scientists, and other subject matter experts throughout the entire data product build process." (Eberhard Hechler et al, "Data Fabric and Data Mesh Approaches with AI", 2023)

"Data Fabric and Data Mesh provide a unified enterprise data architecture and solution for consolidating dispersed data from a hybrid cloud environment through automated data discovery, smart data integration, and intelligent cataloging." (Eberhard Hechler et al, "Data Fabric and Data Mesh Approaches with AI", 2023)

"Data Fabric architecture utilizes active metadata, knowledge graphs, and semantic enrichment, combining intelligent information integration and transformation technologies to intelligently support data consumers, for example, business users."  (Eberhard Hechler et al, "Data Fabric and Data Mesh Approaches with AI", 2023)

"Data Fabric is an integrated layer of data sources and connection processes based on active metadata." (Eberhard Hechler et al, "Data Fabric and Data Mesh Approaches with AI", 2023)

"Data lineage and provenance are often used interchangeably. Both terms refer to the entire lifecycle of the data, including the five Ws: (a) where the data originates, (b) where the data has been and where is the destination, (c) who made changes to the data, (d) when the data was created or updated, and (e) where the data is stored and used. Knowing answers to these questions is critical to data consumers to trust analytics outcomes derived from data." (Eberhard Hechler et al, "Data Fabric and Data Mesh Approaches with AI", 2023)

"Data management is the process of developing, implementing, and monitoring systems, procedures, and practices to deliver and enhance the value of data and assets throughout their lifecycle, while data and AI governance is defined as the exercise of authority and control during the management of data and assets." (Eberhard Hechler et al, "Data Fabric and Data Mesh Approaches with AI", 2023)

"Data Mesh self-service capabilities are business- and domain-centric; they are geared toward building, delivering, and managing data products in a concrete business, domain, or industry context." (Eberhard Hechler et al, "Data Fabric and Data Mesh Approaches with AI", 2023)

"Definition of data and AI governance policies, rules, and classifications is critical to break down data silos, allow for a uniform data consumption, and prevent misuse of data. It includes monitoring of compliance and enforcement of data and AI rules and policies on an ongoing basis, as well as ensuring compliance with regulations and laws." (Eberhard Hechler et al, "Data Fabric and Data Mesh Approaches with AI", 2023)

"Drift measures the drop in accuracy and drop in data consistency by comparing accuracy during runtime with the accuracy during training and by comparing key characteristics of the dataset used for training with the dataset during runtime." (Eberhard Hechler et al, "Data Fabric and Data Mesh Approaches with AI", 2023)

"Exploiting semantic knowledge graphs can support interpretability and explainability of nearly all AI model types (including DL models) by discovering and depicting semantic and non-obvious relationships or depicting an ML model in a simplified and more readable, explainable way." (Eberhard Hechler et al, "Data Fabric and Data Mesh Approaches with AI", 2023)

"Gaining more insight into data, simplifying data access, enabling shopping-for-data, augmenting traditional data governance, generating active metadata, and accelerating development of products and services are enabled by infusing AI into the Data Fabric architecture. An AI-infused Data Fabric is not only leveraging AI but also likewise an architecture to manage and deal with AI artefacts, including AI models, pipelines, etc." (Eberhard Hechler et al, "Data Fabric and Data Mesh Approaches with AI", 2023)

"In Exploiting semantic knowledge graphs can support interpretability and explainability of nearly all AI model types (including DL models) by discovering and depicting semantic and non-obvious relationships or depicting an ML model in a simplified and more readable, explainable way., a Data Mesh solution organizes data around business domain owners and transforms relevant data assets (data sources) to data products that can be consumed by distributed business users from various business domains or functions. These data products are created, governed, and used in an autonomous, decentralized, and self-service manner. Self-service capabilities, which we have already referenced as a Data Fabric capability, enable business organizations to entertain a data marketplace with shopping-for-data characteristics." (Eberhard Hechler et al, "Data Fabric and Data Mesh Approaches with AI", 2023)

"It is essential to realize that the Data Fabric architecture enables the Data Mesh solution via its rich knowledge catalog, semantic search and discovery, smart integration capabilities, and semantic knowledge graphs. Trustworthy AI, for instance, is enabled via the Data Fabric as well." (Eberhard Hechler et al, "Data Fabric and Data Mesh Approaches with AI", 2023) 

"[...] it is the Data Fabric architecture that enables the Data Mesh. In other words, the Data Fabric is the architectural underpinning to implement a Data Mesh solution." (Eberhard Hechler et al, "Data Fabric and Data Mesh Approaches with AI", 2023)

"Over 80% of models are never operationalized because the efforts involved in deploying them are enormous and the models are deployed and found to produce drift or fairness issues that outweigh the benefits."  (Eberhard Hechler et al, "Data Fabric and Data Mesh Approaches with AI", 2023)

"Semantic enrichment is the process of adding meaning to data, which is represented as additional metadata in the knowledge catalog. The intent of semantic enrichment is to simplify and optimize some of the key Data Fabric and Data Mesh tasks, such as search and discovery of assets, access, and consumption of assets by applications and business users to build corresponding data products." (Eberhard Hechler et al, "Data Fabric and Data Mesh Approaches with AI", 2023)

"The AI lifecycle comprises of business problem understanding, collecting data, preparing data, building the model, deploying the model, monitoring the model, and governing the model." (Eberhard Hechler et al, "Data Fabric and Data Mesh Approaches with AI", 2023)

"The aim of a Data Mesh solution is to establish a data marketplace where data can be searched for, discovered, and consumed as a product." (Eberhard Hechler et al, "Data Fabric and Data Mesh Approaches with AI", 2023)

"The Data Fabric architecture needs to guarantee this single version of the truth within the application and transactional landscape, which – depending on the deployment option of an MDM solution – could also mean to assemble this single version of the truth based on core information that is dispersed and maintained in various data stores." (Eberhard Hechler et al, "Data Fabric and Data Mesh Approaches with AI", 2023)

"The Data Fabric architecture can help enterprises address the challenges of data and AI governance effectively, including the orchestration and exchange of metadata across organizational implementations. First, Data Fabric pulls data from disparate data sources and orchestrates metadata exchange across organizational systems, thus providing a holistic view of data and AI at the enterprise level, which lays a solid technology foundation for a consistent and unified enterprise-level data and AI governance. Likewise, a Data Fabric architecture serves as a foundation for a Data Mesh solution, which is supporting organizational or departmental data and AI governance initiatives. Second, the advanced automation and AI technologies employed by a Data Fabric architecture can greatly simplify the implementation of data and AI governance at the enterprise or organizational level, enabling organizational federated Data Mesh initiatives, where orchestration and exchange of metadata across organizations need to be implemented as well." (Eberhard Hechler et al, "Data Fabric and Data Mesh Approaches with AI", 2023)

"The goal of semantic enrichment is to simplify and optimize some of the key Data Fabric and Data Mesh tasks, such as search and discovery of assets, access, and consumption of assets by applications and business users." (Eberhard Hechler et al, "Data Fabric and Data Mesh Approaches with AI", 2023)

"The terms Data Fabric and Data Mesh are often viewed as different, conflicting, or at the best overlapping data architectures or frameworks, data management concepts, or approaches to discover, explore, govern, and consume data. However, these concepts are related to each other, where each concept emphasizes specific imperatives or objectives."(Eberhard Hechler et al, "Data Fabric and Data Mesh Approaches with AI", 2023)

"The term data governance is used for the processes and responsibilities that define, manage, and enforce access, privacy, availability, and security of the organization’s data. It typically includes a set of policies, rules, and data classifications and functionality to monitor and enforce compliance. As stated earlier, we use the term AI governance in a broader sense, also including AI artefacts." (Eberhard Hechler et al, "Data Fabric and Data Mesh Approaches with AI", 2023)

"The value of a Data Mesh solution is that it assigns the creation of data products to data engineers and subject matter experts upstream who are most familiar with the business domains and corresponding needs." (Eberhard Hechler et al, "Data Fabric and Data Mesh Approaches with AI", 2023)

"While a Data Fabric is an architecture that facilitates the end-to-end integration of various data and AI pipelines across hybrid cloud environments through the use of intelligent and automated systems and applications, a Data Mesh should be seen as a solution, which is geared toward delivering data-as-a-product in an organizational federated approach." (Eberhard Hechler et al, "Data Fabric and Data Mesh Approaches with AI", 2023)

🎯🏭🗒️Sonia Mezzetta - Collected Quotes

"A data architecture needs to have the robustness and ability to support multiple data management and operational models to provide the necessary business value and agility to support an enterprise’s business strategy and capabilities." (Sonia Mezzetta, "Principles of Data Fabric: Become a data-driven organization by implementing Data Fabric solutions efficiently", 2023)

"A data strategy must align with the business goals and overall framework of how data will be used and managed within an organization. It needs to include standards for how data will be discovered, integrated, accessed, shared, and protected. It needs to address how data will meet regulatory compliance policies, Master Data Management, and data democratization. There needs to be an assurance that both data and metadata have a quality control framework in place to achieve data trust. A data strategy needs to have a clear path on how an organization will accomplish data monetization." (Sonia Mezzetta, "Principles of Data Fabric: Become a data-driven organization by implementing Data Fabric solutions efficiently", 2023)

"A data strategy is a living document that needs to be continuously updated to align with business goals. It should have a clear maintenance process with frequent reviews and identification of authors and stakeholders that will contribute to the data strategy. This also includes the handling of exceptions to a data strategy process for any one-off decisions in special circumstances. A data strategy document must always be easily assessable, to the point, and understandable." (Sonia Mezzetta, "Principles of Data Fabric: Become a data-driven organization by implementing Data Fabric solutions efficiently", 2023)

"Apply DataOps principles to the development and delivery of data. DataOps is a best practice framework that accelerates the development of data and quality across its entire life cycle with high efficiency and quality. This is especially important when integrating data across distributed complex systems and environments." (Sonia Mezzetta, "Principles of Data Fabric: Become a data-driven organization by implementing Data Fabric solutions efficiently", 2023)

"Automated data orchestration is a key DataOps principle. An example of orchestration can take ETL jobs and a Python script to ingest and transform data based on a specific sequence from different source systems. It can handle the versioning of data to avoid breaking existing data consumption pipelines already in place." (Sonia Mezzetta, "Principles of Data Fabric: Become a data-driven organization by implementing Data Fabric solutions efficiently", 2023)

"Data Fabric’s building blocks represent groupings of different components and characteristics. They are high-level blocks that describe a package of capabilities that address specific business needs. The building blocks are Data Governance and its knowledge layer, Data Integration, and Self-Service." (Sonia Mezzetta, "Principles of Data Fabric: Become a data-driven organization by implementing Data Fabric solutions efficiently", 2023)

"Data Fabric is a composable architecture made up of different tools, technologies, and systems. It has an active metadata and event-driven design that automates Data Integration while achieving interoperability. Data Governance, Data Privacy, Data Protection, and Data Security are paramount to its design and to enable Self-Service data sharing. The following figure summarizes the different characteristics that constitute a Data Fabric design." (Sonia Mezzetta, "Principles of Data Fabric: Become a data-driven organization by implementing Data Fabric solutions efficiently", 2023)

"Data Fabric focuses on Self-Service data access via active metadata leveraging a composable set of tools and technologies. It offers the ability to discover, understand, and access data across hybrid and multi-cloud data landscapes with automation and Data Governance. It is primarily process and technology centric with flexibility in supporting diverse organizational models. On the other hand, Data Mesh is organizationally and process driven. It requires a technical implementation approach to execute its design. Data Mesh is at a higher level and Data Fabric is at a lower level. Data Fabric is capable of fulfilling Data Mesh’s key principles." (Sonia Mezzetta, "Principles of Data Fabric: Become a data-driven organization by implementing Data Fabric solutions efficiently", 2023)

"Data Fabric is a distributed and composable architecture that is metadata and event driven. It’s use case agnostic and excels in managing and governing distributed data. It integrates dispersed data with automation, strong Data Governance, protection, and security. Data Fabric focuses on the Self-Service delivery of governed data." (Sonia Mezzetta, "Principles of Data Fabric: Become a data-driven organization by implementing Data Fabric solutions efficiently", 2023)

"Data Fabric is a distributed data architecture that connects scattered data across tools and systems with the objective of providing governed access to fit-for-purpose data at speed. Data Fabric focuses on Data Governance, Data Integration, and Self-Service data sharing. It leverages a sophisticated active metadata layer that captures knowledge derived from data and its operations, data relationships, and business context. Data Fabric continuously analyzes data management activities to recommend value-driven improvements. Data Fabric works with both centralized and decentralized data systems and supports diverse operational models." (Sonia Mezzetta, "Principles of Data Fabric: Become a data-driven organization by implementing Data Fabric solutions efficiently", 2023)

"[Data Fabric] is not a single technology, such as data virtualization. […] It is not a single tool like a data catalog and it doesn’t have to be a single data storage system like a data warehouse. It represents a diverse set of tools, technologies, and storage systems that work together in a connected ecosystem via a distributed data architecture, with active metadata as the glue. It doesn’t just support centralized data management but also federated and decentralized data management. It excels in connecting distributed data. Data Fabric is not the same as Data Mesh. They are different data architectures that tackle the complexities of distributed data management using different but complementary approaches." (Sonia Mezzetta, "Principles of Data Fabric: Become a data-driven organization by implementing Data Fabric solutions efficiently", 2023)

"Data Fabric supports a federated, decentralized, or centralized organization. To participate in Data Fabric, metadata is contributed in an automated manner and knowledge is populated from it to propel data management. Data Fabric is different from a Data Mesh design in that it supports decentralized, federated, and centralized organizations. Data Fabric’s objectives are to help an organization to evolve to a more mature level of data management by leveraging active metadata, which is a core prerequisite." (Sonia Mezzetta, "Principles of Data Fabric: Become a data-driven organization by implementing Data Fabric solutions efficiently", 2023)

"Data Mesh is a design concept based on federated data and business domains. It applies product management thinking to data management with the outcome being Data Products. It’s technology agnostic and calls for a domain-centric organization with federated Data Governance." (Sonia Mezzetta, "Principles of Data Fabric: Become a data-driven organization by implementing Data Fabric solutions efficiently", 2023)

"Establish an organization’s data maturity level and progress toward ongoing improvement. An organization needs to first understand what its current data maturity level is to determine the areas of improvement to create a forward-looking plan. A data maturity assessment offers a position on the current data maturity that serves as an indicator of the health of an organization. A data maturity assessment can be used as a tool to drive continuous improvement by measuring progress. The key thing here is to always strive for continuous improvement to achieve success." (Sonia Mezzetta, "Principles of Data Fabric: Become a data-driven organization by implementing Data Fabric solutions efficiently", 2023)

"I emphasize this point as there are views in the industry that Data Fabric is a centralized storage architecture, which is not the case from my point of view. A Data Fabric architecture is driven by the needs and direction of the business architecture." (Sonia Mezzetta, "Principles of Data Fabric: Become a data-driven organization by implementing Data Fabric solutions efficiently", 2023)

"Manage data as a strategic asset that evolves into a data product. The premise here is to stop managing data as a byproduct and create an ecosystem that manages data as a valuable strategic asset that can evolve into a data product. Data producers are accountable for managing the life cycle of data from creation to end of life and ensuring it creates business value along the way for data consumers. This requires data that is governed, trusted, protected, secure, and easily accessible. Move data from technical data assets to Data Products by operationalizing data for high scale sharing." (Sonia Mezzetta, "Principles of Data Fabric: Become a data-driven organization by implementing Data Fabric solutions efficiently", 2023)

"Where Data Mesh differs from Data Fabric is that it has fixed requirements for the Self-Service platform focused on organizing and managing Data Products by business domain. Another difference is Data Fabric supports managing data as an asset and as a product. A Data Product can be composed of assets that have been governed and managed in a Data Fabric architecture. Data Fabric does not have these fixed requirements, although it inherently supports isolating data and Data Governance enforcement via metadata by business domain. You can think of a Data Mesh Self-Service data platform as supporting separate, independent companies (business domains), although the key criteria are that it does not create data silos and attains data sharing across these companies in a secure, quick, and easy manner. In Data Mesh, Data Products are created and managed by federated business domains and a data platform requires capabilities that enable data and policy federation. This is where a Data Fabric solution can also address Data Mesh’s requirements." (Sonia Mezzetta, "Principles of Data Fabric: Become a data-driven organization by implementing Data Fabric solutions efficiently", 2023)