04 December 2015

🪙Business Intelligence: Dashboards (Just the Quotes)

"The real value of dashboard products lies in their ability to replace hunt‐and‐peck data‐gathering techniques with a tireless, adaptable, information‐flow mechanism. Dashboards transform data repositories into consumable information." (Gregory L Hovis, "Stop Searching for InformationMonitor it with Dashboard Technology," DM Direct, 2002)

"Dashboards and visualization are cognitive tools that improve your 'span of control' over a lot of business data. These tools help people visually identify trends, patterns and anomalies, reason about what they see and help guide them toward effective decisions. As such, these tools need to leverage people's visual capabilities. With the prevalence of scorecards, dashboards and other visualization tools now widely available for business users to review their data, the issue of visual information design is more important than ever." (Richard Brath & Michael Peters, "Dashboard Design: Why Design is Important," DM Direct, 2004)

“Dashboards aren't all that different from some of the other means of presenting information, but when properly designed the single-screen display of integrated and finely tuned data can deliver insight in an especially powerful way.” (Richard Brath & Michael Peters, "Dashboard Design: Why Design is Important," DM Direct, 2004)

"Alerts are integral to the dashboard concept in that they transform the dashboard from a graphical information presentation into a live console for managing organizational processes and performance. Effective dashboard deployment must facilitate easy management of alerts. This management process involves three components: (1) rules, (2) actions, and (3) recipients." (Shadan Malik, "Enterprise Dashboards: Design and best practices for IT", 2005)

"Although it does so much more, the central purpose of a dashboard is to warn the user when any relevant metrics are out of acceptable boundaries. In the dashboard terminology, these alerts consisting of rules and actions add critical value to an enterprise dashboard deployment complemented with strong visual indicators of warnings." (Shadan Malik, "Enterprise Dashboards: Design and best practices for IT", 2005)

"The term dashboard has acquired a vibrant new meaning in the field of information management as leading organizations worldwide embrace the idea of empowerment through improved real-time information systems. In the current corporate vocabulary, a dashboard is a rich computer interface with charts, reports, visual indicators, and alert mechanisms that are consolidated into a dynamic and relevant information platform." (Shadan Malik, "Enterprise Dashboards: Design and best practices for IT", 2005)

"An effective dashboard is the product not of cute gauges, meters, and traffic lights, but rather of informed design: more science than art, more simplicity than dazzle. It is, above all else, about communication." (Stephen Few, "Information Dashboard Design", 2006)

"Most dashboards fail to communicate efficiently and effectively, not because of inadequate technology (at least not primarily), but because of poorly designed implementations. No matter how great the technology, a dashboard's success as a medium of communication is a product of design, a result of a display that speaks clearly and immediately. Dashboards can tap into the tremendous power of visual perception to communicate, but only if those who implement them understand visual perception and apply that understanding through design principles and practices that are aligned with the way people see and think." (Stephen Few, "Information Dashboard Design", 2006) 

"Having a purposeless or poorly performing dashboard is more common than not. This happens when the underlying architecture is not designed properly to support the needs of dashboard interaction. There is an obvious disconnect between the design of the data warehouse and the design of the dashboards. The people who design the data warehouse do not know what the dashboard will do; and the people who design the dashboards do not know how the data warehouse was designed, resulting in a lack of cohesion between the two. A similar disconnect can also exist between the dashboard designer and the business analyst, resulting in a dashboard that may look beautiful and dazzling but brings very little business value." (Nils H Rasmussen et al, "Business Dashboards: A visual catalog for design and deployment", 2009)

"In general, it still holds true that 'there is no such thing as a free lunch'. What this means is that the most advanced dashboard solutions with the most features and flexibility are generally also the technologies that require more setup and more skill sets from the administrators and the end users. In some cases companies 'dumb down' their dashboard application in the initial stages of deployment so as not to scare their users with too many options. Later, when a dashboard culture has developed, they open up more of the functionality." (Nils H Rasmussen et al, "Business Dashboards: A visual catalog for design and deployment", 2009)

"Dashboarding is an area that has been using information design to communicate key business metrics for decades. In terms of purpose, these interfaces have embodied many of the best practices surrounding visual communication. Yet their aesthetic and creative value is often lacking. This is an area of great opportunity in business communication; while well intentioned, the dashboard’s traditional format and appearance could benefit from a bit of a makeover." (Jason Lankow et al, "Infographics: The power of visual storytelling", 2012)

"There are myriad questions that we can ask from data today. As such, it’s impossible to write enough reports or design a functioning dashboard that takes into account every conceivable contingency and answers every possible question." (Phil Simon, "The Visual Organization: Data Visualization, Big Data, and the Quest for Better Decisions", 2014)

"A dashboard is like the executive summary of a report. We read executive summaries and skip the body of the report if the summary is more or less in line with our expectations. Trouble is, measurement is never exhaustive. It is only when we dive in that we realize what areas may have been missed." (Sriram Narayan, "Agile IT Organization Design: For Digital Transformation and Continuous Delivery", 2015)

"[…] an overall green status indicator doesn’t mean anything most of the time. All it says is that the things under measurement seem okay. But there always will be many more things not under measurement. To celebrate green indicators is to ignore the unknowns. […] The tendency to roll up metrics into dashboards promotes ignorance of the real situation on the ground. We forget that we only see what is under measurement. We only act when something is not green." (Sriram Narayan, "Agile IT Organization Design: For Digital Transformation and Continuous Delivery", 2015)

"Rolling up fine-grained metrics to create high-level dashboards puts pressure on teams to keep the fine-grained metrics green even when it might not be the best use of their time." (Sriram Narayan, "Agile IT Organization Design: For Digital Transformation and Continuous Delivery", 2015)

"A performance dashboard is a practical tool to improve management effectiveness and efficiency, not just a pretty retrospective picture in an annual report." (Pearl Zhu, "Performance Master: Take a Holistic Approach to Unlock Digital Performance", 2017)

"All human storytellers bring their subjectivity to their narratives. All have bias, and possibly error. Acknowledging and defusing that bias is a vital part of successfully using data stories. By debating a data story collaboratively and subjecting it to critical thinking, organizations can get much higher levels of engagement with data and analytics and impact their decision making much more than with reports and dashboards alone." (James Richardson, 2017)

"Dashboards are a type of multiform visualization used to summarize and monitor data. These are most useful when proxies have been well validated and the task is well understood. This design pattern brings a number of carefully selected attributes together for fast, and often continuous, monitoring - dashboards are often linked to updating data streams. While many allow interactivity for further investigation, they typically do not depend on it. Dashboards are often used for presenting and monitoring data and are typically designed for at-a-glance analysis rather than deep exploration and analysis." (Danyel Fisher & Miriah Meyer, "Making Data Visual", 2018)

"Infographics combine art and science to produce something that is not unlike a dashboard. The main difference from a dashboard is the subjective data and the narrative or story, which enhances the data-driven visual and engages the audience quickly through highlighting the required context." (Travis Murphy, "Infographics Powered by SAS®: Data Visualization Techniques for Business Reporting", 2018)

"Dashboards are collections of several linked visualizations all in one place. The idea is very popular as part of business intelligence: having current data on activity summarized and presented all inone place. One danger of cramming a lot of disparate information into one place is that you will quickly hit information overload. Interactivity and small multiples are definitely worth considering as ways of simplifying the information a reader has to digest in a dashboard. As with so many other visualizations, layering the detail for different readers is valuable." (Robert Grant, "Data Visualization: Charts, Maps and Interactive Graphics", 2019)

"[Dashboards] are popular methods for displaying multiple visualizations and statistical information. Dashboards often take the form of some organizational instrument that offers both at-a-glance and detailed views of many different analytical and information dimensions. Dashboards are not a unique chart type themselves, but rather should be considered compositions that comprise multiple chart types." (Andy Kirk, "Data Visualisation: A Handbook for Data Driven Design" 2nd Ed., 2019)

"Understanding the entire data ecosystem, from the production of a data point to its consumption in a dashboard or a visualization, provides the ability to invoke action, which is more valuable than the mere sum of its parts." (Jesús Barrasa et al, "Knowledge Graphs: Data in Context for Responsive Businesses", 2021)

"A well-designed dashboard needs to provide a similar experience; information cannot be placed just anywhere on the dashboard. Charts that relate to one another are usually positioned close to one another. Important charts often appear larger and more visually prominent than less important ones. In other words, there are natural sizes for how a dashboard comprises charts based on the task and context." (Vidya Setlur & Bridget Cogley, "Functional Aesthetics for data visualization", 2022)

"As we enter into certain types of analytical conversations, we expect the conversations to flow in a predictable and cohesive manner. A KPI dashboard, for example, uses redundant structures across specific dimensions or measures to convey information. A dashboard with a top-down exposition style provides high-level information first and clarifies downward, while a bottom-up dashboard starts with the details and clarifies them against the larger picture." (Vidya Setlur & Bridget Cogley, "Functional Aesthetics for data visualization", 2022)

"Chart choices can also create weight within the entire composition. Presenting information as a comprehensive visualization, such as in a dashboard, requires thinking beyond individual charts. In writing, we not only craft sentences, but write the composition as an entire piece. Certain sentences may drive the writing more, but all sentences play a role in conveying the message." (Vidya Setlur & Bridget Cogley, "Functional Aesthetics for data visualization", 2022)

"The sizes of charts in space reflect how we convey information to a reader. In a dashboard context, the content, size, and space that the various charts occupy should reflect the form and function of the main message. As you saw with the bento box metaphor from the introduction, there needs to be deliberate thought put into the placement and size of each individual chart so that they all work together in harmony." (Vidya Setlur & Bridget Cogley, "Functional Aesthetics for data visualization", 2022)

"When integrating written text with charts in a functionally aesthetic way, the reader should be able to find the key takeaways from the chart or dashboard, taking into account the context, constraints, and reading objectives of the overall message."  (Vidya Setlur & Bridget Cogley, "Functional Aesthetics for data visualization", 2022)

"It turns out that the visual part of [gauges] does not convey significant information; it merely serves as an aesthetically pleasing frame for the total value. However, on a dashboard, we typically need to save space and highlight key information, and a standard card would be better suited for this task." (Alex Kolokolov & Maxim Zelensky, "Data Visualization with Microsoft Power BI", 2024)

"Data may indeed be the new oil. But just like crude oil, data needs refining. It must be transformed into information. This is why we clean, combine, model, and visualize data. The output of all this work - whether you do it on your own, get some help, or use a (semi-)automatic process - includes reports and dashboards that provide insights into various aspects of the organization’s dealings, which decision-makers can then consume to make critical business decisions." (Jeroen ter Heerdt et al, "Microsoft Power BI Visual Calculations: Simplifying DAX", 2026)

See also the definitions on "dashboards". 

🪙Business Intelligence: Data Products (Just the Quotes)

"Data scientists combine entrepreneurship with patience, the willingness to build data products incrementally, the ability to explore, and the ability to iterate over a solution. They are inherently interdisciplinary. They can tackle all aspects of a problem, from initial data collection and data conditioning to drawing conclusions. They can think outside the box to come up with new ways to view the problem, or to work with very broadly defined problems: 'there’s a lot of data, what can you make from it?'" (Mike Loukides, "What Is Data Science?", 2011)

"Discovery is the key to building great data products, as opposed to products that are merely good." (Mike Loukides, "The Evolution of Data Products", 2011)

"New interfaces for data products are all about hiding the data itself, and getting to what the user wants." (Mike Loukides, "The Evolution of Data Products", 2011)

"[...] a good definition of a data product is a product that facilitates an end goal through the use of data. It’s tempting to think of a data product purely as a data problem. After all, there’s nothing more fun than throwing a lot of technical expertise and fancy algorithmic work at a difficult problem." (Dhanurjay Patil, "Data Jujitsu: The Art of Turning Data into Product", 2012)

"As data scientists, we prefer to interact with the raw data. We know how to import it, transform it, mash it up with other data sources, and visualize it. Most of your customers can’t do that. One of the biggest challenges of developing a data product is figuring out how to give data back to the user. Giving back too much data in a way that’s overwhelming and paralyzing is 'data vomit'. It’s natural to build the product that you would want, but it’s very easy to overestimate the abilities of your users. The product you want may not be the product they want." (Dhanurjay Patil, "Data Jujitsu: The Art of Turning Data into Product", 2012)

"Generalizing beyond advertising, when building any data product in which the data is obfuscated (where there isn’t a clear relationship between the user and the result), you can compromise on precision, but not on recall. But when the data is exposed, focus on high precision." (Dhanurjay Patil, "Data Jujitsu: The Art of Turning Data into Product", 2012)

"Ideas for data products tend to start simple and become complex; if they start complex, they become impossible." (Dhanurjay Patil, "Data Jujitsu: The Art of Turning Data into Product", 2012)

"In an emergency, a data product that just produces more data is of little use. Data scientists now have the predictive tools to build products that increase the common good, but they need to be aware that building the models is not enough if they do not also produce optimized, implementable outcomes." (Jeremy Howard et al, "Designing Great Data Products", 2012)

"The best way to avoid data vomit is to focus on actionability of data. That is, what action do you want the user to take? If you want them to be impressed with the number of things that you can do with the data, then you’re likely producing data vomit. If you’re able to lead them to a clear set of actions, then you’ve built a product with a clear focus." (Dhanurjay Patil, "Data Jujitsu: The Art of Turning Data into Product", 2012)

"The key aspect of making a data product is putting the 'product' first and 'data' second. Saying it another way, data is one mechanism by which you make the product user-focused. With all products, you should ask yourself the following three questions: (1) What do you want the user to take away from this product? (2) What action do you want the user to take because of the product? (3) How should the user feel during and after using your product?" (Dhanurjay Patil, "Data Jujitsu: The Art of Turning Data into Product", 2012)

"You can give your data product a better chance of success by carefully setting the users’ expectations. [...] One under-appreciated facet of designing data products is how the user feels after using the product. Does he feel good? Empowered? Or disempowered and dejected?" (Dhanurjay Patil, "Data Jujitsu: The Art of Turning Data into Product", 2012)

"A core premise of data mesh is federating data ownership among domain data owners who are responsible for their data as a product. Offering the data as a product requires the data to be discoverable and to have explicitly stated quality characteristics and a clearly defined access method. Such requirements are at the core of what data catalogs support. With support for data labeling, curation, and crowdsourced feedback, data catalogs are well positioned to offer data as a product. Furthermore, data catalogs support the enforcement of compliant data usage, which becomes more important when data ownership is not managed centrally." (Fadi Maali & Jason Lim, "Implementing a Modern Data Catalog to Power Data Intelligence: Make Trustworthy Data Central to Your Organization", 2022)

"Since data engineering is such a crucial field, you may be wondering who the main players are and what skill sets they possess. Building a data product involves several folks, all of whom need to come together with seamless handoffs to ensure a successful end product or service is created. It would be a mistake to create silos and increase both the number and complexity of integration points as each additional integration is a potential failure point." (Anindita Mahapatra, "Simplifying Data Engineering and Analytics with Delta", 2022)

"A data product is based on semantically related raw data that is transformed into a meaningful business context and easily discoverable and consumable by business users." (Eberhard Hechler et al, "Data Fabric and Data Mesh Approaches with AI", 2023)

"Data products should remain stable and be decoupled from the operational/transactional applications. This requires a mechanism for detecting schema drift, and avoiding disruptive changes. It also requires versioning and, in some cases, independent pipelines to run in parallel, giving your data consumers time to migrate from one version to another." (Piethein Strengholt, "Data Management at Scale: Modern Data Architecture with Data Mesh and Data Fabric" 2nd Ed., 2023)

"In Exploiting semantic knowledge graphs can support interpretability and explainability of nearly all AI model types (including DL models) by discovering and depicting semantic and non-obvious relationships or depicting an ML model in a simplified and more readable, explainable way., a Data Mesh solution organizes data around business domain owners and transforms relevant data assets (data sources) to data products that can be consumed by distributed business users from various business domains or functions. These data products are created, governed, and used in an autonomous, decentralized, and self-service manner. Self-service capabilities, which we have already referenced as a Data Fabric capability, enable business organizations to entertain a data marketplace with shopping-for-data characteristics." (Eberhard Hechler et al, "Data Fabric and Data Mesh Approaches with AI", 2023)

"Data Mesh emphasizes ensuring reliable, consistent, and interoperable data products. When data is treated as a product, quality is non-negotiable. High-quality data must meet the expectations and requirements of its users, both internally and externally. Additionally, data products must be designed with other products in mind, adhering to principles like loose coupling for easy interchangeability and high cohesion for strong functional relatedness. This feature enables the integration of different data products, ensuring seamless interoperability and greater usability. Data products should be reliable, complete, accurate, and accurate. They should also be integrated, compatible, and consistent rather than isolated, incompatible, or conflicting." (Pradeep Menon, "Data Mesh Principles, patterns, architecture, and strategies for data-driven decision making", 2024)

"Domain-oriented ownership is a core principle of data mesh. It entails that data producers, experts in their business domains, are responsible for the entire lifecycle of their produced data. Specifically, they take ownership of the data from the point of ingestion through transformation, serving, quality assurance, and governance. Moreover, they are responsible for the data products created from their data, which serve as units of data consumption for other domains or users." (Pradeep Menon, "Data Mesh Principles, patterns, architecture, and strategies for data-driven decision making", 2024)

"First, good data products align to decentralized domain ownership - they should align to a domain (large or small) with a clear boundary, and should have an empowered owner. Second, good data products are treated as their name implies - as a product and not a project. Good data products, like other products, have a life cycle, have clear consumers, and a clear value proposition. Good data products are self-serve, meaning that users can get what they need from a data product without undue participation from third parties. Good data products also have a federated governance mechanism, that is to say, that provides for local autonomy and decision making - at the data product level by the data product owner and their team. It also means that while being federated, data product owners/teams are responsible and accountable to ensuring their data product adheres to enterprise guidelines and standards as needed." (Jean-Georges Perrin & Eric Broda, "Implementing Data Mesh: Principles and Practice to Design, Build, and Implement Data Mesh", 2024)

"The role of an empowered owner is multifaceted. They are tasked with ensuring that the data product aligns with both specific business requirements and the overarching governance framework. This alignment is crucial for maintaining the integrity and usefulness of the data product, ensuring it remains a valuable asset within the organization’s data landscape." (Jean-Georges Perrin & Eric Broda, "Implementing Data Mesh: Principles and Practice to Design, Build, and Implement Data Mesh", 2024)

"To explain a data mesh in one sentence, a data mesh is a centrally managed network of decentralized data products. The data mesh breaks the central data lake into decentralized islands of data that are owned by the teams that generate the data. The data mesh architecture proposes that data be treated like a product, with each team producing its own data/output using its own choice of tools arranged in an architecture that works for them. This team completely owns the data/output they produce and exposes it for others to consume in a way they deem fit for their data." (Aniruddha Deswandikar,"Engineering Data Mesh in Azure Cloud", 2024)

"When data is considered a product, it creates opportunities for collaboration across different domains. This collaboration involves working with other teams to create, share, and use data products that span multiple areas of expertise, interest, or value. Data Mesh promotes cross-domain collaboration by focusing on the consumers rather than the producers. Data products are made available through standardized interfaces and protocols that support various modes of consumption and are governed by domain experts who understand the context and nuances of their data." (Pradeep Menon, "Data Mesh Principles, patterns, architecture, and strategies for data-driven decision making", 2024)

"Data products are part of the enterprise data infrastructure. They are managed, versioned, and monitored much like APIs or microservices. This shift is supported by modern cloud-native infrastructure, which enables decentralized development, consistent governance, and scalable consumption. Data products don’t need to be stored in one location. They might be in a data lake, a data warehouse, a database, or a data lakehouse. They can become discoverable and accessible through a centralized data catalog or marketplace acting as a portal where users can find, understand, and request access to relevant data products." (Fern Halper, "Data Makes the World Go 'Round", 2026)

"The goal of a data product is to provide value, not just data. That value may come from enabling better decision-making, powering customer-facing applications, enriching AI models, or facilitating compliance. Importantly, data products are developed with users in mind, whether internal analysts, business stakeholders, external partners, or automated systems. This user orientation distinguishes data products from raw datasets or traditional reports." (Fern Halper, "Data Makes the World Go 'Round", 2026)

"Data product usage is growing quickly, doubling every year. Obviously, since we made the investment, we'll work with our customer to find applications." (Ken Shelton)

🪙Business Intelligence: Measures (Just the Quotes)

"The most important and frequently stressed prescription for avoiding pitfalls in the use of economic statistics, is that one should find out before using any set of published statistics, how they have been collected, analysed and tabulated. This is especially important, as you know, when the statistics arise not from a special statistical enquiry, but are a by-product of law or administration. Only in this way can one be sure of discovering what exactly it is that the figures measure, avoid comparing the non-comparable, take account of changes in definition and coverage, and as a consequence not be misled into mistaken interpretations and analysis of the events which the statistics portray." (Ely Devons, "Essays in Economics", 1961)

"If we view organizations as adaptive, problem-solving structures, then inferences about effectiveness have to be made, not from static measures of output, but on the basis of the processes through which the organization approaches problems. In other words, no single measurement of organizational efficiency or satisfaction - no single time-slice of organizational performance can provide valid indicators of organizational health." (Warren G Bennis, "General Systems Yearbook", 1962)

"[Management by objectives is] a process whereby the superior and the subordinate managers of an enterprise jointly identify its common goals, define each individual's major areas of responsibility in terms of the results expected of him, and use these measures as guides for operating the unit and assessing the contribution of each of its members." (Robert House, "Administrative Science Quarterly", 1971)

"A mature science, with respect to the matter of errors in variables, is not one that measures its variables without error, for this is impossible. It is, rather, a science which properly manages its errors, controlling their magnitudes and correctly calculating their implications for substantive conclusions." (Otis D Duncan, "Introduction to Structural Equation Models", 1975)

"Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes." (Charles Goodhart, "Problems of Monetary Management: the U.K. Experience", 1975)

"The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor." (Donald T Campbell, "Assessing the impact of planned social change", 1976)

"Reengineering is the fundamental rethinking and radical redesign of business processes to achieve dramatic improvements in critical contemporary measures of performance such as cost, quality, service and speed." (James A Champy & Michael M Hammer, "Reengineering the Corporation", 1993)

"Industrial managers faced with a problem in production control invariably expect a solution to be devised that is simple and unidimensional. They seek the variable in the situation whose control will achieve control of the whole system: tons of throughput, for example. Business managers seek to do the same thing in controlling a company; they hope they have found the measure of the entire system when they say 'everything can be reduced to monetary terms'." (Stanford Beer, "Decision and Control", 1994)

"A strategy is a set of hypotheses about cause and effect. The measurement system should make the relationships (hypotheses) among objectives (and measures) in the various perspectives explicit so that they can be managed and validated. The chain of cause and effect should pervade all four perspectives of a Balanced Scorecard." (Robert S Kaplan & David P Norton, "The Balanced Scorecard", Harvard Business Review, 1996)

"The Balanced Scorecard has its greatest impact when it is deployed to drive organizational change. [...] The Balanced Scorecard is primarily a mechanism for strategy implementation, not for strategy formulation. It can accommodate either approach for formulating business unit strategy-starting from the customer perspective, or starting from excellent internal-business-process capabilities. For whatever approach that SBU senior executives use to formulate their strategy, the Balanced Scorecard will provide an invaluable mechanism for translating that strategy into specific objectives, measures, and targets, and monitoring the implementation of that strategy during subsequent periods." (Robert S Kaplan & David P Norton, "The Balanced Scorecard", Harvard Business Review, 1996)

"The Balanced Scorecard translates mission and strategy into objectives and measures, organized into four different perspectives: financial, customer, internal business process, and learning and growth. The scorecard provides a framework, a language, to communicate mission and strategy; it uses measurement to inform employees about the drivers of current and future success." (Robert S Kaplan & David P Norton, "The Balanced Scorecard", Harvard Business Review, 1996)

"When a measure becomes a target, it ceases to be a good measure." (Marilyn Strathern, "‘Improving ratings’: audit in the British University system", 1997)

"Since the average is a measure of location, it is common to use averages to compare two data sets. The set with the greater average is thought to ‘exceed’ the other set. While such comparisons may be helpful, they must be used with caution. After all, for any given data set, most of the values will not be equal to the average." (Donald J Wheeler, "Understanding Variation: The Key to Managing Chaos" 2nd Ed., 2000)

"First, good statistics are based on more than guessing. [...] Second, good statistics are based on clear, reasonable definitions. Remember, every statistic has to define its subject. Those definitions ought to be clear and made public. [...] Third, good statistics are based on clear, reasonable measures. Again, every statistic involves some sort of measurement; while all measures are imperfect, not all flaws are equally serious. [...] Finally, good statistics are based on good samples." (Joel Best, "Damned Lies and Statistics: Untangling Numbers from the Media, Politicians, and Activists", 2001)

"Statistics depend on collecting information. If questions go unasked, or if they are asked in ways that limit responses, or if measures count some cases but exclude others, information goes ungathered, and missing numbers result. Nevertheless, choices regarding which data to collect and how to go about collecting the information are inevitable." (Joel Best, "More Damned Lies and Statistics: How numbers confuse public issues", 2004)

"If the KPIs you currently have are not creating change, throw them out because there is a good chance that they may be wrong. They are probably measures that were thrown together without the in-depth research and investigation KPIs truly deserve." (David Parmenter, "Pareto’s 80/20 Rule for Corporate Accountants", 2007)

"Key performance indicators (KPIs) are the vital navigation instruments used by managers to understand whether their business is on a successful voyage or whether it is veering off the prosperous path. The right set of indicators will shine light on performance and highlight areas that need attention. ‘What gets measured gets done’ and ‘if you can’t measure it, you can’t manage it’ are just two of the popular sayings used to highlight the critical importance of metrics. Without the right KPIs managers are sailing blind." (Bernard Marr, "Key Performance Indicators (KPI): The 75 measures every manager needs to know", 2011)

"A statistical index has all the potential pitfalls of any descriptive statistic - plus the distortions introduced by combining multiple indicators into a single number. By definition, any index is going to be sensitive to how it is constructed; it will be affected both by what measures go into the index and by how each of those measures is weighted." (Charles Wheelan, "Naked Statistics: Stripping the Dread from the Data", 2012)

"Even if you have a solid indicator of what you are trying to measure and manage, the challenges are not over. The good news is that 'managing by statistics' can change the underlying behavior of the person or institution being managed for the better. If you can measure the proportion of defective products coming off an assembly line, and if those defects are a function of things happening at the plant, then some kind of bonus for workers that is tied to a reduction in defective products would presumably change behavior in the right kinds of ways. Each of us responds to incentives (even if it is just praise or a better parking spot). Statistics measure the outcomes that matter; incentives give us a reason to improve those outcomes." (Charles Wheelan, "Naked Statistics: Stripping the Dread from the Data", 2012)

"Once these different measures of performance are consolidated into a single number, that statistic can be used to make comparisons […] The advantage of any index is that it consolidates lots of complex information into a single number. We can then rank things that otherwise defy simple comparison […] Any index is highly sensitive to the descriptive statistics that are cobbled together to build it, and to the weight given to each of those components. As a result, indices range from useful but imperfect tools to complete charades." (Charles Wheelan, "Naked Statistics: Stripping the Dread from the Data", 2012)

"Financial measures are a quantification of an activity that has taken place; we have simply placed a value on the activity. Thus, behind every financial measure is an activity. I call financial measures result indicators, a summary measure. It is the activity that you will want more or less of. It is the activity that drives the dollars, pounds, or yen. Thus financial measures cannot possibly be KPIs." (David Parmenter, "Key Performance Indicators: Developing, implementing, and using winning KPIs" 3rd Ed., 2015)

"'Getting it right the first time' is a rare achievement, and ascertaining the organization’s winning KPIs and associated reports is no exception. The performance measure framework and associated reporting is just like a piece of sculpture: you can be criticized on taste and content, but you can’t be wrong. The senior management team and KPI project team need to ensure that the project has a just-do-it culture, not one in which every step and measure is debated as part of an intellectual exercise." (David Parmenter, "Key Performance Indicators: Developing, implementing, and using winning KPIs" 3rd Ed., 2015)

"In order to get measures to drive performance, a reporting framework needs to be developed at all levels within the organization." (David Parmenter, "Key Performance Indicators: Developing, implementing, and using winning KPIs" 3rd Ed., 2015)

"Most organizational measures are very much past indicators measuring events of the last month or quarter. These indicators cannot be and never were KPIs." (David Parmenter, "Key Performance Indicators: Developing, implementing, and using winning KPIs" 3rd Ed., 2015)

"We need indicators of overall performance that need only be reviewed on a monthly or bimonthly basis. These measures need to tell the story about whether the organization is being steered in the right direction at the right speed, whether the customers and staff are happy, and whether we are acting in a responsible way by being environmentally friendly. These measures are called key result indicators (KRIs)." (David Parmenter, "Key Performance Indicators: Developing, implementing, and using winning KPIs" 3rd Ed., 2015)

"GIGO is a famous saying coined by early computer scientists: garbage in, garbage out. At the time, people would blindly put their trust into anything a computer output indicated because the output had the illusion of precision and certainty. If a statistic is composed of a series of poorly defined measures, guesses, misunderstandings, oversimplifications, mismeasurements, or flawed estimates, the resulting conclusion will be flawed." (Daniel J Levitin, "Weaponized Lies", 2017)

"To be any good, a sample has to be representative. A sample is representative if every person or thing in the group you’re studying has an equally likely chance of being chosen. If not, your sample is biased. […] The job of the statistician is to formulate an inventory of all those things that matter in order to obtain a representative sample. Researchers have to avoid the tendency to capture variables that are easy to identify or collect data on - sometimes the things that matter are not obvious or are difficult to measure." (Daniel J Levitin, "Weaponized Lies", 2017)

02 December 2015

🪙Business Intelligence: Reporting (Just the Quotes)

"A man's judgment cannot be better than the information on which he has based it. Give him no news, or present him only with distorted and incomplete data, with ignorant, sloppy, or biased reporting, with propaganda and deliberate falsehoods, and you destroy his whole reasoning process and make him somewhat less than a man." (Arthur H Sulzberger, [speech] 1948)

"The secret language of statistics, so appealing in a fact-minded culture, is employed to sensationalize, inflate, confuse, and oversimplify. Statistical methods and statistical terms are necessary in reporting the mass data of social and economic trends, business conditions, 'opinion' polls, the census. But without writers who use the words with honesty and understanding and readers who know what they mean, the result can only be semantic nonsense." (Darell Huff, "How to Lie with Statistics", 1954)

"To be worth much, a report based on sampling must use a representative sample, which is one from which every source of bias has been removed." (Darell Huff, "How to Lie with Statistics", 1954)

"It is probable that one day we shall begin to draw organization charts as a series of linked groups rather than as a hierarchical structure of individual 'reporting' relationships." (Douglas McGregor, "The Human Side of Enterprise", 1960)

"[...] as the planning process proceeds to a specific financial or marketing state, it is usually discovered that a considerable body of 'numbers' is missing, but needed numbers for which there has been no regular system of collection and reporting; numbers that must be collected outside the firm in some cases. This serendipity usually pays off in a much better management information system in the form of reports which will be collected and reviewed routinely." (William H. Franklin Jr., Financial Strategies, 1987)

"Intangible assets [...] surpass physical assets in most business enterprises, both in value and contribution to growth, yet they are routinely expensed in the financial reports and hence remain absent from corporate balance sheets. This asymmetric treatment of capitalizing (considering as assets) physical and financial investment while expensing intangibles leads to biased and deficient reporting of firms’ performance and value." (Baruch Lev, "Intangibles: Management, Measurement, and Reporting", 2000)

"Project planning is the key to effective project management. Detailed and accurate planning of a project produces the managerial information that is the basis of project justification (costs, benefits, strategic impact, etc.) and the defining of the business drivers (scope, objectives) that form the context for the technical solution. In addition, project planning also produces the project schedules and resource allocations that are the framework for the other project management processes: tracking, reporting, and review." (Rob Thomsett, "Radical Project Management", 2002)

"Many management reports are not a management tool; they are merely memorandums of information. As a management tool, management reports should encourage timely action in the right direction, by reporting on those activities the Board, management, and staff need to focus on. The old adage 'what gets measured gets done' still holds true." (David Parmenter, "Pareto’s 80/20 Rule for Corporate Accountants", 2007)

"Reporting to the Board is a classic 'catch-22' situation. Boards complain about getting too much information too late, and management complains that up to 20% of their time is tied up in the Board reporting process. Boards obviously need to ascertain whether management is steering the ship correctly and the state of the crew and customers before they can relax and 'strategize' about future initiatives. The process of assessing the current status of the organization from the most recent Board report is where the principal problem lies. Board reporting needs to occur more efficiently and effectively for both the Board and management." (David Parmenter, "Pareto’s 80/20 Rule for Corporate Accountants", 2007)

"Readability in visualization helps people interpret data and make conclusions about what the data has to say. Embed charts in reports or surround them with text, and you can explain results in detail. However, take a visualization out of a report or disconnect it from text that provides context (as is common when people share graphics online), and the data might lose its meaning; or worse, others might misinterpret what you tried to show." (Nathan Yau, "Data Points: Visualization That Means Something", 2013)

"Another way to secure statistical significance is to use the data to discover a theory. Statistical tests assume that the researcher starts with a theory, collects data to test the theory, and reports the results - whether statistically significant or not. Many people work in the other direction, scrutinizing the data until they find a pattern and then making up a theory that fits the pattern." (Gary Smith, "Standard Deviations", 2014)

"These practices - selective reporting and data pillaging - are known as data grubbing. The discovery of statistical significance by data grubbing shows little other than the researcher’s endurance. We cannot tell whether a data grubbing marathon demonstrates the validity of a useful theory or the perseverance of a determined researcher until independent tests confirm or refute the finding. But more often than not, the tests stop there. After all, you won’t become a star by confirming other people’s research, so why not spend your time discovering new theories? The data-grubbed theory consequently sits out there, untested and unchallenged." (Gary Smith, "Standard Deviations", 2014)

"A dashboard is like the executive summary of a report. We read executive summaries and skip the body of the report if the summary is more or less in line with our expectations. Trouble is, measurement is never exhaustive. It is only when we dive in that we realize what areas may have been missed." (Sriram Narayan, "Agile IT Organization Design: For Digital Transformation and Continuous Delivery", 2015)

"'Getting it right the first time' is a rare achievement, and ascertaining the organization’s winning KPIs and associated reports is no exception. The performance measure framework and associated reporting is just like a piece of sculpture: you can be criticized on taste and content, but you can’t be wrong. The senior management team and KPI project team need to ensure that the project has a just-do-it culture, not one in which every step and measure is debated as part of an intellectual exercise." (David Parmenter, "Key Performance Indicators: Developing, implementing, and using winning KPIs" 3rd Ed., 2015)

"In order to get measures to drive performance, a reporting framework needs to be developed at all levels within the organization." (David Parmenter, "Key Performance Indicators: Developing, implementing, and using winning KPIs" 3rd Ed., 2015)

"Statistics, because they are numbers, appear to us to be cold, hard facts. It seems that they represent facts given to us by nature and it’s just a matter of finding them. But it’s important to remember that people gather statistics. People choose what to count, how to go about counting, which of the resulting numbers they will share with us, and which words they will use to describe and interpret those numbers. Statistics are not facts. They are interpretations. And your interpretation may be just as good as, or better than, that of the person reporting them to you." (Daniel J Levitin, "Weaponized Lies", 2017)

"Data may indeed be the new oil. But just like crude oil, data needs refining. It must be transformed into information. This is why we clean, combine, model, and visualize data. The output of all this work - whether you do it on your own, get some help, or use a (semi-)automatic process - includes reports and dashboards that provide insights into various aspects of the organization’s dealings, which decision-makers can then consume to make critical business decisions." (Jeroen ter Heerdt et al, "Microsoft Power BI Visual Calculations: Simplifying DAX", 2026)

🪙Business Intelligence: Measure (Just the Quotes)

"[…] when you can measure what you are speaking about, and express it in numbers, you know something about it; but when you cannot express it in numbers, your knowledge is of a meager and unsatisfactory kind; it may be the beginning of knowledge, but you have scarcely in your thoughts advanced to the state of science." (William T Kelvin, "Electrical Units of Measurement", 1883)

"Quantify. If whatever it is you’re explaining has some measure, some numerical quantity attached to it, you’ll be much better able to discriminate among competing hypotheses. What is vague and qualitative is open to many explanations." (Carl Sagan, "The Demon-Haunted World: Science as a Candle in the Dark", 1995)

"Clearly, the mean is greatly influenced by extreme values, but it can be appropriate for many situations where extreme values do not arise. To avoid misuse, it is essential to know which summary measure best reflects the data and to use it carefully. Understanding the situation is necessary for making the right choice. Know the subject!" (Herbert F Spirer et al, "Misused Statistics" 2nd Ed, 1998)

"Balanced Scorecard theory is flawed because it presents managers with a scorecard which gives no score – that is no single-valued measure how they have performed. Thus managers evaluated with such a system […] have no way to make principled or purposeful decisions." (Michael Jensen,"Value Maximization, Stakeholder Theory, and the Corporate Objective Function", Journal of Applied Corporate Finance, 2001)

"Changing measures are a particularly common problem with comparisons over time, but measures also can cause problems of their own. [...] We cannot talk about change without making comparisons over time. We cannot avoid such comparisons, nor should we want to. However, there are several basic problems that can affect statistics about change. It is important to consider the problems posed by changing - and sometimes unchanging - measures, and it is also important to recognize the limits of predictions. Claims about change deserve critical inspection; we need to ask ourselves whether apples are being compared to apples - or to very different objects." (Joel Best, "Damned Lies and Statistics: Untangling Numbers from the Media, Politicians, and Activists", 2001)

"Our culture, obsessed with numbers, has given us the idea that what we can measure is more important than what we can't measure. Think about that for a minute. It means that we make quantity more important than quality." (Donella Meadows, "Thinking in Systems: A Primer", 2008)

"What gets measured gets managed - even when it’s pointless to measure and manage it, and even if it harms the purpose of the organisation to do so." (Simon Caulkin, "The rule is simple: be careful what you measure", 2008) [source]

"What gets measured gets managed - so be sure you have the right measures, because the wrong ones kill." (Simon Caulkin, "The rule is simple: be careful what you measure", 2008) [source]

"Once these different measures of performance are consolidated into a single number, that statistic can be used to make comparisons […] The advantage of any index is that it consolidates lots of complex information into a single number. We can then rank things that otherwise defy simple comparison […] Any index is highly sensitive to the descriptive statistics that are cobbled together to build it, and to the weight given to each of those components. As a result, indices range from useful but imperfect tools to complete charades." (Charles Wheelan, "Naked Statistics: Stripping the Dread from the Data", 2012)

"Until a new metric generates a body of data, we cannot test its usefulness. Lots of novel measures hold promise only on paper." (Kaiser Fung, "Numbersense: How To Use Big Data To Your Advantage", 2013)

"Selecting the right measure and measuring things right are both art and science. And KPIs influence management behavior as well as business culture." (Pearl Zhu, "CIO Master: Unleash the Digital Potential of It", 2016)

"It’d be nice to fondly imagine that high-quality statistics simply appear in a spreadsheet somewhere, divine providence from the numerical heavens. Yet any dataset begins with somebody deciding to collect the numbers. What numbers are and aren’t collected, what is and isn’t measured, and who is included or excluded are the result of all-too-human assumptions, preconceptions, and oversights." (Tim Harford, "The Data Detective: Ten easy rules to make sense of statistics", 2020)

"People do care about how they are measured. What can we do about this? If you are in the position to measure something, think about whether measuring it will change people’s behaviors in ways that undermine the value of your results. If you are looking at quantitative indicators that others have compiled, ask yourself: Are these numbers measuring what they are intended to measure? Or are people gaming the system and rendering this measure useless?" (Carl T Bergstrom & Jevin D West, "Calling Bullshit: The Art of Skepticism in a Data-Driven World", 2020)

"As long as measurements are abused as a tool of control, measuring will remain the weakest area in a manager’s performance." (Peter Drucker)

"For although it is certainly true that quantitative measurements are of great importance, it is a grave error to suppose that the whole of experimental physics can be brought under this heading. We can start measuring only when we know what to measure: qualitative observation has to precede quantitative measurement, and by making experimental arrangements for quantitative measurements we may even eliminate the possibility of new phenomena appearing." (Heinrich B G Casimir)

🪙Business Intelligence: Analytics (Just the Quotes)

"Data are essential, but performance improvements and competitive advantage arise from analytics models that allow managers to predict and optimize outcomes. More important, the most effective approach to building a model rarely starts with the data; instead it originates with identifying the business opportunity and determining how the model can improve performance." (Dominic Barton & David Court, "Making Advanced Analytics Work for You", 2012) 

"Even with simple and usable models, most organizations will need to upgrade their analytical skills and literacy. Managers must come to view analytics as central to solving problems and identifying opportunities - to make it part of the fabric of daily operations." (Dominic Barton & David Court, "Making Advanced Analytics Work for You", 2012)

"There is another important distinction pertaining to mining data: the difference between (1) mining the data to find patterns and build models, and (2) using the results of data mining. Students often confuse these two processes when studying data science, and managers sometimes confuse them when discussing business analytics. The use of data mining results should influence and inform the data mining process itself, but the two should be kept distinct." (Foster Provost & Tom Fawcett, "Data Science for Business", 2013)

"In general, analytic processing is known as 'heuristic' processing. In heuristic processing the requirements for analysis are discovered by the results of the current iteration of processing. […] In heuristic processing you start with some requirements. You build a system to analyze those requirements. Then, after you have results, you sit back and rethink your requirements after you have had time to reflect on the results that have been achieved. You then restate the requirements and redevelop and reanalyze again. Each time you go through the redevelopment exercise is called an 'iteration'. You continue the process of building different iterations of processing until such time as you achieve the results that satisfy the organization that is sponsoring the exercise." (William H Inmon & Daniel Linstedt, "Data Architecture: A Primer for the Data Scientist: Big Data, Data Warehouse and Data Vault", 2015)

"It is important to remember that predictive data analytics models built using machine learning techniques are tools that we can use to help make better decisions within an organization and are not an end in themselves. It is paramount that, when tasked with creating a predictive model, we fully understand the business problem that this model is being constructed to address and ensure that it does address it." (John D Kelleher et al, "Fundamentals of Machine Learning for Predictive Data Analytics: Algorithms, worked examples, and case studies", 2015)

"Machine learning takes many different forms and goes by many different names: pattern recognition, statistical modeling, data mining, knowledge discovery, predictive analytics, data science, adaptive systems, self-organizing systems, and more. Each of these is used by different communities and has different associations. Some have a long half-life, some less so." (Pedro Domingos, "The Master Algorithm", 2015)

"The human side of analytics is the biggest challenge to implementing big data." (Paul Gibbons, "The Science of Successful Organizational Change", 2015)

"One important thing to bear in mind about the outputs of data science and analytics is that in the vast majority of cases they do not uncover hidden patterns or relationships as if by magic, and in the case of predictive analytics they do not tell us exactly what will happen in the future. Instead, they enable us to forecast what may come. In other words, once we have carried out some modelling there is still a lot of work to do to make sense out of the results obtained, taking into account the constraints and assumptions in the model, as well as considering what an acceptable level of reliability is in each scenario." (Jesús Rogel-Salazar, "Data Science and Analytics with Python", 2017)

"One of the biggest truths about the real–time analytics is that nothing is actually real–time; it's a myth. In reality, it's close to real–time. Depending upon the performance and ability of a solution and the reduction of operational latencies, the analytics could be close to real–time, but, while day-by-day we are bridging the gap between real–time and near–real–time, it's practically impossible to eliminate the gap due to computational, operational, and network latencies." (Shilpi Saxena & Saurabh Gupta, "Practical Real-time Data Processing and Analytics", 2017)

"The tension between bias and variance, simplicity and complexity, or underfitting and overfitting is an area in the data science and analytics process that can be closer to a craft than a fixed rule. The main challenge is that not only is each dataset different, but also there are data points that we have not yet seen at the moment of constructing the model. Instead, we are interested in building a strategy that enables us to tell something about data from the sample used in building the model." (Jesús Rogel-Salazar, "Data Science and Analytics with Python", 2017)

"Analytics provides a way to demonstrate the linkage between people and business outcomes. HR analytics (also called people analytics or talent analytics) use measurement and analysis techniques to understand, improve, and optimize the people side of business. Data are the raw numbers you track. [...] Metrics focus on counting, tracking, and presenting past data. Analytics uses statistics to help you see patterns in the data." (Shonna D Watters et al, "The Practical Guide for HR Analytics: Using data to inform, transform, and empower HR decisions", 2019)

"Data analytics is a powerful tool to increase the likelihood that you have the right problem. Both quantitative and qualitative data serve a purpose in supporting a hypothesis. They allow you to objectively measure and identify patterns and relationships." (Shonna D Watters et al, "The Practical Guide for HR Analytics: Using data to inform, transform, and empower HR decisions", 2019)

"Big data is revolutionizing the world around us, and it is easy to feel alienated by tales of computers handing down decisions made in ways we don’t understand. I think we’re right to be concerned. Modern data analytics can produce some miraculous results, but big data is often less trustworthy than small data. Small data can typically be scrutinized; big data tends to be locked away in the vaults of Silicon Valley. The simple statistical tools used to analyze small datasets are usually easy to check; pattern-recognizing algorithms can all too easily be mysterious and commercially sensitive black boxes." (Tim Harford, "The Data Detective: Ten easy rules to make sense of statistics", 2020)

"For advanced analytics, a well-designed data pipeline is a prerequisite, so a large part of your focus should be on automation. This is also the most difficult work. To be successful, you need to stitch everything together." (Piethein Strengholt, "Data Management at Scale: Best Practices for Enterprise Architecture", 2020)

"Data literacy is not a change in an individual’s abilities, talents, or skills within their careers, but more of an enhancement and empowerment of the individual to succeed with data. When it comes to data and analytics succeeding in an organization’s culture, the increase in the workforces’ skills with data literacy will help individuals to succeed with the strategy laid in front of them. In this way, organizations are not trying to run large change management programs; the process is more of an evolution and strengthening of individual’s talents with data. When we help individuals do more with data, we in turn help the organization’s culture do more with data." (Jordan Morrow, "Be Data Literate: The data literacy skills everyone needs to succeed", 2021)

"In the world of data and analytics, people get enamored by the nice, shiny object. We are pulled around by the wind of the latest technology, but in so doing we are pulled away from the sound and intelligent path that can lead us to data and analytical success. The data and analytical world is full of examples of overhyped technology or processes, thinking this thing will solve all of the data and analytical needs for an individual or organization. Such topics include big data or data science. These two were pushed into our minds and down our throats so incessantly over the past decade that they are somewhat of a myth, or people finally saw the light. In reality, both have a place and do matter, but they are not the only solution to your data and analytical needs. Unfortunately, though, organizations bit into them, thinking they would solve everything, and were left at the alter, if you will, when it came time for the marriage of data and analytical success with tools." (Jordan Morrow, "Be Data Literate: The data literacy skills everyone needs to succeed", 2021)

"Pure data science is the use of data to test, hypothesize, utilize statistics and more, to predict, model, build algorithms, and so forth. This is the technical part of the puzzle. We need this within easch organization. By having it, we can utilize the power that these technical aspects bring to data and analytics. Then, with the power to communicate effectively, the analysis can flow throughout the needed parts of an organization." (Jordan Morrow, "Be Data Literate: The data literacy skills everyone needs to succeed", 2021)

"A cloud data warehouse is an enterprise data warehouse offered as a managed service (PaaS) on public clouds with optimized integrations for data ingestion, analytics processing, and BI analytics." (Rukmani Gopalan, "The Cloud Data Lake: A Guide to Building Robust Cloud Data Architecture", 2022)

"Advanced analytics pivots around machine learning methods. Machine learning employs statistical learning methods to perform analysis on data. These statistical methods utilize algorithms that predict what may happen based on historical data or extract complex mathematical relationships from data to generate insights." (Pradeep Menon, "Data Lakehouse in Action", 2022)

"Data-driven organizations exhibit a culture of analytics. This cannot be confined to just a few premiere groups but rather to the entire organization. There are both cultural and technical challenges to overcome and this is where people, processes, and tools need to come together to bring around sustainable changes. Every business needs a strategy for business transformation." (Anindita Mahapatra, "Simplifying Data Engineering and Analytics with Delta", 2022)

"Data engineering is the process of converting raw data into analytics-ready data that is more accessible, usable, and consumable than its raw format. Modern companies are increasingly becoming data-driven, which means they use data to make business decisions to give them better insights into their customers and business operations. They can use these to improve profitability, reduce costs, and give them a competitive edge in the market. Behind the scenes, a series of tasks and processes are performed by a host of data personas who build reliable pipelines to source, transform, and analyze data so that it is a repeatable and mostly automated process." (Anindita Mahapatra, "Simplifying Data Engineering and Analytics with Delta", 2022)

"Enterprises typically become interested in data catalogs when they have a specific use case or need in mind. Data governance, self-service analytics, and cloud data migration are common examples. Having a specific need or use case helps focus efforts and measure impact. However, as with other technical efforts within enterprises, it is essential to prepare for long-term sustainable success and to have a plan to maximize successful adoption." (Fadi Maali & Jason Lim, "Implementing a Modern Data Catalog to Power Data Intelligence: Make Trustworthy Data Central to Your Organization", 2022)

"Graph analytics is generally very effective in clustering users, merchants, and communities to provide an effective implementation of behavior analytics. On the other hand, second-party fraud can be identified with the implementation of monitoring employee behavior, as well as compliance checks. Graph analytics can indeed be useful for these use cases. Similar to the first-party models, employee behavior can also be analyzed using graph machine learning, although the dataset may need to encode a number of other sources of information besides transactional data. From a compliance standpoint, process mining techniques that still rely on a graph representation of the various procedural steps/pathways can be effective in identifying fraudulent behavior or non-compliant processes. Finally, third-party fraud, especially in the form of phishing attacks, can also be addressed using graph machine learning. In this context, understanding the network from which the phishing attack comes as well as the URLs being used (which can also benefit from a graph representation) can be critical for building an effective phishing classifier." (Aldo Marzullo et al, "Graph Machine Learning" 2nd Ed., 2025)

"With Fabric, organizations can unlock the full potential of AI and machine learning in their data workflows. First, it provides users with all the tools necessary to create and deploy AI and machine learning models; users can use the frameworks and languages of their choice. Next, it enables these users to benefit from native integration of models that enrich the data present within Fabric with advanced cognitive analytics, such as vision and language, for example, and leverage the new capabilities of generative AI. Finally, it supports users at every stage of their data project with intelligent assistants that help create data integration flows, develop transformations or analyses, build data visualization reports, and even answer business questions by leveraging existing reports and semantic models to deliver contextual insights instantly." (Christopher Maneu et al, "The Definitive Guide to Microsoft Fabric From discovery to building a unified, secure, and scalable data platform", 2025)

30 November 2015

🪙Business Intelligence: Delta Lakes (Just the Quotes)

"Many argue that model drift is best monitored by monitoring the data drift in incoming data and the drift in the generated features. As and when the ground truth is available, it is joined by some primary key criteria with the inference data in a Delta table. Again, the update and merge operation support in Delta makes this a breeze. Now the actual and predicted values of the inference data are computed to see how well the model is doing in terms of the quality of insight generation. The feature engineering pipeline is completely in-house and is easier to monitor for drift. The model interpretability may indicate that some columns contributing to the predictive power are incorrect, and it may be necessary to add or remove features. In such cases, a threshold of tolerance is violated, which signals a need for model retraining." (Anindita Mahapatra, "Simplifying Data Engineering and Analytics with Delta", 2022)

"Traditional data lakes provide the necessary scalability, but not the real-time concurrency and latency needed for BI use cases. Delta comes to the rescue once again by providing performance at scale with a host of optimization techniques, such as caching, data compaction, and indexing. Previously, a subset of the curated data would be pushed to a warehouse to satisfy the latency and concurrency requirements of known queries. What this meant was that if a consumer needed a different access pattern or a slightly older dataset that was not available, they would have to request that their IT or data team get involved. This took data democratization a step backward. Ideally, we should allow people to access any data that they have privileges to. Delta Lake goes a step forward and allows BI tools to access data directly from the lake instead of accessing a sliver of the data in their expensive warehouses." (Anindita Mahapatra, "Simplifying Data Engineering and Analytics with Delta", 2022)

"Delta Lake brings capabilities such as transactional reliability and support for UPSERTs and MERGEs to data lakes while maintaining the dynamic horizontal scalability and separation of storage and compute of data lakes. Delta Lake is one solution for building data lakehouses, an open data architecture combining the best of data warehouses and data lakes." (Bennie Haelen & Dan Davis, "Delta Lake: Up and Running - Modern Data Lakehouse Architectures with Delta Lake", 2023)

"Like data lakes, the lakehouse architecture leverages low-cost cloud storage systems with the inherent flexibility and horizontal scalability of those systems. The goal of a lakehouse is to use existing high-performance data formats, such as Parquet, while also enabling ACID transactions (and other features). To add these capabilities, lakehouses use an open-table format, which adds features like ACID transactions, record-level operations, indexing, and key metadata to those existing data formats. This enables data assets stored on low-cost storage systems to have the same reliability that used to be exclusive to the domain of an RDBMS. Delta Lake is an example of an open-table format that supports these types of capabilities." (Bennie Haelen & Dan Davis, "Delta Lake: Up and Running - Modern Data Lakehouse Architectures with Delta Lake", 2023)

"The main goal of the transaction log is to enable multiple readers and writers to operate on a given version of a dataset file simultaneously and to provide additional information, like data skipping indexes to the execution engine for more performant operations. The Delta Lake transaction log always shows the user a consistent view of the data and serves as a single source of truth. It is the central repository that tracks all changes the user makes to a Delta table." (Bennie Haelen & Dan Davis, "Delta Lake: Up and Running - Modern Data Lakehouse Architectures with Delta Lake", 2023)

"Unstructured and semi-structured data are often critical for AI and machine learning use cases, whereas structured and semi-structured data are critical for BI use cases. Because it natively supports all three types of data classifications, you can create a unified system that supports these diverse workloads in a data lake. These workloads can complement each other in a well-designed processing architecture [...]. A data lake helps solve many of the challenges related to data volumes, types, and cost, and while Delta Lake runs on top of a data lake, it is optimized to run best on a cloud data lake." (Bennie Haelen & Dan Davis, "Delta Lake: Up and Running - Modern Data Lakehouse Architectures with Delta Lake", 2023)

"Delta Lake is a transactional storage software layer that runs on top of an existing data lake and adds RDW-like features that improve the lake’s reliability, security, and performance. Delta Lake itself is not storage. In most cases, it’s easy to turn a data lake into a Delta Lake; all you need to do is specify, when you are storing data to your data lake, that you want to save it in Delta Lake format (as opposed to other formats, like CSV or JSON)." (James Serra, "Deciphering Data Architectures", 2024)

"A lakehouse is a data storage space that hosts and manages all types of data in one place (structured, semi-struc-tured, and unstructured), allowing different tools to normalize and examine this data according to organizational requirements and/or individual choices. A lakehouse thus combines the best aspects of a data lake and a data warehouse by eliminating data duplication and friction related to ingestion, transformation, and sharing of data within the organization, all in the open format, Delta Lake." (Christopher Maneu et al, "The Definitive Guide to Microsoft Fabric From discovery to building a unified, secure, and scalable data platform", 2025)

27 November 2015

🪙Business Intelligence: Control (Just the Quotes)

"90 percent of all problems can be solved by using the techniques of data stratification, histograms, and control charts. Among the causes of nonconformance, only one-fifth or less are attributable to the workers." (Kaoru Ishikawa, The Quality Management Journal Vol. 1, 1993)

"Alert-level control is another feature used to establish relevance within the content domain. Alerts help manage exceptions and alert the user of any unusual change or threshold value reached for any KPI. So, the action resulting from alerts needs to be assigned to those users who need to be informed of the exceptions." (Shadan Malik, "Enterprise Dashboards: Design and best practices for IT", 2005)

"The chaos without a data strategy is not as obvious, but the indicators abound: dirty data, redundant data, inconsistent data, the inability to integrate, poor performance, terrible availability, little accountability, users who are increasingly dissatisfied with the performance of IT, and the general feeling that things are out of control." (Sid Adelman et al, "Data Strategy", 2005)

"Metrics can serve two purposes: identifying problems and measuring performance. When the goal is to identify problems and pinpoint areas of operational inefficiency and ineffectiveness, defining the right metric requires a bit of detective work. It requires you to uncover the data residue of a problem and to determine what evidence can be found and how exactly it shows up. When the goal is to measure performance, the right success metrics focus on measures that can be controlled and where improvement in the metric is an unambiguously good thing." (Zach Gemignani et al, "Data Fluency", 2014)

"When statisticians, trained in math and probability theory, try to assess likely outcomes, they demand a plethora of data points. Even then, they recognize that unless it’s a very simple and controlled action such as flipping a coin, unforeseen variables can exert significant influence." (Zachary Karabell, "The Leading Indicators: A short history of the numbers that rule our world", 2014)

"Data mart: A subset of a data warehouse that’s usually oriented to a business group or process rather than enterprise-wide views. They have value as part of the overall enterprise data architecture, but can cause problems when they sprout uncontrolled as data silos with their own data definitions, creating data shadow systems." (Rick Sherman, "Business Intelligence Guidebook: From Data Integration to Analytics, 2015)

"Measurement is often seen as a tool that helps reduce the complexity of the world. Organizations, with their uncertainty and confusion, are full of people, patterns and trends; and measurement seems to offer a promise of bringing order, rationality and control into this chaos." (Dina Gray et al, "Measurement Madness: Recognizing and avoiding the pitfalls of performance measurement", 2015)

"Data Mesh attempts to strike a balance between team autonomy and inter-term interoperability and collaboration, with a few complementary techniques. It gives domain teams autonomy to have control of their local decision making, such as choosing the best data model for their data products. While it uses the computational governance policies to impose a consistent experience across all data products; for example, standardizing on the data modeling language that all domains utilize." (Zhamak Dehghani, "Data Mesh: Delivering Data-Driven Value at Scale", 2021)

"Data mesh [...] reduces points of centralization that act as coordination bottlenecks. It finds a new way of decomposing the data architecture without slowing the organization down with synchronizations. It removes the gap between where the data originates and where it gets used and removes the accidental complexities - aka pipelines - that happen in between the two planes of data. Data mesh departs from data myths such as a single source of truth, or one tightly controlled canonical data model." (Zhamak Dehghani, "Data Mesh: Delivering Data-Driven Value at Scale", 2021)

"Modern complex organizations increasingly confront the challenge of ensuring data quality at scale without centralizing validation activities into a single bottlenecked team. The data mesh paradigm and federated controls emerge as pivotal architectural styles and organizational patterns that enable decentralized, self-serve data quality validation while preserving coherence and reliability across diverse data products." (William Smith, "Great Expectations for Modern Data Quality: The Complete Guide for Developers and Engineers", 2025)

"As long as measurements are abused as a tool of control, measuring will remain the weakest area in a manager’s performance." (Peter Drucker)

26 November 2015

🪙Business Intelligence: Trends (Just the Quotes)

"When numbers in tabular form are taboo and words will not do the work well as is often the case. There is one answer left: Draw a picture. About the simplest kind of statistical picture or graph, is the line variety. It is very useful for showing trends, something practically everybody is interested in showing or knowing about or spotting or deploring or forecasting." (Darell Huff, "How to Lie with Statistics", 1954)

"Alerts and KPI thresholds are two sides of the same coin. Alerts are actions taken once a KPI threshold is reached. However, alerts are not defined for every threshold boundary. For the most part, they serve as a warning system when a KPI shows poor performance or an undesired trend. Alerts must always be accompanied by attention-capturing actions such as automatic e-mails and/or visual indication such as blinking or animation on the dashboard. The other variable for alerts is the recipient. There may be one or more appropriate recipients for each alert." (Shadan Malik, "Enterprise Dashboards: Design and best practices for IT", 2005)

"For the purpose of choosing adequate forecasting methods, it is useful to dissect a time series into a systematic part and a non-systematic part. The systematic part is typically divided into three components: level , trend , and seasonality. The non-systematic part is called noise. The systematic components are assumed to be unobservable, as they characterize the underlying series, which we only observe with added noise." (Galit Shmueli, "Practical Time Series Forecasting: A Hands-On Guide", 2011)

"Some forecasting methods directly model these components by making assumptions about their structure. For example, a popular assumption about trend is that it is linear or exponential over parts, or all, of the given time period. Another common assumption is about the noise structure: many statistical methods assume that the noise follows a normal distribution. The advantage of methods that rely on such assumptions is that when the assumptions are reasonably met, the resulting forecasts will be more robust and the models more understandable." (Galit Shmueli, "Practical Time Series Forecasting: A Hands-On Guide", 2011)

"The process of visually exploring data can be summarized in a single sentence: find patterns and trends lurking in the data and then observe the deviations from those patterns. Interesting stories may arise from both the norm - also called the smooth - and the exceptions." (Alberto Cairo, "The Functional Art", 2011)

"Measurement is often seen as a tool that helps reduce the complexity of the world. Organizations, with their uncertainty and confusion, are full of people, patterns and trends; and measurement seems to offer a promise of bringing order, rationality and control into this chaos." (Dina Gray et al, "Measurement Madness: Recognizing and avoiding the pitfalls of performance measurement", 2015)

"Exploratory data analysis is the search for patterns and trends in a given data set. Visualization techniques play an important part in this quest. Looking carefully at your data is important for several reasons, including identifying mistakes in collection/processing, finding violations of statistical assumptions, and suggesting interesting hypotheses." (Steven S Skiena, "The Data Science Design Manual", 2017)

"These practices - selective reporting and data pillaging - are known as data grubbing. The discovery of statistical significance by data grubbing shows little other than the researcher’s endurance. We cannot tell whether a data grubbing marathon demonstrates the validity of a useful theory or the perseverance of a determined researcher until independent tests confirm or refute the finding. But more often than not, the tests stop there. After all, you won’t become a star by confirming other people’s research, so why not spend your time discovering new theories? The data-grubbed theory consequently sits out there, untested and unchallenged." (Gary Smith, "Standard Deviations", 2014)

"As presenters of data visualizations, often we just want our audience to understand something about their environment – a trend, a pattern, a breakdown, a way in which things have been progressing. If we ask ourselves what we want our audience to do with that information, we might have a hard time coming up with a clear answer sometimes. We might just want them to know something." (Ben Jones, "Avoiding Data Pitfalls: How to Steer Clear of Common Blunders When Working with Data and Presenting Analysis and Visualizations", 2020)

"Statistical metrics can show us facts and trends that would be impossible to see in any other way, but often they’re used as a substitute for relevant experience, by managers or politicians without specific expertise or a close-up view." (Tim Harford, "The Data Detective: Ten easy rules to make sense of statistics", 2020)

"Overall [...] everyone also has a need to analyze data. The ability to analyze data is vital in its understanding of product launch success. Everyone needs the ability to find trends and patterns in the data and information. Everyone has a need to ‘discover or reveal (something) through detailed examination’, as our definition says. Not everyone needs to be a data scientist, but everyone needs to drive questions and analysis. Everyone needs to dig into the information to be successful with diagnostic analytics. This is one of the biggest keys of data literacy: analyzing data." (Jordan Morrow, "Be Data Literate: The data literacy skills everyone needs to succeed", 2021)

"Data visualization, by contrast, focuses on the visual representation of data in such a way that its values, structure, nature, type, and variability are accurately expressed by means of graphs. It aims to support the exploration and understanding of data, the identi‑fication of patterns, trends, distributions, correlations, and anomalies, the communicationof insights, and aid in decision‑making." (Leandro N de Castro, "Exploratory Data Analysis: Descriptive Analysis, Visualization, and Dashboard Design", 2025)

"Differences in orientation can help us differentiate between items (e.g., data points, lines, objects, etc.) or extract information about the data. For example, using vertical bars in a bar chart can help differentiate between categories, while using horizontal bars can emphasize the magnitude of the data. Angles and direction can be used to convey information, such as trends, movement, sense of depth, or changes in values." (Leandro N de Castro, "Exploratory Data Analysis: Descriptive Analysis, Visualization, and Dashboard Design", 2025)

🪙Business Intelligence: Issues (Just the Quotes)

"Charts and graphs are a method of organizing information for a unique purpose. The purpose may be to inform, to persuade, to obtain a clear understanding of certain facts, or to focus information and attention on a particular problem. The information contained in charts and graphs must, obviously, be relevant to the purpose. For decision-making purposes. information must be focused clearly on the issue or issues requiring attention. The need is not simply for 'information', but for structured information, clearly presented and narrowed to fit a distinctive decision-making context. An advantage of having a 'formula' or 'model' appropriate to a given situation is that the formula indicates what kind of information is needed to obtain a solution or answer to a specific problem." (Cecil H Meyers, "Handbook of Basic Graphs: A modern approach", 1970)

"In using a database, first look at the metadata, then look at the data. [...] The old computer acronym GIGO (Garbage In, Garbage Out) applies to the use of large databases. The issue is whether the data from the database will answer the research question. In order to determine this, the investigator must have some idea about the nature of the data in the database - that is, the metadata." (Gerald van Belle, "Statistical Rules of Thumb", 2002)

"Dashboards and visualization are cognitive tools that improve your 'span of control' over a lot of business data. These tools help people visually identify trends, patterns and anomalies, reason about what they see and help guide them toward effective decisions. As such, these tools need to leverage people's visual capabilities. With the prevalence of scorecards, dashboards and other visualization tools now widely available for business users to review their data, the issue of visual information design is more important than ever." (Richard Brath & Michael Peters, "Dashboard Design: Why Design is Important," DM Direct, 2004)

"[…] dirt and stains are more noticeable on white or light-colored clothing. In the same way, dirty data and data quality issues have existed for a long time. But due to the inherent nature of operational data these issues have not been as visible or immense enough to affect the bottom line. Just as dark clothing hides spills and stains, dirty data have been hidden or ignored in operational data for decades." (Margaret Y Chu, "Blissful Data", 2004)

"Gauging the quality of the operational data becomes an important first step in predicting potential dirty data issues for an organization. But many organizations are reluctant to commit the time and expense to assess their data. Some organizations wait until dirty data issues blow up in their faces. The greater the pain being experienced, the bigger the commitment to improving data quality." (Margaret Y Chu, "Blissful Data", 2004)

"Visualization is an issue at the heart of good dashboard software. Good visualization can be the difference between information overload and information insight. Commonly used graphs (charts) are one example of visualization. However, present-day technology has raised the bar of visualization beyond commonplace charts and data widgets. The three key characteristics requiring evaluation within the area of visualization are: (1) Visual intelligence ( 2) Geographic mapping (3) Screen resolution." (Shadan Malik, "Enterprise Dashboards: Design and best practices for IT", 2005)

"Data governance is about putting people in charge of fixing and preventing data issues and using technology to help aid the process. Any time data is synchronized, merged, and exchanged, there have to be ground rules guiding this. Data governance serves as the method to organize the people, processes, and technologies for data-driven programs like data quality; they are a necessary part of any data quality effort." (Neera Bhansali, "Data Governance: Creating Value from Information Assets", 2014)

"Although performance measurement is often linked to tools such as scorecards, dashboards, performance targets, indicators and information systems, it would be naïve to consider the measurement of performance as just a technical issue. Indeed, measurement is often used as a way of attempting to bring clarity to complex and confusing situations." (Dina Gray et al, "Measurement Madness: Recognizing and avoiding the pitfalls of performance measurement", 2015)

"Data governance presents a clear shift in approach, signals a dedicated focus on data management, distinctly identifies accountability for data, and improves communication through a known escalation path for data questions and issues. In fact, data governance is central to data management in that it touches on essentially every other data management function. In so doing, organizational change will be brought to a group is newly - and seriously - engaging in any aspect of data management." (Mike Fleckenstein & Lorraine Fellows, "Modern Data Strategy", 2018)

"Data is owned by the enterprise, not by systems or individuals. The enterprise should recognize and formalize the responsibilities of roles, such as data stewards, with specific accountabilities for managing data. A data governance framework and guidelines must be developed to allow data stewards to coordinate with their peers and to communicate and escalate issues when needed. Data should be governed cooperatively to ensure that the interests of data stewards and users are represented and also that value to the enterprise is maximized." (Mike Fleckenstein & Lorraine Fellows, "Modern Data Strategy", 2018)

"The desire to collect as much data as possible must be balanced with an approximation of which data sources are useful to address a business issue. It is worth mentioning that often the value of internal data is high. Most internal data has been cleansed and transformed to suit the mission. It should not be overlooked simply because of the excitement of so much other available data." (Mike Fleckenstein & Lorraine Fellows, "Modern Data Strategy", 2018)

"Any change in the reporting requirement had to go through a long-winded process of data model changes, ETL code changes, and respective changes to the reporting system. Often, the ETL process was a specialized skill and became a bottleneck for reducing data to insight turnover time. The nature of analytics is unique. The more you see the output, the more you demand. Many EDW projects were deemed a failure. The failure was not from a technical perspective, but from a business perspective. Operationally, the design changes required to cater to these fast-evolving requirements were too difficult to handle." (Pradeep Menon, "Data Lakehouse in Action", 2022)

"It is often said that data scientists and data analysts spend only 20% of their time doing data analysis work, with 80% consumed by data 'issues'. The bulk of their time is spent finding, evaluating, understanding, and preparing data before analysis can begin. A data catalog inverts this principle by enabling data analysts and data scientists to spend 20% of their time looking for data and 80% performing analysis." (Fadi Maali & Jason Lim, "Implementing a Modern Data Catalog to Power Data Intelligence: Make Trustworthy Data Central to Your Organization", 2022)

"The main challenges include relentlessly chasing data issues that include schema and quality changes (data drift). Sometimes, fixing these issues can cause outages and delays to existing jobs. This is tied tightly to the underlying infrastructure, process, and technology and can be vulnerable to any changes there. For example, a temporary glitch in the cloud ecosystem will result in a failure of the data pipeline." (Anindita Mahapatra, "Simplifying Data Engineering and Analytics with Delta", 2022)

"Consumability is a very important requirement because it will directly affect the experience domain consumers will have in a streaming data mesh. If other domains cannot easily consume streaming data products, then they may opt out of the streaming data mesh and decide to build their own integrations by hand, bypassing any issues they encounter with the data mesh. Some factors to consider when ingesting data derivatives that will affect the consumability of other domains are as follows: (*) Lack of scalability (*) Lack of interoperability" (Hubert Dulay & Stephen Mooney, "Streaming Data Mesh", 2023)

"Choosing the right data ingestion strategy is a significant business decision that partially determines how well your organization can leverage its data for business decision making and operations. The stakes are high; the wrong strategy can lead to poor data quality, performance issues, increased costs, and even regulatory compliance breaches." (James Serra, "Deciphering Data Architectures", 2024)

"The transformation phase represents the most resource-intensive stage of most data projects, often consuming 60-80% of total project time and effort. This significant investment stems from the inherent complexity of converting raw, inconsistent data into clean, structured, and enriched information ready for business use. Every data quality issue must be identified and resolved, every business rule must be correctly implemented, and every integration point must be properly validated. This meticulous work serves as the essential bridge between raw data ingestion and meaningful business insights." (Christopher Maneu et al, "The Definitive Guide to Microsoft Fabric From discovery to building a unified, secure, and scalable data platform", 2025)

"To prevent issues and misunderstandings, it’s crucial to set clear standards and guidelines for creating data products across different teams. These guidelines should focus on important aspects of data product design, including interoperability, data quality, data modeling, metadata, and governance." (Piethein Strengholt, "Building Medallion Architectures: Designing with Delta Lake and Spark", 2025)

25 November 2015

🪙Business Intelligence: Governance (Just the Quotes)

"The data architecture is the most important technical aspect of your business intelligence initiative. Fail to build an information architecture that is flexible, with consistent, timely, quality data, and your BI initiative will fail. Business users will not trust the information, no matter how powerful and pretty the BI tools. However, sometimes it takes displaying that messy data to get business users to understand the importance of data quality and to take ownership of a problem that extends beyond business intelligence, to the source systems and to the organizational structures that govern a company’s data." (Cindi Howson, "Successful Business Intelligence: Secrets to making BI a killer App", 2008)

"Data lake is an ecosystem for the realization of big data analytics. What makes data lake a huge success is its ability to contain raw data in its native format on a commodity machine and enable a variety of data analytics models to consume data through a unified analytical layer. While the data lake remains highly agile and data-centric, the data governance council governs the data privacy norms, data exchange policies, and the ensures quality and reliability of data lake." (Saurabh Gupta et al, "Practical Enterprise Data Lake Insights", 2018)

"Data swamp, on the other hand, presents the devil side of a lake. A data lake in a state of anarchy is nothing but turns into a data swamp. It lacks stable data governance practices, lacks metadata management, and plays weak on ingestion framework. Uncontrolled and untracked access to source data may produce duplicate copies of data and impose pressure on storage systems." (Saurabh Gupta et al, "Practical Enterprise Data Lake Insights", 2018)

"Data Mesh attempts to strike a balance between team autonomy and inter-term interoperability and collaboration, with a few complementary techniques. It gives domain teams autonomy to have control of their local decision making, such as choosing the best data model for their data products. While it uses the computational governance policies to impose a consistent experience across all data products; for example, standardizing on the data modeling language that all domains utilize." (Zhamak Dehghani, "Data Mesh: Delivering Data-Driven Value at Scale", 2021)

"Data lakes have been in existence for a while now, so their need is no longer questioned. What is more relevant is the specifics of the solution's implementation. Consolidating all the siloed data by itself does not constitute a data lake. However, it is a starting point. Layering in governance makes the data consumable and is a step toward a curated data lake. Big data systems provide scale out of the box but force us to make some accommodations for data quality. Age-old aspects of transactional integrity were compromised on a distributed system because it was very hard to maintain ACID compliance. Due to this, BASE properties were favored. All of this was moving the needle in the wrong direction and from pristine data lakes we were moving toward data swamps, where the data could not be trusted and hence insights that were generated on the data could not be trusted either." (Anindita Mahapatra, "Simplifying Data Engineering and Analytics with Delta", 2022)

"A data fabric is a pattern that is very similar to a data mesh in that both provide solutions encompassing data governance and self-service: discovery, access, security, integration, transformation, and lineage. [...] In simple terms, a data fabric is a metadriven means of connecting disparate sets of data and related tools to provide a cohesive data experience and to deliver data in a self-service manner." (Hubert Dulay & Stephen Mooney, "Streaming Data Mesh", 2023)

"Data Fabric focuses on Self-Service data access via active metadata leveraging a composable set of tools and technologies. It offers the ability to discover, understand, and access data across hybrid and multi-cloud data landscapes with automation and Data Governance. It is primarily process and technology centric with flexibility in supporting diverse organizational models. On the other hand, Data Mesh is organizationally and process driven. It requires a technical implementation approach to execute its design. Data Mesh is at a higher level and Data Fabric is at a lower level. Data Fabric is capable of fulfilling Data Mesh’s key principles." (Sonia Mezzetta, "Principles of Data Fabric: Become a data-driven organization by implementing Data Fabric solutions efficiently", 2023)

"Data Fabric’s building blocks represent groupings of different components and characteristics. They are high-level blocks that describe a package of capabilities that address specific business needs. The building blocks are Data Governance and its knowledge layer, Data Integration, and Self-Service." (Sonia Mezzetta, "Principles of Data Fabric: Become a data-driven organization by implementing Data Fabric solutions efficiently", 2023)

"The allure of Data Lakes was their ability to store vast amounts of raw data. However, this advantage can become counterproductive without stringent governance and management protocols. In their zeal to harness the power of Big Data, some organizations indiscriminately dump data into their lakes. Without proper classification, curation, and quality checks, these lakes can become swamps - murky repositories filled with valuable data, redundant information, and outdated datasets. Navigating these data swamps becomes a significant challenge, leading to prolonged data retrieval times, increased chances of using obsolete or incorrect data, and a decline in the agility and efficiency of data-driven decision-making processes rather than facilitating quick and insightful analytics." (Pradeep Menon, "Data Mesh Principles, patterns, architecture, and strategies for data-driven decision making", 2024)

"A lakehouse combines the scalability and flexibility of a data lake with the governance, structure, and performance of a data warehouse. It allows organizations to store both structured and unstructured data in one platform while supporting robust analytics, machine learning, and BI workloads. These lakehouses get by the data swamp problem by providing features that are database like. That includes support for ACID transactions." (Fern Halper, "Data Makes the World Go 'Round", 2026)

"AI maturity depends as much on people as it does on tools. Organizations need to build AI literacy across roles, from analysts to executives, so that teams can ask the right questions, interpret AI results, and act on them. Cross-functional teams with skills in data engineering, machine learning, business domain knowledge, and governance are essential. AI systems should be treated as products and not one-off projects." (Fern Halper, "Data Makes the World Go 'Round", 2026)

Related Posts Plugin for WordPress, Blogger...

About Me

My photo
Koeln, NRW, Germany
IT Professional with more than 25 years experience in IT in the area of full life-cycle of Web/Desktop/Database Applications Development, Software Engineering, Consultancy, Data Management, Data Quality, Data Migrations, Reporting, ERP implementations & support, Team/Project/IT Management, etc.