🤖〽️Prompt Engineering: Copilot Unabridged (Part 135:Trust and Relationship in Contemporary AI Through the Lens of the DIKW Pyramid)

Prompt Engineering Series  

Prompt: "write a post of 600 words on how trust and relationship fit in nowadays AI considered from the perspective of DIKW pyramid and include an introduction, respectively a closing statement"

Introduction

As artificial intelligence becomes a daily companion - embedded in search engines, productivity tools, customer service, and creative work - the question of trust has moved to the center of the conversation. People don’t just want AI that is powerful; they want AI that is reliable, respectful, and predictable. Trust is the foundation of any meaningful relationship, and although AI is not a human partner, it still participates in relational dynamics. To understand how trust and relationship fit into modern AI, the DIKW pyramid (Data, Information, Knowledge, Wisdom) offers a useful lens. It reveals how trust is built - or los - at each stage of AI’s interaction with the world.

Trust at the Data Level

At the base of the DIKW pyramid lies data, and trust begins here. Users want to know that their data is handled responsibly, stored securely, and used ethically. Even though AI systems do not have intentions or emotions, the way data is collected and managed shapes the foundation of trust.

If data is biased, incomplete, or misused, trust erodes before the AI even speaks. Conversely, transparent data practices - clear boundaries, privacy protections, and responsible sourcing - create the first layer of relational confidence. Trust at this level is structural: it depends on the integrity of the system’s foundation.

Trust at the Information Level

When data becomes information, trust shifts toward clarity and predictability. AI systems must communicate in ways that are understandable, consistent, and context‑appropriate. Users expect:

• Clear explanations
• Stable behavior
• Honest acknowledgment of uncertainty
• Respectful tone

AI does not 'feel' trust, but it can behave in ways that foster it. Information-level trust is built through transparency - showing how the system interprets inputs, why it refuses certain requests, and how it handles sensitive topics. This is where the relationship begins to take shape: users start to understand what the AI can and cannot do.

Trust at the Knowledge Level

At the knowledge stage, AI connects information into coherent responses, predictions, or recommendations. This is where relational trust deepens. Users rely on AI to help them think, plan, and create. But trust at this level depends on:

• Reliability across diverse contexts
• Guardrails that prevent harmful outputs
• Consistency in reasoning
• Alignment with human expectations

AI can simulate knowledge, but it does not understand meaning. This makes trust fragile: users must feel confident that the system’s outputs are grounded in responsible design rather than arbitrary pattern‑matching. The relationship here is functional but meaningful - users trust the AI as a tool that behaves responsibly.

Trust at the Wisdom Level

Wisdom, the top of the DIKW pyramid, involves judgment, ethics, and long‑term thinking. AI does not possess wisdom, but it can support wise decision‑making by:

• Encouraging human oversight
• Avoiding speculation
• Recognizing sensitive boundaries
• Prioritizing safety and well‑being

This is where the relational dimension becomes most visible. Users trust AI not because it is wise, but because it behaves as if it understands the gravity of certain situations. It knows when to step back, when to defer to human expertise, and when to avoid harmful territory. This humility is essential for a healthy relationship between humans and AI.

Why Trust and Relationship Matter

Trust is not a technical feature; it is a human experience. AI systems cannot reciprocate trust, but they can behave in ways that make users feel safe, respected, and supported. The DIKW pyramid shows that trust is built layer by layer - from responsible data practices to wise behavior at the point of interaction.

Closing Statement

Trust and relationship in today’s AI are not about emotional bonds but about consistent, transparent, and responsible behavior. Through the DIKW pyramid, we see how trust emerges from the foundation of data, grows through clear information, strengthens through reliable knowledge, and is safeguarded by wise boundaries. AI cannot feel trust, but it can earn it through design choices that respect human values. In doing so, it becomes not just a tool, but a dependable partner in navigating an increasingly complex digital world.

Disclaimer: The whole text was generated by Copilot (under Windows 11) at the first attempt. This is just an experiment to evaluate feature's ability to answer standard general questions, independently on whether they are correctly or incorrectly posed. Moreover, the answers may reflect hallucinations and other types of inconsistent or incorrect reasoning.

Previous Post <<||>> Next Post

🤖〽️Prompt Engineering: Copilot Unabridged (Part 133: Intent and Purpose in Contemporary AI Through the Lens of the DIKW Pyramid)

Prompt Engineering Series

Prompt: "write a post of 600 words on how intent and purpose fit in nowadays AI considered from the perspective of DIKW pyramid and include an introduction, respectively a closing statement"

Introduction

Artificial Intelligence (AI) has become a powerful force in shaping how we work, communicate, and make decisions. Yet as AI systems grow more capable, a fundamental question becomes increasingly important: Where do intent and purpose fit in? Unlike humans, AI does not possess intrinsic goals, motivations, or desires. It operates according to patterns, instructions, and constraints defined by people. To understand how intent and purpose influence modern AI, the DIKW pyramid - Data, Information, Knowledge, Wisdom - offers a useful framework. It helps us see how human intention guides AI at every level, and why purpose remains a uniquely human contribution that AI cannot replicate.

Intent and Purpose at the Data Level

At the base of the DIKW pyramid lies data, the raw material of AI. Data itself has no intent. It is collected, labeled, and curated by humans who decide what to gather, what to exclude, and what the data is meant to represent. This is where purpose first enters the picture. The choice of data reflects human priorities: what we want the AI to learn, what problems we want it to solve, and what outcomes we hope to achieve.

For example, training a model on medical images expresses a purpose - improving diagnostics. Training a model on customer interactions expresses a different purpose - enhancing service or efficiency. AI does not choose these goals; humans do.

Intent and Purpose at the Information Level

When data is organized into information, intent becomes even more visible. Information is shaped by the metrics we choose, the patterns we highlight, and the questions we ask. AI can extract information, but it does not decide which information is meaningful. That decision reflects human intent.

For instance, accuracy, fairness, or safety metrics each reflect different priorities. Choosing one over another signals what we value. AI can compute these metrics, but it cannot determine which ones matter. Purpose guides the transformation of data into information.

Intent and Purpose at the Knowledge Level

Knowledge emerges when information is interpreted and applied. AI can simulate knowledge by generating explanations, making predictions, or offering recommendations. But the use of that knowledge is driven by human intent.

At this level, purpose shapes:

• How AI systems are deployed
• What decisions they support
• What boundaries they must respect
• How they adapt to different contexts

AI can provide knowledge-like output, but it does not understand why that knowledge is important. It cannot prioritize long-term outcomes, ethical considerations, or human well-being on its own. Purpose acts as the compass that directs how AI-generated knowledge is used.

Intent and Purpose at the Wisdom Level

Wisdom, the top of the DIKW pyramid, involves judgment, ethics, and the ability to navigate complexity. This is where intent and purpose become deeply human. Wisdom requires values, empathy, and moral reasoning - qualities AI does not possess.

AI can support wise decision-making by offering structured insights, highlighting risks, or encouraging reflection. But it cannot originate wisdom. It cannot determine what is right, what is fair, or what aligns with human flourishing. At this level, intent and purpose are inseparable from human agency.

Why Intent and Purpose Matter in Today’s AI

Intent and purpose are the invisible forces that shape every layer of the DIKW pyramid. They determine:

• What data is collected
• How information is interpreted
• How knowledge is applied
• How wisdom is exercised

AI amplifies human intent - whether good or bad. This makes clarity of purpose essential. Without it, AI risks becoming powerful but directionless, efficient but misaligned, capable but ungrounded.

Closing Statement

Intent and purpose are not optional add-ons to AI; they are the guiding principles that give AI meaning and direction. Through the lens of the DIKW pyramid, we see that AI can process data, generate information, and simulate knowledge, but it cannot define its own goals or values. That responsibility remains with us. By grounding AI in clear, thoughtful human intent, we ensure that these systems serve meaningful purposes and contribute positively to society.

Disclaimer: The whole text was generated by Copilot (under Windows 11) at the first attempt. This is just an experiment to evaluate feature's ability to answer standard general questions, independently on whether they are correctly or incorrectly posed. Moreover, the answers may reflect hallucinations and other types of inconsistent or incorrect reasoning.

Previous Post <<||>> Next Post

🤖〽️Prompt Engineering: Copilot Unabridged (Part 132: Meaning in Contemporary AI Through the Lens of the DIKW Pyramid)

Prompt Engineering Series

Prompt: "write a post of 600 words on how meaning fits in nowadays AI considered from the perspective of DIKW pyramid and include an introduction, respectively a closing statement"

Introduction

As Artificial Intelligence (AI) becomes increasingly woven into daily life, one question rises above the technical details: Can AI understand meaning? We know AI can process data, generate information, and even simulate knowledge. But meaning - the human ability to interpret, contextualize, and connect ideas - sits at the heart of how we make sense of the world. To explore how meaning fits into modern AI, the DIKW pyramid (Data, Information, Knowledge, Wisdom) offers a powerful lens. It helps us see not only what AI can do, but also where its limitations lie and how meaning emerges - or fails to emerge - within its structure.

Meaning at the Data Level

At the base of the DIKW pyramid lies data: raw symbols, numbers, words, pixels. For AI, data is not meaningful in itself. A sentence, an image, or a sound clip is simply a pattern to be processed. Meaning at this level is entirely absent. AI does not 'see' a cat in a picture; it detects statistical regularities that correlate with the label cat. Humans bring meaning to data through perception and experience. AI brings computation.

This distinction matters because it shows that meaning does not originate at the data level. It must be constructed higher up the pyramid.

Meaning at the Information Level

When data is organized and contextualized, it becomes information. AI excels here. It can summarize text, classify images, extract entities, and identify relationships. But even at this stage, meaning is still not fully present. AI can tell you what is in the data, but not why it matters.

For example, AI can identify that a sentence expresses sadness, but it does not feel sadness or understand the lived experience behind it. Meaning at the information level is functional rather than experiential. AI can manipulate information in ways that appear meaningful, but the meaning is inferred by humans, not generated by the system.

Meaning at the Knowledge Level

Knowledge emerges when information is connected, interpreted, and applied. This is where AI begins to approximate meaning more convincingly. Large language models can integrate patterns across vast amounts of text, generating responses that seem coherent, contextual, and relevant. They can explain concepts, draw analogies, and adapt to different tones.

But this is simulated meaning. AI does not possess understanding; it predicts plausible continuations based on patterns. It can generate knowledge‑like output without internal comprehension. Meaning at this level is relational: AI constructs connections between pieces of information, but the interpretation still resides with the human user.

Meaning at the Wisdom Level

Wisdom, the top of the DIKW pyramid, involves judgment, values, and the ability to navigate ambiguity. This is where meaning becomes deeply human. Wisdom requires lived experience, emotional depth, and ethical grounding - qualities AI does not possess.

AI can support wise decision‑making by providing structured information, highlighting risks, or encouraging reflection. But it cannot embody wisdom. It cannot assign meaning based on personal history or moral intuition. At this level, meaning is inseparable from human consciousness.

Where Meaning Truly Fits

Meaning in today’s AI is not something the system has; it is something the system enables. AI provides the scaffolding - data processing, pattern recognition, contextual generation - that allows humans to derive meaning more efficiently or more deeply. Meaning emerges in the interaction between human interpretation and AI‑generated structure.

In other words, AI participates in the process of meaning‑making without possessing meaning itself.

Closing Statement

Meaning in today’s AI is a collaborative phenomenon. The DIKW pyramid shows that while AI can transform data into information and simulate knowledge, true meaning—and the wisdom that flows from it - remains a uniquely human domain. AI can support, amplify, and organize meaning, but it cannot originate it. Understanding this distinction helps us design AI systems that enhance human insight rather than attempt to replace it. In a world increasingly shaped by intelligent tools, preserving the human role in meaning‑making is not just important - it is essential.

Disclaimer: The whole text was generated by Copilot (under Windows 11) at the first attempt. This is just an experiment to evaluate feature's ability to answer standard general questions, independently on whether they are correctly or incorrectly posed. Moreover, the answers may reflect hallucinations and other types of inconsistent or incorrect reasoning.

Previous Post <<||>> Next Post

🤖〽️Prompt Engineering: Copilot Unabridged (Part 131: What Is Missing from the DIKW Pyramid in Today’s AI?)

Prompt Engineering Series

Prompt: "write a post of 600 words on what is missing from DIKW pyramid besides facts and measurements in nowadays AI and include an introduction, respectively a closing statement"

Introduction

The DIKW pyramid (Data, Information, Knowledge, Wisdom) has long served as a conceptual model for understanding how raw facts evolve into meaningful insight. In the age of Artificial Intelligence), the DIKW framework is more relevant than ever, helping us think about how AI systems process inputs and generate outputs. Yet as AI grows more capable and more deeply embedded in society, it becomes increasingly clear that the DIKW pyramid, while useful, is incomplete. It captures the progression from data to wisdom, but it does not fully address the human, contextual, and relational elements that modern AI must navigate. To understand what AI still lacks, we must look at what lies outside the pyramid’s structure.

Beyond Data and Measurement: What the DIKW Pyramid Leaves Out

1. Meaning and Interpretation

The DIKW pyramid assumes that meaning naturally emerges as we move upward from data to wisdom. But in AI, meaning is not inherent - it is constructed. AI systems do not 'understand' in the human sense; they detect patterns. What is missing is the interpretive layer that humans apply automatically: cultural nuance, emotional tone, social context, and lived experience. These elements shape how people interpret information, but they are not explicitly represented in the DIKW model.

2. Human Intent and Purpose

The pyramid describes how information becomes knowledge, but not why it matters. AI systems operate without intrinsic goals or values; they rely on human-defined objectives. What’s missing is intentionality - the human purpose that gives information direction. Without understanding intent, AI can generate outputs that are technically correct but contextually misaligned. Purpose is the compass that guides wisdom, yet it sits outside the DIKW structure.

3. Ethics and Moral Judgment

Wisdom, as defined in the DIKW pyramid, implies good judgment. But the model does not explicitly address ethics, fairness, or moral reasoning. In today’s AI landscape, these are essential. AI systems must navigate sensitive topics, avoid harm, and respect human dignity. Ethical reasoning is not simply an extension of knowledge; it is a distinct dimension that requires principles, values, and societal norms. The DIKW pyramid does not capture this moral layer, yet it is indispensable for responsible AI.

4. Trust and Relationship

AI does not operate in a vacuum. It interacts with people, influences decisions, and shapes experiences. Trust - built through transparency, consistency, and responsible behavior - is a critical factor in how AI is perceived and adopted. The DIKW pyramid focuses on cognitive transformation, not relational dynamics. But trust is not data, information, knowledge, or wisdom; it is a social construct. Without it, even the most sophisticated AI systems fail to gain acceptance.

5. Creativity and Imagination

AI can generate novel combinations of patterns, but creativity in the human sense involves imagination, intuition, and the ability to envision possibilities beyond existing data. The DIKW pyramid is linear and hierarchical, while creativity is nonlinear and exploratory. Modern AI often operates within the boundaries of what it has seen. What’s missing is the capacity to transcend those boundaries with genuine originality.

6. Embodiment and Experience

Humans learn through physical experience - through senses, emotions, and interactions with the world. AI lacks embodiment. It does not feel, perceive, or experience. The DIKW pyramid does not account for this gap, yet embodiment shapes how humans develop wisdom. Without lived experience, AI’s 'wisdom' remains metaphorical, not literal.

Closing Statement

The DIKW pyramid remains a valuable framework for understanding how AI processes information, but it does not capture the full spectrum of what modern AI needs to navigate human complexity. Meaning, intent, ethics, trust, creativity, and lived experience all lie outside the pyramid’s structure, yet they are essential for building AI systems that are not only intelligent but also responsible and aligned with human values. Recognizing what is missing helps us design AI that complements human judgment rather than merely mimicking it. In doing so, we move toward a future where AI supports not just smarter decisions, but wiser ones.

Disclaimer: The whole text was generated by Copilot (under Windows 11) at the first attempt. This is just an experiment to evaluate feature's ability to answer standard general questions, independently on whether they are correctly or incorrectly posed. Moreover, the answers may reflect hallucinations and other types of inconsistent or incorrect reasoning.

Previous Post <<||>> Next Post

🧭💹Business Intelligence: Perspectives (Part 21: Data Visualization Revised)

Data Visualization Series

Creating data visualizations nowadays became so easy that anybody can do it with a minimum of effort and knowledge, which on one side is great for the creators but can be easily become a nightmare for the readers, respectively users. Just dumping data in visuals can be barely called data visualization, even if the result is considered as such. The problems of visualization are multiple – the lack of data culture, the lack of understanding processes, data and their characteristics, the lack of being able to define and model problems, the lack of educating the users, the lack of managing the expectations, etc.

There are many books on data visualization though they seem an expensive commodity for the ones who want rapid enlightenment, and often the illusion of knowing proves maybe to be a barrier. It's also true that many sets of data are so dull, that the lack of information and meaning is compensated by adding elements that give a kitsch look-and-feel (aka chartjunk), shifting the attention from the valuable elements to decorations. So, how do we overcome the various challenges? 

Probably, the most important step when visualizing data is to define the primary purpose of the end product. Is it to inform, to summarize or to navigate the data, to provide different perspectives at macro and micro level, to help discovery, to explore, to sharpen the questions, to make people think, respectively understand, to carry a message, to be artistic or represent truthfully the reality, or maybe is just a filler or point of attraction in a textual content?

Clarifying the initial purpose is important because it makes upfront the motives and expectations explicit, allowing to determine the further requirements, characteristics, and set maybe some limits in what concern the time spent and the qualitative and/or qualitative criteria upon which the end result should be eventually evaluated. Narrowing down such aspects helps in planning and the further steps performed. 

Many of the steps are repetitive and past experience can help reduce the overall effort. Therefore, professionals in the field, driven by intuition and experience probably don't always need to go through the full extent of the process. Conversely, what is learned and done poorly, has high chances of delivering poor quality. 

A visualization can be considered as effective when it serves the intended purpose(s), when it reveals with minimal effort the patterns, issues or facts hidden in the data, when it allows people to explore the data, ask questions and find answers altogether. One can talk also about efficiency, especially when readers can see at a glance the many aspects encoded in the visualization. However, the more the discovery process is dependent on data navigation via filters or other techniques, the more difficult it becomes to talk about efficiency.

Better criteria to judge visualizations is whether they are meaningful and useful for the readers, whether the readers understood the authors' intent, the further intrinsic implication, though multiple characteristics can be associated with these criteria: clarity, specificity, correctedness, truthfulness, appropriateness, simplicity, etc. All these are important in lower or higher degree depending on the broader context of the visualization.

All these must be weighted in the bigger picture when creating visualizations, though there are probably also exceptions, especially on the artistic side, where artists can cut corners for creating an artistic effect, though also in here the authors need to be truthful to the data and make sure that their work don't distort excessively the facts. Failing to do so might not have an important impact on the short term considerably, though in time the effects can ripple with unexpected effects.

Previous Post <<||>> Next Post

🧭💹Business Intelligence: Perspectives (Part 19: Data Visualization between Art, Pragmatism and Kitsch)

Business Intelligence Series

The data visualizations (aka dataviz) presented in the media, especially the ones coming from graphical artists, have the power to help us develop what is called graphical intelligence, graphical culture, graphical sense, etc., though without a tutor-like experience the process is suboptimal because it depends on our ability of identifying what is important and which are the steps needed for decoding and interpreting such work, respectively for integrating their messages in our overall understanding about the world.

When such skillset is lacking, without explicit annotations or other form of support, the reader might misinterpret or fail to observe important visual cues even for simple visualizations, with all the implications deriving from this – a false understanding, and further aspects deriving from it, this being probably the most important aspect to consider. Unfortunately, even the most elaborate work can fail if the reader doesn’t have a basic understanding of all that’s implied in the process.

The books of Willard Brinton, Ana Rogers, Jacques Bertin, William Cleveland, Leland Wilkinson, Stephen Few, Albert Cairo, Soctt Berinato and many others can help the readers build a general understanding of the dataviz process and how data visualizations or simple graphics can be used/misused effectively, though each reader must follow his/her own journey. It’s also true that the basics can be easily learned, though the deeper one dives, the more interesting and nontrivial the journey becomes. Fortunately, the average reader can stick to the basics and many visualizations are simple enough to be understood.

To grasp the full extent of the implications, one can make comparisons with the domain of poetry where the author uses basic constructs like metaphor, comparisons, rhythm and epithets to create, communicate and imprint in reader’s mind old and new meanings, images and feelings altogether. Artistic data visualizations tend to offer similar charge as poetry does, even if the impact might not appeal so much to our artistic sensibility. Though dataviz from this perspective is or at least resembles an art form.

Many people can write verses, though only a fraction can write good meaningful poetry, from which a smaller fraction get poems, respectively even fewer get books published. Conversely, not everything can be expressed in verses unless one finds good metaphors and other aspects that can be leveraged in the process. Same can be said about good dataviz.

One can argue that in dataviz the author can explore and learn especially by failing fast (seeing what works and what doesn’t). One can also innovate, though the creator has probably a limited set of tools and rules for communication. Enabling readers to see the obvious or the hidden in complex visualizations or contexts requires skill and some kind of mastery of the visual form.

Therefore, dataviz must be more pragmatic and show the facts. In art one has the freedom to distort or move things around to create new meanings, while in dataviz it’s important for the meaning to be rooted in 'truth', at least by definition. The more the creator of a dataviz innovates, the higher the chances of being misunderstood. Moreover, readers need to be educated in interpreting the new meanings and get used to their continuous use.

Kitsch is a term applied to art and design that is perceived as naïve imitation to the degree that it becomes a waste of resources even if somebody pays the tag price. There’s a trend in dataviz to add elements to visualizations that don’t bring any intrinsic value – images, colors and other elements can be misused to the degree that the result resembles kitsch, and the overall value of the visualization is diminished considerably.

Previous Post <<||>> Next Post

📊Graphical Representation: Graphics We Live By II (Discount Rates in MS Excel)

Graphical Representation Series

It's difficult, if not impossible, to give general rules on how data visualizations should be built. However, the data professional can use a set of principles, which are less strict than rules, and validate one's work against them. Even then one might need to make concessions and go against the principles or common sense, though such cases should be few, at least in theory. One of such important principles is reflected in Tufte's statement that "if the statistics are boring, then you've got the wrong numbers" [1].

So, the numbers we show should not be boring, but that's the case with most of the numbers we need to show, or we consume in news and other media sources. Unfortunately, in many cases we need to go with the numbers we have and find a way to represent them, ideally by facilitating the reader to make sense of the respective data. This should be our first goal when visualizing data. Secondly, because everybody talks about insights nowadays, one should identify the opportunity for providing views into the data that go beyond the simple visualization, even if this puts more burden on data professional's shoulder. Frankly, from making sense of a set of data and facilitating an 'Aha' moment is a long leap. Thirdly, one should find or use the proper tools and channels for communicating the findings. 

A basic requirement for the data professional to be able to address these goals is to have clear definitions of the numbers, have a good understanding of how the numbers reflect the reality, respectively how the numbers can be put into the broader context. Unfortunately, all these assumptions seem to be a luxury. On the other side, the type of data we work with allows us to address at least the first goal. Upon case, our previous experience can help, though there will be also cases in which we can try to do our best. 

Let's consider a simple set of data retrieved recently from another post - Discount rates (in percentage) per State, in which the values for 5 neighboring States are considered (see the first two columns from diagram A). Without knowing the meaning of data, one could easily create a chart in Excel or any other visualization tool. Because the State has categorical values, probably some visualization tools will suggest using bar and not column charts. Either by own choice or by using the default settings, the data professional might end up with a column chart (see diagram B), which is Ok for some visualizations. 

One can start with a few related questions:

(1) Does it make sense to use a chart to represent 5 values which have small variability (the difference between the first and last value is of only 6%)? 

(2) Does it make sense to use a chart only for the sake of visualizing the data?

(3) Where is the benefit for using a chart as long there's no information conveyed? 

One can see similar examples in the media where non-aggregated values are shown in a chart just for the sake of visualizing the data. Sometimes the authors compensate for the lack of meaning with junk elements, fancy titles or other tricks. Usually, sense-making in a chart takes longer than looking at the values in a table as there are more dimensions or elements to consider. For a table there's the title, headers and the values, nothing more! For a chart one has in addition the axes and some visualization elements that can facilitate or complicate visualization's decoding. Where to add that there are also many tricks to distort the data. 

Tables tend to maximize the amount of digital ink used to represent the data, and minimize the amount used to represent everything else not important to understanding. It's what Tufte calls the data-to-ink ratio (see [1]), a second important principle. This can be translated in (a) removing the border of the chart area, (b) minimizing the number of gridlines shown, (c) minimizing the number of ticks on the axis without leading to information lost, (d) removing redundant information, (e) or information that doesn't help the reader. 

However, the more data is available in the table, the more difficult it becomes to navigate the data. But again, if the chart shows the individual data without any information gained, a table might be still more effective. One shouldn't be afraid to show a table where is the case!

(4) I have a data visualization, what's next?

Ideally, the data professional should try to obtain the maximum of effect with minimum of elements. If this principle aims for the efficiency of design, a fourth related principle aims for the efficiency of effort - one should achieve a good enough visualization with a minimum of effort. Therefore, it's enough maybe if we settle to any of the two above results. 

On the other side, maybe by investing a bit more effort certain aspects can be improved. In this area beginners start playing with the colors, formatting the different elements of the chart. Unfortunately, even if color plays a major role in the encoding and decoding of meaning, is often misused/overused. 

(5) Is there any meaning in the colors used?

In the next examples taken from the web (diagram C and D), the author changed the color of the column with the minimal value to red to contrast it with the other values. Red is usually associated with danger, error, warning, or other similar characteristics with negative impact. The chances are high that the reader will associate the value with a negative connotation, even if red is used also for conveying important information (usually in text). Moreover, the reader will try to interpret the meaning of the other colors. In practice, the color grey has a neutral tone (and calming effect on the mind). Therefore, it's safe to use grey in visualization (see diagram D in contrast with diagram C). Some even advise setting grey as default for the visualization and changing the colors as needed later. 

In these charts, the author signalized in titles that red denotes the lowest value, though it just reduces the confusion. One can meet titles in which several colors are used, reminding of a Christmas tree. Frankly, this type of encoding is not esthetically pleasing, and it can annoy the reader. 

(6) What's in a name?

The titles and, upon case, the subtitles are important elements in communicating what the data reflects. The title should be in general short and succinct in the information it conveys, having the role of introducing, respectively identifying the chart, especially when multiple charts are used. Some charts can also use a subtitle, which can be longer than the title and have more of a storytelling character by highlighting the message and/or the finding in the data. In diagrams C and D the subtitles were considered as tiles, which is not considerably wrong. 

In the media and presentations with influencing character, subtitles help the user understand the message or the main findings, though it's not appropriate for hardcoding the same in dynamic dashboards. Even if a logic was identified to handle the various scenarios, this shifts users' attention, and the chance is high that they'll stop further investigating the visualization. A data professional should present the facts with minimal interference in how the audience and/or users perceive the data. 

As a recommendation, one should aim for clear general titles and avoid transmitting own message in charts. As a principle this can be summarized as "aim for clarity and equidistance".

(7) What about meaning?

Until now we barely considered the meaning of data. Unfortunately, there's no information about what the Discount rate means. It could be "the minimum interest rate set by the US Federal Reserve (and some other national banks) for lending to other banks" or "a rate used for discounting bills of exchange", to use the definitions given by the Oxford dictionary. Searching on the web, the results lead to discount rates for royalty savings, resident tuitions, or retail for discount transactions. Most probably the Discount rates from the data set refer to the latter.

We need a definition of the Discount rate to understand what the values represent when they are ordered. For example, when Texas has a value of 25% (see B), does this value have a negative or a positive impact when compared with other values? It depends on how it's used in the associated formula. The last two charts consider that the minimum value has a negative impact, though without more information the encoding might be wrong! 

Important formulas and definitions should be considered as side information in the visualization, accompanying text or documentation! If further resources are required for understanding the data, then links to the required resources should be provided as well. At least this assures that the reader can acquire the right information without major overhead. 

(8) What do readers look for? 

Frankly, this should have been the first question! Readers have different expectations from data visualizations. First of all, it's the curiosity - how the data look in row and/or aggregated form, or in more advanced form how are they shaped (e.g. statistical characteristics like dispersion, variance, outliers). Secondly, readers look in the first phase to understand mainly whether the "results" are good or bad, even if there are many shades of grey in between. Further on, there must be made distinction between readers who want to learn more about the data, models, and processes behind, respectively readers who just want a confirmation of their expectations, opinions and beliefs (aka bias). And, in the end, there are also people who are not interested in the data and what it tells, where the title and/or subtitle provide enough information. 

Besides this there are further categories of readers segmented by their role in the decision making, the planning and execution of operational, tactical, or strategic activities. Each of these categories has different needs. However, this exceeds the scope of our analysis. 

Returning to our example, one can expect that the average reader will try to identify the smallest and highest Discount rates from the data set, respectively try to compare the values between the different States. Sorting the data and having the values close to each other facilitates the comparison and ranking, otherwise the reader needing to do this by himself/herself. This latter aspect and the fact that bar charts better handle the display of categorical data such as length and number, make from bar charts the tool of choice (see diagram E). So, whenever you see categorical data, consider using a bar chart!

Despite sorting the data, the reader might still need to subtract the various values to identify and compare the differences. The higher the differences between the values, the more complex these operations become. Diagram F is supposed to help in this area, the comparison to the minimal value being shown in orange. Unfortunately, small variances make numbers' display more challenging especially when the visualization tools don't offer display alternatives.

For showing the data from Diagram F were added in the table the third and fourth columns (see diagram A). There's a fifth column which designates the percentage from a percentage (what's the increase in percentages between the current and minimal value). Even if that's mathematically possible, the gain from using such data is neglectable and can create confusion. This opens the door for another principle that applies in other areas as well: "just because you can, it doesn't mean you should!". One should weigh design decisions against common sense or one's intuition on how something can be (mis)used and/or (mis)understood!

The downside of Diagram F is that the comparisons are made only in relation to the minimum value. The variations are small and allow further comparisons. The higher the differences, the more challenging it becomes to make further comparisons. A matrix display (see diagram G) which compares any two values will help if the number of points is manageable. The upper side of the numbers situated on and above the main diagonal were grayed (and can be removed) because they are either nonmeaningful, or the negatives of the numbers found below the diagram. Such diagrams are seldom used, though upon case they prove to be useful.

Choropleth maps (diagram H) are met almost everywhere data have a geographical dimension. Like all the other visuals they have their own advantages (e.g. relative location on the map) and disadvantages (e.g. encoding or displaying data). The diagram shows only the regions with data (remember the data-to-ink ratio principle).

(9) How about the shape of data?

When dealing with numerical data series, it's useful to show aggregated summaries like the average, quartiles, or standard deviation to understand how the data are shaped. Such summaries don't really make sense for our data set given the nature of the numbers (five values with small variance). One can still calculate them and show them in a box plot, though the benefit is neglectable. 

(10) Which chart should be used?

As mentioned above, each chart has advantages and disadvantages. Given the simplicity and the number of data points, any of the above diagrams will do. A table is simple enough despite not using any visualization effects. Also, the bar charts are simple enough to use, with a plus maybe for diagram F which shows a further dimension of the data. The choropleth map adds the geographical dimension, which could be important for some readers. The matrix table is more appropriate for technical readers and involves more effort to understand, at least at first sight, though the learning curve is small. The column charts were considered only for exemplification purposes, though they might work as well. 

In the end one should go with own experience and consider the audience and the communication channels used. One can also choose 2 different diagrams, especially when they are complementary and offer an additional dimension (e.g. diagrams F and H), though the context may dictate whether their use is appropriate or not. The diagrams should be simple to read and understand, but this doesn't mean that one should stick to the standard visuals. The data professional should explore other means of representing the data, a fresh view having the opportunity of catching the reader's attention.

As a closing remark, nowadays data visualization tools allow building such diagrams without much effort. Conversely, it takes more effort to go beyond the basic functionality and provide more value for thyself and the readers. One should be able to evaluate upfront how much time it makes sense to invest. Hopefully, the few methods, principles and recommendations presented here will help further!

Previous Post <<||>> Next Post

Resources:
[1] Edward R Tufte (1983) "The Visual Display of Quantitative Information"

📉Graphical Representation: Meaning (Just the Quotes)

"It is desirable in all chart work to have certain conventions by which colors would be understood to have certain definite meanings. Thus, following railroad practice, red could generally be used in chart work to indicate dangerous or unfavorable conditions, and green to indicate commended features or favorable conditions. Where neither commendation nor adverse criticism is intended, colors such as blue, yellow, brown, etc., could be used." (Willard C Brinton, "Graphic Methods for Presenting Facts", 1919)

"Graphic forms help us to perform and influence two critical functions of the mind: the gathering of information and the processing of that information. Graphs and charts are ways to increase the effectiveness and the efficiency of transmitting information in a way that enhances the reader's ability to process that information. Graphics are tools to help give meaning to information because they go beyond the provision of information and show relationships, trends, and comparisons. They help to distinguish which numbers and which ideas are more important than others in a presentation." (Robert Lefferts, "Elements of Graphics: How to prepare charts and graphs for effective reports", 1981)

"The more complex the shape of any object. the more difficult it is to perceive it. The nature of thought based on the visual apprehension of objective forms suggests, therefore, the necessity to keep all graphics as simple as possible. Otherwise, their meaning will be lost or ambiguous, and the ability to convey the intended information and to persuade will be inhibited." (Robert Lefferts, "Elements of Graphics: How to prepare charts and graphs for effective reports", 1981)

"Understanding is accomplished through: (a) the use of relative size of the shapes used in the graphic; (b) the positioning of the graphic-line forms; (c) shading; (d) the use of scales of measurement; and (e) the use of words to label the forms in the graphic. In addition, in order for a person to attach meaning to a graphic it must also be simple, clear, and appropriate." (Robert Lefferts, "Elements of Graphics: How to prepare charts and graphs for effective reports", 1981)

"There are two kinds of misrepresentation. In one. the numerical data do not agree with the data in the graph, or certain relevant data are omitted. This kind of misleading presentation. while perhaps hard to determine, clearly is wrong and can be avoided. In the second kind of misrepresentation, the meaning of the data is different to the preparer and to the user." (Anker V Andersen, "Graphing Financial Information: How accountants can use graphs to communicate", 1983)

"Understandability implies that the graph will mean something to the audience. If the presentation has little meaning to the audience, it has little value. Understandability is the difference between data and information. Data are facts. Information is facts that mean something and make a difference to whoever receives them. Graphic presentation enhances understanding in a number of ways. Many people find that the visual comparison and contrast of information permit relationships to be grasped more easily. Relationships that had been obscure become clear and provide new insights." (Anker V Andersen, "Graphing Financial Information: How accountants can use graphs to communicate", 1983)

"There is a technical difference between a bar chart and a histogram in that the number represented is proportional to the length of bar in the former and the area in the latter. This matters if non-uniform binning is used. Bar charts can be used for qualitative or quantitative data, whereas histograms can only be used for quantitative data, as no meaning can be attached to the width of the bins if the data are qualitative." (Roger J Barlow, "Statistics: A guide to the use of statistical methods in the physical sciences", 1989)

"The more clues to meaning that are supplied elsewhere, the less the need for cluttersome scales." (Eric Meyer, "Designing Infographics", 1997) 

"[...] the form of a technological object must depend on the tasks it should help with. This is one of the most important principles to remember when dealing with infographics and visualizations: The form should be constrained by the functions of your presentation. There may be more than one form a data set can adopt so that readers can perform operations with it and extract meanings, but the data cannot adopt any form. Choosing visual shapes to encode information should not be based on aesthetics and personal tastes alone." (Alberto Cairo, "The Functional Art", 2011)

"To keep accuracy and efficiency of your diagrams appealing to a potential audience, explicitly describe the encoding principles we used. Titles, labels, and legends are the most common ways to define the meaning of the diagram and its elements." (Vasily Pantyukhin, "Principles of Design Diagramming", 2015)

"Essentially, magnitude is the size of the effect. It’s a way to determine if the results are meaningful. Without magnitude, it’s hard to get a sense of how much something matters. […] the magnitude of an effect can change, depending on the relationship." (John H Johnson & Mike Gluck, "Everydata: The misinformation hidden in the little data you consume every day", 2016)

"Are your insights based on data that is accurate and reliable? Trustworthy data is correct or valid, free from significant defects and gaps. The trustworthiness of your data begins with the proper collection, processing, and maintenance of the data at its source. However, the reliability of your numbers can also be influenced by how they are handled during the analysis process. Clean data can inadvertently lose its integrity and true meaning depending on how it is analyzed and interpreted." (Brent Dykes, "Effective Data Storytelling: How to Drive Change with Data, Narrative and Visuals", 2019)

"Before you can even consider creating a data story, you must have a meaningful insight to share. One of the essential attributes of a data story is a central or main insight. Without a main point, your data story will lack purpose, direction, and cohesion. A central insight is the unifying theme (telos appeal) that ties your various findings together and guides your audience to a focal point or climax for your data story. However, when you have an increasing amount of data at your disposal, insights can be elusive. The noise from irrelevant and peripheral data can interfere with your ability to pinpoint the important signals hidden within its core." (Brent Dykes, "Effective Data Storytelling: How to Drive Change with Data, Narrative and Visuals", 2019)

"When narrative is coupled with data, it helps to explain to your audience what’s happening in the data and why a particular insight is important. Ample context and commentary are often needed to fully appreciate an analysis finding. The narrative element adds structure to the data and helps to guide the audience through the meaning of what’s being shared." (Brent Dykes, "Effective Data Storytelling: How to Drive Change with Data, Narrative and Visuals", 2019)