16 November 2026

🤖Prompt Engineering: Complexity (Just the Quotes)

"Chain-of-thought prompting is a method that forces LLMs to reason through a series of steps, resulting in more structured, transparent, and precise outputs. The goal is to break down complex tasks into smaller, interconnected subtasks, allowing the LLM to address each subtask in a stepby-step manner. This not only helps the model to 'focus' on specific aspects of the problem, but also encourages it to generate intermediate outputs, making it easier to identify and debug potential issues along the way. Another significant advantage of chain-of-thought prompting is the improved interpretability and transparency of the LLM-generated response. By offering insights into the model’s reasoning process, we, as users, can better understand and qualify how the final output was derived, which promotes trust in the model’s decision-making abilities." (Sinan Ozdemir, "Quick Start Guide to Large Language Models", 2024)

"Large language models (LLMs) are AI models that are usually (but not necessarily) derived from the Transformer architecture and are designed to understand and generate human language, code, and much more. These models are trained on vast amounts of text data, allowing them to capture the complexities and nuances of human language. LLMs can perform a wide range of language-related tasks, from simple text classification to text generation, with high accuracy, fluency, and style." (Sinan Ozdemir, "Quick Start Guide to Large Language Models: Strategies and Best Practices for Using ChatGPT and Other LLMs", 2024) 

"Underfitting in machine learning is a condition in which a model is too simple to capture the underlying patterns in the training data, leading to poor performance on both the training and test data. It typically occurs when the model lacks sufficient complexity or when it is not trained for long enough. In the context of LLMs, underfitting could happen if the model fails to grasp the context or subtleties of the training data, resulting in outputs that are too general, off-topic, or nonsensical in response to prompts." (Sinan Ozdemir, "Quick Start Guide to Large Language Models: Strategies and Best Practices for Using ChatGPT and Other LLMs", 2024)

"A knowledge graph is a data structure that uses nodes to represent concepts and entities and relationships to connect these nodes. [...] Knowledge graphs are highly versatile, capable of storing both structured information (such as employee details, task statuses, and company hierarchies) and unstructured information (such as article contents). This dual capability [...] makes them uniquely suited for complex RAG applications. [...] By integrating structured and unstructured information in the same framework, knowledge graphs enable the seamless blending of both worlds, making them a powerful tool for answering a broad range of questions efficiently and accurately in RAG applications. Moreover, explicit connec-tions between unstructured and structured data unlock advanced retrieval strategies such as linking entities in text to graph nodes or contextualizing structured results with source passages that would be difficult or impossible to achieve using either type of data alone."(Tomaz Bratanic & Oskar Hane,"Essential GraphRAG: Knowledge Graph-Enhanced RAG", 2025)

"Agentic intelligence feels incredibly powerful in demos but breaks in production. Indeed, it is very fragile without solid infrastructure. Every day, I personally see tons of clever orchestrations around dumb prompt chains tied up in a brittle, underused LLMOps infrastructure. But building this infrastructure means acknowledging the costs: performance overhead, strict interface contracts, and state complexity, as well as a need for more LLMOps engineers to create the best practices, tooling, and frameworks to run these systems reliably, safely, and robustly." (Abi Aryan, "LLMOps: Managing Large Language Models in Production", 2025)

"Agentic workflows break when the logic is messy - if, say, the plans don’t decompose or memory is poorly structured. However, infrastructure-level LLM applications introduce even more failure points and complexity. If the protocols don’t sync with each other, or the data flows start leaking, or the model boundaries are unclear... there are far too many failure points to count. While most people have been jumping on the bandwagon to adopt MCPs or A2A, very few are equipped to handle the LLMOps issues these tools introduce." (Abi Aryan, "LLMOps: Managing Large Language Models in Production", 2025)

"An embedding is a way to represent complex data as a set of numbers in a simpler, lower-dimensional space. Think of it as translating data into a format that a computer can easily understand and work with. Embedding models provide a uniform way to represent different types of data. Input to an embedding model can be any complex data, and the output is a vector. For instance, in dealing with text, an embedding model will take words or sentences and turn them into vectors, which are lists of numbers. The model is trained to ensure that these number lists capture essential aspects of the original words, such as their meaning or context." (Tomaz Bratanic & Oskar Hane,"Essential GraphRAG: Knowledge Graph-Enhanced RAG", 2025)

"Another problem that can be confusing is that LLMs seldom put out the same thing twice. [...] Traditional databases are straightforward - you ask for something specific, and you get back exactly what was stored. Search engines work similarly, finding existing information. LLMs work differently. They analyze massive amounts of text data to understand statistical patterns in language. The model processes information through multiple layers, each capturing different aspects - from simple word patterns to complex relationships between ideas." (Jeremy C Morgan, "Coding with AI: Examples in Python", 2025)

"In prompt engineering, we customize the prompts or questions we give the model to get more accurate or insightful responses. The way a prompt is structured has a massive impact on how well a model understands the task at hand and, ultimately, how well it performs. Given LLMs’ versatility, prompt engineering has become an important skill for getting the most out of these models across different domains and tasks. The key is to understand how different prompt structures lead to different model behaviors. There are various strategies - ranging from simple one-shot prompting to more complex techniques like chain-of-thought prompting - that can significantly improve the effectiveness of LLMs." (Abi Aryan, "LLMOps: Managing Large Language Models in Production", 2025)

"LLMs are rapidly improving, not only in their ability to understand and generate language but also in how effectively they can use external tools for data retrieval, transformation, and manipulation. As these models become more capable, they will be able to perform increasingly complex tasks with minimal prompting. However, their effectiveness still depends on the quality, design, and integration of the tools you provide. It’s your job to implement those tools thoughtfully and efficiently, ensuring they are well suited to your system’s goals and constraints." (Tomaz Bratanic & Oskar Hane,"Essential GraphRAG: Knowledge Graph-Enhanced RAG", 2025)

"LLMs are trained on large volumes of data, which inherently provides them with an immense knowledge base and understanding of different languages. Yet, LLMs at their core are complex text completion engines. Since this knowledge and understanding of language is compressed in a very high-dimensional latent space. LLMs end up using these in a very fluid and intelligible way (which often leads to hallucinations). In order to guide LLMs to focus on specific topics or pieces of information to solve certain tasks, (for instance, question-answering from a given piece of text), it is important to provide contextual information explicitly. While most current generations of LLMs have extremely wide context windows, it is recommended to preprocess context into overlapping smaller chunks for better results, reduced latency, and so on. For similar reasons, it is also recommended to preprocess contextual information in clear and task-specific formats. This aspect of context preprocessing is extremely useful in Retrieval-Gugmented Generation (RAG) scenarios." (Joseph Babcock & Raghav Bali, "Generative AI with Python and PyTorch" 2nd. Ed., 2025)

"LLM deployment failures often trace back not to the model itself, but to the prompts it receives. In production environments, prompts are rarely fixed, handcrafted snippets. Instead, they are dynamically generated, assembled from templates, and parameterized based on upstream data sources or evolving user state. This dynamism introduces complexity and variability that can subtly undermine the system’s performance if not carefully managed." (Abi Aryan, "LLMOps: Managing Large Language Models in Production", 2025)

"Recursive prompting is a systematic approach to achieving higher-quality outputs through iterative refinement. Rather than accepting the first response, it uses a step-by-step process of evaluation and improvement, making it particularly valuable for complex tasks such as code development, writing, and problem-solving. Our example demonstrated how a basic factorial function evolved from a simple implementation to a robust, optimized solution through multiple iterations of targeted refinements." (Jeremy C Morgan, "Coding with AI: Examples in Python", 2025)

"Semantic Kernel is a framework designed to simplify integrating LLMs into applications that require dynamic knowledge, reasoning, and state tracking. It’s particularly useful when you want to build complex, modular AI systems that can interact with external APIs, knowledge bases, or decision-making processes. Semantic Kernel focuses on building more flexible AI systems that can handle a variety of tasks beyond just generating text. It allows for modularity, enabling developers to easily combine different components - such as embeddings, prompt templates, and custom functions - in a cohesive manner." (Abi Aryan, "LLMOps: Managing Large Language Models in Production", 2025)

"The art of mega-prompts spanning multiple written pages and looking like essays has become commonplace for complex tasks when building applications to get things `just right'. Unfortunately, they bring with them lots of issues: errors, portability, complexity, and more. The GenAI world didn’t plan for mega-prompts. They have simply evolved into what they’ve become today because practitioners kept wanting to do more and more complex things, and their only way to express those intents was with a prompt. But step back and look at some of these prompts [...] Lurking just below the surface are a bunch of classical computing concepts like data, programming instructions, control flows, memory, and stora - all the components typically associated with classical computing elements." (Rob Thomas et al, "AI Value Creators: Beyond the Generative AI User Mindset", 2025)

"Transformers are complex models built like LEGO blocks using multiple smart and specialized components. [...] Briefly, a vanilla transformer model consists of separate stacks of encoders and decoders. Each encoder block includes multi-head self-attention, enabling the model to capture relationships between tokens regardless of their positions. Residual connections help maintain gradient flow, preventing the vanishing gradient problem. Layer normalization ensures training stability, and feed-forward layers introduce non-linearity and learn complex token interactions. Decoder blocks contain the same components but also include an encoder-decoder attention mechanism to incorporate context from the encoder. The model uses embedding layers to convert tokens into a continuous latent space for contextual learning and positional encoding to preserve the order of tokens in the sequence." (Joseph Babcock & Raghav Bali, "Generative AI with Python and PyTorch" 2nd. Ed., 2025)

"When there are hidden layers between the input and output, the problem becomes more complex: when do we change the internal weights to compute the activations that feed into the final output? How do we modify them in relation to the input weights? The insight of the backpropagation technique is that we can use the chain rule from calculus to efficiently compute the derivatives of each parameter of a network with respect to a loss function and, combined with a learning rule, this provides a scalable way to train multilayer networks." (Joseph Babcock & Raghav Bali, "Generative AI with Python and PyTorch" 2nd. Ed., 2025)

"A solid data foundation is critical for AI success. The foundation can grow in phases, but it needs to be there. AI will add pressure to the data foundation. Modern workloads require data that is connected, interpretable, high quality, and governed. Modern workloads utilize diverse data, and this is especially true with the popularity of GenAI. Trustworthy AI depends on knowing where the data came from and how it has changed. Modern architectures including the lakehouse and the data fabric exist because the traditional ways of managing data no longer fit the scale or complexity of what organizations need to do today. In other words, there is no path toward enterprise AI without a strong data foundation. This was echoed in the expert advices. Experts spoke about the dangers of 'vanity projects', particularly with GenAI, when foundational data issues were ignored. Others described how their companies moved faster specifically because they had already invested in governance, semantic layers, lineage, and observability." (Fern Halper, "Data Makes the World Go 'Round", 2026)

"[...] a transformer is a neural network that uses attention to understand and generate sequences of data efficiently and flexibly, allowing it to capture complex relationships across long inputs. Unlike older models, transformers process all parts of the input in parallel, making them much faster and more scalable than older types of neural networks." (Chris Minnick, "A Developer’s Guide to Integrating Generative AI into Applications", 2026)

"Intelligent systems connect users to AI and ML to achieve meaningful objectives. An intelligent system is one in which intelligence evolves and improves over time, particularly when it improves by watching how users interact with the system.[...] The primary objective of the intelligent system is to support users in accomplishing complex tasks - not by replacing them, but by enhancing their decision-making capabilities. [...] An intelligent system must also have the ability to learn from user interactions and explicit feedback, as well as utilize contextual information. The system should contin-uously develop, use, and maintain an evolving knowledge base. This evolution is driven not only by data sources but also by ongoing interactions with users." (Alessandro Negro et al, "Knowledge Graphs and LLMs in Action", 2026)

"LangGraph handles the perception, reasoning, and action flow while maintaining memory and context across tasks. In this sense, it functions as both the development environment and the orchestration layer - coordinating the steps of perception, reasoning, and action while managing connections to external systems. Underneath this, emerging standards like MCP ensure that agents can connect securely and consistently to tools and data sources, making agentic architectures portable and scalable across platforms. Taken together, the orchestration layer and emerging interoperability standards like MCP form the foundation for scalable agentic AI. They make it possible for agents to perceive, reason, act, and learn in coordinated ways across complex environments, translating autonomous intelligence into practical, enterprise-grade capability." (Fern Halper, "Data Makes the World Go 'Round", 2026)

"Prompt engineering may meet your accuracy requirements, but the prompts end up being extremely long as you encounter more and more corner cases. It’s not unusual for such prompts to amount to two or three pages of detailed instructions. The complex system instructions required to achieve that level of accuracy incur a large token count, and that makes the application too expensive to scale. [...] As your prompts become more detailed, maintenance overhead increases because the overall system is extremely sensitive to the specific wording of prompts. You’ll need to do integration testing every time the foundational model changes, and often even when one of your tools changes." (Valliappa Lakshmanan & Hannes Hapke, "Generative AI Design Patterns: Solutions to Common Challenges When Building GenAI Agents and Applications", 2026)

"Traditional paradigms build systems for specific purposes with structured, homogeneous databases. This approach works for tailored needs but is impractical for complex domains that need to adapt to user characteristics and integrate heterogeneous data. KGs capture connections, enabling relationship discovery through graph pattern matching and traversal. Both the Resource Description Framework (RDF) and Labeled Property Graphs (LPGs) provide machine-readable formats that humans can interpret. KGs emphasize rich, meaningful data representations usable by both humans and machines, enabling a paradigm shift where intelligent behavior is encoded in a unique source of truth." (Alessandro Negro et al, "Knowledge Graphs and LLMs in Action", 2026)

15 November 2026

🤖Prompt Engineering: Meaning (Just the Quotes)

"An embedding is a way to represent complex data as a set of numbers in a simpler, lower-dimensional space. Think of it as translating data into a format that a computer can easily understand and work with. Embedding models provide a uniform way to represent different types of data. Input to an embedding model can be any complex data, and the output is a vector. For instance, in dealing with text, an embedding model will take words or sentences and turn them into vectors, which are lists of numbers. The model is trained to ensure that these number lists capture essential aspects of the original words, such as their meaning or context." (Tomaz Bratanic & Oskar Hane,"Essential GraphRAG: Knowledge Graph-Enhanced RAG", 2025)

"Context is crucial for how language models understand and generate code. The model processes your input by analyzing relationships between different parts of the code and documentation to determine meaning and intent. [...] The model evaluates context by calculating mathematical relationships between elements in your input. However, it may miss important domain knowledge, coding standards, or architectural patterns that experienced developers understand implicitly." (Jeremy C Morgan, "Coding with AI: Examples in Python", 2025)

"Vector databases are designed to store and index highdimensional embeddings - dense numeric vectors that capture the semantic meaning of text, images, audio, or other content. Instead of looking for exact matches, they use approximate nearest neighbor (ANN) algorithms to return the items whose vectors lie closest to a query vector in that multidimensional space. This makes them the engine behind semantic search, recommendation systems, image-or-audio similarity matching, and retrieval augmented generation (RAG) pipelines that supply LLM prompts with relevant context in milliseconds." (Abi Aryan, "LLMOps: Managing Large Language Models in Production", 2025)

"Drift is the phenomenon in which an AI model becomes less accurate over time because of changes in the statistical properties of data in the real world versus the data the model was trained on. For example, a model trained on customer data with a specific age distribution might encounter a different age distribution in the future. This is known as data drift. AI models can also suffer from concept drift, in which the meaning of a term changes over time. Ongoing evaluations are essential for detecting drift, and both human and automated reviews can be used. Hybrid evaluation strategies encourage combining human judgment on output quality (tone, sensitivity, misinterpretations) with automated checks for syntactic correctness and speed." (Chris Minnick, "A Developer’s Guide to Integrating Generative AI into Applications", 2026)

"Embeddings are mathematical representations of text, images, or other input types. They capture semantic meaning by mapping words, phrases, or documents to vectors in a high-dimensional space. LLMs have become exceptionally proficient at creating semantically representative embeddings because this capability forms the foundation for LLM generation. [...] While the latest LLMs provide massive context windows, encoding entire documents in a single embedding vector is often not advisable. You’ll lose the nuances of the different parts of a text, lose critical information, and hinder the RAG system from retrieving relevant information. Plus, it gets very expensive to use large context windows at scale. It’s preferable to chunk documents into information-dense and self-consistent chunks of text." (Valliappa Lakshmanan & Hannes Hapke, "Generative AI Design Patterns: Solutions to Common Challenges When Building GenAI Agents and Applications", 2026)

"Foundational models can manipulate symbols and words effectively, but this may be because they are generalizing from the manipulation they have encountered in the training data, not because they grasp semantic meanings and the logical relationships between them in the way humans do. While you can use foundational models to perform many tasks, that’s because those tasks are similar to the tasks that the foundational model was trained to do." (Valliappa Lakshmanan & Hannes Hapke, "Generative AI Design Patterns: Solutions to Common Challenges When Building GenAI Agents and Applications", 2026)

"Sentence embeddings are dense vector representations of entire sentences or phrases, capturing their semantic meaning in a fixed-dimensional space. Unlike word embeddings, which represent individual words, sentence embeddings are designed to encode the meaning of longer text sequences. Two prominent techniques for generating sentence embeddings are SBERT(sentence-BERT) and dense retriever. [...] SBERT is a modification of the BERT architecture specifically designed for generating sentence embeddings. Traditional BERT outputs contextualized word embeddings, but SBERT fine-tunes BERT to produce fixed-size sentence embeddings by applying a pooling operation (e.g., mean pooling, max pooling, or CLS token pooling) over the token embeddings." (Saloni Garg et al, "RAG Artificial Intelligence: Retrieval-Augmented Generation in Generative AI", 2026)

"Some word meanings are stable, while others tend to move over time. [...] As meaning changes, so do the vector representations of these words in the embedding space. Depending on which dataset was used to train the embedding model, vector representations can fluctuate as cultures evolve-providing an interesting way to measure the impact of cultural changes over the years. [...] Contextual meaning of words forms another layer of dynamics in embedding models. Some words may not occupy a single point in space but may vary depending on how they are used. [...] The embedding space and semantic laws of language can teach us much. They help explain how language can be stable enough for reliable communication and yet flexible enough for creativity, allowing adaptation as cultures change. Just as physical laws govern how objects move through space, the semantic laws of language govern word meaning and use, allowing words to shift and evolve while maintaining a connection to context." (Robert Barton & Jerome Henry, "Demystifying Generative AI A Practical and Intuitive Introduction", 2026)

"The foundational form of RAG, often called naive RAG, follows a straightforward pattern. A pipeline retrieves supporting context from external sources such as enterprise documents, knowledge bases, or structured datasets and appends that information to the model’s prompt before inference. In the most common implementation, each document is converted into an embedding, a numerical representation of its semantic meaning, using either the same foundation model or a specialized embedding model. When a user submits a query, the system performs a vector similarity search to find documents whose embeddings most closely match the query’s vector representation, and the retrieved content is concatenated with the user query before being passed to the language model." (Bennie Haelen, "ML and Generative AI in the Data Lakehouse Building and Deploying AI Applications at Scale", 2026)

"The transformer was revolutionary in many ways. At its heart is a powerful mechanism called self-attention, which allows it to understand the context and meaning of not just individual words but entire sequences of words. By learning the context of words for an input of almost any size, a transformer understands the actual meaning of what it reads, and it can also generate new outputs that are rich in meaning, creative, and useful." (Robert Barton & Jerome Henry, "Demystifying Generative AI A Practical and Intuitive Introduction", 2026)

🪙Business Intelligence: Data Cleaning (Just the Quotes)

"Unfortunately, just collecting the data in one place and making it easily available isn’t enough. When operational data from transactions is loaded into the data warehouse, it often contains missing or inaccurate data. How good or bad the data is a function of the amount of input checking done in the application that generates the transaction. Unfortunately, many deployed applications are less than stellar when it comes to validating the inputs. To overcome this problem, the operational data must go through a 'cleansing' process, which takes care of missing or out-of-range values. If this cleansing step is not done before the data is loaded into the data warehouse, it will have to be performed repeatedly whenever that data is used in a data mining operation." (Joseph P Bigus,"Data Mining with Neural Networks: Solving business problems from application development to decision support", 1996)

"Processes must be implemented to prevent bad data from entering the system as well as propagating to other systems. That is, dirty data must be intercepted at its source. The operational systems are often the source of informational data; thus dirty data must be fixed at the operational data level. Implementing the right processes to cleanse data is, however, not easy." (Margaret Y Chu, "Blissful Data", 2004)

"Data archeology (finding bad data), data cleansing (correcting bad data), and data quality enforcement (preventing data defects at the source) should be business objectives. Therefore, data quality initiatives are business initiatives and require the involvement of business people, such as information consumers and data originators." (Sid Adelman et al, "Data Strategy", 2005)

"Data cleansing is dangerous mainly because data quality problems are usually complex and interrelated. Fixing one problem may create many others in the same or other related data elements." (Arkady Maydanchik, "Data Quality Assessment", 2007)

"Data quality program is a collection of initiatives with the common objective of maximizing data quality and minimizing negative impact of the bad data. [...] objective of any data quality program is to ensure that data quality docs not deteriorate during conversion and consolidation projects, Ideally, we would like to do even more and use the opportunity to improve data quality since data cleansing is much easier to perform before conversion than afterwards." (Arkady Maydanchik, "Data Quality Assessment", 2007)

"If you think of a Data Mart as a store of bottled water, cleansed and packaged and structured for easy consumption, the Data Lake is a large body of water in a more natural state. The contents of the Data Lake stream in from a source to fill the lake, and various users of the lake can come to examine, dive in, or take samples." (James Dixon 2010)

"Many organizations need to create data warehouses - massive data stores of timeseries data for decision support. Data are imported from various external and internal resources and are cleansed and organized in a manner consistent with the organization’s needs. After the data are populated in the data warehouse, data marts can be loaded for a specific area or department. Alternatively, data marts can be created first, as needed, and then integrated into an EDW." (Ramesh Sharda et al, "Business Intelligence: A Managerial Perspective on Analytics" 3rd Ed., 2014)

"Data quality in warehousing and BI is typically defined in terms of the 4 C’s - is the data clean, correct, consistent, and complete? When it comes to big data, there are two schools of thought that have different views and expectations of data quality. The first school believes that the gold standard of the 4 C’s must apply to all data (big and little) used for clinical care and performance metrics. The second school believes that in big data environments, a stringent data quality standard is impossible, too costly, or not required. While diametrically opposite opinions may play well in panel discussions, they do little to reconcile the realities of healthcare data quality." (Prashant Natarajan et al, "Demystifying Big Data and Machine Learning for Healthcare", 2017) 

"The desire to collect as much data as possible must be balanced with an approximation of which data sources are useful to address a business issue. It is worth mentioning that often the value of internal data is high. Most internal data has been cleansed and transformed to suit the mission. It should not be overlooked simply because of the excitement of so much other available data." (Mike Fleckenstein & Lorraine Fellows, "Modern Data Strategy", 2018)

"Understanding data is really about two types of expertise: data expertise and analytics expertise. Data expertise involves working directly with data - data extraction, cleaning, transformation, and management. Analytics expertise involves data analysis, data visualization, and validation." (Shonna D Watters et al, "The Practical Guide for HR Analytics: Using data to inform, transform, and empower HR decisions", 2019)

"[...] data mesh introduces a fundamental shift that the owners of the data products must communicate and guarantee an acceptable level of quality and trustworthiness - specific to their domain - as an intrinsic characteristic of their data product. This means cleansing and running automated data integrity tests at the point of the creation of a data product." (Zhamak Dehghani, "Data Mesh: Delivering Data-Driven Value at Scale", 2021)

"Data lineage is the path the data took from its source origin, the stops it made along the way, and its destination. This includes information on all the systems it passed through, how it was cleansed, what it was enriched with, and how it was secured. Capturing all that metadata is difficult because many of those systems and applications don’t share information. It’s up to you to assemble the data’s path by pulling metadata from all those systems/applications and assembling them in hopes that you find the path your data took from its current location (destination) to its source system. Lineage is probably the hardest piece of metadata to acquire for either streaming or batching data pipelines." (Hubert Dulay & Stephen Mooney, "Streaming Data Mesh", 2023)

"Data lineage provides the entire history of the data: origin, transformations, enrichments, users who engineered the transformations, etc. Consumers of the data need to trust that the data they will be using is the correct data. Data lineage provides a perspective that creates trust. It does so by mapping out the steps for policies, standards, processes, roles, and responsibilities that were involved with the sourcing, transformation, enrichment, and cleansing of the data." (Hubert Dulay & Stephen Mooney, "Streaming Data Mesh", 2023)

"A data warehouse is a centralized repository of structured, cleaned, and verified data that has been extracted, transformed, and loaded from various sources. These steps are commonly called ETL, which stands for Extract, Transform, Load. This data processing methodology involves extracting data from multiple sources, transforming it to meet business needs, and loading it into a destination for analysis and consultation." (Christopher Maneu et al, "The Definitive Guide to Microsoft Fabric From discovery to building a unified, secure, and scalable data platform", 2025)

"Data transformation sits at the heart of every successful data platform, serving as the critical bridge between data ingestion and data consumption. While basic transformations might involve simple cleaning and formatting, advanced transformation techniques encompass complex operations such as data enrichment, sophisticated deduplication, machine learning-based predictions, and the creation of derived metrics that weren’t present in the original data sources. These processes are essential for organizations looking to extract maximum value from their data investments." (Christopher Maneu et al, "The Definitive Guide to Microsoft Fabric From discovery to building a unified, secure, and scalable data platform", 2025)

"Data may indeed be the new oil. But just like crude oil, data needs refining. It must be transformed into information. This is why we clean, combine, model, and visualize data. The output of all this work - whether you do it on your own, get some help, or use a (semi-)automatic process - includes reports and dashboards that provide insights into various aspects of the organization’s dealings, which decision-makers can then consume to make critical business decisions." (Jeroen ter Heerdt et al, "Microsoft Power BI Visual Calculations: Simplifying DAX", 2026)

14 November 2026

🖍️Robert Barton - Collected Quotes

"A high learning rate lets the model adjust quickly but risks overshooting the optimal values and getting stuck in a cycle of jumping around without minimizing the error function. A low learning rate will give you better precision but could slow down training and increase computational cost, or it could get the model stuck in a local minimum, where it essentially loses sight of the big picture of what it is trying to accomplish. Choosing the right learning rate is something of a trade-off. It may take many adjustments to find the optimal learning rate if doing so by hand, but the job can also be made easier by optimization algorithms, which try to guess the optimal learning rate. Such methods include Bayesian optimizations and Grid Search, among others." (Robert Barton & Jerome Henry, "Demystifying Generative AI A Practical and Intuitive Introduction", 2026)

"All machine learning techniques share the same basic idea: A computer program learns and improves by observing or measuring data. This learning is then used to make predictions when new data is presented to the program. The process involves three key elements that work together: a model, a form of learning (training) that is applied to the model, and actions on the model once it is trained (inference). The type, or family, of AI or machine learning that you are working with depends on the goal of this process. This taxonomy can be a bit complex because technical breakthroughs are sometimes remembered as their own family categories, even though they are simply more efficient ways of achieving some well-known goals. However, the most common way to organize the machine learning families is to distinguish between supervised learning, unsupervised learning, and reinforcement learning techniques." (Robert Barton & Jerome Henry, "Demystifying Generative AI A Practical and Intuitive Introduction", 2026)

"Clean data is the cornerstone of any successful deep learning project. Noisy or dirty data, which may contain errors, inconsistencies, or irrelevant information, can mislead the model during training, causing it to learn incorrect patterns or fail to converge altogether. A model that does not converge shows poor performance and is unusable. A model that learns incorrect patterns produces erroneous outputs, leading to poor decisions that can be costly in terms of time, money, and resources. Ensuring data quality involves addressing several key issues. First, clean data must be as close as possible to error free. [...] Second, consistent formatting is critical.[...] Finally, handling outliers is an essential step" (Robert Barton & Jerome Henry, "Demystifying Generative AI A Practical and Intuitive Introduction", 2026) 

"Each input element to a neural network [...] is called a feature. The neural network has an equal number of input layer neurons and features. The input layer does not perform any computation but rather sends each feature into the first hidden layer, which operates as the first computational layer. Some features may be more important than others, so each feature is affected by a weight as it is processed by the computational layer. One of the main goals of the training phase is to dynamically determine the right weight for each feature. The weight itself is not a feature; it is a parameter. Features relate to the data, and parameters relate to how the neural network model treats the data." (Robert Barton & Jerome Henry, "Demystifying Generative AI A Practical and Intuitive Introduction", 2026)

"If AI as a whole mirrors the human thought process, then reinforcement learning represents its most direct connection to human psychology. RL captures one of our most fundamental learning mechanisms: the ability to learn through experience, guided by rewards and penalties. This is not just a superficial parallel. RL’s core ideas were in fact directly inspired by the study of psychology, and many of the most powerful RL algorithms are remarkably aligned with the psychology of human decision-making." (Robert Barton & Jerome Henry, "Demystifying Generative AI A Practical and Intuitive Introduction", 2026)

"Monte Carlo methods are designed to solve optimization problems that are modeled as MDPs. However, unlike Bellman-style methods, which require a complete model of the environment up front (that is, well-defined transition probabilities and rewards), Monte Carlo methods are model free, meaning the agent learns the transition probabilities as it explores the environment through each episode. [...] In cases where episodes are very long and produce only sparse data, Monte Carlo methods simply don’t get enough meaningful data to train well. [...] Monte Carlo methods work well in modeling systems that are defined by episodes that are reasonably stable and frequent enough to provide useful data. However, if long episodes and sparse rewards are involved, Monte Carlo struggles to build an optimal policy." (Robert Barton & Jerome Henry, "Demystifying Generative AI A Practical and Intuitive Introduction", 2026)

"In deep learning, the quality of the data often works in tandem with the quality of the model. Even the most sophisticated neural network architectures cannot perform well without properly prepared data. Data preparation involves cleaning, organizing, normalizing, and sometimes augmenting the raw data to ensure that it is suitable for training a model. This process is one of the most crucial steps in the machine learning pipeline because it directly impacts the model’s ability to learn meaningful patterns and generalize to new, unseen data." (Robert Barton & Jerome Henry, "Demystifying Generative AI A Practical and Intuitive Introduction", 2026)

"In the context of reinforcement learning, Monte Carlo methods are a class of algorithms that evaluate or improve a policy by sampling the output of complete episodes of interaction with the environment. During each episode, actions are taken based on the agent’s current policy, often including random actions to encourage exploration, and then the total return (the sum of the rewards) is recorded. Over many episodes, the algorithm averages these returns and is able to estimate an improved policy. The policy is only updated at the end of an episode, once it completes, and not incrementally during the episode itself." (Robert Barton & Jerome Henry, "Demystifying Generative AI A Practical and Intuitive Introduction", 2026) 

"One of the main reasons today’s generative models appear sophisticated is that they build on foundational concepts of neural computation, including layers of artificial neurons, activation functions, weight adjustment through backpropagation, and careful tuning of hyperparameters. These mechanisms free the models from the limitations of linear relationships between input and output. [...] Moreover, deep learning is not just about model architecture; it is also about how these models learn. Training a neural network is a delicate process that is both art and engineering. It involves managing noisy data, fine-tuning hyperparameters, preventing overfitting, and navigating delicate trade-offs between model complexity and generalization. In generative AI in particular, the model’s behavior is only as good as the training dynamics the model underwent. Imperfect training leads a model to hallucinate, collapse into mode repetition, or generate biased and unhelpful outputs." (Robert Barton & Jerome Henry, "Demystifying Generative AI A Practical and Intuitive Introduction", 2026)

"Some word meanings are stable, while others tend to move over time. [...] As meaning changes, so do the vector representations of these words in the embedding space. Depending on which dataset was used to train the embedding model, vector representations can fluctuate as cultures evolve-providing an interesting way to measure the impact of cultural changes over the years. [...] Contextual meaning of words forms another layer of dynamics in embedding models. Some words may not occupy a single point in space but may vary depending on how they are used. [...] The embedding space and semantic laws of language can teach us much. They help explain how language can be stable enough for reliable communication and yet flexible enough for creativity, allowing adaptation as cultures change. Just as physical laws govern how objects move through space, the semantic laws of language govern word meaning and use, allowing words to shift and evolve while maintaining a connection to context." (Robert Barton & Jerome Henry, "Demystifying Generative AI A Practical and Intuitive Introduction", 2026)

"Supervised learning is broken down into two classes of algorithms: regression and classification. Regression focuses on predicting a continuous value, such as the price of a stock or a house (and takes its name from a common statistical technique used in this context). The learning process is similar to how humans learn: by observation. [...] Classification, on the other hand, is about determining whether a given input belongs to one category or another. The outcome can be binary (for example, yes or no) or drawn from a larger set of predefined classes." (Robert Barton & Jerome Henry, "Demystifying Generative AI A Practical and Intuitive Introduction", 2026)

"Temporal difference learning is based on a simple but powerful idea:. Instead of waiting until the end of an episode to update its knowledge, it updates its estimates continually—every time the agent takes an action it records the current state and the action it took, and observes how the state changed and what reward was given. As it observes, it makes small adjustments to its guesses about how valuable the current state is." (Robert Barton & Jerome Henry, "Demystifying Generative AI A Practical and Intuitive Introduction", 2026)

"The depth of an ANN is also why the hidden layers are called 'hidden'. The term does not mean that we seek to hide them from view but simply that we do not consider their individual output directly. The system takes data into a visible input layer and produces a result in the output layer. There may be many layers in between that contribute to that result, but unless you were studying the particular structure of the neural network, you would not examine the results the individual layers produce. Your system would merely use the result as an input into the next layer." (Robert Barton & Jerome Henry, "Demystifying Generative AI A Practical and Intuitive Introduction", 2026)

"The first big innovation in the transformer architecture is its ability to process tokens in parallel rather than sequentially, one at a time. This was a huge breakthrough, as it allowed faster training and effectively overcame the long-range dependency problem. Rather than reading one sentence at a time as humans do, a transformer reads a whole input sequence at once - spanning hundreds of thousands of words in some cases. To be clear, there are still limitations to how much a transformer can read in parallel, known as the context window, but generally this window is very large. The ability to examine words in parallel allows the transformer to understand how words relate to each other, even if they are far apart in the same sentence, paragraph, or even whole book." (Robert Barton & Jerome Henry, "Demystifying Generative AI A Practical and Intuitive Introduction", 2026)

"The name of the SVM technique comes from the way it considers the data points. When looking at two groups of points (and searching for their boundary), there are points in one group that are far away from the other group. From the perspective of defining the hyperplane between the groups, these points are not very interesting. The points that are more interesting are the ones closest to the other group. An SVM builds vectors (called support vectors) from these edge points toward the other group. The process involves balancing the size (magnitude) of these vectors on both sides to find a line right in the middle (thus maximizing the space, or margin, to each group), between the two groups, using a mathematical approach called Lagrange multipliers." (Robert Barton & Jerome Henry, "Demystifying Generative AI A Practical and Intuitive Introduction", 2026)

"The process of deciding which features to include and which ones to ignore is known as feature engineering. It is a critical step in supervised learning. More features can make the model more powerful and accurate, but they also make it more complex, harder to interpret, and slower to train. As in many other parts of AI, deciding the number of features for a particular problem is often a balancing act." (Robert Barton & Jerome Henry, "Demystifying Generative AI A Practical and Intuitive Introduction", 2026)

"The transformer was revolutionary in many ways. At its heart is a powerful mechanism called self-attention, which allows it to understand the context and meaning of not just individual words but entire sequences of words. By learning the context of words for an input of almost any size, a transformer understands the actual meaning of what it reads, and it can also generate new outputs that are rich in meaning, creative, and useful." (Robert Barton & Jerome Henry, "Demystifying Generative AI A Practical and Intuitive Introduction", 2026)

"Training involves feeding a large set of data (called the training set) into the model. [...] This learning phase is somewhat like a brute-force process, where the program tries a large set of numbers until it finds the “least bad” values for the parameters to model the data. We use the expression least bad because in most cases, the training phase does not find perfect numbers." (Robert Barton & Jerome Henry, "Demystifying Generative AI A Practical and Intuitive Introduction", 2026)

"Unlike symbolic reasoning, which relies on sets of handcrafted logic rules, machine learning involves training a computer by giving it data; the computer improves automatically as it sees more examples. With machine learning, a computer learns to identify patterns by observing data, allowing it to create rules by itself. As it processes more data, its performance continually improves - often dramatically. With enough data, a model trained this way can accomplish tasks that expert systems could never approach." (Robert Barton & Jerome Henry, "Demystifying Generative AI A Practical and Intuitive Introduction", 2026)

"What makes neural networks so powerful is their ability to model not only linear relationships but also complex, nonlinear relationships in data. They can learn from examples in ways that are often impossible for traditional algorithms. Instead of relying on hard-coded rules or shallow statistical approximations, neural networks adjust internal parameters through repeated exposure to data, which allows them to learn patterns, capture nuance, and, ultimately, generate entirely new content. This is the core mechanism behind many generative AI solutions." (Robert Barton & Jerome Henry, "Demystifying Generative AI A Practical and Intuitive Introduction", 2026)

"When training an AI model, you may achieve a higher accuracy rate, but the importance of that accuracy depends on the consequences of being wrong. If the model occasionally issues an alert when none is needed (a false positive), the result may be mild annoyance for the driver. Perhaps they slow down unnecessarily or dismiss the warning. But if the model fails to issue an alert when a real hazard is present (a false negative), the consequences could be severe and result in a collision with another vehicle or an obstacle. This issue highlights a core principle in AI: Acceptable accuracy depends on the cost of mistakes." (Robert Barton & Jerome Henry, "Demystifying Generative AI A Practical and Intuitive Introduction", 2026)

🤖Prompt Engineering: Efficiency (Just the Quotes)

"A knowledge graph is a data structure that uses nodes to represent concepts and entities and relationships to connect these nodes. [...] Knowledge graphs are highly versatile, capable of storing both structured information (such as employee details, task statuses, and company hierarchies) and unstructured information (such as article contents). This dual capability [...] makes them uniquely suited for complex RAG applications. [...] By integrating structured and unstructured information in the same framework, knowledge graphs enable the seamless blending of both worlds, making them a powerful tool for answering a broad range of questions efficiently and accurately in RAG applications. Moreover, explicit connec-tions between unstructured and structured data unlock advanced retrieval strategies such as linking entities in text to graph nodes or contextualizing structured results with source passages that would be difficult or impossible to achieve using either type of data alone."(Tomaz Bratanic & Oskar Hane,"Essential GraphRAG: Knowledge Graph-Enhanced RAG", 2025)

"As with many other deep learning-based approaches, another major challenge is in interpretability. While knowledge graphs provide a structured and transparent way to store relationships, LLMs operate as a black box, making it difficult to understand how specific outputs are generated. [...] Data alignment is also a key issue, as structured knowledge graphs and unstructured text data must be carefully preprocessed to ensure consistency.  Differences in data formats, ontology mismatches, and information redundancy can create inefficiencies when integrating these two paradigms. Developing robust pipelines that seamlessly connect graph-based insights with LLM-generated text remains an open challenge." (Aldo Marzullo et al, "Graph Machine Learning" 2nd Ed., 2025)

"[...] LLMs are pre-trained on large corpora and sometimes fine-tuned on smaller datasets for specific tasks. Recall that one of the factors behind the Transformer’s effectiveness as a language model is that it is highly parallelizable, allowing for faster training and efficient processing of text. What really sets the Transformer apart from other deep learning architectures is its ability to capture long-range dependencies and relationships between tokens using attention. In other words, attention is a crucial component of Transformer-based LLMs, and it enables them to effectively retain information between training loops and tasks (i.e., transfer learning), while being able to process lengthy swatches of text with ease." (Sinan Ozdemir, "Quick Start Guide to Large Language Models: Strategies and Best Practices for Using ChatGPT and Other LLMs", 2024) 

"LLMs are rapidly improving, not only in their ability to understand and generate language but also in how effectively they can use external tools for data retrieval, transformation, and manipulation. As these models become more capable, they will be able to perform increasingly complex tasks with minimal prompting. However, their effectiveness still depends on the quality, design, and integration of the tools you provide. It’s your job to implement those tools thoughtfully and efficiently, ensuring they are well suited to your system’s goals and constraints." (Tomaz Bratanic & Oskar Hane,"Essential GraphRAG: Knowledge Graph-Enhanced RAG", 2025)

"The pre-training step is by far the biggest in terms of data and compute requirements for the whole of the LLM’s lifecycle. Yet fine-tuning is quite resource-intensive when we compare it to traditional machine learning and deep learning workflows. Fine-tuning is also a very important step in improving the quality of the models; hence, it makes sense to understand how we can optimize this step without impacting the performance. Efficiencies in this step also enable us to iterate faster, thereby improving adaptability in many fast-moving domains. In this section, we will focus on some interesting efficient method." (Joseph Babcock & Raghav Bali, "Generative AI with Python and PyTorch" 2nd. Ed., 2025)

"A major aspect of the alignment problem is values misalignment: AI systems may lack sufficient understanding of human ethics, cultural norms, or social contexts, making it difficult to translate our values directly into algorithms. For instance, if an AI is programmed to prioritise efficiency without balancing safety or ethical considerations, it might make decisions that, while effective, disregard human welfare. This is further complicated by 'specification gaming', where an AI might exploit loopholes in its programming to achieve objectives in unintended or counterproductive ways. An AI instructed to avoid obstacles, for instance, could redefine what counts as an 'obstacle' and take routes that, while technically following the rule, lead to negative outcomes." (Alan Watkins & G C Cooke, "Smarter than You Winning in Business with Superintelligent AI", 2026)

"A RAG system needs to identify relevant text chunks from the knowledge base. It does that by building a datastore of chunks and organizing the chunks in a way that makes searching for chunks that are relevant to a query efficient and fast. This preparatory step of building an efficient data store is called indexing, and the runtime step of searching the datastore for relevant chunks is called retrieval. [...] Whenever a query is received, a question-answering pipeline is invoked This consists of two steps: retrieval and generation. In the retrieval step, the retriever finds chunks in the document store that are the most relevant to the query. In the generation step, these relevant chunks are added to the context of the prompt and used along with the user query to generate the answer. In this way, the generated answer is grounded in the relevant information from trusted knowledge sources. The response consists of the generated text as well as the relevant chunks that were used to generate the answer." (Valliappa Lakshmanan & Hannes Hapke, "Generative AI Design Patterns: Solutions to Common Challenges When Building GenAI Agents and Applications", 2026)

"KGs are sophisticated graph structures that represent real-world entities (people, places, diseases, proteins), define meaningful connections between them, and provide context. KGs provide structured, explainable knowledge representation but are challenging to build and query; LLMs offer natural language processing capabilities but suffer from hallucinations, stale information, and a lack of domain-specific grounding. Together, they are a 'killer combination': LLMs can extract entities and relationships from unstructured text to build KGs more efficiently, providing more autonomous and powerful graph querying and analysis. Meanwhile, KGs provide reliable, up-to-date domain knowledge to ground LLM responses and prevent hallucinations." (Alessandro Negro et al, "Knowledge Graphs and LLMs in Action", 2026)

"One of the most important aspects of designing with GenAI is determining when to let the machine act and when to keep a human in the loop. As LLMs and AI agents become increasingly capable of not just generating text but also performing actions (like calling APIs, filling out forms, and triggering workflows), designers and developers face a growing responsibility: deciding what should be automated, not just what can be. Although automation promises speed, efficiency, and scalability, over-automation can lead to unintended consequences and critical errors. This is especially true when actions taken by the AI affect financial, legal, or personal data. The goal of human-centered design isn’t to block AI progress. Instead, it should channel AI in ways that empower users while keeping them informed and in control." (Chris Minnick, "A Developer’s Guide to Integrating Generative AI into Applications", 2026)

"The self-attention mechanism is the cornerstone of transformer models, enabling them to process sequential data like text more effectively than traditional recurrent neural networks (RNNs) or convolutional neural networks (CNNs). Unlike RNNs, which process sequences step-by-step, self-attention allows the model to consider the entire input sequence simultaneously. This parallel processing capability makes transformers highly efficient and scalable. transformers highly efficient and scalable. At its core, self-attention computes relationships between all words in a sentence, assigning higher weights to words that are more relevant to each other" (Saloni Garg et al, "RAG Artificial Intelligence: Retrieval-Augmented Generation in Generative AI", 2026)

12 November 2026

🖍️Tomaz Bratanic - Collected Quotes

"A knowledge graph is a data structure that uses nodes to represent concepts and entities and relationships to connect these nodes. [...] Knowledge graphs are highly versatile, capable of storing both structured information (such as employee details, task statuses, and company hierarchies) and unstructured information (such as article contents). This dual capability [...] makes them uniquely suited for complex RAG applications. [...] By integrating structured and unstructured information in the same framework, knowledge graphs enable the seamless blending of both worlds, making them a powerful tool for answering a broad range of questions efficiently and accurately in RAG applications. Moreover, explicit connec-tions between unstructured and structured data unlock advanced retrieval strategies such as linking entities in text to graph nodes or contextualizing structured results with source passages that would be difficult or impossible to achieve using either type of data alone." (Tomaz Bratanic & Oskar Hane,"Essential GraphRAG: Knowledge Graph-Enhanced RAG", 2025)

"An embedding is a way to represent complex data as a set of numbers in a simpler, lower-dimensional space. Think of it as translating data into a format that a computer can easily understand and work with. Embedding models provide a uniform way to represent different types of data. Input to an embedding model can be any complex data, and the output is a vector. For instance, in dealing with text, an embedding model will take words or sentences and turn them into vectors, which are lists of numbers. The model is trained to ensure that these number lists capture essential aspects of the original words, such as their meaning or context." (Tomaz Bratanic & Oskar Hane,"Essential GraphRAG: Knowledge Graph-Enhanced RAG", 2025)

"Hallucinations occur because LLMs are not reasoning engines. They are probabilistic language models trained to predict what sounds like a good next token, based on patterns in their training data. They don’t know facts the way humans do. Rather, they generate text by guessing the most likely continuation, regardless of whether it’s true. This fundamental difference between statistical pattern matching and actual understanding is what separates LLMs from human cognition." (Tomaz Bratanic & Oskar Hane,"Essential GraphRAG: Knowledge Graph-Enhanced RAG", 2025)

"In the RAG process, the user starts by asking a question. Behind the scenes, the system turns that question into a search query and retrieves relevant information from sources like company documents, knowledge articles, or databases. Advanced retrieval algorithms find the most suitable content, which is then combined with the original question to form an enriched prompt. This prompt is sent to an LLM, which gener-ates a response based on both the question and the retrieved context. The entire retrieval process is automatic, and no extra input is required beyond the original ques-tion from the user. This makes RAG both seamless and effective, improving factual accuracy while reducing the chance of hallucinated answers." (Tomaz Bratanic & Oskar Hane,"Essential GraphRAG: Knowledge Graph-Enhanced RAG", 2025)

"Knowledge graphs are increasingly used to store both structured and unstructured data, a scenario that has become even more common with the advent of LLMs. In this context, LLMs can be used to extract structured data from unstructured sources like text documents. However, storing the original unstructured documents and the extracted structured data within the graph preserves the richness of the original data while enabling more precise querying and analysis of the extracted information." (Tomaz Bratanic & Oskar Hane,"Essential GraphRAG: Knowledge Graph-Enhanced RAG", 2025)

"[...] LLMs are great at understanding natural language and following instructions in the prompt. In the RAG workflow, the goal shifts to task-oriented response generation, where LLMs follow a set of instructions. The process involves utilizing a retrieval tool to fetch relevant documents from a specific knowledge base. The LLM then generates answers based on the provided documents, ensuring responses are accurate, contextually relevant, and aligned with specific guidelines. This systematic approach transforms the answer generation process into a targeted task of inspecting and using the retrieved information to produce the final answer." (Tomaz Bratanic & Oskar Hane,"Essential GraphRAG: Knowledge Graph-Enhanced RAG", 2025)

"LLMs are rapidly improving, not only in their ability to understand and generate language but also in how effectively they can use external tools for data retrieval, transformation, and manipulation. As these models become more capable, they will be able to perform increasingly complex tasks with minimal prompting. However, their effectiveness still depends on the quality, design, and integration of the tools you provide. It’s your job to implement those tools thoughtfully and efficiently, ensuring they are well suited to your system’s goals and constraints." (Tomaz Bratanic & Oskar Hane,"Essential GraphRAG: Knowledge Graph-Enhanced RAG", 2025)

"The result from a semantic classification of text is called an embedding. Any text you want to match using vector similarity search must be converted into an embedding. This is done using an embedding model, and it’s important that the embedding model stays the same throughout the RAG application. If you want to change the embedding model, you must repopulate the vector index. Embeddings are lists of numbers, and the length of the list is called the embedding dimension. The embedding dimension is important because it determines how much information the embedding can hold. The higher the embedding dimension, the more computationally expensive it is to work with the embedding, both when generating the embedding as well as when performing vector similarity search."(Tomaz Bratanic & Oskar Hane,"Essential GraphRAG: Knowledge Graph-Enhanced RAG", 2025)

"While a vector index isn’t strictly required for vector similarity search, it’s highly recommended. A vector index is a data structure (like a map) that stores vectors in a way that makes it easy to search for similar vectors. When using a vector index, the retriever method is often referred to as an approximate nearest neighbor search. This is because the vector index doesn’t find the exact nearest neighbors, but it finds vectors that are very close to the nearest neighbor. This is a tradeoff between speed and accuracy. The vector index is much faster than a brute-force search, but it’s not as accurate." (Tomaz Bratanic & Oskar Hane,"Essential GraphRAG: Knowledge Graph-Enhanced RAG", 2025)

🤖Prompt Engineering: Security (Just the Quotes)

"Prompt injection is a security vulnerability that is specific to AI systems, especially LLM systems, in which malicious users try to manipulate prompts to make a model behave in a certain unintended way. They may try to get it to leak data, execute unauthorized tasks (especially with agentic systems), or ignore constraints. This is possible because LLMs are typically encapsulated inside applications using metaprompts, which are developer-created instructions that define the model’s behavior. Metaprompts usually contain safeguard instructions, such as 'do not use curse words', and placeholders where the input submitted by the user is pasted. The user’s input is combined with the metaprompts into a larger prompt that then goes to the model." (Abi Aryan, "LLMOps: Managing Large Language Models in Production", 2025)

"Authentication and authorization are important parts of any communication between clients and servers. We need to have a clear separation between different roles and to be able to specify what capabilities a particular client is allowed to invoke. MCP, however, is more of a language interoperability protocol - it doesn’t enforce any security principles. [...] Collaboration In MCP, communication between the client and the server is mostly one way, whereas in the real world, you may want collaborative, interactive workflows." (Valliappa Lakshmanan & Hannes Hapke, "Generative AI Design Patterns: Solutions to Common Challenges When Building GenAI Agents and Applications", 2026)

"LangGraph handles the perception, reasoning, and action flow while maintaining memory and context across tasks. In this sense, it functions as both the development environment and the orchestration layer - coordinating the steps of perception, reasoning, and action while managing connections to external systems. Underneath this, emerging standards like MCP ensure that agents can connect securely and consistently to tools and data sources, making agentic architectures portable and scalable across platforms. Taken together, the orchestration layer and emerging interoperability standards like MCP form the foundation for scalable agentic AI. They make it possible for agents to perceive, reason, act, and learn in coordinated ways across complex environments, translating autonomous intelligence into practical, enterprise-grade capability." (Fern Halper, "Data Makes the World Go 'Round", 2026)

"Like penetration testing in cybersecurity - the authorized simulation of attacks to uncover weaknesses in a system before real attackers can exploit them - prompt hacking can be a force for good when done with permission and clear boundaries. Red-team testing is a structured effort to find flaws and vulnerabilities in an AI system, performed in a controlled environment and in collaboration with the system’s developers. Ethical 'red teamers' adopt an attacker’s mindset to systematically probe an AI’s defenses, but they do so to improve safety, not to cause harm. Guides on red teaming recommend starting with baseline tests, defining a threat model, and then iterating with more sophisticated attacks. Sharing findings with the development team helps fix vulnerabilities before malicious actors discover them." (Chris Minnick, "A Developer’s Guide to Integrating Generative AI into Applications", 2026)

"RAG applications must be built with semantics, metadata, and governance in mind. The retrieved information must be high-quality, secure, and appropriate for the user’s role. Equally important is monitoring and management: checking whether source data has changed, ensuring vector stores remain accurate, and watching for hallucinations or data leakage. Organizations are definitely starting to experiment with RAG models today; some are putting them into production applications. Some believe that using RAG helps mitigate hallucinations because it is grounded in trusted organizational data." (Fern Halper, "Data Makes the World Go 'Round", 2026)

"The Model Context Protocol (MCP) is emerging as a standard for connecting AI models to external tools and data sources. Historically, developers had to build custom connectors for each model and API, limiting portability and slowing innovation. MCP simplifies this by creating a common interface, essentially a universal plug like USB-C, that allows compliant models and tools to communicate seamlessly. Frameworks like LangChain/Graph and others are expected to adopt MCP to handle connectivity under the hood, providing developers with interoperability while maintaining secure and auditable access to enterprise systems." (Fern Halper, "Data Makes the World Go 'Round", 2026)

"The most common form of prompt hacking is prompt injection. This occurs when an attacker embeds malicious instructions inside a seemingly innocuous prompt to override the developer’s intended task. The root of the problem is architectural: current language models treat all text, including system prompts, developer instructions, and user input, as one continuous sequence. There is no built-in notion of 'trusted' versus 'untrusted' input, so a later instruction can supersede an earlier one. The simplest version is direct injection, where the attacker places the malicious directive directly in the user’s input. A classic example exploits the fact that models prioritize more recent or specific instructions [...] Indirect injection hides instructions in external content, such as a web page or document the model is asked to summarize, so the model executes code it wasn’t supposed to see. Specialized variants include code injection, where a code-generation assistant is tricked into outputting malicious commands, and recursive injection, where the output of one model carries an injection to the next in a chain. All of these attacks exploit the model’s inability to separate context from instruction." (Chris Minnick, "A Developer’s Guide to Integrating Generative AI into Applications", 2026)

"The power of RAG systems stems from their ability to access and process vast amounts of data. However, this very capability introduces significant risks regarding the privacy of individuals and the security of sensitive information. Unlike a simple chatbot, a RAG system often has access to proprietary corporate data, internal documentation, and potentially personal user information within its knowledge base. A data breach or misuse of this information can lead to severe financial, legal, and reputational damage. Furthermore, a global patchwork of stringent regulations now governs how personal data must be handled, making compliance a central pillar of AI system design, not an afterthought. Ethical deployment requires an architecture built on privacy by design and by default, ensuring user trust is maintained through robust technical and procedural safeguards." (Saloni Garg et al, "RAG Artificial Intelligence: Retrieval-Augmented Generation in Generative AI", 2026)

"With autonomous agents there is the risk that they can take action that isn’t governed. These systems introduce autonomy, interdependence between agents, and possibly emergent behavior. [...] Because agents act autonomously, a single misconfigured or misaligned agent can propagate errors at scale. In multiagent environments, one faulty output can trigger many downstream mistakes. Other risks include tool misuse, conflicting goals among agents (and other interoperability issues), and operational opacity, when teams cannot easily determine which agent took which action or why. Governance must extend to agent registration, version control, permissioning, and simulation testing before deployment. Likewise, accountability may also blur. If an agent takes a dangerous action, who is responsible? There are, of course, cybersecurity risks as agents pose a new attack surface."  (Fern Halper, "Data Makes the World Go 'Round", 2026)

11 November 2026

🤖Prompt Engineering: Bias (Just the Quotes)

"LLMs can inadvertently produce toxic content or biased language, leak private information, or be vulnerable to jailbreak prompts. These risks carry serious legal and reputational consequences. To mitigate them, evaluation tools must integrate automated filters and classifiers that flag problematic outputs in real time, as we discussed earlier in the chapter. Metrics such as safety scores, toxicity indices, and bias measurements should be collected alongside model metadata for auditing purposes." (Abi Aryan, "LLMOps: Managing Large Language Models in Production", 2025

"LLM-centric workloads change everything. Now the raw material is heterogeneous text, code, images, audio, and chat logs whose value depends on semantic richness - that is, the informational value of the content - rather than a rigid structure. Pipelines must tokenize, chunk, embed, and version this content; store it in vector indexes for similarity search; and apply filters for personally identifiable information, toxicity, and licensing constraints. Instead of ETL jobs, teams run continuous ingestion and reembedding loops so that RAG systems stay fresh, and they log every prompt–response pair so that the inputs and outputs can be evaluated and improve the future performance of this system. Data quality in this context is judged by grounding, factuality, and bias metrics - attributes that require automated red-teaming and humanin-the-loop (HITL) review rather than the data structure violation checks of the past." (Abi Aryan, "LLMOps: Managing Large Language Models in Production", 2025)

"LLMs excel at understanding context and making associations among words, phrases, and concepts to provide relevant information based on the input query or prompt. While structured knowledge bases rely on humancurated data, LLMs can  automatically extract knowledge from unstructured text. When trained on diverse textual sources, they can process a vast amount of information without explicit human intervention. However, this also introduces a challenge, as the model can learn biased or incorrect information from the training data." (Abi Aryan, "LLMOps: Managing Large Language Models in Production", 2025)

"Because LLMs are probabilistic, it’s possible for them to generate responses that are biased, toxic, false, or off-topic for your application. If a user submits prompts to an LLM that contain profanity or hate speech, it increases the chances that the generated content will contain inappropriate language. Such interactions can create legal liability or brand damage. LLMs are also vulnerable to attack by malicious users who attempt to convince the LLM to reveal sensitive corporate data or personally identifiable information (PII)." (Chris Minnick, "A Developer’s Guide to Integrating Generative AI into Applications", 2026)

"Before releasing the fine-tuned model to a production environment, you need to evaluate its performance and make sure it hasn’t picked up any counterproductive constructs, like biased language. Also, whenever a new version of your foundational model is released, you must fine-tune the model again and redo the training. Finally, you need to track the lineage of the training and validation data used to train the fine-tuned model. These are additional tasks that require careful execution, and using in-context learning is far simpler since it only requires you to add a handful of examples." (Valliappa Lakshmanan & Hannes Hapke, "Generative AI Design Patterns: Solutions to Common Challenges When Building GenAI Agents and Applications", 2026)

"Bias can creep in through multiple channels. Training data often reflects the cultures and languages most represented on the internet, which can leave out or misrepresent others. Prompts themselves can encode bias through how tasks or roles are framed. And even the architecture of a model can favor certain patterns of output over others, reinforcing statistical tendencies rather than producing balanced results. As a developer, you can’t eliminate bias entirely. But you can recognize where it comes from, test for it, and design safeguards to minimize harm." (Chris Minnick, "A Developer’s Guide to Integrating Generative AI into Applications", 2026)  

"Generative AI has a lot of problems because it needs a lot of data to work. Generative models like GPT and GANs need a lot of training data to learn patterns and make good outputs. This data dependency can cause problems like overfitting, which happens when the model does well on training data but doesn’t work well with new, unseen data. Also, the quality of the content that is generated is directly related to the diversity and representativeness of the training data. This means that biased or incomplete datasets can lead to outputs that are wrong or unfair." (Saloni Garg et al, "RAG Artificial Intelligence: Retrieval-Augmented Generation in Generative AI", 2026)

"If ethical lapses or AI failures occur, the impact on a business can be significant. Misinformation, biases, or harmful content generated by AI can lead to reputational damage, customer distrust, and potential regulatory scrutiny. The public relations fallout from an AI-driven error or ethical misstep can erode consumer confidence, resulting in lost revenue and lasting harm to brand image. Businesses, therefore, need to proactively address ethical considerations in AI implementation, not only to ensure compliance but also to protect and strengthen their reputation in a highly competitive, and increasingly transparent, marketplace." (Bennie Haelen, "ML and Generative AI in the Data Lakehouse Building and Deploying AI Applications at Scale", 2026)

"[...] LLMs raise serious concerns about ethics, bias and fairness, errors in reasoning, hallucinations, and misuse (e.g., misinformation and disinformation). These concerns are exacerbated by modern LLMs being both literal and figurative 'black boxes': Literal black boxes because many advanced AI systems are proprietary and the weights (trained parameters of the models) are not released to the public; and figurative black boxes because even the open-source AI models are so complicated that understanding them and developing safety guardrails has thus far proven extremely difficult." (Mike X Cohen,"50 ML Projects To Understand LLMs", 2026)

"The promise of AI is its ability to process information objectively and at scale. However, this promise is fundamentally threatened by the twin challenges of bias and misinformation. AI systems are not born in a vacuum; they are created by humans and trained on data produced by humans. Consequently, they are prone to inheriting and even amplifying our prejudices, errors, and the systemic inequalities present in that data. In a RAG system, this risk is a two-fold problem: first in the retrieval of information, and second in the generation of a response based on that retrieval. A failure to address these issues doesn’t just lead to technically incorrect outputs; it can perpetuate social harm, erode public trust, and lead to the widespread dissemination of falsehoods. Therefore, understanding and mitigating bias and misinformation is not an optional add-on but a core requirement for any ethically deployed AI system." (Saloni Garg et al, "RAG Artificial Intelligence: Retrieval-Augmented Generation in Generative AI", 2026)

09 November 2026

🖍️Chris Minnick - Collected Quotes

"[...] a transformer is a neural network that uses attention to understand and generate sequences of data efficiently and flexibly, allowing it to capture complex relationships across long inputs. Unlike older models, transformers process all parts of the input in parallel, making them much faster and more scalable than older types of neural networks." (Chris Minnick, "A Developer’s Guide to Integrating Generative AI into Applications", 2026)

"Although GenAI gets most of the attention for its ability to create new content, successful AI applications often combine multiple approaches. Generative models aren’t always the optimal choice for every component of an AI system. Traditional ML models, rule-based systems, and algorithmic approaches can offer significant advantages in speed, accuracy, cost-effectiveness, and reliability when used strategically alongside GenAI." (Chris Minnick, "A Developer’s Guide to Integrating Generative AI into Applications", 2026)

"As with any software system, designing GenAI interfaces with accessibility in mind is essential to inclusive, humane design. It may also be required by law. AI features in applications often introduce new interaction patterns, dynamic content, and inferred behaviors that, without care, can unintentionally exclude users with visual, auditory, motor, or cognitive impairments. Many accessibility principles extend naturally to AI scenarios, and GenAI can help automate inclusive experiences when responsibly implemented. Generative models may produce rich, interactive content, but AI-only overlays or content modifications risk causing confusion or misrepresentation." (Chris Minnick, "A Developer’s Guide to Integrating Generative AI into Applications", 2026)

"Because LLMs are probabilistic, it’s possible for them to generate responses that are biased, toxic, false, or off-topic for your application. If a user submits prompts to an LLM that contain profanity or hate speech, it increases the chances that the generated content will contain inappropriate language. Such interactions can create legal liability or brand damage. LLMs are also vulnerable to attack by malicious users who attempt to convince the LLM to reveal sensitive corporate data or personally identifiable information (PII)." (Chris Minnick, "A Developer’s Guide to Integrating Generative AI into Applications", 2026)

"Bias can creep in through multiple channels. Training data often reflects the cultures and languages most represented on the internet, which can leave out or misrepresent others. Prompts themselves can encode bias through how tasks or roles are framed. And even the architecture of a model can favor certain patterns of output over others, reinforcing statistical tendencies rather than producing balanced results. As a developer, you can’t eliminate bias entirely. But you can recognize where it comes from, test for it, and design safeguards to minimize harm." (Chris Minnick, "A Developer’s Guide to Integrating Generative AI into Applications", 2026)  

"Deep learning, a subset of ML, employs layers of artificial neural networks to identify patterns in data. A key attribute of deep learning is its ability to enable models to automatically learn hierarchical features from raw data, rather than relying on manually engineered features from structured data. Prior to deep learning, defining relevant features such as edges, color histograms, or keywords was typically necessary to train a model to recognize objects or sentiment. In contrast, deep learning models autonomously learn these features during training by optimizing the parameters of each layer to minimize error on the task at hand." (Chris Minnick, "A Developer’s Guide to Integrating Generative AI into Applications", 2026)

"Drift is the phenomenon in which an AI model becomes less accurate over time because of changes in the statistical properties of data in the real world versus the data the model was trained on. For example, a model trained on customer data with a specific age distribution might encounter a different age distribution in the future. This is known as data drift. AI models can also suffer from concept drift, in which the meaning of a term changes over time. Ongoing evaluations are essential for detecting drift, and both human and automated reviews can be used. Hybrid evaluation strategies encourage combining human judgment on output quality (tone, sensitivity, misinterpretations) with automated checks for syntactic correctness and speed." (Chris Minnick, "A Developer’s Guide to Integrating Generative AI into Applications", 2026)

"Generative models are designed to be persuasive storytellers, not fact checkers. When you ask a model a question, it predicts what words are most likely to follow based on patterns in its training data. Sometimes those patterns reflect reality; other times they include errors or biases. The result is that large language models (LLMs) can produce highly plausible sentences that are utterly untrue. They may cite papers that don’t exist, invent statistics, or confidently answer questions they have no basis to answer. In the AI community, this phenomenon is often called a hallucination, but that term is both imprecise and potentially stigmatizing. Hallucinations refer to sensory experiences without external stimuli and should not be conflated with algorithmic errors. More accurate labels like fabrication, confabulation, and non sequitur highlight that the AI response doesn’t logically follow from its training data." (Chris Minnick, "A Developer’s Guide to Integrating Generative AI into Applications", 2026)

"Hybrid integration combines a client-side model or significant frontend data processing with backend data processing. In practice, hybrid integration often blends the use of two or more different types of AI models. For example, blending the creativity of a generative model with the potential accuracy of reinforcement learning can lead to more trustworthy results than you might get from a generative model by itself. Other combinations of different types of models could include integrating neural networks with knowledge graphs or integrating symbolic AI with machine learning." (Chris Minnick, "A Developer’s Guide to Integrating Generative AI into Applications", 2026)

"In backend service integration, the application accesses an AI model or API through a server-side service. Using server-side resources to access the model improves the application’s security, simplifies monitoring and logging, and improves the scalability of the application. The trade-off is that it increases the complexity of the backend, and network round trips introduce latency into the application. Backend service integration is the right approach for applications that require access to a powerful AI model and that don’t require much client-side logic, such as search applications, chatbots, and data analysis." (Chris Minnick, "A Developer’s Guide to Integrating Generative AI into Applications", 2026)

"Mixture of experts (MoE) architectures are neural networks that contain many different 'expert' submodels. Instead of using them all at once, the model chooses a few experts to handle each input. [...] A gating network in an MoE architecture handles routine tasks and decides which experts are activated for each prompt. Most MoE language models and their expert submodels are transformer-based. The most important benefit of MoE is that it can be made more capable by adding additional experts, rather than increasing the size of a single model. Because only a subset of experts is activated at a time, unnecessary computation is reduced." (Chris Minnick, "A Developer’s Guide to Integrating Generative AI into Applications", 2026)

"One of the most important aspects of designing with GenAI is determining when to let the machine act and when to keep a human in the loop. As LLMs and AI agents become increasingly capable of not just generating text but also performing actions (like calling APIs, filling out forms, and triggering workflows), designers and developers face a growing responsibility: deciding what should be automated, not just what can be. Although automation promises speed, efficiency, and scalability, over-automation can lead to unintended consequences and critical errors. This is especially true when actions taken by the AI affect financial, legal, or personal data. The goal of human-centered design isn’t to block AI progress. Instead, it should channel AI in ways that empower users while keeping them informed and in control." (Chris Minnick, "A Developer’s Guide to Integrating Generative AI into Applications", 2026)

"Prompt engineering is the practice of crafting, optimizing, and managing prompts to guide large language models (LLMs) toward producing useful, specific, and high-quality outputs. It ranges from simple one-off instructions to structured templates and programmatic prompt management in production systems. [...] Prompt engineering is all about providing context to better align your question with the type of output you want the LLM to generate." (Chris Minnick, "A Developer’s Guide to Integrating Generative AI into Applications", 2026)

"RAG is a technique that combines the ability to make use of specialized domain knowledge with the techniques of prompt engineering. In a RAG application, documents that the application should have access to are split into chunks, such as overlapping blocks of text, and embedded as high-dimensional vectors. [...] A RAG application then stores the embeddings created from the documents along with metadata about each embedding, including a link to its source document and position in the document, in a vector database. When a user submits a prompt to an application with RAG capabilities, the application encodes the user’s query using the same embedding model used to encode the documents and finds the closest matches (aka nearest neighbors) between the user prompt and the chunks in the vector database. Finally, a new prompt is created that combines the retrieved snippets of text with a templated instruction. This new prompt is sent to the LLM for inference." (Chris Minnick, "A Developer’s Guide to Integrating Generative AI into Applications", 2026)

"Self-attention is a mechanism where, for each token, the model looks at all other tokens in the input and computes weighted importance scores (attention weights) to decide how much each other token should contribute to its representation. Prior to self-attention, each token would be evaluated sequentially. This capability makes transformers especially good at handling long-range dependencies in text and is one of the reasons they have become the foundation for nearly all modern LLMs." (Chris Minnick, "A Developer’s Guide to Integrating Generative AI into Applications", 2026)

"Technical features are the design and training decisions that are made that determine aspects such as a model’s use-case suitability and performance metrics. By understanding the basics of these technical features, you can better evaluate, assess, and predict how models will perform. These technical features include architecture, parameter count, training data, fine-tuning and specialized tuning, context window, and unique functionalities." (Chris Minnick, "A Developer’s Guide to Integrating Generative AI into Applications", 2026)

"The most common form of prompt hacking is prompt injection. This occurs when an attacker embeds malicious instructions inside a seemingly innocuous prompt to override the developer’s intended task. The root of the problem is architectural: current language models treat all text, including system prompts, developer instructions, and user input, as one continuous sequence. There is no built-in notion of 'trusted' versus 'untrusted' input, so a later instruction can supersede an earlier one. The simplest version is direct injection, where the attacker places the malicious directive directly in the user’s input. A classic example exploits the fact that models prioritize more recent or specific instructions [...] Indirect injection hides instructions in external content, such as a web page or document the model is asked to summarize, so the model executes code it wasn’t supposed to see. Specialized variants include code injection, where a code-generation assistant is tricked into outputting malicious commands, and recursive injection, where the output of one model carries an injection to the next in a chain. All of these attacks exploit the model’s inability to separate context from instruction." (Chris Minnick, "A Developer’s Guide to Integrating Generative AI into Applications", 2026) 

31 October 2026

⚡️🗒️Power BI: CALCULATE [Notes]

Disclaimer: This is work in progress intended to consolidate information from various sources for learning purposes. For the latest information please consult the documentation (see the links below)! 

Last updated: 31-Oct-2026

[DAX] CALCULATE

  • {def} function that evaluates an expression in a modified filter context [4]
    • filter arguments are used as the context for evaluating the expression CALCULATE belongs to
      • filter arguments for CALCULATE replace all existing filters over the same columns [6]
      • the new context effected by the filter argument for CALCULATE affects only existing filters on columns mentioned as part of the filter argument [6]
        • filters on columns other than those mentioned in the arguments of CALCULATE remain in effect and unaltered [6]
    • creates a filter context that includes a filter for every column of the iterated table
    • when a DAX formula is executed, multiple levels of filter contexts are combined [2] 
    • requires an active row context [2]
      • either CALCULATE is executed or a measure is invoked [2]
        • implies a context transition in these conditions [2]
          • the row context is transformed into an equivalent filter context [2]
        •  most of the time, the context transition happens because a measure is being invoked rather than because of an explicit CALCULATE [2]
    • it allows developers to
      • add, keep, remove or replace filters
  • {concept} outer filter context
    • the filter context that is active when CALCULATE is invoked[2
  • {concept} inner filter context
    • the new filter context created by CALCULATE and active only during the execution of its first argument [2]
    • leverages the context transition leads to better code [2]
    • propagates through relationships, moving from one table to another [2]
  • {type} implicit CALCULATE [2]
    • functions can conveniently hide CALCULATE or CALCULATETABLE functions from users but still invokes them [2]
      • e.g. TOTALYTD
    • performs a context transition by transforming any existing row contexts into an equivalent filter context
      • the measure referenced is calculated within a filter context created by context transition [2]
      • happens immediately before the measure starts being executed, as if CALCULATE automatically surrounds the measure [2]
  • {type} explicit CALCULATE
    • actively invokes context transition [2]
    • {warning} a random CALCULATE added to the code forces the context transition [2]
      • may lead to unexpected results  
      • {recommendation} don't add CALCULATE unnecessarily [2]
        • context transition can lead to situations in which the calculation works but all control over why it works is lost [2]
  • {benefit} leverage metrics without changing the underlying data by modifying the filter context 
    • allows to create alternative views of the same data model without creating new tables or columns
    • measures become flexible and reusable 
    • because calculations are performed dynamically at query time
  • {benefit} add/remove filters to calculate measures independent of user selections
    • KPIs/indicators can remain stable regardless of report slicers
  • {benefit} simplify complex row-to-aggregation calculations enable context transition
    • allows aggregation logic to operate correctly inside iterators and calculated columns
    • reduces complex manual filtering logic
  • {benefit} use relationships more effectively by activating inactive relationships
    • a single model can support multiple business perspectives without duplicating tables
  • {benefit} create dynamic business metrics 
    • metrics respond intelligently to slicers, filters, drill-downs, and report interactions.
  • {benefit} improve code readability
    • measures become easier to understand, maintain, and troubleshoot
    • using variables and explicit filter modifications makes business logic more transparent to future developers and analysts
  • {benefit} simplify complex DAX
    • functions like CALCULATE, variables, and context manipulation reduce nested expressions and repeated calculations
    • this generally leads to shorter, more maintainable formulas. 
  • {benefit} enable the creation of complex measures that can adapt to changes in filters
    • allows one measure to support many analytical scenarios without requiring multiple versions
    • measures automatically recalculate based on report context, allowing
  • {syntax} CALCULATE(<expression>, <filter1>, <filter2>,...)
    • {parameter} expression to compute
      • only at the end does CALCULATE compute its first argument after creating the new filter context [2]
    • {parameter} filters
      • starting from the second argument, are defined the filters used to create/modify the filter context
        • the filters are applied before the first argument can be computed
          • therefore, they must be evaluated first [2]
      • starts evaluating the 'filter' parameters in the order in which they were provided
      • accepts any number of arguments as filters. They are all intersected, meaning that all the filters work together at once [2]
      • when CALCULATE places a filter on a column that already had filters set by previous functions (or by the visual), the filter placed replaces the previous filter [2] 
        • ⇐ behavior can be changed by using KEEPFILTERS
      • the filter can be one of three types
        • condition
          • one or more values are specified for a set of columns [2]
          • the conditions are tables
            • it evaluates this table and restricts the data context to only the rows that match the rows present in that filter table [2]
        • modifier
          • instructs CALCULATE to perform an action (akabehave in a certain way) 
            • via REMOVEFILTERS or USERELATIONSHIP
        • table
          • filter is applied to the columns present in the table with an IN condition
        • placing a filter on one table automatically propagates its effect to any table with a relationship with the original table, following the cross-filter direction of the relationship [2]
  • {warning} filter context can change unexpectedly
    • CALCULATE modifies the existing filter context before evaluating an expression 
  • {warning} context transition can produce surprising results
    • when CALCULATE is used in a row context (calculated columns, iterators such as SUMX()), it automatically performs context transition
  • {warning} overriding existing filters

  • {warning} performance degradation
    • complex CALCULATE() measures may repeatedly recalculate filter contexts
  • {warning} ambiguous logic from multiple filters
    • the filter arguments can interact in ways that are not intuitive

References:
[1] Greg Deckler & Brett Powell (2024) "Microsoft Power BI Cookbook: Convert raw data into business insights with updated techniques, use cases, and best practices" 3rd Ed.
[2] Alberto Ferrari & Marco Russo (2025) The Definitive Guide to DAX: Mastering the semantic model expression language for Microsoft Power BI, Fabric and Excel
[3] Jeroen ter Heerdt et al (2026) "Microsoft Power BI Visual Calculations: Simplifying DAX"
[4] Microsoft Learn (2026) DAX: CALCULATE [link] 
[5] Cooper Dunn (2025) Microsoft DP-600 Exam Practice Tests: 500 Questions for Implementing Analytics Solutions Using Microsoft Fabric
[6] https://learn.microsoft.com/en-us/dax/keepfilters-function-dax


 
Related Posts Plugin for WordPress, Blogger...

About Me

My photo
Koeln, NRW, Germany
IT Professional with more than 25 years experience in IT in the area of full life-cycle of Web/Desktop/Database Applications Development, Software Engineering, Consultancy, Data Management, Data Quality, Data Migrations, Reporting, ERP implementations & support, Team/Project/IT Management, etc.