🔭Data Science: AI Systems (Just the Quotes)

"Inference is to bring about a new thought, which in logic amounts to drawing a conclusion, and more generally involves using what we already know, and what we see or observe, to update prior beliefs. […] Inference is also a leap of sorts, deemed reasonable […] Inference is a basic cognitive act for intelligent minds. If a cognitive agent (a person, an AI system) is not intelligent, it will infer badly. But any system that infers at all must have some basic intelligence, because the very act of using what is known and what is observed to update beliefs is inescapably tied up with what we mean by intelligence. If an AI system is not inferring at all, it doesn’t really deserve to be called AI." (Erik J Larson, "The Myth of Artificial Intelligence: Why Computers Can’t Think the Way We Do", 2021)

"The idea that we can predict the arrival of AI typically sneaks in a premise, to varying degrees acknowledged, that successes on narrow AI systems like playing games will scale up to general intelligence, and so the predictive line from artificial intelligence to artificial general intelligence can be drawn with some confidence. This is a bad assumption, both for encouraging progress in the field toward artificial general intelligence, and for the logic of the argument for prediction." (Erik J Larson, "The Myth of Artificial Intelligence: Why Computers Can’t Think the Way We Do", 2021)

"We think of context as the network surrounding a data point of interest that is relevant to a specific AI system. […] AI benefits greatly from context to enable probabilistic decision making for real-time answers, handle adjacent scenarios for broader applicability, and be maximally relevant to a given situation. But all systems, including AI, are only as good as their inputs." (Jesús Barrasa et al, "Knowledge Graphs: Data in Context for Responsive Businesses", 2021)

"Prompt injection is a security vulnerability that is specific to AI systems, especially LLM systems, in which malicious users try to manipulate prompts to make a model behave in a certain unintended way. They may try to get it to leak data, execute unauthorized tasks (especially with agentic systems), or ignore constraints. This is possible because LLMs are typically encapsulated inside applications using metaprompts, which are developer-created instructions that define the model’s behavior. Metaprompts usually contain safeguard instructions, such as 'do not use curse words', and placeholders where the input submitted by the user is pasted. The user’s input is combined with the metaprompts into a larger prompt that then goes to the model." (Abi Aryan, "LLMOps: Managing Large Language Models in Production", 2025)

"A major aspect of the alignment problem is values misalignment: AI systems may lack sufficient understanding of human ethics, cultural norms, or social contexts, making it difficult to translate our values directly into algorithms. For instance, if an AI is programmed to prioritise efficiency without balancing safety or ethical considerations, it might make decisions that, while effective, disregard human welfare. This is further complicated by 'specification gaming', where an AI might exploit loopholes in its programming to achieve objectives in unintended or counterproductive ways. An AI instructed to avoid obstacles, for instance, could redefine what counts as an 'obstacle' and take routes that, while technically following the rule, lead to negative outcomes." (Alan Watkins & G C Cooke, "Smarter than You Winning in Business with Superintelligent AI", 2026)

"AI governance manages the AI system itself. This includes the models, algorithms, and applications that use the data. It includes controls over model development, deployment, and monitoring, including documentation, versioning, validation, drift detection, and incident response. AI governance provides the structure for managing risk and accountability across the model life cycle, ensuring that AI systems perform as intended and remain compliant with organizational and regulatory requirements. It also includes responsible AI as a component of trusted AI governance and ensures that these principles are implemented as part of AI systems." (Fern Halper, "Data Makes the World Go 'Round", 2026)

"As AI systems become more autonomous and integrated into decision-making processes, the question of accountability grows in importance. Businesses and developers must ensure that AI systems operate within predefined ethical boundaries and that there are mechanisms to identify and correct issues when things go wrong. Because AI models can make decisions that directly impact people’s lives (such as credit approvals, hiring decisions, or healthcare recommendations), organizations must maintain accountability for these outcomes. Clear guidelines should be in place to determine who is responsible when an AI system makes a mistake or causes harm." (Bennie Haelen, "ML and Generative AI in the Data Lakehouse Building and Deploying AI Applications at Scale", 2026)

"Knowledge representation is the 'language' that AI systems use to structure, encode, and communicate information about the domain they are modelling. AI systems need it to reason or predict, and we use it to interpret the results they produce. The choice of how to represent knowledge can significantly affect the capabilities and limitations of the AI system." (Alessandro Negro et al, "Knowledge Graphs and LLMs in Action", 2026)

"[...] LLMs raise serious concerns about ethics, bias and fairness, errors in reasoning, hallucinations, and misuse (e.g., misinformation and disinformation). These concerns are exacerbated by modern LLMs being both literal and figurative 'black boxes': Literal black boxes because many advanced AI systems are proprietary and the weights (trained parameters of the models) are not released to the public; and figurative black boxes because even the open-source AI models are so complicated that understanding them and developing safety guardrails has thus far proven extremely difficult." (Mike X Cohen,"50 ML Projects To Understand LLMs", 2026)

"Semantic Kernel is a framework designed to simplify integrating LLMs into applications that require dynamic knowledge, reasoning, and state tracking. It’s particularly useful when you want to build complex, modular AI systems that can interact with external APIs, knowledge bases, or decision-making processes. Semantic Kernel focuses on building more flexible AI systems that can handle a variety of tasks beyond just generating text. It allows for modularity, enabling developers to easily combine different components - such as embeddings, prompt templates, and custom functions - in a cohesive manner." (Abi Aryan, "LLMOps: Managing Large Language Models in Production", 2025)

"The AI adoption challenges that businesses are facing share a common thread: AI is not a magic solution. Many businesses fall into the trap of expecting AI to solve their problems quickly, without fully understanding its limitations or properly integrating it into their operations. Data quality plays a crucial role in the success of AI systems; if the data are biased, incomplete, or outdated, the results will reflect those shortcomings. In addition, the human element is vital. AI should augment, not replace, human intelligence in areas that require empathy, creativity, or complex decision-making, such as dealing with the many experts right across the organisation. Additionally, AI systems need to be adaptable, accounting for external factors such as cultural differences, market changes, and unpredictable human behaviours." (Alan Watkins & G C Cooke, "Smarter than You Winning in Business with Superintelligent AI", 2026)

"The third battleground is the race to Artificial General Intelligence (AGI) – machines capable of performing any intellectual task that humans can. Although still in early skirmishes, this battle is intensifying, with leading technology companies and researchers making significant strides. [...] The race towards AGI is characterised by rapid technological advancements, differing expert opinions on timelines, ethical and regulatory considerations, and geopolitical dynamics. As AI systems become more sophisticated, the importance of responsible development and international cooperation becomes increasingly critical." (Alan Watkins & G C Cooke, "Smarter than You Winning in Business with Superintelligent AI", 2026)

🪙Business Intelligence: Control (Just the Quotes)

"90 percent of all problems can be solved by using the techniques of data stratification, histograms, and control charts. Among the causes of nonconformance, only one-fifth or less are attributable to the workers." (Kaoru Ishikawa, The Quality Management Journal Vol. 1, 1993)

"Alert-level control is another feature used to establish relevance within the content domain. Alerts help manage exceptions and alert the user of any unusual change or threshold value reached for any KPI. So, the action resulting from alerts needs to be assigned to those users who need to be informed of the exceptions." (Shadan Malik, "Enterprise Dashboards: Design and best practices for IT", 2005)

"The chaos without a data strategy is not as obvious, but the indicators abound: dirty data, redundant data, inconsistent data, the inability to integrate, poor performance, terrible availability, little accountability, users who are increasingly dissatisfied with the performance of IT, and the general feeling that things are out of control." (Sid Adelman et al, "Data Strategy", 2005)

"Metrics can serve two purposes: identifying problems and measuring performance. When the goal is to identify problems and pinpoint areas of operational inefficiency and ineffectiveness, defining the right metric requires a bit of detective work. It requires you to uncover the data residue of a problem and to determine what evidence can be found and how exactly it shows up. When the goal is to measure performance, the right success metrics focus on measures that can be controlled and where improvement in the metric is an unambiguously good thing." (Zach Gemignani et al, "Data Fluency", 2014)

"When statisticians, trained in math and probability theory, try to assess likely outcomes, they demand a plethora of data points. Even then, they recognize that unless it’s a very simple and controlled action such as flipping a coin, unforeseen variables can exert significant influence." (Zachary Karabell, "The Leading Indicators: A short history of the numbers that rule our world", 2014)

"Data mart: A subset of a data warehouse that’s usually oriented to a business group or process rather than enterprise-wide views. They have value as part of the overall enterprise data architecture, but can cause problems when they sprout uncontrolled as data silos with their own data definitions, creating data shadow systems." (Rick Sherman, "Business Intelligence Guidebook: From Data Integration to Analytics, 2015)

"Measurement is often seen as a tool that helps reduce the complexity of the world. Organizations, with their uncertainty and confusion, are full of people, patterns and trends; and measurement seems to offer a promise of bringing order, rationality and control into this chaos." (Dina Gray et al, "Measurement Madness: Recognizing and avoiding the pitfalls of performance measurement", 2015)

"Data Mesh attempts to strike a balance between team autonomy and inter-term interoperability and collaboration, with a few complementary techniques. It gives domain teams autonomy to have control of their local decision making, such as choosing the best data model for their data products. While it uses the computational governance policies to impose a consistent experience across all data products; for example, standardizing on the data modeling language that all domains utilize." (Zhamak Dehghani, "Data Mesh: Delivering Data-Driven Value at Scale", 2021)

"Data mesh [...] reduces points of centralization that act as coordination bottlenecks. It finds a new way of decomposing the data architecture without slowing the organization down with synchronizations. It removes the gap between where the data originates and where it gets used and removes the accidental complexities - aka pipelines - that happen in between the two planes of data. Data mesh departs from data myths such as a single source of truth, or one tightly controlled canonical data model." (Zhamak Dehghani, "Data Mesh: Delivering Data-Driven Value at Scale", 2021)

"Modern complex organizations increasingly confront the challenge of ensuring data quality at scale without centralizing validation activities into a single bottlenecked team. The data mesh paradigm and federated controls emerge as pivotal architectural styles and organizational patterns that enable decentralized, self-serve data quality validation while preserving coherence and reliability across diverse data products." (William Smith, "Great Expectations for Modern Data Quality: The Complete Guide for Developers and Engineers", 2025)

"As long as measurements are abused as a tool of control, measuring will remain the weakest area in a manager’s performance." (Peter Drucker)

🪙Business Intelligence: Trends (Just the Quotes)

"When numbers in tabular form are taboo and words will not do the work well as is often the case. There is one answer left: Draw a picture. About the simplest kind of statistical picture or graph, is the line variety. It is very useful for showing trends, something practically everybody is interested in showing or knowing about or spotting or deploring or forecasting." (Darell Huff, "How to Lie with Statistics", 1954)

"Alerts and KPI thresholds are two sides of the same coin. Alerts are actions taken once a KPI threshold is reached. However, alerts are not defined for every threshold boundary. For the most part, they serve as a warning system when a KPI shows poor performance or an undesired trend. Alerts must always be accompanied by attention-capturing actions such as automatic e-mails and/or visual indication such as blinking or animation on the dashboard. The other variable for alerts is the recipient. There may be one or more appropriate recipients for each alert." (Shadan Malik, "Enterprise Dashboards: Design and best practices for IT", 2005)

"For the purpose of choosing adequate forecasting methods, it is useful to dissect a time series into a systematic part and a non-systematic part. The systematic part is typically divided into three components: level , trend , and seasonality. The non-systematic part is called noise. The systematic components are assumed to be unobservable, as they characterize the underlying series, which we only observe with added noise." (Galit Shmueli, "Practical Time Series Forecasting: A Hands-On Guide", 2011)

"Some forecasting methods directly model these components by making assumptions about their structure. For example, a popular assumption about trend is that it is linear or exponential over parts, or all, of the given time period. Another common assumption is about the noise structure: many statistical methods assume that the noise follows a normal distribution. The advantage of methods that rely on such assumptions is that when the assumptions are reasonably met, the resulting forecasts will be more robust and the models more understandable." (Galit Shmueli, "Practical Time Series Forecasting: A Hands-On Guide", 2011)

"The process of visually exploring data can be summarized in a single sentence: find patterns and trends lurking in the data and then observe the deviations from those patterns. Interesting stories may arise from both the norm - also called the smooth - and the exceptions." (Alberto Cairo, "The Functional Art", 2011)

"Measurement is often seen as a tool that helps reduce the complexity of the world. Organizations, with their uncertainty and confusion, are full of people, patterns and trends; and measurement seems to offer a promise of bringing order, rationality and control into this chaos." (Dina Gray et al, "Measurement Madness: Recognizing and avoiding the pitfalls of performance measurement", 2015)

"Exploratory data analysis is the search for patterns and trends in a given data set. Visualization techniques play an important part in this quest. Looking carefully at your data is important for several reasons, including identifying mistakes in collection/processing, finding violations of statistical assumptions, and suggesting interesting hypotheses." (Steven S Skiena, "The Data Science Design Manual", 2017)

"These practices - selective reporting and data pillaging - are known as data grubbing. The discovery of statistical significance by data grubbing shows little other than the researcher’s endurance. We cannot tell whether a data grubbing marathon demonstrates the validity of a useful theory or the perseverance of a determined researcher until independent tests confirm or refute the finding. But more often than not, the tests stop there. After all, you won’t become a star by confirming other people’s research, so why not spend your time discovering new theories? The data-grubbed theory consequently sits out there, untested and unchallenged." (Gary Smith, "Standard Deviations", 2014)

"As presenters of data visualizations, often we just want our audience to understand something about their environment – a trend, a pattern, a breakdown, a way in which things have been progressing. If we ask ourselves what we want our audience to do with that information, we might have a hard time coming up with a clear answer sometimes. We might just want them to know something." (Ben Jones, "Avoiding Data Pitfalls: How to Steer Clear of Common Blunders When Working with Data and Presenting Analysis and Visualizations", 2020)

"Statistical metrics can show us facts and trends that would be impossible to see in any other way, but often they’re used as a substitute for relevant experience, by managers or politicians without specific expertise or a close-up view." (Tim Harford, "The Data Detective: Ten easy rules to make sense of statistics", 2020)

"Overall [...] everyone also has a need to analyze data. The ability to analyze data is vital in its understanding of product launch success. Everyone needs the ability to find trends and patterns in the data and information. Everyone has a need to ‘discover or reveal (something) through detailed examination’, as our definition says. Not everyone needs to be a data scientist, but everyone needs to drive questions and analysis. Everyone needs to dig into the information to be successful with diagnostic analytics. This is one of the biggest keys of data literacy: analyzing data." (Jordan Morrow, "Be Data Literate: The data literacy skills everyone needs to succeed", 2021)

"Data visualization, by contrast, focuses on the visual representation of data in such a way that its values, structure, nature, type, and variability are accurately expressed by means of graphs. It aims to support the exploration and understanding of data, the identi‑fication of patterns, trends, distributions, correlations, and anomalies, the communicationof insights, and aid in decision‑making." (Leandro N de Castro, "Exploratory Data Analysis: Descriptive Analysis, Visualization, and Dashboard Design", 2025)

"Differences in orientation can help us differentiate between items (e.g., data points, lines, objects, etc.) or extract information about the data. For example, using vertical bars in a bar chart can help differentiate between categories, while using horizontal bars can emphasize the magnitude of the data. Angles and direction can be used to convey information, such as trends, movement, sense of depth, or changes in values." (Leandro N de Castro, "Exploratory Data Analysis: Descriptive Analysis, Visualization, and Dashboard Design", 2025)

🔭Data Science: Issues (Collected Quotes)

"We can consider three broad classes of statistical pitfalls. The first involves sources of bias. These are conditions or circumstances which affect the external validity of statistical results. The second category is errors in methodology, which can lead to inaccurate or invalid results. The third class of problems concerns interpretation of results, or how statistical results are applied (or misapplied) to real world issues." (Clay Helberg, "Pitfalls of Data Analysis (or How to Avoid Lies and Damned Lies)", 1995)

"There are two problems with sampling - one obvious, and  the other more subtle. The obvious problem is sample size. Samples tend to be much smaller than their populations. [...] Obviously, it is possible to question results based on small samples. The smaller the sample, the less confidence we have that the sample accurately reflects the population. However, large samples aren't necessarily good samples. This leads to the second issue: the representativeness of a sample is actually far more important than sample size. A good sample accurately reflects (or 'represents') the population." (Joel Best, "Damned Lies and Statistics: Untangling Numbers from the Media, Politicians, and Activists", 2001)

"Given the important role that correlation plays in structural equation modeling, we need to understand the factors that affect establishing relationships among multivariable data points. The key factors are the level of measurement, restriction of range in data values (variability, skewness, kurtosis), missing data, nonlinearity, outliers, correction for attenuation, and issues related to sampling variation, confidence intervals, effect size, significance, sample size, and power." (Randall E Schumacker & Richard G Lomax, "A Beginner’s Guide to Structural Equation Modeling" 3rd Ed., 2010)

"There are several key issues in the field of statistics that impact our analyses once data have been imported into a software program. These data issues are commonly referred to as the measurement scale of variables, restriction in the range of data, missing data values, outliers, linearity, and nonnormality." (Randall E Schumacker & Richard G Lomax, "A Beginner’s Guide to Structural Equation Modeling" 3rd Ed., 2010)

"Machine learning is well suited for the unpredictable future, because most algorithms learn from new information. But as new information is found, it can also come in unstable forms, and new issues can arise that weren’t thought of before. We don’t know what we don’t know. When processing new information, it’s sometimes hard to tell whether our model is working." (Matthew Kirk, "Thoughtful Machine Learning", 2015)

"Hypotheses build the foundation for data analytics. Develop alternative hypotheses to explain the issue at hand. These hypotheses will guide your data collection." (Shonna D Watters et al, "The Practical Guide for HR Analytics: Using data to inform, transform, and empower HR decisions", 2019)

"A widely adopted technique for solving the data sparsity issue and the cold-start problem is based on graph representation, navigation, and processing. Graph navigation methods (like the pathfinding example [...]) and graph algorithms (such as PageRank) are applied to fill some gaps and create a denser representation of [a] dataset." (Alessandro Negro, "Graph-Powered Machine Learning", 2021)

"It is often said that data scientists and data analysts spend only 20% of their time doing data analysis work, with 80% consumed by data 'issues'. The bulk of their time is spent finding, evaluating, understanding, and preparing data before analysis can begin. A data catalog inverts this principle by enabling data analysts and data scientists to spend 20% of their time looking for data and 80% performing analysis." (Fadi Maali & Jason Lim, "Implementing a Modern Data Catalog to Power Data Intelligence: Make Trustworthy Data Central to Your Organization", 2022)

"Over 80% of models are never operationalized because the efforts involved in deploying them are enormous and the models are deployed and found to produce drift or fairness issues that outweigh the benefits."  (Eberhard Hechler et al, "Data Fabric and Data Mesh Approaches with AI", 2023)

"A common pitfall in ML delivery is the horizontal slicing of work, where we sequentially deliver functional layers of a technical solution - e.g., data lake, ML platform, ML models, UX interfaces - from the bottom-up. This is a risky delivery approach because customers can only experience the product and provide valuable feedback after months and even years of significant engineering investment. In addition, horizontal slicing naturally leads to late integration issues when horizontal slices come together, increasing the risk of release delays." (David Tan et al,"Effective Machine Learning Teams: Best Practices for ML Practitioners", 2024)

"Observability extends the concept of reliability. It’s about having the ability to monitor the health and performance of the data product. By using tools to track various metrics like response times and error rates, organizations can proactively manage the data product’s health. This proactive management plays a crucial role in maintaining the product’s reliability, as it allows for the early identification and resolution of potential issues before they escalate." (Jean-Georges Perrin & Eric Broda, "Implementing Data Mesh: Principles and Practice to Design, Build, and Implement Data Mesh", 2024)

"Driven by a fear of missing out (FOMO), many companies have launched AI initiatives. But many failed. For example, customer service chatbots have been introduced to handle inquiries, reduce costs and improve efficiency. But chatbots failed to grasp the complexity of customer issues; they created frustration rather than solving problems. Customers were stuck in repetitive loops, being asked the same questions, and unable to reach a human when needed. Instead of enhancing the customer experience, they delivered spikes in customer complaints instead." (Alan Watkins & G C Cooke, "Smarter than You Winning in Business with Superintelligent AI", 2026)

🖍️Alan Watkins - Collected Quotes

"A major aspect of the alignment problem is values misalignment: AI systems may lack sufficient understanding of human ethics, cultural norms, or social contexts, making it difficult to translate our values directly into algorithms. For instance, if an AI is programmed to prioritise efficiency without balancing safety or ethical considerations, it might make decisions that, while effective, disregard human welfare. This is further complicated by 'specification gaming', where an AI might exploit loopholes in its programming to achieve objectives in unintended or counterproductive ways. An AI instructed to avoid obstacles, for instance, could redefine what counts as an 'obstacle' and take routes that, while technically following the rule, lead to negative outcomes." (Alan Watkins & G C Cooke, "Smarter than You Winning in Business with Superintelligent AI", 2026)

"A transformer is a type of deep learning model designed to handle sequential data, such as text, more efficiently than previous RNNs. It uses a self-attention mechanism introduced [...] to process all parts of a sequence simultaneously, rather than one by one. This allows it to capture relationships between words (or data points) more effectively. The development of transformers revolutionised natural language processing and was a key leap forward. Transformers also used an encoder–decoder architecture. The encoder transforms the input into an abstract representation, and the decoder generates the output. So, by tokenising text into subword units and using parallel self-attention, it enables far greater throughput than RNNs. This design eliminates the need for recurrence (in RNNs) and achieves state-of-the-art results in tasks like translation and allows for a 1,000× speed-up using GPUs." (Alan Watkins & G C Cooke, "Smarter than You Winning in Business with Superintelligent AI", 2026)

"An alternative future is a more federated open-source AI where commoditised LLM can be run independently by anyone but loosely coordinated between each other if that is mutually advantageous. This requires sophisticated governance of an ecosystem where the rules that maintain the healthy stability of the ecosystem are defined and adhered to. This model allows anyone to put any LLM on a normal computer and plug into the AI. This is the path that already seems most sensible to business leaders and government officials, who recognise that their private data and iterative datasets hold value that can increase productivity and improve the bottom line or assist citizens in their lives." (Alan Watkins & G C Cooke, "Smarter than You Winning in Business with Superintelligent AI", 2026)

"Driven by a fear of missing out (FOMO), many companies have launched AI initiatives. But many failed. For example, customer service chatbots have been introduced to handle inquiries, reduce costs and improve efficiency. But chatbots failed to grasp the complexity of customer issues; they created frustration rather than solving problems. Customers were stuck in repetitive loops, being asked the same questions, and unable to reach a human when needed. Instead of enhancing the customer experience, they delivered spikes in customer complaints instead." (Alan Watkins & G C Cooke, "Smarter than You Winning in Business with Superintelligent AI", 2026)

"In game theory, the prisoner’s dilemma provides a powerful framework for understanding why companies might hesitate to adopt AI, even if it appears advantageous in the long run. For example, two competitors might invest in AI-driven automation to gain a potential edge, but if they do they also incur increased costs. Or they could both refrain from adopting AI-automation and avoid the associated risks and expenses. The dilemma arises because each company fears that if it chooses a different path to its competitors, it could be at a significant disadvantage, losing market share, efficiency, or innovation capability." (Alan Watkins & G C Cooke, "Smarter than You Winning in Business with Superintelligent AI", 2026)

"In RAG methods, the AI model itself doesn’t actually 'remember' or learn the proprietary data directly. Instead, the proprietary data are stored separately in what’s called a vector database. When the model is asked a question, it first performs a quick search of the proprietary database, finds relevant pieces of information, and then uses these to generate its response. The model’s core parameters remain entirely unchanged and are never updated with this private data. In this sense, the model hasn’t learned or 'seen' your proprietary data in its internal parameters, it only temporarily consults it as a reference to formulate an answer." (Alan Watkins & G C Cooke, "Smarter than You Winning in Business with Superintelligent AI", 2026)

"One major concern is goal misalignment, where an AI interprets its goals in unintended ways. This demonstrates the challenge of designing objectives that are both specific and safe and consider the wider context. Another issue is instrumental convergence, the idea that an AI might develop certain intermediate goals, like acquiring resources or ensuring its survival, that help it achieve its primary objective but also make it harder to control. These concerns tie into the broader value alignment problem: the difficulty of ensuring that AI systems act in accordance with human ethics and priorities." (Alan Watkins & G C Cooke, "Smarter than You Winning in Business with Superintelligent AI", 2026)

"So training is all about learning from data, while inference is about using that learning to make real-world predictions or classifications. Training builds the model’s capability, while inference applies that capability to new data." (Alan Watkins & G C Cooke, "Smarter than You Winning in Business with Superintelligent AI", 2026)

"Synthetic data are artificially created data that closely resembles real-world data, generated using algorithms, simulations, or machine learning models. Unlike real data, which are collected from actual events or user interactions, synthetic data are intentionally designed to share the statistical properties and patterns of real data. This makes it valuable for training and testing machine learning models, particularly in fields where real data may be limited, sensitive, or difficult to obtain. [...] One of the key benefits of synthetic data is its ability to overcome challenges related to data scarcity and privacy. In fields like healthcare and finance, privacy regulations often restrict access to sensitive data. By using synthetic data, developers can create and train models while maintaining user privacy. Additionally, synthetic data can be generated to cover rare or unique scenarios, which may not be well-represented in real-world data, making models more robust and better at handling fringe cases." (Alan Watkins & G C Cooke, "Smarter than You Winning in Business with Superintelligent AI", 2026)

"The AI adoption challenges that businesses are facing share a common thread: AI is not a magic solution. Many businesses fall into the trap of expecting AI to solve their problems quickly, without fully understanding its limitations or properly integrating it into their operations. Data quality plays a crucial role in the success of AI systems; if the data are biased, incomplete, or outdated, the results will reflect those shortcomings. In addition, the human element is vital. AI should augment, not replace, human intelligence in areas that require empathy, creativity, or complex decision-making, such as dealing with the many experts right across the organisation. Additionally, AI systems need to be adaptable, accounting for external factors such as cultural differences, market changes, and unpredictable human behaviours." (Alan Watkins & G C Cooke, "Smarter than You Winning in Business with Superintelligent AI", 2026)

"This AI-driven version of the prisoner’s dilemma captures a fundamental tension in business decision-making: competitive pressures can compel firms to take actions that aren’t necessarily aligned with their best interests. It highlights the risk of acting defensively out of fear and creating an unsustainable cycle of AI investment without strategic benefit, rather than fostering cooperative approaches that could yield mutual gains, such as industry-wide standards, ethical AI practices, or shared innovation efforts."(Alan Watkins & G C Cooke, "Smarter than You Winning in Business with Superintelligent AI", 2026)

"The crux of the disagreement lies in the perceived balance of risk and reward. Extinctionists stress that the stakes of getting AI wrong are so high, potentially existential, that precaution must take precedence. Expansionists, however, contend that an overly cautious approach could stifle innovation and prevent humanity from achieving its full potential, including partnering with AI to prevent alignment problems. Bridging this divide requires finding strategies to pursue the benefits of AI while addressing the legitimate concerns about its risks, a balance that continues to fuel heated debate in AI ethics and policy circles." (Alan Watkins & G C Cooke, "Smarter than You Winning in Business with Superintelligent AI", 2026)

"The third battleground is the race to Artificial General Intelligence (AGI) – machines capable of performing any intellectual task that humans can. Although still in early skirmishes, this battle is intensifying, with leading technology companies and researchers making significant strides. [...] The race towards AGI is characterised by rapid technological advancements, differing expert opinions on timelines, ethical and regulatory considerations, and geopolitical dynamics. As AI systems become more sophisticated, the importance of responsible development and international cooperation becomes increasingly critical." (Alan Watkins & G C Cooke, "Smarter than You Winning in Business with Superintelligent AI", 2026)

"Training is the process of teaching a model to recognise patterns and relationships in data and adjust over time to improve accuracy. During this phase, the model is fed large amounts of labelled data (data where the correct output is already known), and it tries to learn the associations between input data and its corresponding outputs. As the model processes the data, it repeatedly adjusts its internal parameters, like weights and biases, to reduce the difference between its predictions and the actual results (just like the human mind adjusts.) This iterative optimisation continues until the model reaches an acceptable level of accuracy." (Alan Watkins & G C Cooke, "Smarter than You Winning in Business with Superintelligent AI", 2026)

🔭Data Science: AI Models (Just the Quotes)

"Exploiting semantic knowledge graphs can support interpretability and explainability of nearly all AI model types (including DL models) by discovering and depicting semantic and non-obvious relationships or depicting an ML model in a simplified and more readable, explainable way." (Eberhard Hechler et al, "Data Fabric and Data Mesh Approaches with AI", 2023)

"Gaining more insight into data, simplifying data access, enabling shopping-for-data, augmenting traditional data governance, generating active metadata, and accelerating development of products and services are enabled by infusing AI into the Data Fabric architecture. An AI-infused Data Fabric is not only leveraging AI but also likewise an architecture to manage and deal with AI artefacts, including AI models, pipelines, etc." (Eberhard Hechler et al, "Data Fabric and Data Mesh Approaches with AI", 2023)

"In Exploiting semantic knowledge graphs can support interpretability and explainability of nearly all AI model types (including DL models) by discovering and depicting semantic and non-obvious relationships or depicting an ML model in a simplified and more readable, explainable way., a Data Mesh solution organizes data around business domain owners and transforms relevant data assets (data sources) to data products that can be consumed by distributed business users from various business domains or functions. These data products are created, governed, and used in an autonomous, decentralized, and self-service manner. Self-service capabilities, which we have already referenced as a Data Fabric capability, enable business organizations to entertain a data marketplace with shopping-for-data characteristics." (Eberhard Hechler et al, "Data Fabric and Data Mesh Approaches with AI", 2023)

"Large language models (LLMs) are AI models that are usually (but not necessarily) derived from the Transformer architecture and are designed to understand and generate human language, code, and much more. These models are trained on vast amounts of text data, allowing them to capture the complexities and nuances of human language. LLMs can perform a wide range of language-related tasks, from simple text classification to text generation, with high accuracy, fluency, and style." (Sinan Ozdemir, "Quick Start Guide to Large Language Models: Strategies and Best Practices for Using ChatGPT and Other LLMs", 2024)

"As AI systems become more autonomous and integrated into decision-making processes, the question of accountability grows in importance. Businesses and developers must ensure that AI systems operate within predefined ethical boundaries and that there are mechanisms to identify and correct issues when things go wrong. Because AI models can make decisions that directly impact people’s lives (such as credit approvals, hiring decisions, or healthcare recommendations), organizations must maintain accountability for these outcomes. Clear guidelines should be in place to determine who is responsible when an AI system makes a mistake or causes harm." (Bennie Haelen, "ML and Generative AI in the Data Lakehouse Building and Deploying AI Applications at Scale", 2026)

"In RAG methods, the AI model itself doesn’t actually 'remember' or learn the proprietary data directly. Instead, the proprietary data are stored separately in what’s called a vector database. When the model is asked a question, it first performs a quick search of the proprietary database, finds relevant pieces of information, and then uses these to generate its response. The model’s core parameters remain entirely unchanged and are never updated with this private data. In this sense, the model hasn’t learned or 'seen' your proprietary data in its internal parameters, it only temporarily consults it as a reference to formulate an answer." (Alan Watkins & G C Cooke, "Smarter than You Winning in Business with Superintelligent AI", 2026)

"[...] LLMs raise serious concerns about ethics, bias and fairness, errors in reasoning, hallucinations, and misuse (e.g., misinformation and disinformation). These concerns are exacerbated by modern LLMs being both literal and figurative 'black boxes': Literal black boxes because many advanced AI systems are proprietary and the weights (trained parameters of the models) are not released to the public; and figurative black boxes because even the open-source AI models are so complicated that understanding them and developing safety guardrails has thus far proven extremely difficult." (Mike X Cohen,"50 ML Projects To Understand LLMs", 2026)

"The Model Context Protocol (MCP) is emerging as a standard for connecting AI models to external tools and data sources. Historically, developers had to build custom connectors for each model and API, limiting portability and slowing innovation. MCP simplifies this by creating a common interface, essentially a universal plug like USB-C, that allows compliant models and tools to communicate seamlessly. Frameworks like LangChain/Graph and others are expected to adopt MCP to handle connectivity under the hood, providing developers with interoperability while maintaining secure and auditable access to enterprise systems." (Fern Halper, "Data Makes the World Go 'Round", 2026)

🪙Business Intelligence: Governance (Just the Quotes)

"The data architecture is the most important technical aspect of your business intelligence initiative. Fail to build an information architecture that is flexible, with consistent, timely, quality data, and your BI initiative will fail. Business users will not trust the information, no matter how powerful and pretty the BI tools. However, sometimes it takes displaying that messy data to get business users to understand the importance of data quality and to take ownership of a problem that extends beyond business intelligence, to the source systems and to the organizational structures that govern a company’s data." (Cindi Howson, "Successful Business Intelligence: Secrets to making BI a killer App", 2008)

"Data lake is an ecosystem for the realization of big data analytics. What makes data lake a huge success is its ability to contain raw data in its native format on a commodity machine and enable a variety of data analytics models to consume data through a unified analytical layer. While the data lake remains highly agile and data-centric, the data governance council governs the data privacy norms, data exchange policies, and the ensures quality and reliability of data lake." (Saurabh Gupta et al, "Practical Enterprise Data Lake Insights", 2018)

"Data swamp, on the other hand, presents the devil side of a lake. A data lake in a state of anarchy is nothing but turns into a data swamp. It lacks stable data governance practices, lacks metadata management, and plays weak on ingestion framework. Uncontrolled and untracked access to source data may produce duplicate copies of data and impose pressure on storage systems." (Saurabh Gupta et al, "Practical Enterprise Data Lake Insights", 2018)

"Data Mesh attempts to strike a balance between team autonomy and inter-term interoperability and collaboration, with a few complementary techniques. It gives domain teams autonomy to have control of their local decision making, such as choosing the best data model for their data products. While it uses the computational governance policies to impose a consistent experience across all data products; for example, standardizing on the data modeling language that all domains utilize." (Zhamak Dehghani, "Data Mesh: Delivering Data-Driven Value at Scale", 2021)

"Data lakes have been in existence for a while now, so their need is no longer questioned. What is more relevant is the specifics of the solution's implementation. Consolidating all the siloed data by itself does not constitute a data lake. However, it is a starting point. Layering in governance makes the data consumable and is a step toward a curated data lake. Big data systems provide scale out of the box but force us to make some accommodations for data quality. Age-old aspects of transactional integrity were compromised on a distributed system because it was very hard to maintain ACID compliance. Due to this, BASE properties were favored. All of this was moving the needle in the wrong direction and from pristine data lakes we were moving toward data swamps, where the data could not be trusted and hence insights that were generated on the data could not be trusted either." (Anindita Mahapatra, "Simplifying Data Engineering and Analytics with Delta", 2022)

"A data fabric is a pattern that is very similar to a data mesh in that both provide solutions encompassing data governance and self-service: discovery, access, security, integration, transformation, and lineage. [...] In simple terms, a data fabric is a metadriven means of connecting disparate sets of data and related tools to provide a cohesive data experience and to deliver data in a self-service manner." (Hubert Dulay & Stephen Mooney, "Streaming Data Mesh", 2023)

"Data Fabric focuses on Self-Service data access via active metadata leveraging a composable set of tools and technologies. It offers the ability to discover, understand, and access data across hybrid and multi-cloud data landscapes with automation and Data Governance. It is primarily process and technology centric with flexibility in supporting diverse organizational models. On the other hand, Data Mesh is organizationally and process driven. It requires a technical implementation approach to execute its design. Data Mesh is at a higher level and Data Fabric is at a lower level. Data Fabric is capable of fulfilling Data Mesh’s key principles." (Sonia Mezzetta, "Principles of Data Fabric: Become a data-driven organization by implementing Data Fabric solutions efficiently", 2023)

"Data Fabric’s building blocks represent groupings of different components and characteristics. They are high-level blocks that describe a package of capabilities that address specific business needs. The building blocks are Data Governance and its knowledge layer, Data Integration, and Self-Service." (Sonia Mezzetta, "Principles of Data Fabric: Become a data-driven organization by implementing Data Fabric solutions efficiently", 2023)

"The allure of Data Lakes was their ability to store vast amounts of raw data. However, this advantage can become counterproductive without stringent governance and management protocols. In their zeal to harness the power of Big Data, some organizations indiscriminately dump data into their lakes. Without proper classification, curation, and quality checks, these lakes can become swamps - murky repositories filled with valuable data, redundant information, and outdated datasets. Navigating these data swamps becomes a significant challenge, leading to prolonged data retrieval times, increased chances of using obsolete or incorrect data, and a decline in the agility and efficiency of data-driven decision-making processes rather than facilitating quick and insightful analytics." (Pradeep Menon, "Data Mesh Principles, patterns, architecture, and strategies for data-driven decision making", 2024)

"A lakehouse combines the scalability and flexibility of a data lake with the governance, structure, and performance of a data warehouse. It allows organizations to store both structured and unstructured data in one platform while supporting robust analytics, machine learning, and BI workloads. These lakehouses get by the data swamp problem by providing features that are database like. That includes support for ACID transactions." (Fern Halper, "Data Makes the World Go 'Round", 2026)

"AI maturity depends as much on people as it does on tools. Organizations need to build AI literacy across roles, from analysts to executives, so that teams can ask the right questions, interpret AI results, and act on them. Cross-functional teams with skills in data engineering, machine learning, business domain knowledge, and governance are essential. AI systems should be treated as products and not one-off projects." (Fern Halper, "Data Makes the World Go 'Round", 2026)

⛩️Douglas T Ross - Collected Quotes

"Automatic design has the computer do too much and the human do too little, whereas automatic programming has the human do too much and the computer do too little. Both techniques are important, but are not representative for what we wish to mean by computer-aided design." (Douglas T Ross, "Computer-Aided Design: A Statement of Objectives", 1960)

"Computer-aided design is not automatic design, although it must include many automatic design features. By automatic design we mean design procedures which are capable of being completely specified in a form which a computer can execute without human intervention." (Douglas T Ross, "Computer-Aided Design: A Statement of Objectives", 1960)

"It is very difficult to define what is meant by computer-aided design since the complete definition is, in fact, the sum and substance of the total project effort which has only begun. It is much easier to describe, what is not computer-aided design as we mean it." (Douglas T Ross, "Computer-Aided Design: A Statement of Objectives", 1960)

"The objective of the Computer-Aided Design Project is to evolve a machine systems which will permit the human designer and the computer to work together on creative design problems."  (Douglas T Ross, "Computer-Aided Design: A Statement of Objectives", 1960)

"Mechanical drawings and blueprints are not mere pictures, but a complete and rich language. In blueprint language, scientific, mathematical, and geometric formulations, notations, mensurations, and naming do not merely describe an object or process, they actually model it. Because of broad differences in subject, purpose, roles, and the needs of the people who use them, many forms of blueprint have evolved, but all rigorously present well structured information in understandable form." (Douglas T Ross, "Structured analysis (SA): A language for communicating ideas", IEEE Transactions on Software Engineering Vol. 3 No. 1, 1977)

"Structured analysis (SA) combines blueprint-like graphic language with the nouns and verbs of any other language to provide a hierarchic, top-down, gradual exposition of detail in the form of an SA model. The things and happenings of a subject are expressed in a data decomposition and an activity decomposition, both of which employ the same graphic building block, the SA box, to represent a part of a whole. SA arrows, representing input, output, control, and mechanism, express the relation of each part to the whole." (Douglas T Ross, "Structured analysis (SA): A language for communicating ideas", IEEE Transactions on Software Engineering Vol. 3 No. 1, 1977)

"The natural law of good communications takes the following, quite different, form in SA: Everything worth saying about anything worth saying something about must be expressed in six or fewer pieces." (Douglas T Ross, "Structured analysis (SA): A language for communicating ideas", IEEE Transactions on Software Engineering Vol. 3 No. 1, 1977)

"There are certain basic, known principles about how people's minds go about the business of understanding, and communicating understanding by means of language, which have been known and used for many centuries. No matter how these principles are addressed, they always end up with hierarchic decomposition as being the heart of good storytelling." (Douglas T Ross, "Structured analysis (SA): A language for communicating ideas", IEEE Transactions on Software Engineering Vol. 3 No. 1, 1977)

"We never have any understanding of any subject matter except in terms of our own mental constructs of ‘things’ and ‘happenings’ of that subject matter." (Douglas T Ross, "Structured analysis (SA): A language for communicating ideas", IEEE Transactions on Software Engineering Vol. 3 No. 1, 1977)

"A general theme for what I'm trying to convey and what actually drove me and my very industrious and creative project members over all these years, is… that there is much more to it than pictures. It has to be a picture language. There has to be meaning there, and the meaning is useful. You're trying to solve problems. So it really comes down to man machine problem solving. Better means of communication and expression is what always has driven our work." (Douglas T Ross, "Retrospectives: The Early Years in Computer Graphics at at MIT", Lincoln Lab and Harvard, 1989)

"There is a rigorous science, just waiting to be recognized and developed, which encompasses the whole of 'the software problem,' as defined, including the hardware, software, languages, devices, logic, data, knowledge, users, users, and effectiveness, etc. for end-users, providers, enablers, commissioners, and sponsors, alike." (Douglas T Ross,, 1989)

🤖Prompt Engineering: Domains (Just the Quotes)

"The idea behind transfer learning is that the pre-trained model has already learned a lot of information about the language and relationships between words, and this information can be used as a starting point to improve performance on a new task. Transfer learning allows LLMs to be fine-tuned for specific tasks with much smaller amounts of task-specific data than would be required if the model were trained from scratch. This greatly reduces the amount of time and resources needed to train LLMs." (Sinan Ozdemir, "Quick Start Guide to Large Language Models: Strategies and Best Practices for Using ChatGPT and Other LLMs", 2024)

"As the tech industry moves from non-generative models to generative models, it is shifting away from feature engineering, or creating features to model the data and experimenting with different hyperparameters to optimize performance. Generative models, and specifically LLMs, do not require feature engineering. Today, the core requirements are usually prompt engineering or building a RAG pipeline - skills that lie within the domain of AI engineers." (Abi Aryan, "LLMOps: Managing Large Language Models in Production", 2025)

"Context is crucial for how language models understand and generate code. The model processes your input by analyzing relationships between different parts of the code and documentation to determine meaning and intent. [...] The model evaluates context by calculating mathematical relationships between elements in your input. However, it may miss important domain knowledge, coding standards, or architectural patterns that experienced developers understand implicitly." (Jeremy C Morgan, "Coding with AI: Examples in Python", 2025)

"Despite their impressive capabilities, LLMs are not without limitations. One of the most significant challenges is the problem of hallucination, where an LLM generates factually incorrect or misleading information that appears plausible. This is particularly problematic in domains requiring high factual accuracy, such as healthcare, finance, and legal applications. To mitigate hallucinations and enhance the reliability of LLM outputs,  Retrieval-Augmented Generation (RAG) has emerged as a powerful technique. RAG works by dynamically retrieving relevant information from an external knowledge source (such as a knowledge graph) at inference time, rather than just relying on pre-trained knowledge. This approach ensures that the model has access to up-to-date and accurate data, grounding answers in verified information rather than generating content purely from its internal representations." (Aldo Marzullo et al, "Graph Machine Learning" 2nd Ed., 2025)

"In prompt engineering, we customize the prompts or questions we give the model to get more accurate or insightful responses. The way a prompt is structured has a massive impact on how well a model understands the task at hand and, ultimately, how well it performs. Given LLMs’ versatility, prompt engineering has become an important skill for getting the most out of these models across different domains and tasks. The key is to understand how different prompt structures lead to different model behaviors. There are various strategies - ranging from simple one-shot prompting to more complex techniques like chain-of-thought prompting - that can significantly improve the effectiveness of LLMs." (Abi Aryan, "LLMOps: Managing Large Language Models in Production", 2025)

 "There are three techniques for model domain adaptation: prompt engineering, RAG, and fine-tuning. Strictly speaking, RAG is a form of dynamic prompt engineering where developers use a retrieval system to add content to an existing prompt, but RAG systems are used so often that it’s worth discussing them separately. One critical difference with fine-tuning is that you must have access to the model’s weights, information that is usually not available with cloud-based, proprietary LLMs." (Abi Aryan, "LLMOps: Managing Large Language Models in Production", 2025)

"Generative artificial intelligence (GenAI), powered by large language models (LLMs) like Google’s Gemini and OpenAI’s GPT, has transformed how we work and live, revolutionizing business after business. Despite this success, generative AI falls short in domains where specific domain knowledge, high accuracy, and explainability are essential. And it has other significant limitations, including hallucinations and a lack of context and relations. This is where knowledge graphs (KGs) come in, provid-ing contextual information - such as experiences, environmental characteristics, cultural aspects, and social normsneeded to build the 'third wave of AI' for mission-critical applications." (Alessandro Negro et al, "Knowledge Graphs and LLMs in Action", 2026)

"KGs are sophisticated graph structures that represent real-world entities (people, places, diseases, proteins), define meaningful connections between them, and provide context. KGs provide structured, explainable knowledge representation but are challenging to build and query; LLMs offer natural language processing capabilities but suffer from hallucinations, stale information, and a lack of domain-specific grounding. Together, they are a 'killer combination': LLMs can extract entities and relationships from unstructured text to build KGs more efficiently, providing more autonomous and powerful graph querying and analysis. Meanwhile, KGs provide reliable, up-to-date domain knowledge to ground LLM responses and prevent hallucinations." (Alessandro Negro et al, "Knowledge Graphs and LLMs in Action", 2026)

"RAG is a paradigm that combines the strengths of LLMs with the rich, often unstructured data stored in a lakehouse. Rather than asking an LLM to generate responses purely from its internal parameters and training data, where knowledge can be outdated or incomplete, RAG systems first retrieve relevant documents, records, or data slices from your lakehouse and then feed those pieces into the model as context for its generative step. The result is an AI that can speak confidently about the latest reports, proprietary datasets, or domain-specific knowledge you have stored without having to retrain the model each time your data changes." (Bennie Haelen, "ML and Generative AI in the Data Lakehouse Building and Deploying AI Applications at Scale", 2026)

"Traditional paradigms build systems for specific purposes with structured, homogeneous databases. This approach works for tailored needs but is impractical for complex domains that need to adapt to user characteristics and integrate heterogeneous data. KGs capture connections, enabling relationship discovery through graph pattern matching and traversal. Both the Resource Description Framework (RDF) and Labeled Property Graphs (LPGs) provide machine-readable formats that humans can interpret. KGs emphasize rich, meaningful data representations usable by both humans and machines, enabling a paradigm shift where intelligent behavior is encoded in a unique source of truth." (Alessandro Negro et al, "Knowledge Graphs and LLMs in Action", 2026)

🤖Prompt Engineering: Intelligence (Just the Quotes)

"Deep learning has instead given us machines with truly impressive abilities but no intelligence. The difference is profound and lies in the absence of a model of reality." (Judea Pearl, "The Book of Why: The New Science of Cause and Effect", 2018)

"First, intelligence is situational - there is no such thing as general intelligence. Your brain is one piece in a broader system which includes your body, your environment, other humans, and culture as a whole. Second, it is contextual - far from existing in a vacuum, any individual intelligence will always be both defined and limited by its environment. (And currently, the environment, not the brain, is acting as the bottleneck to intelligence.) Third, human intelligence is largely externalized, contained not in your brain but in your civilization. Think of individuals as tools, whose brains are modules in a cognitive system much larger than themselves - a system that is self-improving and has been for a long time." (Erik J Larson, "The Myth of Artificial Intelligence: Why Computers Can’t Think the Way We Do", 2021)

"Inference is to bring about a new thought, which in logic amounts to drawing a conclusion, and more generally involves using what we already know, and what we see or observe, to update prior beliefs. […] Inference is also a leap of sorts, deemed reasonable […] Inference is a basic cognitive act for intelligent minds. If a cognitive agent (a person, an AI system) is not intelligent, it will infer badly. But any system that infers at all must have some basic intelligence, because the very act of using what is known and what is observed to update beliefs is inescapably tied up with what we mean by intelligence. If an AI system is not inferring at all, it doesn’t really deserve to be called AI." (Erik J Larson, "The Myth of Artificial Intelligence: Why Computers Can’t Think the Way We Do", 2021)

"The idea that we can predict the arrival of AI typically sneaks in a premise, to varying degrees acknowledged, that successes on narrow AI systems like playing games will scale up to general intelligence, and so the predictive line from artificial intelligence to artificial general intelligence can be drawn with some confidence. This is a bad assumption, both for encouraging progress in the field toward artificial general intelligence, and for the logic of the argument for prediction." (Erik J Larson, "The Myth of Artificial Intelligence: Why Computers Can’t Think the Way We Do", 2021)

"The problem-solving view of intelligence helps explain the production of invariably narrow applications of AI throughout its history. Game playing, for instance, has been a source of constant inspiration for the development of advanced AI techniques, but games are simplifications of life that reward simplified views of intelligence. […] Treating intelligence as problem-solving thus gives us narrow applications." (Erik J Larson, "The Myth of Artificial Intelligence: Why Computers Can’t Think the Way We Do", 2021)

"Agentic intelligence feels incredibly powerful in demos but breaks in production. Indeed, it is very fragile without solid infrastructure. Every day, I personally see tons of clever orchestrations around dumb prompt chains tied up in a brittle, underused LLMOps infrastructure. But building this infrastructure means acknowledging the costs: performance overhead, strict interface contracts, and state complexity, as well as a need for more LLMOps engineers to create the best practices, tooling, and frameworks to run these systems reliably, safely, and robustly." (Abi Aryan, "LLMOps: Managing Large Language Models in Production", 2025)

"Agentic AI extends GenAI by embedding intelligence within autonomous or semi-autonomous systems that can plan, reason, and take actions within defined boundaries. Instead of simply generating a report, an agentic system might determine which data it needs, retrieve that data from multiple sources, perform analysis, summarize the results, and then trigger follow-up workflows, all while maintaining auditability and alignment with governance policies." (Fern Halper, "Data Makes the World Go 'Round", 2026)

"Intelligent systems connect users to AI and ML to achieve meaningful objectives. An intelligent system is one in which intelligence evolves and improves over time, particularly when it improves by watching how users interact with the system.[...] The primary objective of the intelligent system is to support users in accomplishing complex tasks - not by replacing them, but by enhancing their decision-making capabilities. [...] An intelligent system must also have the ability to learn from user interactions and explicit feedback, as well as utilize contextual information. The system should contin-uously develop, use, and maintain an evolving knowledge base. This evolution is driven not only by data sources but also by ongoing interactions with users." (Alessandro Negro et al, "Knowledge Graphs and LLMs in Action", 2026)

"LangGraph handles the perception, reasoning, and action flow while maintaining memory and context across tasks. In this sense, it functions as both the development environment and the orchestration layer - coordinating the steps of perception, reasoning, and action while managing connections to external systems. Underneath this, emerging standards like MCP ensure that agents can connect securely and consistently to tools and data sources, making agentic architectures portable and scalable across platforms. Taken together, the orchestration layer and emerging interoperability standards like MCP form the foundation for scalable agentic AI. They make it possible for agents to perceive, reason, act, and learn in coordinated ways across complex environments, translating autonomous intelligence into practical, enterprise-grade capability." (Fern Halper, "Data Makes the World Go 'Round", 2026)

🤖Prompt Engineering: Retrieval Augmented Generation [RAG] (Just the Quotes)

"As the tech industry moves from non-generative models to generative models, it is shifting away from feature engineering, or creating features to model the data and experimenting with different hyperparameters to optimize performance. Generative models, and specifically LLMs, do not require feature engineering. Today, the core requirements are usually prompt engineering or building a RAG pipeline - skills that lie within the domain of AI engineers." (Abi Aryan, "LLMOps: Managing Large Language Models in Production", 2025)

"Despite their impressive capabilities, LLMs are not without limitations. One of the most significant challenges is the problem of hallucination, where an LLM generates factually incorrect or misleading information that appears plausible. This is particularly problematic in domains requiring high factual accuracy, such as healthcare, finance, and legal applications. To mitigate hallucinations and enhance the reliability of LLM outputs,  Retrieval-Augmented Generation (RAG) has emerged as a powerful technique. RAG works by dynamically retrieving relevant information from an external knowledge source (such as a knowledge graph) at inference time, rather than just relying on pre-trained knowledge. This approach ensures that the model has access to up-to-date and accurate data, grounding answers in verified information rather than generating content purely from its internal representations." (Aldo Marzullo et al, "Graph Machine Learning" 2nd Ed., 2025)

 "There are three techniques for model domain adaptation: prompt engineering, RAG, and fine-tuning. Strictly speaking, RAG is a form of dynamic prompt engineering where developers use a retrieval system to add content to an existing prompt, but RAG systems are used so often that it’s worth discussing them separately. One critical difference with fine-tuning is that you must have access to the model’s weights, information that is usually not available with cloud-based, proprietary LLMs." (Abi Aryan, "LLMOps: Managing Large Language Models in Production", 2025)

"RAG is a framework that combines the strengths of traditional information retrieval systems with the generative capabilities of LLMs. In this setup, an LLM is augmented with a retrieval component that fetches relevant information from external data sources, such as knowledge bases or databases, to produce more accurate and contextually relevant responses. This method enhances the LLM’s output by grounding it in authoritative, up-to-date information." (Aldo Marzullo et al, "Graph Machine Learning" 2nd Ed., 2025)

"Vector databases are designed to store and index high-dimensional embeddings - dense numeric vectors that capture the semantic meaning of text, images, audio, or other content. Instead of looking for exact matches, they use approximate nearest neighbor (ANN) algorithms to return the items whose vectors lie closest to a query vector in that multidimensional space. This makes them the engine behind semantic search, recommendation systems, image-or-audio similarity matching, and retrieval augmented generation (RAG) pipelines that supply LLM prompts with relevant context in milliseconds." (Abi Aryan, "LLMOps: Managing Large Language Models in Production", 2025)

"In RAG methods, the AI model itself doesn’t actually 'remember' or learn the proprietary data directly. Instead, the proprietary data are stored separately in what’s called a vector database. When the model is asked a question, it first performs a quick search of the proprietary database, finds relevant pieces of information, and then uses these to generate its response. The model’s core parameters remain entirely unchanged and are never updated with this private data. In this sense, the model hasn’t learned or 'seen' your proprietary data in its internal parameters, it only temporarily consults it as a reference to formulate an answer." (Alan Watkins & G C Cooke, "Smarter than You Winning in Business with Superintelligent AI", 2026)

"RAG applications must be built with semantics, metadata, and governance in mind. The retrieved information must be high-quality, secure, and appropriate for the user’s role. Equally important is monitoring and management: checking whether source data has changed, ensuring vector stores remain accurate, and watching for hallucinations or data leakage. Organizations are definitely starting to experiment with RAG models today; some are putting them into production applications. Some believe that using RAG helps mitigate hallucinations because it is grounded in trusted organizational data." (Fern Halper, "Data Makes the World Go 'Round", 2026)

"RAG is a paradigm that combines the strengths of LLMs with the rich, often unstructured data stored in a lakehouse. Rather than asking an LLM to generate responses purely from its internal parameters and training data, where knowledge can be outdated or incomplete, RAG systems first retrieve relevant documents, records, or data slices from your lakehouse and then feed those pieces into the model as context for its generative step. The result is an AI that can speak confidently about the latest reports, proprietary datasets, or domain-specific knowledge you have stored without having to retrain the model each time your data changes." (Bennie Haelen, "ML and Generative AI in the Data Lakehouse Building and Deploying AI Applications at Scale", 2026)

"The foundational form of RAG, often called naive RAG, follows a straightforward pattern. A pipeline retrieves supporting context from external sources such as enterprise documents, knowledge bases, or structured datasets and appends that information to the model’s prompt before inference. In the most common implementation, each document is converted into an embedding, a numerical representation of its semantic meaning, using either the same foundation model or a specialized embedding model. When a user submits a query, the system performs a vector similarity search to find documents whose embeddings most closely match the query’s vector representation, and the retrieved content is concatenated with the user query before being passed to the language model." (Bennie Haelen, "ML and Generative AI in the Data Lakehouse Building and Deploying AI Applications at Scale", 2026)

🖍️Nick Bostrom - Collected Quotes

"A genie is a command-executing system: it receives a high-level command, carries it out, then pauses to await the next command. A sovereign is a system that has an open-ended mandate to operate in the world in pursuit of broad and possibly very long-range objectives. Although these might seem like radically different templates for what a superintelligence should be and do, the difference is not as deep as it might at first glance appear." (Nick Bostrom, "Superintelligence", 2014)

"A question distinct from, but related to, the question of kinetics is whether there will be one superintelligent power or many? Might an intelligence explosion propel one project so far ahead of all others as to make it able to dictate the future? Or will progress be more uniform, unfurling across a wide front, with many projects participating but none securing an overwhelming and permanent lead?" (Nick Bostrom, "Superintelligence", 2014)

"A system that has the intelligence amplification superpower could use it to bootstrap itself to higher levels of intelligence and to acquire any of the other intellectual superpowers that it does not possess at the outset. But using an intelligence amplification superpower is not the only way for a system to become a full-fledged superintelligence. A system that has the strategizing superpower, for instance, might use it to devise a plan that will eventually bring an increase in intelligence (e.g. by positioning the system so as to become the focus for intelligence amplification work performed by human programmers and computer science researchers)." (Nick Bostrom, "Superintelligence", 2014)

"A system might thus greatly boost its effective intellectual capability by absorbing pre-produced content accumulated through centuries of human science and civilization: for instance, by reading through the internet. If an AI reaches human level without previously having had access to this material or without having been able to digest it, then the AI’s overall recalcitrance will be low even if it is hard to improve its algorithmic architecture." (Nick Bostrom, "Superintelligence", 2014)

"An agent’s ability to shape humanity’s future depends not only on the absolute magnitude of the agent’s own faculties and resources - how smart and energetic it is, how much capital it has, and so forth - but also on the relative magnitude of its capabilities compared with those of other agents with conflicting goals." (Nick Bostrom, "Superintelligence", 2014) 

"If some day we build machine brains that surpass human brains in general intelligence, then this new superintelligence could become very powerful. And, as the fate of the gorillas now depends more on us humans than on the gorillas themselves, so the fate of our species would depend on the actions of the machine superintelligence." (Nick Bostrom, "Superintelligence: Paths, Dangers, Strategies", 2014)

"Improvements in rationality and intelligence will tend to improve an agent’s decision-making, rendering the agent more likely to achieve its final goals. One would therefore expect cognitive enhancement to emerge as an instrumental goal for a wide variety of intelligent agents. For similar reasons, agents will tend to instrumentally value many kinds of information." (Nick Bostrom, "Superintelligence: Paths, Dangers, Strategies", 2014) 

"Once machines attain some form of human-equivalence in general reasoning ability, how long will it then be before they attain radical superintelligence? Will this be a slow, gradual, protracted transition? Or will it be sudden, explosive?" (Nick Bostrom, "Superintelligence: Paths, Dangers, Strategies", 2014)

"Some paths to superintelligence require great resources and are therefore likely to be the preserve of large well-funded projects. Whole brain emulation, for instance, requires many different kinds of expertise and lots of equipment. Biological intelligence enhancements and brain–computer interfaces would also have a large scale factor: while a small biotech firm might invent one or two drugs, achieving superintelligence along one of these paths (if doable at all) would likely require many inventions and many tests, and therefore the backing of an industrial sector or a well-funded national program. Achieving collective superintelligence by making organizations and networks more efficient requires even more extensive input, involving much of the world economy." (Nick Bostrom, "Superintelligence: Paths, Dangers, Strategies", 2014)

"Suppose that a digital superintelligent agent came into being, and that for some reason it wanted to take control of the world: would it be able to do so?" (Nick Bostrom, "Superintelligence: Paths, Dangers, Strategies", 2014) 

"The fact that there are many paths that lead to superintelligence should increase our confidence that we will eventually get there. If one path turns out to be blocked, we can still progress." (Nick Bostrom, "Superintelligence: Paths, Dangers, Strategies", 2014)

"Various considerations thus point to an increased likelihood that a future power with superintelligence that obtained a sufficiently large strategic advantage would actually use it to form a singleton. The desirability of such an outcome depends, of course, on the nature of the singleton that would be created and also on what the future of intelligent life would look like in alternative multipolar scenarios." (Nick Bostrom, "Superintelligence: Paths, Dangers, Strategies", 2014)

"We can tentatively define a superintelligence as any intellect that greatly exceeds the cog‐ nitive performance of humans in virtually all domains of interest." (Nick Bostrom, "Superintelligence: Paths, Dangers, Strategies", 2014)

"We have seen that a superintelligence could have a great ability to shape the future according to its goals. But what will its goals be? What is the relation between intelligence and motivation in an artificial agent?" (Nick Bostrom, "Superintelligence: Paths, Dangers, Strategies", 2014)

"[...] we use the term 'superintelligence' to refer to intellects that greatly outperform the best current human minds across many very general cognitive domains. This is still quite vague. Different kinds of system with rather disparate performance attributes could qualify as superintelligences under this definition. To advance the analysis, it is helpful to disaggregate this simple notion of superintelligence by distinguishing different bundles of intellectual super-capabilities. There are many ways in which such decomposition could be done. Here we will differentiate between three forms: speed superintelligence, collective superintelligence, and quality superintelligence." (Nick Bostrom, "Superintelligence: Paths, Dangers, Strategies", 2014)

🤖Prompt Engineering: Transformers (Just the Quotes)

"Self-attention, sometimes called intra-attention is an attention mechanism relating different positions of a single sequence in order to compute a representation of the sequence. Self-attention has been used successfully in a variety of tasks including reading comprehension, abstractive summarization, textual entailment and learning task-independent sentence representations.  End-to-end memory networks are based on a recurrent attention mechanism instead of sequence-aligned recurrence and have been shown to perform well on simple-language question answering and language modeling tasks. To the best of our knowledge, however, the Transformer is the first transduction model relying entirely on self-attention to compute representations of its input and output without using sequence-aligned RNNs or convolution." (Ashish Vaswani et al, "Attention Is All You Need", 2017) [source] 

"Attention is a mechanism used in deep learning models (not just Transformers) that assigns different weights to different parts of the input, allowing the model to prioritize and emphasize the most important information while performing tasks like translation or summarization. Essentially, attention allows a model to 'focus' on different parts of the input dynamically, leading to improved performance and more accurate results. Before the popularization of attention, most neural networks processed all inputs equally and the models relied on a fixed representation of the input to make predictions. Modern LLMs that rely on attention can dynamically focus on different parts of input sequences, allowing them to weigh the importance of each part in making predictions." (Sinan Ozdemir, "Quick Start Guide to Large Language Models: Strategies and Best Practices for Using ChatGPT and Other LLMs", 2024)

"Large language models (LLMs) are AI models that are usually (but not necessarily) derived from the Transformer architecture and are designed to understand and generate human language, code, and much more. These models are trained on vast amounts of text data, allowing them to capture the complexities and nuances of human language. LLMs can perform a wide range of language-related tasks, from simple text classification to text generation, with high accuracy, fluency, and style." (Sinan Ozdemir, "Quick Start Guide to Large Language Models: Strategies and Best Practices for Using ChatGPT and Other LLMs", 2024) 

"[...] LLMs are pre-trained on large corpora and sometimes fine-tuned on smaller datasets for specific tasks. Recall that one of the factors behind the Transformer’s effectiveness as a language model is that it is highly parallelizable, allowing for faster training and efficient processing of text. What really sets the Transformer apart from other deep learning architectures is its ability to capture long-range dependencies and relationships between tokens using attention. In other words, attention is a crucial component of Transformer-based LLMs, and it enables them to effectively retain information between training loops and tasks (i.e., transfer learning), while being able to process lengthy swatches of text with ease." (Sinan Ozdemir, "Quick Start Guide to Large Language Models: Strategies and Best Practices for Using ChatGPT and Other LLMs", 2024)

"This ability to zero in on important code is why modern AI coding assistants can offer meaningful suggestions for your specific needs. It’s similar to how skilled developers know which code sections affect a new implementation the most. Each transformer layer learns about various code patterns, ranging from syntax validation to understanding the relationships among functions, classes, and modules." (Jeremy C Morgan, "Coding with AI: Examples in Python", 2025)

"Transformers are complex models built like LEGO blocks using multiple smart and specialized components. [...] Briefly, a vanilla transformer model consists of separate stacks of encoders and decoders. Each encoder block includes multi-head self-attention, enabling the model to capture relationships between tokens regardless of their positions. Residual connections help maintain gradient flow, preventing the vanishing gradient problem. Layer normalization ensures training stability, and feed-forward layers introduce non-linearity and learn complex token interactions. Decoder blocks contain the same components but also include an encoder-decoder attention mechanism to incorporate context from the encoder. The model uses embedding layers to convert tokens into a continuous latent space for contextual learning and positional encoding to preserve the order of tokens in the sequence." (Joseph Babcock & Raghav Bali, "Generative AI with Python and PyTorch" 2nd. Ed., 2025)

"A transformer is a type of deep learning model designed to handle sequential data, such as text, more efficiently than previous RNNs. It uses a self-attention mechanism introduced [...] to process all parts of a sequence simultaneously, rather than one by one. This allows it to capture relationships between words (or data points) more effectively. The development of transformers revolutionised natural language processing and was a key leap forward. Transformers also used an encoder–decoder architecture. The encoder transforms the input into an abstract representation, and the decoder generates the output. So, by tokenising text into subword units and using parallel self-attention, it enables far greater throughput than RNNs. This design eliminates the need for recurrence (in RNNs) and achieves state-of-the-art results in tasks like translation and allows for a 1,000× speed-up using GPUs." (Alan Watkins & G C Cooke, "Smarter than You Winning in Business with Superintelligent AI", 2026)

"GenAI encompasses a variety of model architectures, including diffusion models, normalizing flows, and autoregressive models. However, three types of GenAI have become particularly dominant in practical applications. GANs pit two neural networks against each other to produce realistic data. VAEs encode input data into a latent space and decode it to generate new samples. Transformer-based models, like those in the GPT, Llama, and Gemini families, leverage attention mechanisms to generate coherent and contextually relevant sequences." (Bennie Haelen, "ML and Generative AI in the Data Lakehouse Building and Deploying AI Applications at Scale", 2026)

"Transformers dominate language for several reasons: their autoregressive framework provides a straightforward training objective (predict next token), self-attention captures long-range dependencies across thousands of tokens, parallel training processes entire sequences simultaneously (unlike RNNs’ sequential bottleneck), and scaling behavior produces emergent capabilities at billions of parameters. These properties make transformers uniquely suited for the language understanding and generation tasks central to enterprise AI applications [...]" (Bennie Haelen, "ML and Generative AI in the Data Lakehouse Building and Deploying AI Applications at Scale", 2026)

"Transformer-based models represent the dominant architecture for processing sequential data, particularly text. Unlike GANs and VAEs, which excel at generating visual content, transformers have revolutionized natural language processing (NLP) and are now being applied to code, music, and even protein structures. At their core, transformers use a mechanism called self-attention that allows the model to consider relationships between all elements in a sequence simultaneously. This enables transformers to capture long-range dependencies and contextual nuances that earlier sequential models struggled with." (Bennie Haelen, "ML and Generative AI in the Data Lakehouse Building and Deploying AI Applications at Scale", 2026) 

"Unlike earlier neural network designs that processed text sequentially, transformers use a mechanism called self-attention that allows the model to consider relationships between all words in a passage simultaneously. This parallel processing enables transformers to capture long-range dependencies and contextual nuances that previous architectures struggled with. Positional encoding provides information about word order, while the attention mechanism allows the model to focus on relevant context when processing each word." (Bennie Haelen, "ML and Generative AI in the Data Lakehouse Building and Deploying AI Applications at Scale", 2026)

🖍️Hayden Van Der Post - Collected Quotes

"A critical aspect of neural networks is their ability to learn from data. This learning occurs during the training phase, where the network is exposed to vast datasets, allowing it to adjust its internal parameters - the weights and biases associated with each neuron. The goal of this adjustment is to minimize the difference between the network's predictions and the actual outcomes, a process known as optimization. Through techniques such as gradient descent and backpropagation, neural networks iteratively refine their parameters, enhancing their ability to make accurate predictions or decisions based on new input."(Hayden Van Der Post, "Neural Network: Mastering the Art of Algorithmic Trading", 2024)

"Beyond the choice of model, the accuracy of neural network predictions in trading significantly depends on the quality and relevance of the data fed into them. This underscores the importance of meticulous data preparation, encompassing cleaning, normalization, and feature engineering. By ensuring that the input data is reflective of the market's complexities, traders can fine-tune their neural networks to produce more accurate and actionable predictions." (Hayden Van Der Post, "Neural Network: Mastering the Art of Algorithmic Trading", 2024)

"Deep learning algorithms are exceptionally adept at identifying anomalies within large datasets, making them an indispensable tool for detecting fraudulent transactions and financial irregularities. By learning from historical transaction data, these models can pinpoint patterns and behaviors indicative of fraudulent activities with remarkable accuracy. This ability not only aids in safeguarding assets but also ensures compliance with increasingly stringent regulatory standards aimed at preventing financial fraud and misconduct." (Hayden Van Der Post, "Neural Network: Mastering the Art of Algorithmic Trading", 2024)

"Deep learning is an advanced subset of machine learning, distinguished by its ability to process data through layers of neural networks, each layer abstracting information from the one preceding it. This hierarchical approach enables the model to handle complex, high-dimensional data, learning features and patterns at multiple levels of abstraction. [...] Traditional neural networks, with their shallower architectures, often struggle with the nuances of financial data, limited by their capacity to extrapolate and interpret intricate patterns. Deep learning, however, with its deeper, more sophisticated networks, can navigate these complexities, offering nuanced insights into market dynamics." (Hayden Van Der Post, "Neural Network: Mastering the Art of Algorithmic Trading", 2024)

"Feature selection is the first critical step in model optimization. It's the art of identifying which features in your dataset contribute most significantly to the outcome you're trying to predict. This not only helps in enhancing the model's accuracy but also in reducing computational complexity, leading to more efficient models. [...] While feature selection is about cherry-picking the most useful features, feature engineering is about creating new features that increase the predictive strength of the model. This is where creativity and domain knowledge come into play, especially in financial data, where market sentiment, economic indicators, and other external factors can influence market movements." (Hayden Van Der Post, "Neural Network: Mastering the Art of Algorithmic Trading", 2024)

"Genetic programming represents a frontier in feature engineering, allowing for the automated creation of new features through the application of evolutionary algorithms. By combining existing features in non-linear and complex ways, genetic programming can uncover hidden relationships in the data that were not apparent through manual exploration. This technique, while computationally intensive, holds the promise of discovering novel predictors that can enhance the performance of trading algorithms." (Hayden Van Der Post, "Neural Network: Mastering the Art of Algorithmic Trading", 2024)

"Neural networks are structured into layers, each comprising a collection of neurons. The arrangement begins with an input layer, which receives the raw data. This is followed by one or more hidden layers, where the actual processing happens through a complex web of interconnected neurons. The journey through the layers culminates in an output layer, where the network delivers its final decision or prediction. The hidden layers are the cradle of the network’s learning capability, enabling it to detect patterns, make associations, and refine its predictions through repeated exposure to data." (Hayden Van Der Post, "Neural Network: Mastering the Art of Algorithmic Trading", 2024)

"Predictive analytics, involves the use of data, statistical algorithms, and machine learning techniques to identify the likelihood of future outcomes based on historical data. Neural networks, with their remarkable ability to learn and model complex patterns, have become the backbone of modern forecasting methods. Their application ranges from predicting consumer behavior in retail to forecasting the stock market trends, from anticipating weather patterns to foreseeing potential healthcare outbreaks." (Hayden Van Der Post, "Neural Network: Mastering the Art of Algorithmic Trading", 2024)

"The beauty of neural networks lies in their ability to learn and improve. Through a process known as 'training', a neural network is fed large amounts of data along with feedback on its performance. This feedback guides the network in adjusting its internal parameters, known as weights, to minimize errors in its predictions. This iterative process of learning from mistakes closely mirrors the cognitive and learning processes of the human brain, making neural networks particularly adept at tasks that involve pattern recognition, such as image and speech recognition [...]" (Hayden Van Der Post, "Neural Network: Mastering the Art of Algorithmic Trading", 2024)

"The essence of backpropagation is captured in the gradient descent algorithm, which seeks to minimize the error by iteratively adjusting the weights in the direction that most steeply decreases the error function. This rigorous process requires a meticulous balance; too large a weight adjustment can lead to erratic learning, while too small an adjustment can trap the network in local minima. In algorithmic trading, the capacity to learn from past predictions and refine strategies accordingly is invaluable, allowing for the continual optimization of trading algorithms in alignment with market dynamics." (Hayden Van Der Post, "Neural Network: Mastering the Art of Algorithmic Trading", 2024)

"The power of neural networks lies in their flexibility and adaptability. They are not confined to a single type of problem or dataset but can be tailored to a wide range of applications, from voice recognition and image classification to forecasting financial market movements. This versatility stems from the network's ability to capture and model complex, non-linear relationships within the data it is trained on, making it a potent tool in the arsenal of data scientists and algorithmic traders alike." (Hayden Van Der Post, "Neural Network: Mastering the Art of Algorithmic Trading", 2024)

"Weights and biases are pivotal in shaping the neural network's decision-making process. Weights determine the strength of the connection between two neurons, influencing how much of the input signal is passed forward. Biases, added to the weighted sum before the activation function, allow neurons to adjust their output independently of their input, providing an additional degree of freedom. The process of learning in a neural network involves adjusting these weights and biases based on the error between the network's predictions and the actual data, typically using an algorithm like gradient descent." (Hayden Van Der Post, "Neural Network: Mastering the Art of Algorithmic Trading", 2024)

"While forward propagation provides the prediction, backpropagation is the mechanism through which a neural network learns from its errors and enhances its accuracy. Backpropagation, a form of reverse engineering of the forward propagation process, involves calculating the error between the predicted output and the actual output, and then distributing this error back through the network. This distribution occurs layer by layer, in reverse order from output to input, adjusting the weights of the connections based on the magnitude of the error." (Hayden Van Der Post, "Neural Network: Mastering the Art of Algorithmic Trading", 2024)

🤖Prompt Engineering: Challenges (Just the Quotes)

"Another problem that can be confusing is that LLMs seldom put out the same thing twice. [...] Traditional databases are straightforward - you ask for something specific, and you get back exactly what was stored. Search engines work similarly, finding existing information. LLMs work differently. They analyze massive amounts of text data to understand statistical patterns in language. The model processes information through multiple layers, each capturing different aspects - from simple word patterns to complex relationships between ideas." (Jeremy C Morgan, "Coding with AI: Examples in Python", 2025)

"Chain-of-thought prompting is a method that forces LLMs to reason through a series of steps, resulting in more structured, transparent, and precise outputs. The goal is to break down complex tasks into smaller, interconnected subtasks, allowing the LLM to address each subtask in a stepby-step manner. This not only helps the model to 'focus' on specific aspects of the problem, but also encourages it to generate intermediate outputs, making it easier to identify and debug potential issues along the way. Another significant advantage of chain-of-thought prompting is the improved interpretability and transparency of the LLM-generated response. By offering insights into the model’s reasoning process, we, as users, can better understand and qualify how the final output was derived, which promotes trust in the model’s decision-making abilities." (Sinan Ozdemir, "Quick Start Guide to Large Language Models: Strategies and Best Practices for Using ChatGPT and Other LLMs", 2024) 

"AI isn’t just going to be about our digital world. It’s also about our physical world; and applied properly, imagine what AI can do for the pace of discovery and innovation. It’s not just makeup; imagine what it can do for new materials discovery for medicine, energy, climate, and all the other pressing challenges we face as a species - these are the same challenges of makeup, just described with a different 'language'. And quantum computing evolves, we’re bound to see a synergy of these innovations that we can use to tackle these problem domains and more." (Rob Thomas et al, "AI Value Creators: Beyond the Generative AI User Mindset", 2025)

"LLMs can inadvertently produce toxic content or biased language, leak private information, or be vulnerable to jailbreak prompts. These risks carry serious legal and reputational consequences. To mitigate them, evaluation tools must integrate automated filters and classifiers that flag problematic outputs in real time, as we discussed earlier in the chapter. Metrics such as safety scores, toxicity indices, and bias measurements should be collected alongside model metadata for auditing purposes." (Abi Aryan, "LLMOps: Managing Large Language Models in Production", 2025)

"LLM developers can train the model simply to perform well on the benchmarks, like a student memorizing the answers to an upcoming exam. This is a very serious problem in practice. It’s not uncommon to see an LLM perform well in general benchmarks, only to perform below the level of GPT-3.5 (a now-obsolete but inexpensive model) in a practical application, like describing a scene. When this happens, there’s usually little reason to use the model that has the higher general scores - your users should have the final word. Another problem is that LLMs are highly sensitive to the compatibility of the data used in training and prompts used in evaluation. A seemingly minor change in the prompt can lead to drastically different outputs. This makes it difficult to design prompts that consistently elicit the desired response and assess the LLM’s true capabilities." (Abi Aryan, "LLMOps: Managing Large Language Models in Production", 2025)

"The art of mega-prompts spanning multiple written pages and looking like essays has become commonplace for complex tasks when building applications to get things `just right'. Unfortunately, they bring with them lots of issues: errors, portability, complexity, and more. The GenAI world didn’t plan for mega-prompts. They have simply evolved into what they’ve become today because practitioners kept wanting to do more and more complex things, and their only way to express those intents was with a prompt. But step back and look at some of these prompts [...] Lurking just below the surface are a bunch of classical computing concepts like data, programming instructions, control flows, memory, and stora - all the components typically associated with classical computing elements." (Rob Thomas et al, "AI Value Creators: Beyond the Generative AI User Mindset", 2025)

"The same difficulties that characterize training deep feedforward networks also apply to RNNs; gradients tend to die out over long distances using traditional activation functions (or explode if the gradients become greater than 1). However, unlike feedforward networks, RNNs aren’t trained with traditional backpropagation, but rather a variant known as Backpropagation through Time (BPTT): the network is unrolled, as before, and backpropagation is used, averaging over errors at each time point (since an 'output', the hidden state, occurs at each step). Also, in the case of RNNs, we run into the problem that the network has a very short memory; it only incorporates information from the most recent unit before the current one and has trouble maintaining long-range context. For applications such as translation, this is clearly a problem, as the interpretation of a word at the end of a sentence may depend on terms near the beginning, not just those directly preceding it." (Joseph Babcock & Raghav Bali, "Generative AI with Python and PyTorch" 2nd. Ed., 2025)

"When there are hidden layers between the input and output, the problem becomes more complex: when do we change the internal weights to compute the activations that feed into the final output? How do we modify them in relation to the input weights? The insight of the backpropagation technique is that we can use the chain rule from calculus to efficiently compute the derivatives of each parameter of a network with respect to a loss function and, combined with a learning rule, this provides a scalable way to train multilayer networks." (Joseph Babcock & Raghav Bali, "Generative AI with Python and PyTorch" 2nd. Ed., 2025)

"There is no law of physics tdictates AI must remain expensive. The cost of training and inference isn’t fixed - it is an engineering challenge to solved. Businesses, both incumbents and upstarts, have the ingenuity to push these costs down and make AI more practical and widespread." (Rob Thomas et al, "AI Value Creators: Beyond the Generative AI User Mindset", 2025)

"While the backpropagation procedure provides a way to update interior weights within the network in a principled way, it has several shortcomings that make deep networks difficult to use in practice. One is the problem of vanishing gradients. [...] As the value of the sigmoid function increases or decreases toward the extremes (0 or 1, representing either 'off' or 'on' ), the values of the gradient vanish to near zero. This means that the updates to and , which are products of these gradients from hidden activation functions , shrink toward zero, making the weights change little between iterations and making the parameters of the hidden layer neurons change very slowly during backpropagation. Clearly, one problem here is that the sigmoid function saturates; thus, choosing another nonlinearity might circumvent this problem." (Joseph Babcock & Raghav Bali, "Generative AI with Python and PyTorch" 2nd. Ed., 2025)