17 November 2026

🤖Prompt Engineering: Rules (Just the Quotes)

"How can a cognitive system process environmental input and stored knowledge so as to benefit from experience? More specific versions of this question include the following: How can a system organize its experience so that it has some basis for action even in unfamiliar situations? How can a system determine that rules in its knowledge base are inadequate? How can it generate plausible new rules to replace the inadequate ones? How can it refine rules that are useful but non-optimal? How can it use metaphor and analogy to transfer information and procedures from one domain to another?" (John H Holland et al, "Induction: Processes Of Inference, Learning, And Discovery", 1986)

"When there are hidden layers between the input and output, the problem becomes more complex: when do we change the internal weights to compute the activations that feed into the final output? How do we modify them in relation to the input weights? The insight of the backpropagation technique is that we can use the chain rule from calculus to efficiently compute the derivatives of each parameter of a network with respect to a loss function and, combined with a learning rule, this provides a scalable way to train multilayer networks." (Joseph Babcock & Raghav Bali, "Generative AI with Python and PyTorch" 2nd. Ed., 2025)

"Unlike traditional code completion, which operates on predefined rules, generative AI creates a continuous improvement cycle, which includes the following five basic steps: (1) Developer input: You provide source code, comments, or natural language requirements. (2) Context analysis: The model analyzes patterns in your existingcode and requirements. (3) Prediction: Based on training data and your specific context, the model generates probable code. (4) Developer feedback: You accept, modify, or reject suggestions. (5) Model adaptation: The system incorporates your feedback to improve future suggestions." (Jeremy C Morgan, "Coding with AI: Examples in Python", 2025)

"A major aspect of the alignment problem is values misalignment: AI systems may lack sufficient understanding of human ethics, cultural norms, or social contexts, making it difficult to translate our values directly into algorithms. For instance, if an AI is programmed to prioritise efficiency without balancing safety or ethical considerations, it might make decisions that, while effective, disregard human welfare. This is further complicated by 'specification gaming', where an AI might exploit loopholes in its programming to achieve objectives in unintended or counterproductive ways. An AI instructed to avoid obstacles, for instance, could redefine what counts as an 'obstacle' and take routes that, while technically following the rule, lead to negative outcomes." (Alan Watkins & G C Cooke, "Smarter than You Winning in Business with Superintelligent AI", 2026)

"Although GenAI gets most of the attention for its ability to create new content, successful AI applications often combine multiple approaches. Generative models aren’t always the optimal choice for every component of an AI system. Traditional ML models, rule-based systems, and algorithmic approaches can offer significant advantages in speed, accuracy, cost-effectiveness, and reliability when used strategically alongside GenAI." (Chris Minnick, "A Developer’s Guide to Integrating Generative AI into Applications", 2026)

"An alternative future is a more federated open-source AI where commoditised LLM can be run independently by anyone but loosely coordinated between each other if that is mutually advantageous. This requires sophisticated governance of an ecosystem where the rules that maintain the healthy stability of the ecosystem are defined and adhered to. This model allows anyone to put any LLM on a normal computer and plug into the AI. This is the path that already seems most sensible to business leaders and government officials, who recognise that their private data and iterative datasets hold value that can increase productivity and improve the bottom line or assist citizens in their lives." (Alan Watkins & G C Cooke, "Smarter than You Winning in Business with Superintelligent AI", 2026)

"[...] KGs and LLMs can be the foundation for different types of reasoning, complementing each other in intelligent systems. We can use KGs for tasks that require precise, rule-based reasoning and explicit knowledge representation, and LLMs for tasks involving pattern recognition, context understanding, handling ambiguity or incomplete information, and reasoning about graph structures and their derived metrics. However, neither approach inherently possesses common-sense reasoning capabilities comparable to those of humans, and they often fail to make intuitive leaps or understand the implicit context that would be obvious to a person. These limitations underscore the importance of carefully considering the strengths and weaknesses of each approach when designing intelligent systems and potentially developing a powerful hybrid IAS." (Alessandro Negro et al, "Knowledge Graphs and LLMs in Action", 2026)

"Machine learning was a big change because it let systems learn patterns from data instead of having to follow hardcoded rules. AI could generalize better and change to new inputs thanks to techniques like decision trees and support vector machines. These models still needed a lot of work to get the features right, though, and they couldn’t handle high-dimensional data like images or text very well. The breakthrough happened when deep learning, a type of machine learning that uses neural networks with multiple layers to automatically learn hierarchical representations of data, became popular. Convolutional neural networks (CNNs) for processing images and recurrent neural networks (RNNs) for sequential data changed what AI could do. Architectures like GANs and transformers took things even further by making it possible to do things like make images, understand natural language, and more." (Saloni Garg et al, "RAG Artificial Intelligence: Retrieval-Augmented Generation in Generative AI", 2026)

"ML is a useful - and under-utilized - framework for studying LLMs. For one thing, LLMs are literally composed of simple ML algorithms (linear weighted averages and nonlinear transformations). Furthermore, using ML techniques like regression, classification, and clustering, can help reveal how concepts like grammar rules are represented inside LLMs. And finally, many people find LLMs to be intimidatingly complicated while finding ML to be much more approachable. Thus, using ML to study LLMs involves using simple tools to understand complicated tools." (Mike X Cohen,"50 ML Projects To Understand LLMs", 2026)

No comments:

Related Posts Plugin for WordPress, Blogger...

About Me

My photo
Koeln, NRW, Germany
IT Professional with more than 25 years experience in IT in the area of full life-cycle of Web/Desktop/Database Applications Development, Software Engineering, Consultancy, Data Management, Data Quality, Data Migrations, Reporting, ERP implementations & support, Team/Project/IT Management, etc.