"An embedding is a way to represent complex data as a set of numbers in a simpler, lower-dimensional space. Think of it as translating data into a format that a computer can easily understand and work with. Embedding models provide a uniform way to represent different types of data. Input to an embedding model can be any complex data, and the output is a vector. For instance, in dealing with text, an embedding model will take words or sentences and turn them into vectors, which are lists of numbers. The model is trained to ensure that these number lists capture essential aspects of the original words, such as their meaning or context." (Tomaz Bratanic & Oskar Hane,"Essential GraphRAG: Knowledge Graph-Enhanced RAG", 2025)
"Hallucinations occur because LLMs are not reasoning engines. They are probabilistic language models trained to predict what sounds like a good next token, based on patterns in their training data. They don’t know facts the way humans do. Rather, they generate text by guessing the most likely continuation, regardless of whether it’s true. This fundamental difference between statistical pattern matching and actual understanding is what separates LLMs from human cognition." (Tomaz Bratanic & Oskar Hane," Essential Graphrag: Knowledge Graph-Enhanced RAG", 2025)
"In a nutshell, training LLMs involves optimizing a large number of parameters on very large datasets. This process, known as pretraining, typically employs unsupervised learning objectives, such as predicting missing words in a sentence (masked language modeling) or forecasting subsequent words (causal language modeling). As a side effect, the pre-training phase lets the model learn and 'understand' a language, resulting in a remarkable ability to generalize across various tasks, often achieving state-of-the-art performance. LLMs have demonstrated proficiency in a diverse array of applications, reflecting their versatility and depth of language understanding. Key areas include text generation, language translation, question answering, and summarization, among many others." (Aldo Marzullo et al, "Graph Machine Learning" 2nd Ed., 2025)
"In prompt engineering, we customize the prompts or questions we give the model to get more accurate or insightful responses. The way a prompt is structured has a massive impact on how well a model understands the task at hand and, ultimately, how well it performs. Given LLMs’ versatility, prompt engineering has become an important skill for getting the most out of these models across different domains and tasks. The key is to understand how different prompt structures lead to different model behaviors. There are various strategies - ranging from simple one-shot prompting to more complex techniques like chain-of-thought prompting - that can significantly improve the effectiveness of LLMs." (Abi Aryan, "LLMOps: Managing Large Language Models in Production", 2025)
"[...] LLMs are great at understanding natural language and following instruc-tions in the prompt. In the RAG workflow, the goal shifts to task-oriented response gen-eration, where LLMs follow a set of instructions. The process involves utilizing a retrieval tool to fetch relevant documents from a specific knowledge base. The LLM then generates answers based on the provided documents, ensuring responses are accurate, contextually relevant, and aligned with specific guidelines. This systematic approach transforms the answer generation process into a targeted task of inspecting and using the retrieved information to produce the final answer." (Tomaz Bratanic & Oskar Hane,"Essential GraphRAG: Knowledge Graph-Enhanced RAG", 2025)
"LLMs are rapidly improving, not only in their ability to understand and generate language but also in how effectively they can use external tools for data retrieval, transformation, and manipulation. As these models become more capable, they will be able to perform increasingly complex tasks with minimal prompting. However, their effectiveness still depends on the quality, design, and integration of the tools you provide. It’s your job to implement those tools thoughtfully and efficiently, ensuring they are well suited to your system’s goals and constraints." (Tomaz Bratanic & Oskar Hane,"Essential GraphRAG: Knowledge Graph-Enhanced RAG", 2025)
"LLMs are trained on large volumes of data, which inherently provides them with an immense knowledge base and understanding of different languages. Yet, LLMs at their core are complex text completion engines. Since this knowledge and understanding of language is compressed in a very high-dimensional latent space. LLMs end up using these in a very fluid and intelligible way (which often leads to hallucinations). In order to guide LLMs to focus on specific topics or pieces of information to solve certain tasks, (for instance, question-answering from a given piece of text), it is important to provide contextual information explicitly. While most current generations of LLMs have extremely wide context windows, it is recommended to preprocess context into overlapping smaller chunks for better results, reduced latency, and so on. For similar reasons, it is also recommended to preprocess contextual information in clear and task-specific formats. This aspect of context preprocessing is extremely useful in Retrieval-Gugmented Generation (RAG) scenarios." (Joseph Babcock & Raghav Bali, "Generative AI with Python and PyTorch" 2nd. Ed., 2025)
"LLMs excel at understanding context and making associations among words, phrases, and concepts to provide relevant information based on the input query or prompt. While structured knowledge bases rely on humancurated data, LLMs can automatically extract knowledge from unstructured text. When trained on diverse textual sources, they can process a vast amount of information without explicit human intervention. However, this also introduces a challenge, as the model can learn biased or incorrect information from the training data." (Abi Aryan, "LLMOps: Managing Large Language Models in Production", 2025)
"Prompt engineering is a crucial aspect of working with large language models (LLMs) like OpenAI's GPT, Google's PaLM, and others in the space of AI and machine learning. It involves the art and science of designing inputs (prompts) in a way that maximizes the quality, relevance, and accuracy of the AI-generated output. As the capabilities of AI continue to improve, the task of crafting effective prompts has become an essential skill for anyone leveraging these tools for real-world applications, including natural language understanding, translation, summarization, code generation, and more." (Code Planet, "Python for Large Language Models", 2025)
"A major aspect of the alignment problem is values misalignment: AI systems may lack sufficient understanding of human ethics, cultural norms, or social contexts, making it difficult to translate our values directly into algorithms. For instance, if an AI is programmed to prioritise efficiency without balancing safety or ethical considerations, it might make decisions that, while effective, disregard human welfare. This is further complicated by 'specification gaming', where an AI might exploit loopholes in its programming to achieve objectives in unintended or counterproductive ways. An AI instructed to avoid obstacles, for instance, could redefine what counts as an 'obstacle' and take routes that, while technically following the rule, lead to negative outcomes." (Alan Watkins & G C Cooke, "Smarter than You Winning in Business with Superintelligent AI", 2026)
"[...] KGs and LLMs can be the foundation for different types of reasoning, complementing each other in intelligent systems. We can use KGs for tasks that require precise, rule-based reasoning and explicit knowledge representation, and LLMs for tasks involving pattern recognition, context understanding, handling ambiguity or incomplete information, and reasoning about graph structures and their derived metrics. However, neither approach inherently possesses common-sense reasoning capabilities comparable to those of humans, and they often fail to make intuitive leaps or understand the implicit context that would be obvious to a person. These limitations underscore the importance of carefully considering the strengths and weaknesses of each approach when designing intelligent systems and potentially developing a powerful hybrid IAS." (Alessandro Negro et al, "Knowledge Graphs and LLMs in Action", 2026)
"The first big innovation in the transformer architecture is its ability to process tokens in parallel rather than sequentially, one at a time. This was a huge breakthrough, as it allowed faster training and effectively overcame the long-range dependency problem. Rather than reading one sentence at a time as humans do, a transformer reads a whole input sequence at once - spanning hundreds of thousands of words in some cases. To be clear, there are still limitations to how much a transformer can read in parallel, known as the context window, but generally this window is very large. The ability to examine words in parallel allows the transformer to understand how words relate to each other, even if they are far apart in the same sentence, paragraph, or even whole book." (Robert Barton & Jerome Henry, "Demystifying Generative AI A Practical and Intuitive Introduction", 2026)
"The transformer was revolutionary in many ways. At its heart is a powerful mechanism called self-attention, which allows it to understand the context and meaning of not just individual words but entire sequences of words. By learning the context of words for an input of almost any size, a transformer understands the actual meaning of what it reads, and it can also generate new outputs that are rich in meaning, creative, and useful." (Robert Barton & Jerome Henry, "Demystifying Generative AI A Practical and Intuitive Introduction", 2026)
"Transformers dominate language for several reasons: their autoregressive framework provides a straightforward training objective (predict next token), self-attention captures long-range dependencies across thousands of tokens, parallel training processes entire sequences simultaneously (unlike RNNs’ sequential bottleneck), and scaling behavior produces emergent capabilities at billions of parameters. These properties make transformers uniquely suited for the language understanding and generation tasks central to enterprise AI applications [...]" (Bennie Haelen, "ML and Generative AI in the Data Lakehouse Building and Deploying AI Applications at Scale", 2026)
