"A knowledge graph is a data structure that uses nodes to represent concepts and entities and relationships to connect these nodes. [...] Knowledge graphs are highly versatile, capable of storing both structured information (such as employee details, task statuses, and company hierarchies) and unstructured information (such as article contents). This dual capability [...] makes them uniquely suited for complex RAG applications. [...] By integrating structured and unstructured information in the same framework, knowledge graphs enable the seamless blending of both worlds, making them a powerful tool for answering a broad range of questions efficiently and accurately in RAG applications. Moreover, explicit connec-tions between unstructured and structured data unlock advanced retrieval strategies such as linking entities in text to graph nodes or contextualizing structured results with source passages that would be difficult or impossible to achieve using either type of data alone."(Tomaz Bratanic & Oskar Hane,"Essential GraphRAG: Knowledge Graph-Enhanced RAG", 2025)
"As with many other deep learning-based approaches, another major challenge is in interpretability. While knowledge graphs provide a structured and transparent way to store relationships, LLMs operate as a black box, making it difficult to understand how specific outputs are generated. [...] Data alignment is also a key issue, as structured knowledge graphs and unstructured text data must be carefully preprocessed to ensure consistency. Differences in data formats, ontology mismatches, and information redundancy can create inefficiencies when integrating these two paradigms. Developing robust pipelines that seamlessly connect graph-based insights with LLM-generated text remains an open challenge." (Aldo Marzullo et al, "Graph Machine Learning" 2nd Ed., 2025)
"[...] LLMs are pre-trained on large corpora and sometimes fine-tuned on smaller datasets for specific tasks. Recall that one of the factors behind the Transformer’s effectiveness as a language model is that it is highly parallelizable, allowing for faster training and efficient processing of text. What really sets the Transformer apart from other deep learning architectures is its ability to capture long-range dependencies and relationships between tokens using attention. In other words, attention is a crucial component of Transformer-based LLMs, and it enables them to effectively retain information between training loops and tasks (i.e., transfer learning), while being able to process lengthy swatches of text with ease." (Sinan Ozdemir, "Quick Start Guide to Large Language Models: Strategies and Best Practices for Using ChatGPT and Other LLMs", 2024)
"LLMs are rapidly improving, not only in their ability to understand and generate language but also in how effectively they can use external tools for data retrieval, transformation, and manipulation. As these models become more capable, they will be able to perform increasingly complex tasks with minimal prompting. However, their effectiveness still depends on the quality, design, and integration of the tools you provide. It’s your job to implement those tools thoughtfully and efficiently, ensuring they are well suited to your system’s goals and constraints." (Tomaz Bratanic & Oskar Hane,"Essential GraphRAG: Knowledge Graph-Enhanced RAG", 2025)
"The pre-training step is by far the biggest in terms of data and compute requirements for the whole of the LLM’s lifecycle. Yet fine-tuning is quite resource-intensive when we compare it to traditional machine learning and deep learning workflows. Fine-tuning is also a very important step in improving the quality of the models; hence, it makes sense to understand how we can optimize this step without impacting the performance. Efficiencies in this step also enable us to iterate faster, thereby improving adaptability in many fast-moving domains. In this section, we will focus on some interesting efficient method." (Joseph Babcock & Raghav Bali, "Generative AI with Python and PyTorch" 2nd. Ed., 2025)
"A major aspect of the alignment problem is values misalignment: AI systems may lack sufficient understanding of human ethics, cultural norms, or social contexts, making it difficult to translate our values directly into algorithms. For instance, if an AI is programmed to prioritise efficiency without balancing safety or ethical considerations, it might make decisions that, while effective, disregard human welfare. This is further complicated by 'specification gaming', where an AI might exploit loopholes in its programming to achieve objectives in unintended or counterproductive ways. An AI instructed to avoid obstacles, for instance, could redefine what counts as an 'obstacle' and take routes that, while technically following the rule, lead to negative outcomes." (Alan Watkins & G C Cooke, "Smarter than You Winning in Business with Superintelligent AI", 2026)
"A RAG system needs to identify relevant text chunks from the knowledge base. It does that by building a datastore of chunks and organizing the chunks in a way that makes searching for chunks that are relevant to a query efficient and fast. This preparatory step of building an efficient data store is called indexing, and the runtime step of searching the datastore for relevant chunks is called retrieval. [...] Whenever a query is received, a question-answering pipeline is invoked This consists of two steps: retrieval and generation. In the retrieval step, the retriever finds chunks in the document store that are the most relevant to the query. In the generation step, these relevant chunks are added to the context of the prompt and used along with the user query to generate the answer. In this way, the generated answer is grounded in the relevant information from trusted knowledge sources. The response consists of the generated text as well as the relevant chunks that were used to generate the answer." (Valliappa Lakshmanan & Hannes Hapke, "Generative AI Design Patterns: Solutions to Common Challenges When Building GenAI Agents and Applications", 2026)
"KGs are sophisticated graph structures that represent real-world entities (people, places, diseases, proteins), define meaningful connections between them, and provide context. KGs provide structured, explainable knowledge representation but are challenging to build and query; LLMs offer natural language processing capabilities but suffer from hallucinations, stale information, and a lack of domain-specific grounding. Together, they are a 'killer combination': LLMs can extract entities and relationships from unstructured text to build KGs more efficiently, providing more autonomous and powerful graph querying and analysis. Meanwhile, KGs provide reliable, up-to-date domain knowledge to ground LLM responses and prevent hallucinations." (Alessandro Negro et al, "Knowledge Graphs and LLMs in Action", 2026)
"One of the most important aspects of designing with GenAI is determining when to let the machine act and when to keep a human in the loop. As LLMs and AI agents become increasingly capable of not just generating text but also performing actions (like calling APIs, filling out forms, and triggering workflows), designers and developers face a growing responsibility: deciding what should be automated, not just what can be. Although automation promises speed, efficiency, and scalability, over-automation can lead to unintended consequences and critical errors. This is especially true when actions taken by the AI affect financial, legal, or personal data. The goal of human-centered design isn’t to block AI progress. Instead, it should channel AI in ways that empower users while keeping them informed and in control." (Chris Minnick, "A Developer’s Guide to Integrating Generative AI into Applications", 2026)
"The self-attention mechanism is the cornerstone of transformer models, enabling them to process sequential data like text more effectively than traditional recurrent neural networks (RNNs) or convolutional neural networks (CNNs). Unlike RNNs, which process sequences step-by-step, self-attention allows the model to consider the entire input sequence simultaneously. This parallel processing capability makes transformers highly efficient and scalable. transformers highly efficient and scalable. At its core, self-attention computes relationships between all words in a sentence, assigning higher weights to words that are more relevant to each other" (Saloni Garg et al, "RAG Artificial Intelligence: Retrieval-Augmented Generation in Generative AI", 2026)
