"A RAG system needs to identify relevant text chunks from the knowledge base. It does that by building a datastore of chunks and organizing the chunks in a way that makes searching for chunks that are relevant to a query efficient and fast. This preparatory step of building an efficient data store is called indexing, and the runtime step of searching the datastore for relevant chunks is called retrieval. [...] Whenever a query is received, a question-answering pipeline is invoked This consists of two steps: retrieval and generation. In the retrieval step, the retriever finds chunks in the document store that are the most relevant to the query. In the generation step, these relevant chunks are added to the context of the prompt and used along with the user query to generate the answer. In this way, the generated answer is grounded in the relevant information from trusted knowledge sources. The response consists of the generated text as well as the relevant chunks that were used to generate the answer." (Valliappa Lakshmanan & Hannes Hapke, "Generative AI Design Patterns: Solutions to Common Challenges When Building GenAI Agents and Applications", 2026)
"As an AI engineer, you can ask a foundational model to directly generate the content your application needs by sending the model an appropriate text input, which is known as a prompt. However, you will face certain common problems - the generated content may not match the style you want, may be missing enterprise knowledge that the model doesn’t know about, or may lack certain capabilities." (Valliappa Lakshmanan & Hannes Hapke, "Generative AI Design Patterns: Solutions to Common Challenges When Building GenAI Agents and Applications", 2026)
"Authentication and authorization are important parts of any communication between clients and servers. We need to have a clear separation between different roles and to be able to specify what capabilities a particular client is allowed to invoke. MCP, however, is more of a language interoperability protocol - it doesn’t enforce any security principles. [...] Collaboration In MCP, communication between the client and the server is mostly one way, whereas in the real world, you may want collaborative, interactive workflows." (Valliappa Lakshmanan & Hannes Hapke, "Generative AI Design Patterns: Solutions to Common Challenges When Building GenAI Agents and Applications", 2026)
"Before releasing the fine-tuned model to a production environment, you need to evaluate its performance and make sure it hasn’t picked up any counterproductive constructs, like biased language. Also, whenever a new version of your foundational model is released, you must fine-tune the model again and redo the training. Finally, you need to track the lineage of the training and validation data used to train the fine-tuned model. These are additional tasks that require careful execution, and using in-context learning is far simpler since it only requires you to add a handful of examples." (Valliappa Lakshmanan & Hannes Hapke, "Generative AI Design Patterns: Solutions to Common Challenges When Building GenAI Agents and Applications", 2026)
"Content Optimization is a pattern that uses preference tuning to produce optimally performing content. Preference tuning is a way of training an LLM on a dataset of paired items so that it outputs responses that are closer to the chosen items than the rejected ones. Unlike optimization methods, in which optimization attempts to approximate an independent source of truth, this pattern attempts to hack the model to create content the evaluation method will consider great." (Valliappa Lakshmanan & Hannes Hapke, "Generative AI Design Patterns: Solutions to Common Challenges When Building GenAI Agents and Applications", 2026)
"Embeddings are mathematical representations of text, images, or other input types. They capture semantic meaning by mapping words, phrases, or documents to vectors in a high-dimensional space. LLMs have become exceptionally proficient at creating semantically representative embeddings because this capability forms the foundation for LLM generation. [...] While the latest LLMs provide massive context windows, encoding entire documents in a single embedding vector is often not advisable. You’ll lose the nuances of the different parts of a text, lose critical information, and hinder the RAG system from retrieving relevant information. Plus, it gets very expensive to use large context windows at scale. It’s preferable to chunk documents into information-dense and self-consistent chunks of text." (Valliappa Lakshmanan & Hannes Hapke, "Generative AI Design Patterns: Solutions to Common Challenges When Building GenAI Agents and Applications", 2026)
"Fine-tuning LLMs can lead to catastrophic forgetting, in which the model overemphasizes the examples provided during fine-tuning and loses its previously acquired knowledge. This wipes out LLMs’ primary advantage: their comprehensive world knowledge. You can mitigate this issue by fine-tuning your model on a small dataset, for only a few epochs, and by selecting an appropriate learning rate during the fine-tuning process. Generally, you should begin the fine-tuning learning rate at the value where the pretraining phase concluded (typically around 1e-5)." (Valliappa Lakshmanan & Hannes Hapke, "Generative AI Design Patterns: Solutions to Common Challenges When Building GenAI Agents and Applications", 2026)
"Foundational models are closed systems that are limited by their training data. In many cases, you’ll need to give a foundational model additional information. For example, the information may be based on recent events that had not occurred when the foundational model was being trained, or the information may have been private, confidential, or otherwise unavailable to the foundational model trainers. It’s impractical to retrain an LLM with additional knowledge or even perform continuing pretraining (CPT) on a foundational model to add knowledge to it. The cost of even a single training run is significant, and information changes so fast that CPT would have to be done very frequently." (Valliappa Lakshmanan & Hannes Hapke, "Generative AI Design Patterns: Solutions to Common Challenges When Building GenAI Agents and Applications", 2026)
"Foundational models can manipulate symbols and words effectively, but this may be because they are generalizing from the manipulation they have encountered in the training data, not because they grasp semantic meanings and the logical relationships between them in the way humans do. While you can use foundational models to perform many tasks, that’s because those tasks are similar to the tasks that the foundational model was trained to do." (Valliappa Lakshmanan & Hannes Hapke, "Generative AI Design Patterns: Solutions to Common Challenges When Building GenAI Agents and Applications", 2026)
"GenAI is so powerful and easy to use that even nontechnical users can easily prototype very compelling applications on top of GenAI. However, taking such GenAI prototypes to production is hard because GenAI models are unreliable - they can hallucinate, return different answers to the same input, and can have surprising limitations because of how they are trained." (Valliappa Lakshmanan & Hannes Hapke, "Generative AI Design Patterns: Solutions to Common Challenges When Building GenAI Agents and Applications", 2026)
"In 'traditional' ML, where models are predominantly trained for single problems like classifications or regressions, models require extensive training to adapt their weights or parameters to a new task. But you can use LLMs for a wide variety of tasks by changing just the prompt without changing the weights. Foundational models’ capability to adapt to new tasks based solely on examples or instructions they’re given in a prompt is called in-context learning. It emerges from training on diverse datasets that include various task formats and instructions. When given examples of a task within a prompt, the model identifies the underlying pattern and applies it to new instances. This process doesn’t modify the model’s weights, but rather, it leverages the model’s existing knowledge to perform new tasks." (Valliappa Lakshmanan & Hannes Hapke, "Generative AI Design Patterns: Solutions to Common Challenges When Building GenAI Agents and Applications", 2026)
"Multiagent architectures offer better scalability than monolithic approaches do. Rather than scaling vertically by increasing model size, multiagent systems can scale horizontally by adding more specialized agents as needed. You can update or replace individual agents without having to retrain the entire system, and that allows for more agile development and deployment. Computational resources can also be allocated dynamically based on current needs, with more resources directed to the most critical or computationally intensive subtasks. Multiagent systems are also more robust. Critical capabilities can be replicated across multiple agents to reduce the impact of individual agent failures. Different agents can approach problems with different methodologies, which increases the likelihood of finding effective solutions. Agents can verify each other’s outputs to identify and correct errors that might go undetected in a single-agent system." (Valliappa Lakshmanan & Hannes Hapke, "Generative AI Design Patterns: Solutions to Common Challenges When Building GenAI Agents and Applications", 2026)
"Prompt engineering may meet your accuracy requirements, but the prompts end up being extremely long as you encounter more and more corner cases. It’s not unusual for such prompts to amount to two or three pages of detailed instructions. The complex system instructions required to achieve that level of accuracy incur a large token count, and that makes the application too expensive to scale. [...] As your prompts become more detailed, maintenance overhead increases because the overall system is extremely sensitive to the specific wording of prompts. You’ll need to do integration testing every time the foundational model changes, and often even when one of your tools changes." (Valliappa Lakshmanan & Hannes Hapke, "Generative AI Design Patterns: Solutions to Common Challenges When Building GenAI Agents and Applications", 2026)
"The Chain of Thought pattern uses chain-of-thought (CoT) reasoning, which is a prompting technique that enhances LLMs’ problem-solving capabilities by encouraging them to break down complex problems into intermediate reasoning steps before arriving at a final answer. [...] CoT involves modifying the prompting strategy - instead of asking for a direct answer, the prompt explicitly requests or demonstrates a step-by-step reasoning process. There are three variants: Zero-shot CoT, Few-shot CoT, and Auto-CoT. [...] In Zero-shot CoT, you request a step-by-step reasoning by simply adding the phrase think step-by-step to the prompt. [...] In cases where the model’s pretrained capabilities do not include the specific logic needed, you can try Few-shot CoT, which involves providing the model with examples of step-by-step reasoning for similar problems before asking it to solve a new problem. In Few-shot CoT, you demonstrate the desired reasoning pattern through examples. [...] In the Auto-CoT variant, you add demonstrative examples to a wide variety of questions to a database called an example store. Most people use a document database (in which the examples are indexed by keywords in the question text) or a vector database (in which the examples are indexed by the embedding of the questions) for the example store." (Valliappa Lakshmanan & Hannes Hapke, "Generative AI Design Patterns: Solutions to Common Challenges When Building GenAI Agents and Applications", 2026)
"The Deep Search pattern involves an iterative loop that consists of retrieval and thinking until either a good enough answer is found or the time/cost budget is exhausted. It goes beyond traditional RAG systems in three ways. First, it adds a thinking step between retrieval and generation to determine what additional information, if any, needs to be gathered. Second, instead of a single retrieval step, Deep Search involves iterating through multiple stages of retrieval and generation. Finally, instead of just relying on adding knowledge from an enterprise knowledge base, Deep Search employs tools such as search engines and enterprise APIs to gather the knowledge it needs proactively." (Valliappa Lakshmanan & Hannes Hapke, "Generative AI Design Patterns: Solutions to Common Challenges When Building GenAI Agents and Applications", 2026)
"The idea behind a GraphRAG is that, having discovered a chunk that has a partial answer to your question, you can retrieve related chunks. This can help you round out the context when each chunk is small. You can also index a document with a small chunk size and then store the embeddings in a tree form so that the embedding of a parent node captures the holistic meaning of all the nodes below it. You can even repeat nodes within the RAG if they make sense in different contexts. You can also index the nodes on both keywords and on embeddings to permit different types of retrieval for different types of queries. In addition, you can incorporate query-focused summarization into a GraphRAG by pregenerating and storing summaries for all groups of closely related entities." (Valliappa Lakshmanan & Hannes Hapke, "Generative AI Design Patterns: Solutions to Common Challenges When Building GenAI Agents and Applications", 2026)
"The Reflection pattern is an agentic approach in which an AI system evaluates its own reasoning, decisions, and outputs before finalizing them. This self-monitoring process allows the system to identify errors, improve solutions, and refine its approach to problems. Here, the AI system as a whole is reflecting and improving, and the foundational model used to generate the response remains the same. You’ll sometimes see this process referred to as self-reflection, but there’s no reason why the evaluation has to be done by an LLM at all, let alone by the same LLM that generates the content." (Valliappa Lakshmanan & Hannes Hapke, "Generative AI Design Patterns: Solutions to Common Challenges When Building GenAI Agents and Applications", 2026)
"The type of post-training you can do is intimately connected to the structure of the dataset. If the dataset consists purely of text completions, then you can use it to do unsupervised training, such as teaching a model new vocabulary or completely new associations via CPT. If the dataset consists of ideal responses to inputs (input-output pairs), then you can use it for SFT or instruction tuning. If the dataset consists of two responses to each input and notes which of the two is preferable, then you can use it for preference tuning. You can do any of these forms of post-training in a parameter efficient way or on quantized models." (Valliappa Lakshmanan & Hannes Hapke, "Generative AI Design Patterns: Solutions to Common Challenges When Building GenAI Agents and Applications", 2026)
"Traditional RAG systems 'guide' LLMs in generating responses by providing document chunks that are related to the user’s initial query. Incorporating external knowledge from a document store helps to reduce hallucinations, but its effectiveness relies heavily on the relevance of the retrieved document chunks. This can be problematic if the retrieval step produces irrelevant document chunks or incorrect information. The key goal of corrective retrieval-augmented generation (CRAG) is to detect and correct hallucinations before the LLM attempts response generation. It adds an evaluator to the process that evaluates the quality of the retrieved chunks for a given query before passing on the documents to the response generation step. The evaluator returns a confidence score that triggers different knowledge retrieval actions." (Valliappa Lakshmanan & Hannes Hapke, "Generative AI Design Patterns: Solutions to Common Challenges When Building GenAI Agents and Applications", 2026)
"What is an agent? In computer science, the term agent has long been used to describe software entities that act on behalf of users or other programs. When you invoke a foundational model, you specify a role, provide some context, and ask it to carry out some instruction. In the computer science sense, then, the LLM is acting as your agent." (Valliappa Lakshmanan & Hannes Hapke, "Generative AI Design Patterns: Solutions to Common Challenges When Building GenAI Agents and Applications", 2026)
"When a RAG system generates a response, the generation can blur the boundaries of the original chunks or multiple chunks can contribute to a single statement. Some LLMs tend to paraphrase chunks in ways that will alter the direct connection to the source chunk, and the paraphrasing can make direct text matching difficult." (Valliappa Lakshmanan & Hannes Hapke, "Generative AI Design Patterns: Solutions to Common Challenges When Building GenAI Agents and Applications", 2026)
