"A RAG system needs to identify relevant text chunks from the knowledge base. It does that by building a datastore of chunks and organizing the chunks in a way that makes searching for chunks that are relevant to a query efficient and fast. This preparatory step of building an efficient data store is called indexing, and the runtime step of searching the datastore for relevant chunks is called retrieval. [...] Whenever a query is received, a question-answering pipeline is invoked This consists of two steps: retrieval and generation. In the retrieval step, the retriever finds chunks in the document store that are the most relevant to the query. In the generation step, these relevant chunks are added to the context of the prompt and used along with the user query to generate the answer. In this way, the generated answer is grounded in the relevant information from trusted knowledge sources. The response consists of the generated text as well as the relevant chunks that were used to generate the answer." (Valliappa Lakshmanan & Hannes Hapke, "Generative AI Design Patterns: Solutions to Common Challenges When Building GenAI Agents and Applications", 2026)
"As an AI engineer, you can ask a foundational model to directly generate the content your application needs by sending the model an appropriate text input, which is known as a prompt. However, you will face certain common problems - the generated content may not match the style you want, may be missing enterprise knowledge that the model doesn’t know about, or may lack certain capabilities." (Valliappa Lakshmanan & Hannes Hapke, "Generative AI Design Patterns: Solutions to Common Challenges When Building GenAI Agents and Applications", 2026)
"Before releasing the fine-tuned model to a production environment, you need to evaluate its performance and make sure it hasn’t picked up any counterproductive constructs, like biased language. Also, whenever a new version of your foundational model is released, you must fine-tune the model again and redo the training. Finally, you need to track the lineage of the training and validation data used to train the fine-tuned model. These are additional tasks that require careful execution, and using in-context learning is far simpler since it only requires you to add a handful of examples." (Valliappa Lakshmanan & Hannes Hapke, "Generative AI Design Patterns: Solutions to Common Challenges When Building GenAI Agents and Applications", 2026)
"Content Optimization is a pattern that uses preference tuning to produce optimally performing content. Preference tuning is a way of training an LLM on a dataset of paired items so that it outputs responses that are closer to the chosen items than the rejected ones. Unlike optimization methods, in which optimization attempts to approximate an independent source of truth, this pattern attempts to hack the model to create content the evaluation method will consider great." (Valliappa Lakshmanan & Hannes Hapke, "Generative AI Design Patterns: Solutions to Common Challenges When Building GenAI Agents and Applications", 2026)
"Fine-tuning LLMs can lead to catastrophic forgetting, in which the model overemphasizes the examples provided during fine-tuning and loses its previously acquired knowledge. This wipes out LLMs’ primary advantage: their comprehensive world knowledge. You can mitigate this issue by fine-tuning your model on a small dataset, for only a few epochs, and by selecting an appropriate learning rate during the fine-tuning process. Generally, you should begin the fine-tuning learning rate at the value where the pretraining phase concluded (typically around 1e-5)." (Valliappa Lakshmanan & Hannes Hapke, "Generative AI Design Patterns: Solutions to Common Challenges When Building GenAI Agents and Applications", 2026)
"Foundational models are closed systems that are limited by their training data. In many cases, you’ll need to give a foundational model additional information. For example, the information may be based on recent events that had not occurred when the foundational model was being trained, or the information may have been private, confidential, or otherwise unavailable to the foundational model trainers. It’s impractical to retrain an LLM with additional knowledge or even perform continuing pretraining (CPT) on a foundational model to add knowledge to it. The cost of even a single training run is significant, and information changes so fast that CPT would have to be done very frequently." (Valliappa Lakshmanan & Hannes Hapke, "Generative AI Design Patterns: Solutions to Common Challenges When Building GenAI Agents and Applications", 2026)
"GenAI is so powerful and easy to use that even nontechnical users can easily prototype very compelling applications on top of GenAI. However, taking such GenAI prototypes to production is hard because GenAI models are unreliable - they can hallucinate, return different answers to the same input, and can have surprising limitations because of how they are trained." (Valliappa Lakshmanan & Hannes Hapke, "Generative AI Design Patterns: Solutions to Common Challenges When Building GenAI Agents and Applications", 2026)
"In 'traditional' ML, where models are predominantly trained for single problems like classifications or regressions, models require extensive training to adapt their weights or parameters to a new task. But you can use LLMs for a wide variety of tasks by changing just the prompt without changing the weights. Foundational models’ capability to adapt to new tasks based solely on examples or instructions they’re given in a prompt is called in-context learning. It emerges from training on diverse datasets that include various task formats and instructions. When given examples of a task within a prompt, the model identifies the underlying pattern and applies it to new instances. This process doesn’t modify the model’s weights, but rather, it leverages the model’s existing knowledge to perform new tasks." (Valliappa Lakshmanan & Hannes Hapke, "Generative AI Design Patterns: Solutions to Common Challenges When Building GenAI Agents and Applications", 2026)
"The type of post-training you can do is intimately connected to the structure of the dataset. If the dataset consists purely of text completions, then you can use it to do unsupervised training, such as teaching a model new vocabulary or completely new associations via CPT. If the dataset consists of ideal responses to inputs (input-output pairs), then you can use it for SFT or instruction tuning. If the dataset consists of two responses to each input and notes which of the two is preferable, then you can use it for preference tuning. You can do any of these forms of post-training in a parameter efficient way or on quantized models." (Valliappa Lakshmanan & Hannes Hapke, "Generative AI Design Patterns: Solutions to Common Challenges When Building GenAI Agents and Applications", 2026)
"What is an agent? In computer science, the term agent has long been used to describe software entities that act on behalf of users or other programs. When you invoke a foundational model, you specify a role, provide some context, and ask it to carry out some instruction. In the computer science sense, then, the LLM is acting as your agent." (Valliappa Lakshmanan & Hannes Hapke, "Generative AI Design Patterns: Solutions to Common Challenges When Building GenAI Agents and Applications", 2026)
No comments:
Post a Comment