Skip to content

RAG

  • in RAG
  • 4 min read

Optimizing Tool Retrieval in RAG Systems: A Balanced Approach

RAG Course

This is based on a conversation that came up during office hours from my RAG course for engineering leaders. There's another cohort that's coming up soon, so if you're interested in that, you can sign up here.

When it comes to Retrieval-Augmented Generation (RAG) systems, one of the key challenges is deciding how to select and use tools effectively. As someone who's spent countless hours optimizing these systems, many people ask me whether or not they should think about using retrieval to choose which tools to put into the prompt. What this actually means is that we're interested in making precision and recall trade-offs. I've found that the key lies in balancing recall and precision. Let me break down my approach and share some insights that could help you improve your own RAG implementations.

In this article, we'll cover:

  1. The challenge of tool selection in RAG systems
  2. Understanding the recall vs. precision tradeoff
  3. The "Evergreen Tools" strategy for optimizing tool selection
  • in RAG
  • 4 min read

The RAG Playbook

When it comes to building and improving Retrieval-Augmented Generation (RAG) systems, too many teams focus on the wrong things. They obsess over generation before nailing search, implement RAG without understanding user needs, or get lost in complex improvements without clear metrics. I've seen this pattern repeat across startups of all sizes and industries.

But it doesn't have to be this way. After years of building recommendation systems, instrumenting them, and more recently consulting on RAG applications, I've developed a systematic approach that works. It's not just about what to do, but understanding why each step matters in the broader context of your business.

Here's the flywheel I use to continually infer and improve RAG systems:

  1. Initial Implementation
  2. Synthetic Data Generation
  3. Fast Evaluations
  4. Real-World Data Collection
  5. Classification and Analysis
  6. System Improvements
  7. Production Monitoring
  8. User Feedback Integration
  9. Iteration

Let's break this down step-by-step:

  • in RAG
  • 5 min read

Predictions for the Future of RAG

In the next 6 to 8 months, RAG will be used primarily for report generation. We'll see a shift from using RAG agents as question-answering systems to using them more as report-generation systems. This is because the value you can get from a report is much greater than the current RAG systems in use. I'll explain this by discussing what I've learned as a consultant about understanding value and then how I think companies should describe the value they deliver through RAG.

Rag is the feature, not the benefit.

  • in RAG
  • 5 min read

RAG Course

If you're looking to deepen your understanding of RAG systems and learn how to systematically improve them, consider enrolling in the Systematically Improving RAG Applications course. This 4-week program covers everything from evaluation techniques to advanced retrieval methods, helping you build a data flywheel for continuous improvement.

RAG (Retrieval-Augmented Generation), is a powerful technique that combines information retrieval with LLMs to provide relevant and accurate responses to user queries. By searching through a large corpus of text and retrieving the most relevant chunks, RAG systems can generate answers that are grounded in factual information.

In this post, we'll explore six key areas where you can focus your efforts to improve your RAG search system. These include using synthetic data for baseline metrics, adding date filters, improving user feedback copy, tracking average cosine distance and Cohere reranking score, incorporating full-text search, and efficiently generating synthetic data for testing.

  • in RAG
  • 10 min read

Levels of Complexity: RAG Applications

This guide explores different levels of complexity in Retrieval-Augmented Generation (RAG) applications. We'll cover everything from basic ideas to advanced methods, making it useful for beginners and experienced developers alike.

We'll start with the basics, like breaking text into chunks, creating embeddings, and storing data. Then, we'll move on to more complex topics such as improved search methods, creating structured responses, and making systems work better. By the end, you'll know how to build strong RAG systems that can answer tricky questions accurately.

As we explore these topics, we'll use ideas from other resources, like our articles on data flywheels and improving tool retrieval in RAG systems. These ideas will help you understand how to create systems that keep improving themselves, making your product better and keeping users more engaged.

Key topics we'll explore include:

  1. Basic text processing and embedding techniques
  2. Efficient data storage and retrieval methods
  3. Advanced search and ranking algorithms
  4. Asynchronous programming for improved performance
  5. Observability and logging for system monitoring
  6. Evaluation strategies using synthetic and real-world data
  7. Query enhancement and summarization techniques

This guide aligns with the insights from our RAG flywheel article, which emphasizes the importance of continuous improvement in RAG systems through data-driven iterations and user feedback integration.