Skip to content

Writing

  • in RAG
  • 11 min read

Levels of Complexity: RAG Applications

This guide explores different levels of complexity in Retrieval-Augmented Generation (RAG) applications. We'll cover everything from basic ideas to advanced methods, making it useful for beginners and experienced developers alike.

We'll start with the basics, like breaking text into chunks, creating embeddings, and storing data. Then, we'll move on to more complex topics such as improved search methods, creating structured responses, and making systems work better. By the end, you'll know how to build strong RAG systems that can answer tricky questions accurately.

As we explore these topics, we'll use ideas from other resources, like our articles on data flywheels and improving tool retrieval in RAG systems. These ideas will help you understand how to create systems that keep improving themselves, making your product better and keeping users more engaged.

Key topics we'll explore include:

  1. Basic text processing and embedding techniques
  2. Efficient data storage and retrieval methods
  3. Advanced search and ranking algorithms
  4. Asynchronous programming for improved performance
  5. Observability and logging for system monitoring
  6. Evaluation strategies using synthetic and real-world data
  7. Query enhancement and summarization techniques

This guide aligns with the insights from our RAG flywheel article, which emphasizes the importance of continuous improvement in RAG systems through data-driven iterations and user feedback integration.

Format your own prompts

This is mostly to add onto Hamels great post called Fuck you show me the prompt

I think too many llm libraries are trying to format your strings in weird ways that don't make sense. In an OpenAI call for the most part what they accept is an array of messages.

from pydantic import BaseModel

class Messages(BaseModel):
    content: str
    role: Literal["user", "system", "assistant"]

But so many libaries wanted me you to submit a string block and offer some synatic sugar to make it look like this: They also tend to map the docstring to the prompt. so instead of accessing a string variable I have to access the docstring via __doc__.

A feat of strength MVP for AI Apps

A minimum viable product (MVP) is a version of a product with just enough features to be usable by early customers, who can then provide feedback for future product development.

Today I want to focus on what that looks like for shipping AI applications. To do that, we only need to understand 4 things.

  1. What does 80% actually mean?

  2. What segments can we serve well?

  3. Can we double down?

  4. Can we educate the user about the segments we don’t serve well?

The Pareto principle, also known as the 80/20 rule, still applies but in a different way than you might think.

How to ask for Referrals (Among other things)

How can I help? Do you know anyone that could use my help? Do you know anyone that could use my services?

These are all examples of exceptionally low agency questions. Not only is it difficult to answer the question, you subject your victim to a lot of additional work and thinking in their busy day.

It's like seeing your mom sweating away busy cooking, chopping vegetables and asking "How can I help?" It's a lot of work to manage you, and it's a lot of work to think about what you can do. Now she has to consider what's in your ability, what the unfinished work is, and prioritize that versus the other.

This post is my simple framework on how I ask.

  • in RAG
  • 10 min read

Stop using LGTM@Few as a metric (Better RAG)

I work with a few seed series a startups that are ramping out their retrieval augmented generation systems. I've noticed a lot of unclear thinking around what metrics to use and when to use them. I've seen a lot of people use "LGTM@Few" as a metric, and I think it's a terrible idea. I'm going to explain why and what you should use instead.

For more writing on RAG and evaluation, start with RAG series index.

When giving advice to developers on improving their retrieval augmented generation, I usually say two things:

  1. Look at the Data
  2. Don't just look at the Data

Wise men speak in paradoxes because we are afraid of half-truths. This blog post will try to capture when to look at data and when to stop looking at data in the context of retrieval augmented generation.

I'll cover the different relevancy and ranking metrics, some stories to help you understand them, their trade-offs, and some general advice on how to think.

My year at 1100ng/dL

I'm not a doctor, but I did manage to double my testosterone levels in a year. I'm going to talk about what I did, what I learned, and what I think about it:

  1. It's just a fact that male testosterone levels have been dropping for the past couple of years.
  2. I felt like I was in a rut and I wanted to feel better, and I did.
  3. I was such a psycho about it that I decided to go off the protocol.
  4. Despite that, I still think every man should get their levels tested and see if they can improve them. And just understand how they feel.

Indie Consulting

As I've shared insights on building a consulting practice, marketing strategies, and referral techniques, it's important to understand the unique position of indie consulting in the broader landscape. In this post, we'll explore how indie consulting differs from traditional large-scale consulting firms and why it can offer more value to clients.

Indie consulting is fundamentally distinct from the practices of well-known institutions. For a critical perspective on these large firms, I recommend watching John Oliver's insightful critique of McKinsey or this concise TikTok video that encapsulates the issues with big consulting firms.

In contrast to these large firms, indie consulting focuses on specialized expertise, direct accountability, and long-term value creation for clients. It's about leveraging personal experience and skills to solve specific problems, rather than applying generic frameworks or strategies. This approach aligns closely with the pricing strategies and tools I've discussed in previous posts, all aimed at delivering maximum value to clients.

A Critique on Couches

Here are some fragmented reasons as to why I don't like having a couch.

The couch, often positioned facing a television, symbolizes the societal imposition of a predetermined essence onto our living spaces. This arrangement, reminiscent of Sartre's concept of bad faith, dictates the room's function and restricts its potential. It mirrors the limitations we place upon ourselves when we conform to societal expectations, disregarding our authentic selves.

For real.

Tips for probabilistic software

This writing stems from my experience advising a few startups, particularly smaller ones with plenty of junior software engineers trying to transition into machine learning and related fields. From this work, I've noticed three topics that I want to address. My aim is that, by the end of this article, these younger developers will be equipped with key questions they can ask themselves to improve their ability to make decisions under uncertainty.

  1. Could an experiment just answer my questions?
  2. What specific improvements am I measuring?
  3. How will the result help me make a decision?
  4. Under what conditions will I reevaluate if results are not positive?
  5. Can I use the results to update my mental model and plan future work?