Skip to content

Writing and mumblings

  • in RAG
  • 7 min read

With the advent of large language models (LLM), retrieval augmented generation (RAG) has become a hot topic. However throught the past year of helping startups integrate LLMs into their stack I've noticed that the pattern of taking user queries, embedding them, and directly searching a vector store is effectively demoware.

What is RAG?

Retrieval augmented generation (RAG) is a technique that uses an LLM to generate responses, but uses a search backend to augment the generation. In the past year using text embeddings with a vector databases has been the most popular approach I've seen being socialized.

RAG

Simple RAG that embedded the user query and makes a search.

So let's kick things off by examining what I like to call the 'Dumb' RAG Model—a basic setup that's more common than you'd think.

Kojima's Philosophy in LLMs: From Sticks to Ropes

Hideo Kojima's unique perspective on game design, emphasizing empowerment over guidance, offers a striking parallel to the evolving world of Large Language Models (LLMs). Kojima advocates for giving players a rope, not a stick, signifying support that encourages exploration and personal growth. This concept, when applied to LLMs, raises a critical question: Are we merely using these models as tools for straightforward tasks, or are we empowering users to think critically and creatively?

Good LLM Observability is just plain observability

In this post, I aim to demystify the concept of LLM observability. I'll illustrate how everyday tools employed in system monitoring and debugging can be effectively harnessed to enhance AI agents. Using Open Telemetry, we'll delve into creating comprehensive telemetry for intricate agent actions, spanning from question answering to autonomous decision-making.

If you want to learn about my consulting practice check out my expert calls page. If you're interested in working together please reach out to me via email

What is Open Telemetry?

Essentially, Open Telemetry comprises a suite of APIs, tools, and SDKs that facilitate the creation, collection, and exportation of telemetry data (such as metrics, logs, and traces). This data is crucial for analyzing and understanding the performance and behavior of software applications.

Freediving under ice

Growing up, I wasn't very physically active. However, as I got older and had more time, I made a conscious effort to get in shape and improve my relationship with my body.

I had done plenty of sports before like you know ping pong or rock climbing or jiu jitsu but after I got my hand injuries during covid I really couldn't do any of that...

Recommendations with Flight at Stitch Fix

As a data scientist at Stitch Fix, I faced the challenge of adapting recommendation code for real-time systems. With the absence of standardization and proper performance testing, tracing, and logging, building reliable systems was a struggle.

To tackle these problems, I created Flight – a framework that acts as a semantic bridge and integrates multiple systems within Stitch Fix. It provides modular operator classes for data scientists to develop, and offers three levels of user experience.