Writing and mumblings¶

Best Tools for Indie Consultants

As an indie consultant, having the right tools can make or break your business. Over the years, I've refined my "consulting stack" - a collection of software and services that keep my operations smooth and professional. This post is an extension of my thoughts on AI consulting and freelancing in the AI gold rush.

In this guide, I'll share the key components of my stack and why they matter. Whether you're just starting out or looking to optimize your existing practice, these tools can help streamline your operations and enhance your professional image. I'll cover everything from setting up your business foundation to essential software for day-to-day operations.

By the end, you'll have a comprehensive todo list for setting up your business foundation and implementing your own consulting stack. This advice stems from my personal experience and lessons learned from AI consulting, aimed at helping you avoid common pitfalls and accelerate your success in the consulting world.

Optimizing Tool Retrieval in RAG Systems: A Balanced Approach

When it comes to Retrieval-Augmented Generation (RAG) systems, one of the key challenges is deciding how to select and use tools effectively. As someone who's spent countless hours optimizing these systems, many people ask me whether or not they should think about using retrieval to choose which tools to put into the prompt. What this actually means is that we're interested in making precision and recall trade-offs. I've found that the key lies in balancing recall and precision. Let me break down my approach and share some insights that could help you improve your own RAG implementations.

In this article, we'll cover:

1. The challenge of tool selection in RAG systems
2. Understanding the recall vs. precision tradeoff
3. The "Evergreen Tools" strategy for optimizing tool selection

The RAG Playbook

When it comes to building and improving Retrieval-Augmented Generation (RAG) systems, too many teams focus on the wrong things. They obsess over generation before nailing search, implement RAG without understanding user needs, or get lost in complex improvements without clear metrics. I've seen this pattern repeat across startups of all sizes and industries.

But it doesn't have to be this way. After years of building recommendation systems, instrumenting them, and more recently consulting on RAG applications, I've developed a systematic approach that works. It's not just about what to do, but understanding why each step matters in the broader context of your business.

Here's the flywheel I use to continually infer and improve RAG systems:

1. Initial Implementation
2. Synthetic Data Generation
3. Fast Evaluations
4. Real-World Data Collection
5. Classification and Analysis
6. System Improvements
7. Production Monitoring
8. User Feedback Integration
9. Iteration

Let's break this down step-by-step:

How I want you to write

I'm gonna write something technical.

It's often less about the nitty-gritty details of the tech stuff and more about learning something new or getting a solution handed to me on a silver platter.

Look, when I read, I want something out of it. So when I write, I gotta remember that my readers want something too. This whole piece? It's about cluing in anyone who writes for me, or wants me to write for them, on how I see this whole writing product thing.

I'm gonna lay out a checklist of stuff I'd like to have. It'll make the whole writing gig a bit smoother, you know?

Living My Best Life: A $20 Million Daydream

I\'ve been playing a little thought experiment lately: If I had $20 million in the bank, how would I want to live my life? It\'s not about the money per se, but about imagining a life where financial constraints aren\'t the primary driver of my decisions. Here\'s what I\'ve come up with.

On Getting Recognized: The Unexpected Price of Online Success

I never thought I'd be writing about the challenges of being recognized in public. A year ago, I was just another data scientist trying to build connections and establish myself in the field. Now, I'm grappling with the unintended consequences of my growing online presence. Here's how it all unfolded, and what I've learned about the price of distribution in the digital age.

Art of Looking at RAG Data

In the past year, I've done a lot of consulting on helping companies improve their RAG applications. One of the biggest things I want to call out is the idea of topics and capabilities.

I use this distinction to train teams to identify and look at the data we have to figure out what we need to build next.

My Self-Reflection on Success and Growth

In his essay "What's Going On Here, With This Human?", Graham Duncan discusses the importance of seeing people clearly, both in the context of hiring and in understanding oneself. He suggests asking the question "what's going on here with this person in front of me?" and provides a framework for expanding one's ability to see others more clearly. Inspired by this essay, I asked myself some probing questions to better understand my own strengths, weaknesses, and motivations. Here are my reflections:

Predictions for the Future of RAG

In the next 6 to 8 months, RAG will be used primarily for report generation. We'll see a shift from using RAG agents as question-answering systems to using them more as report-generation systems. This is because the value you can get from a report is much greater than the current RAG systems in use. I'll explain this by discussing what I've learned as a consultant about understanding value and then how I think companies should describe the value they deliver through RAG.

Rag is the feature, not the benefit.

10 Ways to Be Data Illiterate (and How to Avoid Them)

Data literacy is an essential skill in today's data-driven world. As AI engineers, understanding how to properly handle, analyze, and interpret data can make the difference between success and failure in our projects. In this post, we will explore ten common pitfalls that lead to data illiteracy and provide actionable strategies to avoid them. By becoming aware of these mistakes and learning how to address them, you can enhance your data literacy and ensure your work is both accurate and impactful. Let's dive in and discover how to navigate the complexities of data with confidence and competence.