Predictions for the Future of RAG¶

In the next 6 to 8 months, RAG will be used primarily for report generation. We'll see a shift from using RAG agents as question-answering systems to using them more as report-generation systems. This is because the value you can get from a report is much greater than the current RAG systems in use. I'll explain this by discussing what I've learned as a consultant about understanding value and then how I think companies should describe the value they deliver through RAG.

Rag is the feature, not the benefit.

Reports over RAG¶

So why are reports better than RAG? Simply put, RAG systems suck because the value you derive is time saved from finding an answer. This is a one-dimensional value, and it's very hard to sell any value beyond that. Meanwhile, a report is a higher-value product because it is a decision-making tool that enables better resource allocation.

To better illustrate this, I'll give a couple of examples:

If I have one employee I'm paying hourly, they can use a RAG app to run a query, and then they can deliver an answer. This is a perfectly acceptable way of using RAG in one-dimensional static scenarios, such as asking single questions. However, when a research team wants to do interviews (question-answer queries), the deliverable isn't an answer to a set of questions. Instead, it's a report. So, the RAG app can save the time of 8 employees making 50 dollars an hour, whereas the report will cost $20,000. If the report is helping an executive allocate a 5million dollar budget, the price might charge becomes a much smaller portion of that investment? This is true even if the process to generate the report is just a RAG application in a for loop.

The value of these two items is communicated differently. RAG is evaluated as a percentage of wages, while the report is evaluated as a percentage of high-leverage outcomes.

Another way this plays out is if you're hiring. If you're interviewing a client with 6 rounds of interviews, you could use RAG to ask questions, which might work. What might be better is if your organization made a well-defined template on which you can make high-value decisions. Something like "Has this candidate worked in a team before", "Are they independent?", "Do they reflect our company's values?". These are pretty well-known and established parts of the hiring template.

If there were a service that could take this template and all the meeting notes from the six interviews and then generate a report for you and your team to review and utilize in your hiring process, the value would be derived from the decision-making and capital allocated to hire the candidate. A recruiter might take $40,000 on a $250,000 candidate, which means being able to make a better decision as a result of this hiring overview is enormous. The hypothetical hiring app's value is much greater than simple question-answer sets because the outcome of the RAG application is less clear than the outcome of having a high-quality report you can rely on to make key decisions for your business. This is because the end deliverable has a greater value that can be leveraged, even if the process is similar. A good interview panel knows what the question should be, but your hiring copilot should do more and help you get there.

Why you need SOPs¶

Furthermore, how reports are written is incredibly important. Scaling decision-making and processes in a company often involves developing standard operating procedures (SOPs), which are a way of formatting various reports in a unified manner.

One of the reasons I attend workshops, get coaching, or read business books is because the outcome I am looking for is an SOP. For instance, I learned a way to write sales engagement letters that convert better. Now, all of my meetings fit this format and help make me far more money than the $40 dollar book I learned the template from cost. People are taught to give feedback and answer questions in specific ways. You get better outcomes when this output is structured correctly in something like a report or a template. Being able to pay for the right report template can be incredibly valuable because it ensures you're getting the outcome you actually need.

I don't care so much about being able to read a chat transcript of a meeting I had. I care if I can turn that transcript into a format and report that I know will drive my desired business outcomes rather than just save me time. I want the AI to create a memo with clear deliverables for me or summarize the chat transcript to tell me, "This is the objective, this is how we make the decision, and here are the follow-ups."

Ultimately, a report's value goes beyond a wage worker answering questions—it supports high-leverage outcomes like strategic decision-making.

Future outcomes¶

If RAG primarily becomes report generation it means two things are possible: 1. a marketplace of report-generating tools, and 2. the ability to effectively find the right report for your desired outcome. I think that question-answer sets are going to be of limited usefulness, while report generation addresses not only question-answer sets but the value of decision-making. When these reports are available in a marketplace of templates, they add further value because understanding what the template is defining becomes a skill in itself. These are the kinds of skills that people then take workshops on, get coaches for, and buy books about.

Subscribe to my writing¶

I write about a mix of consulting, open source, personal work, and applying llms. I won't email you more than twice a month, not every post I write is worth sharing but I'll do my best to share the most interesting stuff including my own writing, thoughts, and experiences.

Related Posts¶

For more insights on RAG systems and related topics, check out these posts: - The RAG Playbook - A systematic approach to continually improve RAG systems - How to build a terrible RAG system - An inverted thinking exercise on RAG best practices - RAG is more than just embedding search - Exploring advanced RAG techniques beyond simple embeddings