RAG Query Transformation

RAG query transformation means changing the user's question before searching. Users write short, vague or follow-up questions like "what about the second one?" that retrieve poorly. A quick LLM call can turn them into better search queries.

Rewriting Follow-Up Questions

import ollama

def rewrite(history, question):
    prompt = ("Rewrite the last question as a standalone search query using the chat history. "
              "Return only the query.\n\n"
              f"History:\n{history}\n\nQuestion: {question}")
    r = ollama.chat(model="llama3.2", options={"temperature": 0},
                    messages=[{"role": "user", "content": prompt}])
    return r["message"]["content"].strip()

history = "User: What is the leave policy for interns?\nAssistant: Interns get 10 days."
print(rewrite(history, "And for full-time staff?"))

Sample output:
leave policy for full-time staff

Without rewriting, "And for full-time staff?" would be embedded alone and match almost anything. This step is essential for any conversational RAG app.

Multi-Query Retrieval

Generate a few different phrasings, search with each, and merge the results (for example with RRF from the hybrid search lesson). Different wordings reach different chunks.

def multi_query(question, n=3):
    r = ollama.chat(model="llama3.2", messages=[{"role": "user", "content":
        f"Write {n} different search queries for: {question}\nOne per line, no numbering."}])
    return [q.strip() for q in r["message"]["content"].splitlines() if q.strip()][:n]

print(multi_query("Why is my laptop so slow?"))

Sample output:
['laptop running slow causes', 'how to speed up a slow computer', 'high CPU usage laptop fix']

HyDE: Hypothetical Document Embeddings

HyDE asks the model to write a fake answer, then embeds that answer instead of the question. A hypothetical answer often looks more like the real document than a short question does.

def hyde_vector(question):
    fake = ollama.chat(model="llama3.2", messages=[{"role": "user",
        "content": f"Write a short passage that answers: {question}"}])["message"]["content"]
    return ollama.embed(model="nomic-embed-text", input=fake)["embeddings"][0]

(returns a 768-dim vector used for the search)

Other Query Techniques

• Step-back prompting: ask a more general question first for background.
• Decomposition: split a multi-part question into sub-questions.
• Metadata extraction: pull filters like year or product from the question.
• Routing: choose which index or tool to search based on the question.

Frequently asked questions

What is query rewriting in RAG?

Using an LLM to turn the user's question into a better standalone search query, especially for follow-up questions in a chat.

Does HyDE always improve results?

No. It helps when questions are short and documents are long, but can hurt when the model invents misleading details. Test it.

Do query transformations add latency?

Yes, each adds an LLM call. Use a small fast model for rewriting.