RAG Embeddings

RAG embeddings are the numeric fingerprints of meaning that make semantic search possible. An embedding model turns a piece of text into a fixed-length vector, such as 768 numbers, so that texts with similar meaning end up close together.

Creating Embeddings with Ollama

Install the Ollama Python client (pip install ollama) and pull an embedding model (ollama pull nomic-embed-text):

import ollama

resp = ollama.embed(model="nomic-embed-text", input=["I love pizza", "Pizza is my favourite food"])
vectors = resp["embeddings"]
print(len(vectors), "vectors of length", len(vectors[0]))

2 vectors of length 768

ollama.embed accepts a single string or a list and returns one vector per input. Batching many texts in one call is faster than calling it in a loop.

Cosine Similarity

To compare two vectors we usually use cosine similarity: 1 means same direction (same meaning), 0 means unrelated.

import math

def cosine(a, b):
    dot = sum(x * y for x, y in zip(a, b))
    return dot / (math.hypot(*a) * math.hypot(*b))

texts = ["How do I cancel my subscription?",
         "Steps to end your membership plan",
         "Best hiking trails near Islamabad"]
v = ollama.embed(model="nomic-embed-text", input=texts)["embeddings"]
print(round(cosine(v[0], v[1]), 3))
print(round(cosine(v[0], v[2]), 3))

Sample output:
0.812
0.356

The first two sentences share no important words, yet they score high because they mean the same thing. That is the superpower of embeddings over keyword search.

Choosing an Embedding Model

• nomic-embed-text: good general English model, 768 dimensions, runs anywhere.
• mxbai-embed-large: higher quality, 1024 dimensions, a bit slower.
• bge-m3: multilingual and long inputs, good for Urdu, Arabic and other languages.
• Check the MTEB leaderboard for retrieval benchmarks.
• Hosted APIs (OpenAI, Cohere, Voyage) are options if you cannot run models locally.

Embedding Rules of Thumb

• Use the same model for documents and queries, always.
• Some models want prefixes like "search_document:" and "search_query:"; check the model card.
• Embedding inputs have a maximum length; long chunks get truncated silently.
• Store the model name with your index so you know when re-embedding is needed.

Frequently asked questions

What is an embedding in RAG?

A vector of numbers representing the meaning of a text chunk or query. Similar meanings give similar vectors, enabling semantic search.

Why does my similarity score look high for unrelated text?

Many models produce scores in a narrow range, so absolute values mislead. Compare scores relative to each other rather than using a fixed universal threshold.

How many dimensions should embeddings have?

Common sizes are 384 to 1024. More dimensions can capture more detail but use more storage and slower search.