Technical field guide

Index-time decisions are expensive to change; query-time decisions are expensive to run

Index time turns source content into retrieval units, representations, and an index, and changing any of it means reprocessing documents or rebuilding part of that index. Query time retrieves candidates and reranks them, and those settings land directly on latency, cost, recall, and precision. A control loop closes the stack: change one lever, measure it against labeled queries from your own corpus, then keep or roll back.

This guide explains what each control measures, gives an implementation range or model example, and shows the downstream trade-off. The builder then translates your choices into JSON or Markdown that tells a developer or AI what to implement and what must be calibrated first.

Learn first. Configure second.

Build a measurable vector search stack

Choose one decision at each stage. Every option names the real control, its unit, a starting value, and the consequence. The generated contract gives an AI or developer explicit decisions, compatibility checks, and calibration guardrails.

Ranges are starting points, not universal targets. Validate every choice on labeled queries from your own corpus.

System lifecycle

Three parts of the system behind a search

Ingest & Index builds searchable data. Retrieve & Rank runs for each request. Evaluate & Improve measures both phases and tunes the next version. Select a card to see its decisions, terms, and output.

What happens

Split content into retrieval units, convert each unit into one or more searchable representations, then store those representations in an index. Changes here usually require reprocessing documents or rebuilding part of the index.

Question: How should source content become searchable?
You decide: Where chunks begin and end
Which embedding and keyword representations to store
Which index structure and build settings to use
Expected output: A versioned index containing retrieval units, representations, and searchable metadata.

Configure:Segment Represent Index

Terms in this phase

What the technical terms mean

ChunkRetrieval unit

What it is: A small section of a document stored and searched as one unit.
What it does: It sets the amount of context represented by one embedding and returned by one retrieval result.
When to use it: Choose chunk boundaries and token counts that preserve enough meaning without mixing unrelated topics.
Scope: Chunk boundaries are created at Index-time. At Query-time, each returned result is one of those stored retrieval units, so chunk size controls how much context a search can return.

EmbeddingNumeric meaning representation

What it is: A list of numbers produced by a model to represent the meaning of text, images, or other content.
What it does: It lets the system compare content by mathematical distance or similarity.
When to use it: Use embeddings to retrieve conceptually related content even when the query and document use different words.
Scope: Document embeddings are created at Index-time. At Query-time, the query must use a compatible embedding model, vector space, and distance measure.

HNSWHierarchical Navigable Small World

What it is: An approximate nearest-neighbor index that organizes vectors as a layered graph.
What it does: A query follows promising links instead of comparing every stored vector. The m setting controls stored links per node. ef_construction controls index-build effort. ef_search controls query-time effort.
When to use it: Use it when exact scans are too slow. Measure recall against exact search, then tune m, ef_construction, and ef_search against memory, build time, and p95 latency.
Scope: The graph is built at Index-time. The ef_search control changes how thoroughly a query explores that graph at Query-time.

What happens

Retrieve a wider candidate set, optionally apply a more precise reranker, then return the final results. These choices directly affect request latency, cost, recall, and precision.

Question: How should each request find and order evidence?
You decide: How many candidates each retriever returns
Which similarity threshold removes weak candidates
Whether and how a reranker orders the candidate set
Expected output: A ranked result set with scores, source references, and evidence for the next system step.

Configure:Retrieve Rank

Terms in this phase

What the technical terms mean

RAGRetrieval-augmented generation

What it is: A pattern that retrieves source material before an AI generates an answer.
What it does: It gives the model selected context from your own content instead of relying only on model memory.
When to use it: Use it for answers that need private, recent, or source-backed information.
Scope: RAG spans the stack. Retrieval runs at Query-time, but it depends on content prepared at Index-time and passes the results to a generation model.

Dense retrievalEmbedding-based search

What it is: Search that compares one numeric embedding for the query with one embedding for each stored item.
What it does: It finds conceptually similar content even when the query and document use different words.
When to use it: Use it for semantic similarity. Combine it with sparse retrieval when exact names, codes, or rare terms also matter.

Sparse retrievalBM25 and inverted-index search

What it is: Keyword-oriented search that stores which documents contain each term. BM25 is a common scoring method, and an inverted index is the lookup structure behind it.
What it does: It gives strong weight to exact terms while accounting for term rarity and document length.
When to use it: Use it for product names, identifiers, citations, and domain terms that semantic embeddings may blur.

ANNApproximate nearest-neighbor search

What it is: A family of methods that finds likely nearest vectors without comparing every stored vector.
What it does: It reduces search work and latency by accepting a measurable chance of missing an exact neighbor.
When to use it: Use ANN when exact vector scans are too slow. Compare ANN results with exact search to measure index recall.

RRFReciprocal Rank Fusion

What it is: A method that combines ranked lists without requiring their raw scores to use the same scale.
What it does: It rewards items that rank highly in one or more retrieval lists, such as dense and sparse search.
When to use it: Use it to combine retrieval methods before reranking. Tune the fusion constant only on labeled queries.

RerankingSecond-pass scoring

What it is: A more precise scoring pass over a smaller set of retrieved candidates.
What it does: It reorders the candidate set using a model or method that is usually too expensive to run over the full corpus.
When to use it: Use it when first-stage retrieval finds useful candidates but does not put the best results first.

Late interactionMulti-vector reranking with MaxSim

What it is: A ranking method that keeps multiple token-level vectors for a query and document instead of reducing each one to a single vector. MaxSim means taking the best token-to-token similarity matches.
What it does: It preserves fine-grained matching signals and scores them after the smaller candidate set has been retrieved.
When to use it: Use it when single-vector retrieval misses detailed term relationships and the added storage and query cost are acceptable.

Starting profiles

Choose the miss you can least afford

Directional effect

How the current choices push each trade-off

Qualitative, not measured

RecallNear baselineHow often relevant items enter the candidate set.

PrecisionNear baselineHow much of the returned set is relevant.

CostNear baselineEmbedding, storage, model, and compute load.

LatencyNear baselineQuery time, including retrieval and reranking.

Decision output

Copy the decision contract

An AI can implement resolved choices and stop on fields marked for calibration or correction.

Implementation statusrequires calibration2 fields must be measured or supplied.

Resolve before implementation

Replace CALIBRATE with a threshold measured on labeled queries. Similarity scores are model, metric, and corpus specific.
Replace REQUIRED_LABELED_QUERY_SET with the versioned evaluation dataset used to approve this configuration.

{
  "schema_version": "rosslabs.vector-search-decision.v2",
  "profile_name": "balanced-rag",
  "implementation_status": "requires_calibration",
  "unresolved_fields": [
    {
      "path": "query.retrieval.similarity_threshold",
      "placeholder": "CALIBRATE",
      "resolution": "Sweep scores on labeled queries and choose the threshold that meets the product precision-recall target."
    },
    {
      "path": "evaluation.dataset_id",
      "placeholder": "REQUIRED_LABELED_QUERY_SET",
      "resolution": "Create and version a representative query-to-relevant-result dataset before approving the stack."
    }
  ],
  "assumptions": [
    "Numeric values are explicit starting points, not measured predictions.",
    "Chunk size uses the embedding model tokenizer, not words or characters.",
    "All quality targets require a labeled query and relevance set from the target corpus."
  ],
  "indexing": {
    "segmentation": {
      "strategy": "structure_aware",
      "boundary_order": [
        "heading",
        "paragraph",
        "sentence",
        "token"
      ],
      "target_tokens": 600,
      "range_tokens": [
        400,
        800
      ],
      "overlap_percent": 10,
      "overlap_tokens": 60
    },
    "representation": {
      "provider": "openai",
      "model": "text-embedding-3-small",
      "dimensions": 1536,
      "max_input_tokens": 8192,
      "hosting": "managed",
      "distance_metric": "cosine",
      "normalize_vectors": true,
      "signals": [
        "dense",
        "sparse"
      ]
    },
    "nearest_neighbor_index": {
      "engine": "pgvector",
      "algorithm": "hnsw",
      "approximate": true,
      "storage_type": "vector",
      "distance_metric": "cosine",
      "query_operator": "<=>",
      "standard_vector_max_dimensions": 2000,
      "m": 16,
      "ef_construction": 64,
      "ef_search": 100
    }
  },
  "query": {
    "retrieval": {
      "candidate_k_per_leg": 50,
      "similarity_threshold": "CALIBRATE",
      "calibration_metric": "precision_recall_curve",
      "recalibrate_on_model_change": true,
      "sparse_retriever": {
        "engine": "lucene_compatible",
        "method": "bm25",
        "index": "inverted",
        "analyzer": "language_specific_with_keyword_subfields",
        "k1": 1.2,
        "b": 0.75,
        "parameter_status": "starting_heuristic",
        "candidate_k_source": "query.retrieval.candidate_k_per_leg"
      },
      "fusion": {
        "method": "rrf",
        "rrf_k": 60,
        "dense_weight": 1,
        "sparse_weight": 1,
        "parameter_status": "starting_heuristic",
        "tune_on_labeled_queries": true
      }
    },
    "reranking": {
      "provider": "cohere",
      "model": "rerank-v4.0-fast",
      "method": "cross_encoder",
      "final_k": 10
    }
  },
  "evaluation": {
    "dataset_id": "REQUIRED_LABELED_QUERY_SET",
    "primary_metrics": [
      "recall@20",
      "ndcg@10"
    ],
    "example_targets": {
      "recall@20": 0.9,
      "ndcg@10": 0.75,
      "citation_coverage": 0.95
    },
    "guardrail_metrics": [
      "citation_coverage",
      "p95_retrieval_latency_ms",
      "cost_per_query"
    ],
    "target_policy": "Replace examples after measuring the baseline"
  },
  "warnings": [
    "Replace CALIBRATE with a threshold measured on labeled queries. Similarity scores are model, metric, and corpus specific.",
    "Replace REQUIRED_LABELED_QUERY_SET with the versioned evaluation dataset used to approve this configuration."
  ],
  "sources": [
    "https://developers.openai.com/api/docs/guides/embeddings",
    "https://help.openai.com/en/articles/6824809-embeddings-faq",
    "https://github.com/pgvector/pgvector#hnsw",
    "https://huggingface.co/BAAI/bge-m3",
    "https://docs.cohere.com/v2/docs/rerank",
    "https://docs.unstructured.io/concepts/chunking",
    "https://qdrant.tech/documentation/advanced-tutorials/reranking-hybrid-search/"
  ]
}

Primary sources and claim boundaries

Product names and documented defaults were checked on 2026-08-10. Heuristic ranges are labeled as starting points.

OpenAI vector embeddings guidetext-embedding-3-small and 3-large dimensions, 8,192-token input, dimensions parameter, and model trade-offs
OpenAI embeddings FAQnormalization and equivalence of cosine, dot-product ranking, and Euclidean ranking for OpenAI embeddings
pgvector index documentationexact versus approximate search, HNSW defaults, ef_search behavior, IVFFlat, filtering, and dimension limits
BAAI BGE-M3 model card1,024 dimensions, 8,192-token input, multilingual coverage, and dense, sparse, and multi-vector modes
Cohere Rerank model documentationRerank 4 Pro and Fast roles, multilingual support, and query-document reranking
Unstructured chunking documentationstructure-aware and semantic chunking controls, overlap behavior, and similarity-threshold semantics
Qdrant hybrid search documentationdense, sparse, RRF fusion, late interaction, candidate retrieval, and reranking architecture