Vector Search & RAG Engineering Roadmap

Build a reproducible representation pipeline. Treat source authority, structure, permissions, freshness, model, dimensions, and preprocessing as one versioned contract.

• Explain embeddings as vectors whose distance or similarity ranks related items
• Record model, deployment, dimensions, tokenizer, preprocessing, and normalization behavior
• Keep document and query vectors in a compatible versioned embedding space
• Relate unit normalization to cosine, dot product, and L2 ranking
• Compare fixed, paragraph, heading, procedure, table, and parent-child chunking
• Tune size and overlap on labeled boundary cases rather than folklore
• Preserve document/chunk IDs, source URI, section, checksum, product, version, language, and dates
• Preserve tenant and ACL metadata without exposing secrets
• Design idempotent updates, tombstones, deletes, re-embedding, and rollback
• Measure chunks, duplicates, embedding tokens, storage, ingestion latency, and failed items

Use exact eligible neighbors as ground truth, then tune approximate indexes and filters along an evidence-based speed-recall-resource frontier.

• Implement exact nearest-neighbor search as an ANN recall baseline
• Understand how HNSW navigates a multilayer proximity graph
• Tune HNSW construction and search candidates against recall, latency, memory, and writes
• Understand how IVFFlat or IVF-style indexes train lists and probe selected clusters
• Build IVF-style indexes after representative data exists and tune lists/probes
• Match cosine, dot-product, or L2 indexes to the embedding model and query operator
• Test warm/cold latency, p95/p99, throughput, concurrency, inserts, deletes, and rebuilds
• Measure low, medium, and highly selective filters
• Compare prefilter, postfilter, iterative scan, partial index, and partitioning behavior where supported
• Use query plans, backend diagnostics, exact comparisons, and filled top-k counts to troubleshoot

Protect exact identifiers with lexical search, broaden conceptual recall with vectors, fuse ranks, rerank bounded candidates, and carry authorized current provenance into every answer.

• Create separate lexical and vector baselines for exact and conceptual queries
• Run both under the same trusted permission and freshness scope
• Use RRF or a documented fusion method without treating raw score scales as interchangeable
• Preserve codes, names, dates, versions, and specialist terms during query processing
• Rerank only a bounded candidate set and measure incremental gain
• Deduplicate overlapping chunks and preserve parent/neighbor relationships
• Assemble token-bounded context with immutable source and chunk identifiers
• Map material claims to authorized current supporting passages
• Abstain or escalate on missing, stale, conflicting, or insufficient evidence
• Measure source-to-index lag, update/delete propagation, citation validity, latency, and cost

Keep authorization and authority outside the model. Treat every query, source, retrieved passage, model output, citation, and telemetry event as data crossing a controlled boundary.

• Derive tenant, user, and group scope from authenticated identity in trusted code
• Enforce document access before reranking, model context, caching, citations, and telemetry
• Use row security, permission metadata, index partitioning, or supported permission-aware retrieval deliberately
• Run cross-tenant, forged-claim, removed-group, owner-bypass, and stale-ACL negative tests
• Treat retrieved instructions as untrusted evidence, never authority
• Keep secrets, system policy, allowlists, and credentials outside retrieved/model-controlled content
• Block arbitrary tools, files, networks, URLs, and data exports unless explicitly required and authorized
• Minimize and redact prompts, chunks, identity data, and outputs in telemetry
• Apply NIST AI RMF Govern, Map, Measure, and Manage throughout the lifecycle
• Define incident, correction, retention, deletion, evidence publication, and retirement controls

Operate quality as a service characteristic. Separate ingestion, retrieval, ranking, answer, policy, and infrastructure failures, then make every release measurable and reversible.

• Build representative, adversarial, permission, freshness, and no-answer query slices
• Measure recall@k, MRR, nDCG, filter correctness, duplicate rate, and eligible top-k fill
• Measure answer correctness, completeness, claim support, citations, abstention, and safety
• Calibrate automated graders against deterministic checks and human review
• Trace ingestion, embedding, search, filters, fusion, reranking, context, generation, and evaluation
• Dashboard freshness, versions, quality probes, p50/p95/p99 latency, throughput, failures, resources, tokens, and cost
• Inject stale vectors, partial batches, malformed metadata, selective filters, overload, timeouts, and telemetry loss
• Use versioned manifests, canaries, release gates, kill controls, rebuilds, and rollback
• Attribute embeddings, text/vector storage, index memory, queries, reranking, generation, evaluation, and telemetry
• Stop ingestion and delete data, indexes, models, identities, secrets, telemetry, infrastructure, and billing footprint