Vector Data Architecture: Similarity Search and AI Retrieval

When I work with vector data, I start with retrieval quality and access control, not the embedding model alone. I recommend versioning embeddings, storing metadata, and filtering before nearest neighbor search returns anything to the user.

Where embeddings usually live

When I talk about "Where embeddings usually live", I am checking whether Vector Data Architecture can be traced back, trusted, and used by someone making a decision.

OpenSearch, Elasticsearch, pgvector, Pinecone, Milvus, Weaviate, Vertex AI Vector Search, Azure AI Search vector, BigQuery vector search, AlloyDB or PostgreSQL with pgvector.

Managed vector options

Similarity pattern to remember

My recommendation in "Similarity pattern to remember" is to keep the raw source close enough that the answer can be explained later.

Netflix search papers are a good public mental model: search quality is not only keyword match. Modern retrieval mixes text, behavior, metadata, ranking, and similarity so users find the right item even when their words are imperfect.

Merchant support retrieval example

I use "Merchant support retrieval example" to keep Vector Data Architecture grounded in a real system, because abstract patterns are too easy to agree with and too hard to operate.

Imagine a payments company that supports merchants in the US, UK, and EU. The support knowledge base has refund policy PDFs, chargeback playbooks, API docs, processor return code guides, and internal escalation notes. Each document is split into small chunks and embedded, but every chunk also carries metadata: product, region, customer type, policy version, effective date, and visibility level.

A merchant asks, "Can I refund an ACH payment after the bank returned it as R01?" The retrieval service embeds that question, then filters to product = payments, payment method = ACH, region = US, visibility = support tier 2, and documents effective on the ticket date. The nearest chunks come from the ACH return policy, the refund eligibility table, and the negative balance recovery playbook. The answer workflow can now say: do not issue a normal refund after the ACH debit has already returned; explain the R01 insufficient funds return, check whether the merchant balance is negative, and route the case to balance recovery if money movement already failed.

Retrieval threshold query

I use "Retrieval threshold query" to connect the data type to the person, workflow, or system that will consume it.

with matches as (
  select article_id,
         title,
         embedding <=> :question_embedding as distance
  from help_article_embeddings
  where product = 'payments'
    and payment_method = 'ACH'
    and region = 'US'
    and visibility in ('PUBLIC_SUPPORT', 'TIER_2_SUPPORT')
    and effective_from <= :ticket_created_at
    and (effective_to is null or effective_to > :ticket_created_at)
  order by distance
  limit 5
)
select *,
       case when distance <= 0.35 then 'ANSWER' else 'ESCALATE' end as action
from matches;

The query combines semantic distance with metadata filters and an action threshold. Close matches can answer; weak matches escalate instead of hallucinating confidence.

Vector search traps

• Version the embedding model and refresh policy.
• Keep metadata filters close to the vector index.
• Evaluate retrieval with real questions, not only demo prompts.

Related vector reads

• Everything Data: The Asset Behind Modern Software
• YCSB: A Useful Benchmark for Data Stores
• Finding Performance Bottlenecks in a Serverless Microservice Chain