u/ethanchen20250322

A lot of databases now support vector columns, which is great for getting started.

But I’m not sure “just add a vector column” is enough once the AI dataset becomes a real production asset.

The problem is that embeddings are not like ordinary columns. They are large, expensive to regenerate, and often versioned. In a RAG or multimodal pipeline, you may need to:

• backfill embeddings after changing models
• keep embedding_v1 and embedding_v2 for evaluation
• add sparse vectors for hybrid search
• update small metadata fields without rewriting huge vector blobs
• support both batch scans and low-latency ANN lookups
• let Spark/Ray/GPU jobs write derived features back into the dataset

Once you get to that point, a single table with a vector column can become hard to manage. Metadata is usually fine in a columnar layout because you want cheap scans and filters. Large vector fields have very different access patterns, especially after ANN search when you need to fetch specific rows quickly. Raw files like images, PDFs, or videos usually should not be embedded into the table at all. And indexes need to be built, replaced, and compacted on their own schedule.

Loon, the new storage engine in Milvus 3.0 beta and Zilliz Vector Lakebase, is treating the collection as one logical dataset, but storing different parts in different physical layouts:

• metadata in Parquet
• vectors in Vortex
• raw objects in object storage
• everything tied together by row IDs and a versioned Manifest

That feels closer to a lakehouse model for AI data than a traditional “table with a vector column.”

Curious how others are handling this in production. Are vector columns enough for your workloads, or do you end up building separate storage/versioning layers around them?

Most vector DB discussions focus on ANN algorithms: HNSW, IVF, DiskANN, quantization, recall/latency, etc.

But in real AI workloads, the dataset keeps changing. You add captions, swap embedding models, backfill new vector columns, add sparse vectors, fix metadata, delete old rows, and rebuild indexes.

That creates storage problems:

• A new embedding column can mean TB-scale writes.
• A tiny metadata fix should not rewrite huge vector columns.
• Parquet is good for scans, but ANN needs fast row-level reads.
• Spark/Ray/GPU pipelines and the vector DB often create duplicate sources of truth.

Loon, the new storage engine in Milvus 3.0 beta and Zilliz Vector Lakebase, tries to solve this by splitting one logical collection into different physical layouts:

• metadata in Parquet
• vectors in Vortex
• raw objects in object storage
• everything tied together by row IDs and a versioned Manifest

So instead of treating vector data as just a search index, Loon treats it as a constantly evolving AI dataset.

Curious: are you managing vector data as a rebuildable index, or as a versioned storage layer?

I talk with developers building RAG systems pretty often, and one thing keeps coming up.

Most teams start by optimizing retrieval latency. That makes sense. If search is slow, the whole app feels slow.

But once the first version works, another problem shows up: improving the data behind retrieval is often slower than retrieval itself.

That is why the idea behind Vector Lakebase is interesting to me.

The demo pipeline is simple: chunk documents, embed them, index them, retrieve top-k results.

Production RAG is less clean. Teams keep changing chunks, metadata, embeddings, evals, and indexes.

The painful part is that online serving and offline data work are often separate. The search index serves traffic, while deduplication, clustering, re-embedding, quality analysis, and cold data search happen somewhere else.

So data gets exported, processed, copied back, re-indexed, and kept in sync by glue code.

Vector Lakebase is trying to close that gap: keep low-latency retrieval, but bring serving and discovery closer to the same logical data, with different compute modes for online, on-demand, and offline work.

I do not see this as replacing vector databases. It is more like recognizing that RAG is becoming a loop: serve, observe, improve the data, rebuild, evaluate, repeat.

Latency still matters. But once search is fast enough, I’m curious where others feel the bigger pain: query latency or data iteration?

For years, the goal of vector search was simple: make it faster.

Lower latency. Higher QPS. Better indexes. Better recall.

That made sense. Many production AI apps need fast, always-on search, and a vector database is still the core system for those workloads.

But AI changed how people use vector search.

With RAG, agents, support search, logs, and user documents, teams now create a lot more embeddings than before. Some of this data is searched all the time, but a lot of it is not.

It may be stored for months and only searched once in a while.

That made me ask a different question:

Does every vector workload need always-on compute?

For hot data, yes. Low latency and strong performance still matter a lot.

But for cold or warm embeddings, the goal can be different. Storage cost, scale, and on-demand compute may matter more than keeping everything ready all the time.

That is how I think about Vector Lakebase. It is not a replacement for vector databases. It extends vector search to workloads where the data is still useful, but not always hot.

I’m still thinking through this shift, so I’d love to hear how others see it.

How much of your vector data is actually hot?

I have been thinking about a simple tradeoff in vector search.

Today, many teams store a lot of embeddings. These can come from docs, support tickets, logs, user data, or RAG apps.

But not all of this data is searched all the time. Some data is “cold.” It may sit there for weeks or months and only get searched once in a while.

The problem is that many vector databases still need compute to stay on. The index needs to be ready. So you keep paying, even when nobody is searching.

Vector Lakebase uses a different model. The data and index can stay in object storage. Compute can start only when a search is needed.

That can make vector search much cheaper for cold data.

But there is a cost: the first search may need a 5-10 second cold start.

So my question is:

Would you accept a 5-10 second wait if vector search became much cheaper?

I think it depends on the use case.

For a user-facing chatbot, probably not.

For internal search, maybe.

For batch jobs, analytics, or rarely used customer data, it might be fine.

Curious what others think. Do your vector search workloads need very low latency, or do you also have a lot of cold embeddings that are rarely searched?

Vector databases started with a clear job: serve vector search fast. Keep indexes loaded, optimize for low latency, and make semantic retrieval reliable for production apps.

That still makes sense for hot workloads. But embedding data is starting to look less like “just an online index” and more like a durable data layer. Teams are storing vectors alongside raw text, metadata, feedback logs, labels, agent traces, and eval data.

That is why I find the shift from vector database to vector lakebase interesting.

To me, a vector lakebase should mean separating persistent semantic storage from the compute used to search or process it. The same data should support different workloads: real-time retrieval for hot paths, on-demand search for rarely queried data, and batch analytics for clustering, deduping, corpus analysis, or dataset prep.

It also should not just be “vectors in object storage.” It still needs database-like behavior: metadata filtering, scalar fields, indexing, query execution, and support for hybrid retrieval across vectors, text, JSON, and reranking.

Curious how data engineers see this:

• Should embeddings become part of the lakehouse-style data layer?
• Or should vector search stay as a separate serving system?
• What would make “vector lakebase” useful rather than just another term?

[ Removed by Reddit on account of violating the content policy. ]