r/vectordatabase

I’m building a local-first tool that indexes your files for semantic search.

I’m thinking about covering the OpenAI embedding usage for new users, mostly so nobody has to paste in an API key before they’ve searched a single PDF.

But embeddings are cheap enough that I’ve completely lost perspective.

1M tokens a month? 5M? 10M?

What says “actually useful” rather than “congrats, you can embed your README”?

I ended up trying a few different things in MindCache. The parts that survived those many iterations were...i just wanna whether these desgins make sense to people who have worked with retrieval, rag and memory systems and where they might fail.

I decided using four memory types- user, knowledge, episodic, and decision memories, each with different lifecycles, different roles and different token budget in the retrieved context.

Decision analysis + anchors — decisions can evolve overtime so they can be active or superseded or conditional instead of remaining as unrelated memories.
we keep the track of decision memory which is active, superseded or conditional with additional context and using such active decisions related to the query as anchors to further retrieve memories using lexical bm25.

Smart injection — when new memories come they aren't simply assigned to a topic based on similarity.
An LLM-guided ingestion step uses the existing topic structure as context to decide where a memory belongs and how it relates to what is already there.
This lets the hierarchy grow dynamically instead of becoming a collection of isolated memory nodes.

Hierarchical summaries — MindCache adapts the static RAPTOR-style tree idea into a dynamic hierarchy that is incrementally updated as new memories arrive.
I thought organizing memories into broader topics and maintaining summaries at those levels might help with broad queries, where retrieving individual memories one by one may miss the overall context. The topic structure also gives retrieval additional lexical/contextual signals, so a query can match against the organized topic structure as well as the underlying memories..

On my BEAM evaluation, MindCache achieved about 64% average rubric pass rate vs ~53% for Mem0, with stronger results on several categories including summarization,
contradiction resolution, and multi-session reasoning.

I also wrote a short overview of the project if you are interested:
https://medium.com/@faisaliitian/i-built-an-ai-memory-system-because-just-retrieve-more-wasnt-working-0b1dc9a60c01?postPublishedType=initial

Do these design choices make sense ?

Seen a lot of "why would you use Pinecone over pgvector" threads here, and I agree with the consensus — but I want to add a data point one step further than the usual pgvector comparison.

We consolidated vector search into our main DB (SynapCores, an AI-native DB — not pgvector, but same underlying philosophy: vectors as a column type, not a separate service). What's interesting is it doesn't stop at vector search — same connection also gives you a graph engine (Cypher) and agentic SQL functions. So the "why maintain 3 services when 1 does it" argument extends past just vectors vs. Pinecone.

What it actually looks like:

CREATE TABLE docs (
  id INT PRIMARY KEY, title TEXT, body TEXT,
  embedding VECTOR(384)
);

INSERT INTO docs (id, title, body, embedding)
VALUES (1, 'title', 'content', EMBED('content'));

SELECT title, COSINE_SIMILARITY(embedding, EMBED('user question')) AS sim
FROM docs ORDER BY sim DESC LIMIT 5;

No separate vector DB to provision, no sync job keeping it consistent with the source of truth, joins against your regular relational tables in the same query. Built a RAG knowledge base this way this week — 17 docs, 92 chunks, zero new infrastructure, just two tables.

The tradeoff nobody in these pgvector-vs-Pinecone threads mentions enough: a purpose-built vector DB (Pinecone, Qdrant, Milvus) still wins on raw ANN performance at serious scale — HNSW tuning, sharding, purpose-built indexes. If you're doing >10M vectors with tight latency SLAs, that specialization still buys you something. For most RAG/agent use cases people post here about (a few hundred K to low millions of vectors), the "just use your existing DB" camp seems right to me.

Curious if others who ditched a dedicated vector DB have hit a scale where they regretted it and went back?

We're running Qdrant in production and want a hands-on review from someone who knows the internals deeply, not just tutorials, but the actual Qdrant docs, config tuning, and real deployment experience.

Specifically looking for help with:

• Diagnosing our current collection/indexing configuration

• Reviewing our vector search query patterns and relevance tuning

• Identifying bottlenecks in our retrieval pipeline

• Advising on payload filtering, HNSW config, and quantization tradeoffs

If you've contributed to Qdrant, answered Qdrant questions on Discord, or built something meaningful with it in production, I'd love to talk.

For a local setup, I think pgvector is the easy answer if you're already running Postgres. One extension, ACID, and you can filter with a WHERE clause instead of standing up a second service. For a local knowledge base, that seems like plenty. The catch is that it gets complicated once you're past a certain vector count or writes get heavy, index build time climbs, and latency goes with it. I've seen pgvector latency go from about 50ms to 800ms past the 10M mark, though that was on a big instance, not a local box.

What I can't tell is where the line sits for someone running this on their own hardware rather than a cloud node. Locally, you don't get to scale out of the problem, so the wall probably comes sooner.

For people running local RAG:

• What are you on, pgvector or something dedicated like Qdrant/Chroma/Milvus/VectorDB, and at what vector count did you pick?
• Did anyone start on pgvector and hit a wall on a local box?
• For a few hundred thousand to low millions of vectors, is a dedicated engine overkill locally?

Hey everyone,

I’m working on a personal project to build a completely local, lightweight system (codename: Orin) that can process messy unstructured information and segregate/clean it into highly structured, tabular formats (CSV files). Essentially, it's meant to be a better, fully offline version of Atlas.

Here is the exact data structure and the pipeline I am trying to build:

1. The Target Data Schema

The model needs to take raw info and divide it into clear subtopics:

• Columns: Topic | Subtopic1 | Subtopic2 | Subtopic3 | Info
• Example Output:
  • Topic: Flying machine
  • Subtopic1: Airplane
  • Subtopic2: Passenger plane
  • Example Scenario: If incoming news data says "Qatar Airways wins starring award again", the model should automatically categorize it under the correct subtopic hierarchies and store the relevant data in the final Info column.

2. Proposed Pipeline & Architecture

I am planning a Retrieval-Augmented Generation (RAG) approach using a combination of specialized, local agents:

• A Fact Searcher / Main Topic Searcher: To find missing points and gather core data from the dataset.
• A Local Summarizer / Keyword Generator: Acting as a text quantizer to condense the given prompt or raw context.
• A Joke Generator (Optional Component): To add humor or personality to the generated answer output.
• The Core Logic Flow: Prompt → Gathers data for it → Finds missing points → Fills the spots (to Phrase) → Final Answer.

3. The Big Bottleneck: Hardware Constraints & Failed Attempts

Since this system must run locally, finding the right LLM engine and model has been incredibly difficult. Here is what I’ve attempted so far:

• llama.cpp: Would technically work, but performance is a massive issue (it took over 2 hours just to compile 8%).
• TinyStories: Super fast at stitching sentences together, but it only tells stories; it cannot handle this specific data formatting task.
• TinyLlama (llama.co): Unable to get it to work properly / wouldn't run.
• Ollama: Cannot use it seamlessly because it isn't properly optimized or built for my hardware (ARM chips).

I would like to ask the community how to make the better and how to develop it to efficient RAG model For my Project.