u/Sorry-Substance8283

Hello,

I've created a Python library (as middleware) for Neo4j, Langchain, and other frameworks/libraries to create graphs. Basically, the problem I personally had was that my graph created nodes whenever I didn't have exactly the same word (for example, Apple and AAPL were separate), which was annoying. The first approach was to run LLM on all the nodes and find the duplicates, but this is very expensive if the data grows exponentially.

So I built this tool that combines three layers of analysis (RapidFuze, vector analysis, and, if nothing else works, LLM). You can check it out on GitHub if you're interested. I'm looking for feedback to improve it:

https://github.com/jules-gd-dev/autograft-lib

Have a good day, team!

Been working on a Neo4j knowledge graph using LangChain and kept running into the classic issue: the LLM extracts "Apple", "Apple Inc.", and "Apple Incorporated" as three completely separate nodes.

The standard fix is using an LLM-as-a-judge to deduplicate every single entity before insertion, but the token cost scales horribly. I was burning cash just to keep the graph clean.

I ended up writing a small Python middleware that intercepts the entities before they actually hit Neo4j. It tries to short-circuit the resolution locally using RapidFuzz (for exact/alias matches) and numpy for vector similarity. It only falls back to an LLM via litellm if the match is actually ambiguous.

I ran some tests on about 200 enterprise docs. The middleware caught all 742 duplicate entities locally in layers 1 and 2, dropping the resolution API cost to exactly zero.
You just wrap your Neo4jGraph or LlamaIndex store with it and it handles the deduplication silently.

I open-sourced it here if anyone else is struggling with graph pollution: https://github.com/jules-gd-dev/autograft-lib

Would appreciate any thoughts on the architecture, especially if someone knows a better way to do the deterministic matching without pulling too much of the graph schema into memory at scale.

N.B.: The project is on a really early stage, i've been working on it for a week and published it on github just yesterday. You can find some "benchmarks" on GitHub too.

I created a tool that allows you to remove duplicate nodes on a graphrag while consuming 0 tokens (almost zero).

I open-sourced a 3-layer ER middleware to stop LangChain from polluting Neo4j with duplicate nodes.