Bitemporal time-travel + truth-maintenance-style provenance retraction on Postgres/SQLite (open-source TS graph library)
▲ 8 r/sqlite+1 crossposts

Bitemporal time-travel + truth-maintenance-style provenance retraction on Postgres/SQLite (open-source TS graph library)

I just shipped bitemporal provenance for TypeGraph, my open-source graphs-on-SQL library. Three pieces, usable independently but most powerful together:

  • Valid time: when a fact was true in the world (an invoice's effective date, a role grant's window).
  • Recorded/system time: when the system captured that fact (what you knew, as of a commit instant; the SQL:2011 FOR SYSTEM_TIME / Datomic system-time axis).
  • Provenance: why the system still believes a derived fact, and what happens downstream when a source it depended on turns out to be wrong.

Derived facts are the annoying case that surfaces the issue(s) these primitives solve. For example, a Vulnerability node exists because a scanner and a vendor advisory both pointed at it. The graph concluded it; nobody asserted it directly.

ScannerSource ──┐
                ├──▶ Vulnerability (CVE-2026-1234, libvector)
VendorSource  ──┘

So when the scanner turns out to be garbage, you can't treat retracting it as a delete. The vendor might still back that vulnerability. The scanner might have been the only thing propping up a bunch of other facts. You want the graph to sort out which.

What you want: retract a source and it recomputes which derived facts still have grounded support. Retract the vendor too and the vulnerability finally goes non-current, and a "block the deploy" decision sitting on top of it goes with it.

The behavior, then the theory

A fact stays believed while it has at least one justification whose premises are all still supported. Premises bottom out at sources. Retract a source and every justification that leaned on it stops counting; a fact loses currency only once it runs out of surviving justifications.

const provenance = createRetractionCapability(store, {
  source: { kinds: ["ScannerSource", "VendorSource"] },
  justification: { kind: "Justification" },
  fact: { kinds: ["Vulnerability", "DeployDecision"] },
  premiseOf: { kind: "premiseOf" },
  derives: { kind: "derives" },
});

const report = await provenance.retract({ kind: "VendorSource", id: vendorId });
// report.died:        facts that lost all grounded support
// report.survivedVia: facts that still have an alternate justification

This is modeled on truth-maintenance systems. The storage follows the JTMS shape (Doyle 1979, "A Truth Maintenance System"): AND-justifications over premises, sources at the bottom, a fact in the well-founded support set only if some justification has all its premises supported. I use the monotonic, inlist-only fragment, so this is the easy part of Doyle's system; the hard part, non-monotonic belief revision, isn't here. The question retract actually answers, "which facts survive because an alternate justification still holds," is the ATMS question (de Kleer 1986): which combinations of sources hold each fact up. So it's JTMS-shaped storage with an ATMS-flavored query.

Retraction is a normal write, so you get replay for free

Retraction doesn't hard-delete. It recomputes support and flips unsupported facts to non-current, leaving the justification edges in place so you can still see why something used to be believed. Because that write lands on TypeGraph's recorded-time (system-time) substrate, you can replay the belief transition:

const before = await store.recordedNow();
await provenance.retract(badSource);
const after = await store.recordedNow();

await store.asOfRecorded(before).nodes.Vulnerability.getById(id); // believed
await store.asOfRecorded(after).nodes.Vulnerability.getById(id);  // not current

TypeGraph tracks both temporal axes as explicit read lenses, valid time ("when true in the world") and recorded time ("when the database learned it"), and because they're lenses they compose:

store.asOf(validTime).asOfRecorded(recordedTime)

Architecture

No engine-native temporal tables. Postgres needs an extension for system-versioning and SQLite has nothing, so TypeGraph stores history explicitly and reconstructs point-in-time views in the query compiler. That's why one implementation runs on both backends.

Limits

  • Only TypeGraph-managed writes are captured. Raw SQL bypasses it; this isn't a database-level CDC/audit layer.
  • No backfill. Enable history on a fresh graph.
  • Point-in-time reads reconstruct from history relations, so they're slower than current-state reads. It's an audit tool, keep it off hot paths.
  • Per-write overhead runs ~2.5–6x unless you batch writes in one transaction, where it drops to ~1–1.5x.

A naming note

My asOf is valid time, the reverse of SQL:2011 FOR SYSTEM_TIME AS OF and Datomic (d/as-of db t), where a bare as-of is system time. Valid-time reads are the common case here so they took the short name; system time is asOfRecorded.

I'd love to compare with other systems that handle provenance retraction, or truth maintenance generally, modeled directly on ordinary SQL tables instead of a dedicated reasoning engine. There's plenty of JTMS/ATMS literature but not much on mapping it onto relational storage. Pointers welcome.

GitHub: https://github.com/nicia-ai/typegraph Docs: https://typegraph.dev/provenance

Examples: https://typegraph.dev/examples/provenance-retraction/ https://typegraph.dev/examples/bitemporal-time-travel/

u/pdlug — 3 days ago
▲ 25 r/Rag+1 crossposts

BM25 + vectors + graph traversal compiled to one SQL query, with ontology reasoning at query time (TypeGraph, open source)

I've been building retrieval systems and knowledge graphs for decades so when I started building RAG systems years ago it was pretty natural to combine it all. The tooling quickly becomes an issue: most systems have a SQL database of record, then add a vector DB, then a fulltext search index if that's not sufficient. It all comes with a pile of data pipelines to keep it all in sync.

SQL DBs do a pretty solid job of being a graph DB using things like recursive CTEs and (most) already have vectors and fulltext search. They're not as full featured as the standalone versions of each of those tools, but for most RAG systems that's not the limiting factor.

After re-implementing the same tooling over and over I built TypeGraph (open source, I'm the author): a TypeScript knowledge-graph library that runs on the Postgres or SQLite you already have. The RAG-relevant bit is that three retrieval primitives live in the same store and fuse in one SQL round trip:

  • BM25 fulltexttsvector+GIN on Postgres, FTS5 on SQLite
  • Vector similarity — pgvector or sqlite-vec
  • Graph traversal — typed edges, recursive, with depth + cycle handling

​

const hits = await store.search.hybrid("Chunk", {
  limit: 3,
  vector: { fieldPath: "embedding", queryEmbedding, k: 20 },
  fulltext: { query: "AlphaGo Go champion", k: 20, includeSnippets: true },
  fusion: { method: "rrf", k: 60, weights: { vector: 1, fulltext: 1.25 } },
});

Reciprocal Rank Fusion happens at the SQL layer, and every hit carries sub-scores from both halves so you can see what each side contributed.

Schemas are defined in TypeScript using Zod so there's full compile time type safety, editor support, etc. Runtime graph extensions are supported so you can extend the schema dynamically (or have an agent do it) with full validation and change control.

The other big feature the schema-driven approach unlocks is ontology support. The graph already makes the relationships explicit, exact, and queryable (that's just typed nodes and edges). The ontology is the layer on top that lets you reason about those relationships: you declare what a relationship means once, and the engine infers the rest at query time. Expanding, substituting, and excluding matches you'd otherwise hand-code. Some of the relations supported:

  • Taxonomy expansion (broaderThan / subClassOf): declare ContrastiveSelfSupervisedDeepLearning. A query for "DeepLearning" now matches docs tagged only with the narrow descendants — guaranteed, not "probably, if the embedding's good." For example: nothing is tagged "DeepLearning," yet the query finds the GPT-4 / AlphaGo / ChatGPT chunks because reachable() walks the hierarchy in the query.
  • Synonyms / cross-vocabulary (equivalentTo): map "K8s" ≡ "Kubernetes," or reconcile two teams' schemas that named the same concept differently. The query writer never has to know which surface form a document used.
  • Polymorphic retrieval (subClassOf): ask for Media, get Podcast, Article, Video. Add a Newsletter subtype next quarter and every existing retrieval query picks it up — no re-indexing, no re-embedding.
  • Precision, not just recall (disjointWith): declare Apple/organization disjoint from Apple/fruit. Now entity linking can prune impossible candidates instead of letting a fuzzy match drag the wrong sense into context.
  • Inverses (inverseOf): declare manages is the inverse of reportsTo; create one edge, traverse it both ways. Half the relationship-modeling boilerplate disappears.

A few cool things that putting all this together enables for RAG and agents:

  1. Expansion, hybrid, and traversal compose in one query. Expansion isn't a preprocessing fan-out of N extra searches. It compiles into the recursive CTE alongside your BM25 + vector retrieval. One round trip. Ontology expansion and the entity traversal are the same store.query() call:
  2. Ontology as data, not code. It lives in the graph, so it's introspectable. An agent can query "what concepts roll up into X?" or "are these two senses disjoint?" and get a machine-readable answer, instead of that logic being buried in code the model can't see.
  3. Deterministic and explainable. When a doc shows up because it's tagged with a descendant concept, you can say exactly why. Ex: "matched SelfSupervised, which is ⊆ DeepLearning." Try debugging "why did this rank?" when the only answer is a cosine score.

This is the symbolic half that pure-vector RAG gives up and that GraphRAG-style pipelines gesture at. But here it's a first-class declarative thing you query, not an LLM-summarized artifact you regenerate.

Because relationships are first-class, you get the actual path, not just a similarity score. For example, a shortestPath over founder/employer/acquisition edges can return:

OpenAI ─[cofounded]─ Ilya Sutskever ─[studied under]─ Geoffrey Hinton ─[worked at]─ Google ─[acquired]─ DeepMind

Entity resolution (the actually-hard part of building a graph from LLM-extracted entities) is handled with scoped unique constraints + atomic getOrCreateByConstraint, so concurrent ingestion doesn't split "Apple Inc." from "Apple" or merge the company with the fruit. There's a worked example of exactly this — chunk→entity→document traversal + dedup here: https://typegraph.dev/examples/knowledge-graph-rag/

The powerful part I've found with this library is it can go from an embedded SQLite DB to a full scale PostgreSQL cluster with the same tooling. You can ship a DB-per-user-per-project or just maximize your one big DB by layering on better retrieval, all with the same library.

How this relates to "GraphRAG": Microsoft's GraphRAG is an end-to-end pipeline (LLM extraction → Leiden communities → LLM-summarized communities) aimed at global sensemaking across a corpus. TypeGraph is the substrate underneath (typed graph store, hybrid retrieval, traversal, ontology reasoning on your existing DB) and is the better fit for local, structured retrieval with explainability, or as the storage+query layer for your own GraphRAG-style pipeline.

If you want to actually run all of this — hybrid retrieval, ontology-expanded topic matching, shortestPath/reachable/degree/neighbors over a citation graph — there's a single-file research-copilot example: https://typegraph.dev/examples/research-copilot/

Honest limits: it compiles to SQL, so you inherit your DB's performance which is great for thousands–millions of nodes and traversals up to ~10s of hops, wrong tool for billions of edges or PageRank-style analytics. Not a vector DB replacement at massive scale; it's for teams who'd rather not run three systems to do RAG well.

Curious what this sub thinks, especially anyone doing graph RAG who's found the entity-resolution / ontology layer is where it lives or dies.

GitHub: https://github.com/nicia-ai/typegraph

Docs: https://typegraph.dev

u/pdlug — 2 days ago
▲ 12 r/sqlite

TypeGraph: graph queries that compile to a single recursive CTE on Postgres/SQLite (no graph DB needed)

I'm a huge fan of graphs, they tend to simplify a lot of problems (permissions that inherit, content that relates, entities for RAG, etc.) and preserve optionality when the data modeling is uncertain. Most of the time you need a graph, you don't need a graph database. Your app already has a perfectly good SQL DB and you can get pretty far with recursive CTEs. After building this a few dozen times I decided to package it all up with a nice DX and open source it.

TypeGraph (open source, I'm the author) is a graph modeling + query layer that compiles to SQL. It is explicitly not a graph database — you keep your Postgres/SQLite, your transactions, your backups, and you inherit your DB's performance. Comes with some tradeoffs but also a lot of power, like being able to connect from relational to graph.

  • Each algorithm (shortestPath, reachable, canReach, neighbors, degree) compiles to one recursive CTE with cycle detection and depth limits — identical semantics on SQLite and Postgres
  • Postgres CTEs emit NOT MATERIALIZED hints; LIMIT is pushed past GROUP BY in safe aggregation cases
  • Server-side prepared statements (named) cache plans — ~6× faster on multi-hop traversals in my benchmarks
  • refreshStatistics() wraps ANALYZE (per-table on PG) for post-bulk-load plan stability
  • withTransaction(externalTx) shares one transaction across TypeGraph and your existing Drizzle/relational writes — atomic across both models, no data syncing
  • Multi-driver Postgres: node-postgres, postgres-js, Neon WS + HTTP; SQLite via better-sqlite3, libsql, and Cloudflare Durable Objects

Embraces TypeScript and related libraries: a single Zod schema per node/edge is the source of truth for runtime validation, storage, and type inference. Result types come from your select clause. Traversal autocomplete only shows valid target kinds.

It ships a basic ontology with the ontology as data so you can do things like admin.implies(editor) and every query expands automatically, no getEffectivePermissions() duplicated across services.

It does all the fancy AI stuff too (semantic search, fulltext, hybrid retrieval w/ RRF) but it's really just graphs done right on top of a DB you're probably already using

Honest feedback welcome, especially on the type ergonomics.

GitHub: https://github.com/nicia-ai/typegraph · Docs: https://typegraph.dev

reddit.com
u/pdlug — 2 months ago

TypeGraph: graph queries that compile to a single recursive CTE on Postgres/SQLite (no graph DB needed)

I'm a huge fan of graphs, they tend to simplify a lot of problems (permissions that inherit, content that relates, entities for RAG, etc.) and preserve optionality when the data modeling is uncertain. Most of the time you need a graph, you don't need a graph database. Your app already has a perfectly good SQL DB and you can get pretty far with recursive CTEs. After building this a few dozen times I decided to package it all up with a nice DX and open source it.

TypeGraph (open source, I'm the author) is a graph modeling + query layer that compiles to SQL. It is explicitly not a graph database — you keep your Postgres/SQLite, your transactions, your backups, and you inherit your DB's performance. Comes with some tradeoffs but also a lot of power, like being able to connect from relational to graph.

  • Each algorithm (shortestPath, reachable, canReach, neighbors, degree) compiles to one recursive CTE with cycle detection and depth limits — identical semantics on SQLite and Postgres
  • Postgres CTEs emit NOT MATERIALIZED hints; LIMIT is pushed past GROUP BY in safe aggregation cases
  • Server-side prepared statements (named) cache plans — ~6× faster on multi-hop traversals in my benchmarks
  • refreshStatistics() wraps ANALYZE (per-table on PG) for post-bulk-load plan stability
  • withTransaction(externalTx) shares one transaction across TypeGraph and your existing Drizzle/relational writes — atomic across both models, no data syncing
  • Multi-driver Postgres: node-postgres, postgres-js, Neon WS + HTTP; SQLite via better-sqlite3, libsql, and Cloudflare Durable Objects

Embraces TypeScript and related libraries: a single Zod schema per node/edge is the source of truth for runtime validation, storage, and type inference. Result types come from your select clause. Traversal autocomplete only shows valid target kinds.

It ships a basic ontology with the ontology as data so you can do things like admin.implies(editor) and every query expands automatically, no getEffectivePermissions() duplicated across services.

It does all the fancy AI stuff too (semantic search, fulltext, hybrid retrieval w/ RRF) but it's really just graphs done right on top of a DB you're probably already using

Honest feedback welcome, especially on the type ergonomics.

GitHub: https://github.com/nicia-ai/typegraph · Docs: https://typegraph.dev

reddit.com
u/pdlug — 2 months ago