u/Limp-Park7849

My favorite projects are the ones where the founders took a premise and pushed it until it breaks of pays off. In this case, the premise is old enough to be uncontroversial: the log is the source of truth, and everything else (tables, indexes, replicas, caches) is a materialized view you can rebuild by replaying it. Jay Kreps, who co-created Apache Kafka and later ran Confluent, wrote the general version in his 2013 essay The Log, but Postgres has quietly worked this way for far longer. A commit isn't your rows being rewritten in place, it's a single sequential append to the write-ahead log, and the data files exist only as a cache so reads don't have to replay history from byte zero every time. If you want the primary source rather than the slogan, it's the ARIES paper (Mohan et al., 1992), all 69 pages of it, and the length is earned.

The interesting exercise is to take that premise literally and follow it one forced step at a time, because each consequence pulls the next one behind it.

Step one: if the log is the truth and the data files are just a cache of it, why do they sit on the same disk? In a stock Postgres box they do, and almost every operational headache traces back to that single colocation:

* Durability is only as good as what your fsync actually did, and fsync has a long history of lying.
* A read replica is a full physical copy of the database replaying the log.
* High availability is yet another full physical copy.
* A heavy analytical scan competes with transactional traffic for the same buffer pool and CPU.

Those look like four unrelated problems, but they're the same problem wearing different clothes: the log and its cache share a machine.

Step two: stop sharing the machine. Pull the two apart into separate services, one owning the log and one owning the pages. This is the part that's actually been built and run in production rather than sketched on a whiteboard, first in Neon's storage engine and now underneath Databricks' Lakebase, which is built on that same engine:

* The write-ahead log moves to a service called the SafeKeeper. A commit becomes durable the moment it's replicated across a Paxos quorum, not when a single local disk claims it flushed, so durability stops being a property of one disk and becomes a network commit.
* The data files move to a service called the PageServer, which behaves as a write-through cache in front of cloud object storage. On a miss it replays the log from the SafeKeeper to reconstruct the page it needs.

Step three: once both the log and the pages live outside the database process, that process holds no durable state. This is where the consequences start compounding. Stateless compute can be killed, restarted, or forked without losing anything, because the truth was never on that box. Branching a production database stops being a physical copy and becomes a metadata operation you can do in seconds. HA stops meaning "keep a second full node running and babysit it." Storage stops having a ceiling you size against in advance, because underneath it's just object storage.

The latency objection is the first thing a skeptic reaches for, and it deserves a real answer:

* Writes: you haven't added a network hop. Any Postgres you'd trust in production already runs synchronous replication, which is the same round trip. All that changed is where the acknowledgment comes from.
* Reads: the buffer pool answers first, a local disk cache second, and the PageServer only on a genuine miss. Give a compute node the same memory and disk as the old monolith and your hit rate is unchanged, so in steady state you rarely touch the network.

All of that is still just a better single database, though, and the consequential step opens up only after storage has been externalized. The PageServer already has to materialize pages into object storage as part of its normal job, so the real question is what format it writes them in. Write Postgres's native row pages and you get a very nice cloud OLTP database and nothing more. Write open columnar instead (Delta or Iceberg, with Parquet underneath) and the same materialization step that was already happening now produces something an analytical engine can read directly, at no extra cost to the transactional path. Unifying the two worlds at the storage layer rather than inside one engine is what's being called LTAP.

Freshness is where "just run analytics on the operational data" usually falls apart, so it's worth walking through how this one handles it. When an analytical query begins:

1. It asks for the current log sequence number, a single cheap lookup that says exactly how far the log has advanced.
2. It reads everything already materialized up to that LSN straight from object storage, which is the overwhelming majority of the data.
3. It merges in the small recent tail that hasn't materialized yet by fetching it from the PageServer.

The result is a consistent view as of that LSN, assembled almost entirely from cheap object storage, with Postgres having served essentially none of the analytical bytes. It handed over one number and stayed out of the way, which means no CDC pipeline to maintain, no second copy drifting from the first, and no reporting query stealing the buffer pool your transactions depend on.

This is the deliberate contrast with HTAP, which spent years trying to make a single engine excel at both workloads and generally arrived at a weaker optimizer, a thinner ecosystem, and no real isolation between the two. Keeping Postgres for transactions and a lakehouse engine for analytics, while unifying the storage beneath them, sidesteps all three, and it's a more believable bet precisely because it doesn't ask one engine to be good at everything.

If you've ever carried a pager for a Postgres fleet, the throughput numbers aren't what worry you. Here's where I'd push before believing any of it:

* Cold start. Scale-to-zero reads beautifully in a slide and stings on the first query after idle. The number that matters is the p99 on a cold wake, not the average on a warm pool, and whether that's acceptable depends entirely on whether it fronts a dev branch or a customer-facing OLTP path.
* Quorum tail. "No more than synchronous replication" holds on the happy path. The honest question is what p99 write latency does when one SafeKeeper is slow or the quorum is mid-reconfiguration, because the tail pages you, not the median.
* Cold PageServer reads. A miss reconstructs a page by replaying the log, which is fine until it happens right after a failover with cold caches, exactly when you're already having a bad day.
* Vacuum doesn't go away. Externalizing storage doesn't repeal MVCC garbage collection. Dead tuples and GC pressure are still real, now spread across a distributed store instead of one disk.
* Columnar with Postgres semantics. Preserving full MVCC and point-in-time behavior inside a columnar format is where the genuine engineering risk lives. The design keeps row versions and heap addresses under the covers, invisible to the Delta and Iceberg readers, and that one sentence in a blog is where the bodies are buried.

None of those are disqualifying, but they're the questions that decide whether an architecture survives contact with a real workload, and the ones a vendor post won't lead with. It's also fair to say plainly that LTAP is still rolling out, so some of this is roadmap you can't stress-test today.

The deeper caveat is architectural rather than operational. None of these properties fall out for free unless the storage layer was designed around the log from the beginning, which means you can't bolt it onto an existing monolith, and "compute is stateless" is quietly carrying enormous weight, because the SafeKeeper and PageServer are now the hard distributed-systems problem. The difficulty moved to a better place.

My favorite projects are the ones where the founders took a premise and pushed it until it breaks of pays off. In this case, the premise is old enough to be uncontroversial: the log is the source of truth, and everything else (tables, indexes, replicas, caches) is a materialized view you can rebuild by replaying it. Jay Kreps, who co-created Apache Kafka and later ran Confluent, wrote the general version in his 2013 essay The Log, but Postgres has quietly worked this way for far longer. A commit isn't your rows being rewritten in place, it's a single sequential append to the write-ahead log, and the data files exist only as a cache so reads don't have to replay history from byte zero every time. If you want the primary source rather than the slogan, it's the ARIES paper (Mohan et al., 1992), all 69 pages of it, and the length is earned.

The interesting exercise is to take that premise literally and follow it one forced step at a time, because each consequence pulls the next one behind it.

Step one: if the log is the truth and the data files are just a cache of it, why do they sit on the same disk? In a stock Postgres box they do, and almost every operational headache traces back to that single colocation:

• Durability is only as good as what your fsync actually did, and fsync has a long history of lying.
• A read replica is a full physical copy of the database replaying the log.
• High availability is yet another full physical copy.
• A heavy analytical scan competes with transactional traffic for the same buffer pool and CPU.

Those look like four unrelated problems, but they're the same problem wearing different clothes: the log and its cache share a machine.

Step two: stop sharing the machine. Pull the two apart into separate services, one owning the log and one owning the pages. This is the part that's actually been built and run in production rather than sketched on a whiteboard, first in Neon's storage engine and now underneath Databricks' Lakebase, which is built on that same engine:

• The write-ahead log moves to a service called the SafeKeeper. A commit becomes durable the moment it's replicated across a Paxos quorum, not when a single local disk claims it flushed, so durability stops being a property of one disk and becomes a network commit.
• The data files move to a service called the PageServer, which behaves as a write-through cache in front of cloud object storage. On a miss it replays the log from the SafeKeeper to reconstruct the page it needs.

Step three: once both the log and the pages live outside the database process, that process holds no durable state. This is where the consequences start compounding. Stateless compute can be killed, restarted, or forked without losing anything, because the truth was never on that box. Branching a production database stops being a physical copy and becomes a metadata operation you can do in seconds. HA stops meaning "keep a second full node running and babysit it." Storage stops having a ceiling you size against in advance, because underneath it's just object storage.

The latency objection is the first thing a skeptic reaches for, and it deserves a real answer:

• Writes: you haven't added a network hop. Any Postgres you'd trust in production already runs synchronous replication, which is the same round trip. All that changed is where the acknowledgment comes from.
• Reads: the buffer pool answers first, a local disk cache second, and the PageServer only on a genuine miss. Give a compute node the same memory and disk as the old monolith and your hit rate is unchanged, so in steady state you rarely touch the network.

All of that is still just a better single database, though, and the consequential step opens up only after storage has been externalized. The PageServer already has to materialize pages into object storage as part of its normal job, so the real question is what format it writes them in. Write Postgres's native row pages and you get a very nice cloud OLTP database and nothing more. Write open columnar instead (Delta or Iceberg, with Parquet underneath) and the same materialization step that was already happening now produces something an analytical engine can read directly, at no extra cost to the transactional path. Unifying the two worlds at the storage layer rather than inside one engine is what's being called LTAP.

Freshness is where "just run analytics on the operational data" usually falls apart, so it's worth walking through how this one handles it. When an analytical query begins:

1. It asks Postgres for the current log sequence number, a single cheap lookup that says exactly how far the log has advanced.
2. It reads everything already materialized up to that LSN straight from object storage, which is the overwhelming majority of the data.
3. It merges in the small recent tail that hasn't materialized yet by fetching it from the PageServer.

The result is a consistent view as of that LSN, assembled almost entirely from cheap object storage, with Postgres having served essentially none of the analytical bytes. It handed over one number and stayed out of the way, which means no CDC pipeline to maintain, no second copy drifting from the first, and no reporting query stealing the buffer pool your transactions depend on.

This is the deliberate contrast with HTAP, which spent years trying to make a single engine excel at both workloads and generally arrived at a weaker optimizer, a thinner ecosystem, and no real isolation between the two. Keeping Postgres for transactions and a lakehouse engine for analytics, while unifying the storage beneath them, sidesteps all three, and it's a more believable bet precisely because it doesn't ask one engine to be good at everything.

If you've ever carried a pager for a Postgres fleet, the throughput numbers aren't what worry you. Here's where I'd push before believing any of it:

• Cold start. Scale-to-zero reads beautifully in a slide and stings on the first query after idle. The number that matters is the p99 on a cold wake, not the average on a warm pool, and whether that's acceptable depends entirely on whether it fronts a dev branch or a customer-facing OLTP path.
• Quorum tail. "No more than synchronous replication" holds on the happy path. The honest question is what p99 write latency does when one SafeKeeper is slow or the quorum is mid-reconfiguration, because the tail pages you, not the median.
• Cold PageServer reads. A miss reconstructs a page by replaying the log, which is fine until it happens right after a failover with cold caches, exactly when you're already having a bad day.
• Vacuum doesn't go away. Externalizing storage doesn't repeal MVCC garbage collection. Dead tuples and GC pressure are still real, now spread across a distributed store instead of one disk.
• Columnar with Postgres semantics. Preserving full MVCC and point-in-time behavior inside a columnar format is where the genuine engineering risk lives. The design keeps row versions and heap addresses under the covers, invisible to the Delta and Iceberg readers, and that one sentence in a blog is where the bodies are buried.

None of those are disqualifying, but they're the questions that decide whether an architecture survives contact with a real workload, and the ones a vendor post won't lead with. It's also fair to say plainly that LTAP is still rolling out, so some of this is roadmap you can't stress-test today.

The deeper caveat is architectural rather than operational. None of these properties fall out for free unless the storage layer was designed around the log from the beginning, which means you can't bolt it onto an existing monolith, and "compute is stateless" is quietly carrying enormous weight, because the SafeKeeper and PageServer are now the hard distributed-systems problem. The difficulty moved to a better place.

WAL makes a commit "durable." On a single machine it's fast because the write-ahead log goes straight to local disk. In the cloud that disk is ephemeral, it's gone if the instance dies, so the database has to ship every commit's log to remote storage before it can tell you "done." That round trip is a big reason cloud commit latency is what it is.

This VLDB'26 paper (BtrLog) lays out the problem and one fix pretty clearly:

• EBS-style remote disk: easy, but adds latency and cost to every commit.
• Object storage (S3): dirt cheap and durable, but way too slow per-write for transactional stuff.
• BtrLog's middle path: write each log record to a quorum of fast SSD nodes in one network hop (so one slow node can't stall your commit), then lazily roll the logs into big chunks on S3 in the background for cheap storage. This is exactly the Neon architecture but engine agnostic.

The numbers, as commit latency:

• ~70 µs per append vs 260–500 µs for EBS. So 4–5x faster, and about 3x the transaction throughput.

This compute/storage split iis how modern serverless Postgres already works. Neon does this exact pattern (its "safekeepers" are the quorum WAL layer), which is why you can spin up a Postgres that scales to zero and still commit fast. The paper basically asks what if that durable-log layer were a reusable building block instead of buried inside one engine.

When Databricks announced LTAP off the back of Lakebase, it got me thinking about HTAP again. Around 2015 it was sold as one engine for transactions AND analytics. No ETL, no stale data. Sounded amazing. Basically didn’t happen. HTAP revenue staled when the companies that split storage from compute and only did analytics or transactional are the giants now. The need itself never died however, so could LTAP be the answer?

I wanted a real eval set, a pile of question + known-answer pairs I could re-run after every change using the native benchmark feature in Genie.

The ideal and best way to do so is to get actual users run questions in the space and use to build the benchmark but what is this is not an option?

Writing those by hand is slow and thankless, so I got curious: could I just generate them synthetically? Let a model write the questions and the expected answers.

I built that. Then I stared at it for a while and saw the obvious problem I'd walked straight past : the model writing my "correct" answer key is the same kind of model behind Genie. When they misread a question the same way, a wrong answer comes back looking like a match.

I'd basically built a mirror and was grading the reflection.

The stats were worse. My confidence interval was sitting on 42 "data points" that were really 7 questions asked 6 times each. Tight number but no meaning.

Then I started thinking what if instead of trusting the generated SQL, I trust what it returns when you run it.
Take the question, have a few solvers from different model families re-derive the SQL blind, run them all on the real tables, and keep the answer key only if the rows agree.

However, if every solver misreads a question the same way (common when a space redefines a term like "active customer"), they agree and you certify something wrong.
The payoff wasn't a score anyway. It was the handful of answers where Genie disagreed with a key two model families had verified. Those were real gaps in a space I already trusted.

How are you evaluating your Genie spaces today? Hand-written pairs? Benchmarks? Anything synthetic?

My reasoning is that a slide deck forces your story into fixed rectangles. An HTML deck reveals one idea per screen as you scroll, builds a chart in front of the audience, and lets you toggle scenarios live.

In a customer meeting for example I switched a cost projection between three growth assumptions while they watched. Try that with a static bar chart pasted from Excel.

Another key advantages is Iteration. The deck is code. I can diff it, version it, ask Claude for surgical edits, and review the change like a code review.

While technically a .pptx is a binary blob. Every edit means a full regeneration and a manual re-inspection.

And finally volume effect, Claude has seen orders of magnitude more HTML and CSS than python-pptx layout code. Its model of how a flexbox renders beats its model of where EMUs put a text box. So really playing to the strength of the tool here

Been running Fable as the orchestrator and delegating the actual work to Sonnet subagents. I expected a quality drop. So far I can’t see one, and the token bill is way down. Keen to hear what you’ve doing on this front.