Login

u/Holiday_Hat_8605

r/u_Holiday_Hat_8605 r/Observability r/databasedevelopment
▲ 13 r/databasedevelopment

Why are we rebuilding the same database execution engine over and over?

I recently dug into Meta’s Velox, an open-source C++ execution engine designed to act like a reusable “engine block” for data systems.

Instead of Presto, Spark, etc. independently implementing things like vectorized execution, joins, memory management, spilling, and file readers, Velox provides these building blocks as a shared execution layer.

The performance numbers are interesting too:

  • 8.4× faster on TPC-H Q1
  • 9× faster on Q6
  • 6–7× average speedup on Meta production traffic
  • Up to 3× fewer servers

Checkout this interesting deep dive that how Velox actually works and why this architecture could matter for the future of data engines.

https://prestodb.io/blog/2026/08/13/inside-velox-open-source-universal-engine-block-for-big-data-and-ai/

Do you think shared execution layers like Velox will become the norm, or will specialized engines always win?

reddit.com
u/Holiday_Hat_8605 — 4 days ago
▲ 3 r/u_Holiday_Hat_8605+1 crossposts

Presto observability is tricky (JVM GC pauses, resource group queues, disk spill). Here’s a complete open-source monitoring setup + free live workshop.

Hey everyone,

If you’ve ever managed a Presto/Trino cluster in production, you know that generic APM tools often fall short. When a query queues up, a worker node silently drops, or JVM memory spikes, pinpointing the root cause across a distributed cluster can be a nightmare.

Register for Hands-On Workshop: https://prestodb.io/hands-on-workshop-monitoring-presto-at-petabyte-scale/

  • Title: Monitoring Presto at Petabyte Scale
  • Date: Thursday, August 20, 2026
  • Time: 8:30 AM PDT | 11:30 AM EDT | 9:00 PM IST

Common headaches we see in production:

  • Silent Bottlenecks: Queries getting stuck in leaf-level Resource Group queues without visibility.
  • GC Spikes: JVM thread freezes causing missed heartbeats and false worker node failures.
  • Disk Spilling: Unnoticed spilling to disk during heavy joins/aggregations degrading cluster throughput.
  • Alert Fatigue: Getting spammed by Slack alerts for single transient query failures.

To solve this, we put together a fully self-contained, 6-container Docker observability sandbox that integrates Presto, Prometheus, JMX Exporter, Alertmanager, Grafana, and Slack.

What’s inside the open-source setup:

  • JMX Exporter Regex Rules: Whitelists and translates complex Presto MBeans into clean, labeled Prometheus metrics.
  • 24-Panel Grafana Dashboard: Pre-configured panels covering active worker health, JVM heap ratios, P50-P99 latency percentiles, disk spill rates, and multi-tenant resource group queues.
  • PromQL Baseline Guards: Ratio-based alerts that require a minimum traffic threshold before triggering, eliminating false positives.
  • Multi-Tenant Queue Tracking: Label exclusion rules that prevent double-counting queued queries across user, team, and root resource levels.

👉 Register here: https://prestodb.io/hands-on-workshop-monitoring-presto-at-petabyte-scale/

u/Holiday_Hat_8605 — 8 days ago