r/erlang
ex_data_sketch v0.8.0 — Deterministic Foundations
ex_data_sketch v0.8.0 is out. This release invests entirely in the substrate that all 15 existing sketches share, preparing the grounds for release v0.9.0 where we add streaming integrations for Broadway / GenStage support, ETS / DETS / Zarr.
What's new:
Deterministic hashing. Every sketch now goes through a validated, byte-stable hash layer. HLL, ULL, Theta, and CMS accept
hash_strategy: :murmur3for Apache DataSketches interop — this was silently ignored in v0.7.x. XXHash3 remains the default and fastest path (~30 M items/sec at p=14 on the Rust NIF).Binary stability & corruption detection. Serialized sketches now carry a CRC32C trailer and an embedded hash metadata block (EXSK v2). Bit-flip corruption that previously would silently produce wrong estimates is now caught and returns a structured
DeserializationError. v0.8.0 reads v1 frames; v0.7.x cannot read v2 — stage your rollout accordingly.Murmur3 hot path. 8 new Rust NIFs extend in-Rust hashing to Murmur3. The Murmur3 path is within 8% of XXH3 throughput. No more falling off the fast path when you select
:murmur3.Precompiled NIFs for Windows. x86_64 and ARM64 MSVC targets join the matrix. 16 artifacts total (8 targets x 2 NIF versions). No Rust toolchain needed on any supported platform.
Property-locked guarantees. 14 StreamData properties lock HLL/ULL monotonicity and error bounds, KLL/REQ rank consistency, CMS overestimation-only, and Bloom/XorFilter/Cuckoo no-false-negative. A 200-mutation fuzz suite verifies that binary v2 corruption never silently propagates.
Breaking changes (2):
- EXSK v2 is one-way. v0.7.x readers can't decode v2 frames. Deploy readers first, then producers.
hash_strategy: :murmur3is no longer silently overridden to:xxhash3. Sketches that specified Murmur3 will now actually use it — estimates are correct but differ from v0.7.x.
One-liner upgrade:
{:ex_data_sketch, "~> 0.8.0"}
Most users need no code changes. Full migration guide ships in HexDocs.
Stats: 1,317 tests, 171 properties, 92.7% coverage, 0 credo issues.
u/Shoddy_One4465 — 9 days ago
Edge Core: a self-hostable control plane for distributed Linux fleets, built in Elixir
Hey guys! We finally opened up the codebase for something we've been working on for over a year.
I joined a company that spent 3 years (and counting) trying to ship products on locked down edge hardware. Every product kept hitting the same walls: deployments and monitoring were a black box, machines on the same LAN couldn't reliably find each other, and every new app had to reimplement the same WS/MQTT logics just to stay in touch with the cloud.
So we built Edge Core to solve these pain points. In V1, we used Headscale/Tailscale for the VPN. It worked mostly for what we wanted (remote execution, SSH, metrics aggregation, etc.), but couldn't scale past ~100 nodes (mesh explosion with O(n2)) and gave us no isolation between different projects (each project must spin up its own core, though ACLs exist). In V2 (current version), we moved towards Netmaker for a proper mesh/network segmentation solution, added a forward proxy + dynamic proxy chaining for cloud-to-edge communication, and built the whole orchestration layer on top.
Some Elixir specific stuff that might interest you:
- Masterless clustering for the control plane: no (strong) leader election, no Raft consensus. Admins coordinate via `:syn` registry and Postgres. Each admin runs the same deterministic sharding algorithm and converges independently.
- Oban and Quantum for async background jobs
- API-first control plane with clustering HTTP/SOCKS5 proxy servers and first class fleet metrics discovery + scraping that are prometheus compatible
- MCP server that mirrors the full REST API, basically every API endpoint is also an MCP tool that AI agents can drive the whole fleet
- Webhook system and event broker integration for async system events with 7 adapters (NATS, Kafka, AMQP 0.9.1/RabbitMQ, Redis, MQTT, AWS SNS, and GCP Pub/Sub).
- Agent and shared libs are Apache 2.0. Admin is ELv2.
Links:
- Repo: https://github.com/wenet-ec/edge-core
- Docs: https://wenet-ec.github.io/edge-core/
- Learn about edge core's concepts: https://wenet-ec.github.io/edge-core/guide/
- Architecture: https://wenet-ec.github.io/edge-core/architecture/
u/Best_Recover3367 — 11 days ago
▲ 21 r/erlang
I wanted Ecto's ergonomics in Erlang without writing Elixir, so I wrote Kura. It sits on pgo. You define a schema, build queries, get changesets, run migrations.
-module(user).
-behaviour(kura_schema).
-include_lib("kura/include/kura.hrl").
-export([table/0, fields/0]).
table() -> ~"users".
fields() ->
[#kura_field{name = id, type = id, primary_key = true},
#kura_field{name = email, type = string, nullable = false},
#kura_field{name = name, type = string},
#kura_field{name = inserted_at, type = utc_datetime}].
Querying:
Q = kura_query:from(user),
Q1 = kura_query:where(Q, fun(U) -> U#user.age > 18 end),
my_repo:all(kura_query:order_by(Q1, [{desc, inserted_at}])).
It does the things you'd expect: schemas, changesets, composable queries with joins/CTEs/subqueries/window functions, migrations, associations with preloading, embedded JSONB, transaction pipelines, multitenancy via schema prefix, optimistic locking, audit trail, cursor streaming, pagination.
It's not a port. Records and functions, no macro DSL. The README has a coming-from-Ecto cheatsheet.
u/taure1 — 13 days ago