u/pollop-12345

Has anyone tried the experimental ZXC codec?

Has anyone tried the experimental ZXC codec?

I'm the author of ZXC, a lossless compression library built for fast decompression. Since CH 26.7 it ships as an experimental column codec, and I'd like to know if anyone here has tried it, even just on a test table.

https://clickhouse.com/docs/reference/statements/create/table/codec

Quick context: ZXC is asymmetric. Compression is slower, decompression is very fast, and the ratio sits between LZ4 and ZSTD. That suits data written once and scanned many times, which describes a lot of MergeTree workloads.

CH team ran an independent benchmark on ClickBench hits (Graviton4, ZXC 0.13.1, single-threaded). At the default level, ZXC decoded 1.8-2.6x faster than LZ4 on every column tested, with better ratios. https://pastila.nl/?cafebabe/16ae94cb2aba1cd26c656f3ded96d882.md

Fair warning: the format isn't stable until 1.0: (v1.0 is expected at the end of 2026)

I'd like to have feedbacks about:

  • Ratio and query speed vs LZ4 / ZSTD(1) on your data
  • Which column types it helps with, and which it doesn't
  • Any crashes, errors or odd behavior
  • CPU (x86 or ARM)

Thx

Repo: https://github.com/hellobertrand/zxc

u/pollop-12345 — 14 days ago

ZXC - Asymmetric lossless compression: 40%+ faster decompression than LZ4 on ARM64, with better ratios

ZXC is a small C library for lossless compression built around an "asymmetric" trade-off: a heavy, slow encoder that produces a bitstream tuned for maximum decompression throughput. It targets the "write once, read many" reality of software distribution: you compress once on CI, but users decompress on every device, every launch, so it spends the effort upfront to make the read path as fast as possible.

Representative pairings on the Silesia corpus (Apple M2, single-threaded; reproducible via lzbench and TurboBench, into which zxc has been merged):

  • Fastest tier: ZXC -1 vs LZ4 --fast -17: 12,530 vs 5,623 MB/s decode (2.2x), at a marginally smaller size (61.5 vs 62.2).
  • Default tier: ZXC -3 vs LZ4 (default): 7,049 vs 4,783 MB/s decode (1.5x) AND a better ratio (45.8 vs 47.6).
  • Max-density tier: ZXC -6 vs LZ4HC -9: 5,620 vs 4,528 MB/s decode (1.2x) and a smaller output too (36.3 vs 36.8), beating LZ4HC on both axes.

(Level 6 is new in 0.11.0: Huffman-coded literals + an optimal LZ77 parser. Note its decode lead over LZ4HC is an ARM story; on x86 the two are roughly on par on decode, with ZXC still ahead on ratio.)

Other bits: seekable archives (O(1) random access), thread-safe API, a push-based streaming API for async/event-loop integrations, and bindings for Rust / Python / Node / Go / WASM. BSD-3-Clause.

Install: Debian (apt install zxc / libzxc-dev currently in testing/unstable), Homebrew, conan, vcpkg, pip install zxc-compress, cargo add zxc-compress, npm i zxc-compress.

Github Repo : https://github.com/hellobertrand/zxc

Feedback welcome, especially on the decoder design.

u/pollop-12345 — 4 months ago