Announcing Ducktape: duckdb + tech.v3.dataset via Project Panama
Announcing Ducktape!
Pleased to announce that we just released the first version of dynamic-alpha/ducktape for people who feel like helping us test. It is a near drop-in replacement for tmducken but uses java.lang.foreign instead of JNA for FFI. Highlights include:
• Deterministic native memory. Allocations live in scoped Arenas and are released with with-open rather than via GC finalizers. Helps avoid possible doublefrees under GC load
• 12 more DuckDB types -BLOB, HUGEINT, DECIMAL, INTERVAL, ENUM, LIST, STRUCT, MAP, and the full timestamp precision family.
• Streaming appender API - open-appender / append-dataset! / flush-appender! keeps DuckDB's appender alive across batches, amortizing per-call setup. Handy for Kafka consumers, paginated API ingest, file shards - up to 10x faster than repeated insert-dataset! for small batches.
• Performance - Parallel column encode/decode, direct MethodHandle dispatch, partitioned parallel-concat for multi-chunk reads. Lands at 1.1–4.0× vs tmducken across numeric / string / uuid / mixed / wide workloads (1M rows, JDK 25, DuckDB 1.5.2, all significant at 95% CI - full table in the README).