Seeking feedback on my new library SoaTable
I've been working on SoaTable, a header-only C++23 library that stores data as a Structure-of-Arrays (one contiguous array per field) but keeps a row-shaped API on top: you insert() a record, assign<Column>() fields to it, and iterate with view<Position, Velocity>().
The goal was to get columnar/cache-friendly layout without
forcing the code that uses it to think in columns.
Repo: https://github.com/bbalouki/soatable
A few design points that might be worth discussing here:
1- Columns are sparse and optional per row. A record only pays for the fields it actually has. Data-less column types (struct Frozen {};) act as tags and cost \~nothing. Reading a missing field is a defined "not present", not UB.
2- Handles are generational. insert() returns a row\_id that survives erase, insert, and full re-sorts, and a stale handle reports itself invalid instead of aliasing a reused slot (ABA).
3- Joins start from the smallest column. view/select<A, B>() scans the smaller of the two validity sets and probes the other, so selective queries touch far fewer rows.
4- Layout is a policy, not a rewrite. soa\_table (flat, 64B-aligned), aosoa\_table<Tile> (tiled, growth never copies), pmr\_soa\_table (arena/pool), and mmap\_soa\_table (larger-than-RAM) all share the identical row API.
5- Zero-copy escape hatch. column<T>() hands back a std::span over the real aligned storage for SIMD/BLAS/FFT, with a separate validity bitmap.
6- Opt-in everything. Core is one dependency-free header; compute, query/group-by, serialize, concurrent, timeseries, units, and a runtime dynamic\_table are separate headers you include only if you use them. There's also a SOATABLE\_NO\_EXCEPTIONS / no-RTTI build for embedded/flight targets, kept honest by a dedicated CI leg.
Where it earns its keep: large, sparse tables with selective queries (ECS worlds, tick stores, telemetry).
Where it doesn't: if the table is small, every row has every field, and every pass touches every field, a plain std::vector<Struct> is simpler and just as fast. I tried to be upfront about that in the README.
Numbers (selective join, 250k rows, Release; machine-dependent, harness in the repo):
smallest-drawer select \~168µs vs \~1.55ms when forced to start from the largest column, vs \~1.06ms for a hand-rolled columnar scan and \~1.30ms for an AoS branch scan.
It is not an ECS framework (no systems scheduler, no archetypes), it's the storage layer, so it's more comparable to a sparse-set column store than to EnTT/flecs.
C++23 required (GCC 13+, Clang 18+, MSVC VS2022), CMake/Conan/vcpkg.
Feedback on the API, the sparse-column design, and the benchmark methodology is very welcome m, especially from people doing ECS or columnar-analytics work.