u/Anode1_dev

A few times in my career, I ported into plain C or Java the tools that scientists had written and tested in R, SAS, Python or MATLAB, so they could run on any computer, down to embedded ones, in the smallest memory footprint I could get, without depending on anything else, and in a streaming fashion, so that a million records go through the same 2.7 MB as hundreds of millions do. One of them was a hospital length-of-stay model, and that is the one I've just released as open source, in case it's useful to someone here.

It's ordinary least squares - linear regression, fitted by minimising the squared residuals. The terms aren't compiled in. Your CSV header names them, so adding a term to the model means adding a column to the file, like in R, and the same binary fits two terms or 35 (which is what one of my production models needed). With one term the whole method is five lines of arithmetic. The README shows them.

There are checks built in against bad design. The program tests each term for curvature, the fitted value for a missing interaction, and the residuals for spread that grows with the prediction.

It's not a stats package: no inference, no imputation, and an NA stops the run rather than being filled in. Do the modelling in R, Python, whatever. It reproduces NIST's Longley to eleven digits, which is the set that kills naive implementations, and agrees with lm() on every example file. Both run as regression tests in the suite.

Disclosure: written with AI assistance. The design, the prototype, the requirements and the tests are mine. make check diffs every example file against lm() and against the Java implementation, which came from the original that has been running at a large number of facilities since 2011.

The author is not a statistician, so if someone here sees where the diagnostics go wrong, that is the most useful thing to hear. Contributions welcome.

https://github.com/Anode1/linearr

I wrote a small tool for myself. You file a path, a link or a note under keys you choose, and get it back by those keys. What I would like comment on is how it is written, not what it does.

I came to C alongside FORTRAN, Ada 83 and Pascal in the 90s, so I think in functions, data locality and streams before objects. The rule is that memory is bounded by the structs, not by the data. Records go through one at a time, on the stack, in fixed buffers. get, find, set, merge and compact allocate nothing at all, so a 10 GB store and a 10 KB store run in the same footprint. Set operations are k-way merges over sorted posting lists: keep the head of each list and advance.

Six heap sites in 18k lines, each written down with what bounds it.

Two things I know are wrong: main() is 630 lines and the HTTP handler is 520. Both are flat dispatchers, both are too long, and both are written down as debts rather than defended.

On a million records, 85 MB store, one core: building the whole index is 7.9 s in one streaming pass, a full scan is 1.8 s (cat class), and a get on the hottest key, 270k ids, is 2.2 s. That last one used to take hours, because finding a record by id meant scanning the store; an id-to-offset index fixed it. Bulk import is still O(n^(2)) and I say so in the same doc.

C99, GPLv2+, no dependencies, plain Makefile. About 5k lines of tests, run under AddressSanitizer and UBSan on every push.

Style doc: https://github.com/Anode1/ais/blob/main/doc/dev/STYLE.md
Numbers, reproducible with a seeded generator: https://github.com/Anode1/ais/blob/main/doc/performance.txt
Code: https://github.com/Anode1/ais