I wrote a from-scratch ML framework in C++ and trained a 10M param GPT on it that runs in your browser via WASM
I've been building tiramisu, a machine learning framework written from scratch in C++20. Only the stdlib is used at link time.
What's in it:
- Strided tensor engine with zero-copy views
- Reverse-mode autograd with a dynamic tape
- Tiled + AVX2 SIMD matmul
- Full transformer stack (MHA, LayerNorm, GELU FFN)
- CUDA backend with custom kernels
- Python bindings via pybind11
- Compiled to WASM via Emscripten for the browser demo
The 10M parameter Shakespeare GPT in the demo (6 layers, 8 heads, 512-dim) was trained end-to-end using tiramisu on a free Kaggle T4, then int8 quantized to 11MB for the browser.
Demo: https://tiramisu.dnex.dev/shakespeare
Repo: https://github.com/dnexdev/tiramisu
Happy to answer questions on design decisions. Any feedback on the implementation is very welcome.