Built a LLM engine that trains on any GPU — Nvidia, AMD, Intel iGPU — in one C++ binary. No CUDA, no PyTorch. AGPL
Inference run everywhere. Training still means CUDA and a framework stack. I thought that was a convention, not a law of physics, so I built sym_engine: a single C++ binary (~6,000 lines + 21 Vulkan compute shaders) that pretrains, finetunes, evals, and runs inference on any Vulkan-capable GPU. No autograd — gradients are derived, forward and backward both visible in the source.
What I've actually verified, not what I hope: training on an RTX 5090, an AMD APU, and an Intel iGPU - the video shows two of them training simultaneously on one machine, CPU at 1%. Same binary, same shaders, fp32: RTX 5090 - 10.4M model, ~44,000 tok/s. Intel iGPU (64 EU) - 1.2M model, ~50,000 tok/s; 10.4M model, ~1,800 tok/s. The spread is the hardware's bandwidth, not the software — when the model fits the silicon's budget, an iGPU trains at desktop-GPU pace. Physics can't fit what won't fit. Largest trained so far: 318M params 32 GB VRAM
The target is edge devices - that was the goal from day one. To prove it, I gamified the engine into an Android app demoed in the link also: (CPU path on-device; the Adreno Vulkan path is in progress) a virtual pet whose brain is its own 120K-parameter model, training and inferring in real time on the phone as you play. Its save file is literally the weights. That's the point of the architecture: models that learn where they run - on-device, offline, no cloud, no API.
What it's not: a llama.cpp competitor. That runs other people's models fast; this grows your own small models on whatever silicon you have. No KV cache by design - training and inference share one code path. Known limits: single GPU per instance, Python needed once for tokenizer prep.
AGPL-3.0, sole copyright, not accepting contributions but forks welcome. Repo: https://github.com/chrismelanov85/SYM\_ENGINE