I got tired of manually benchmarking ONNX vs CoreML vs PyTorch every project, so I built a CLI for it
Every time I ship a YOLO model I end up asking the same question should this be ONNX, CoreML, or just PyTorch? Does FP16 actually help here or is it just marginal?
I've answered this by hand, badly, on four different projects this year, and thrown the results away every time. First i have to optimize a model for my liking and then figure a way to reduce its size.
So I'm building exportrace - you run one command, it benchmarks your model across every export backend available on your actual machine (PyTorch, ONNX, CoreML, CUDA, TensorRT depending on your setup), and gives you FPS, latency, and accuracy delta vs FP32, plus a ranked recommendation.
Consumer hardware only - your laptop or dev box, not Jetson/Pi. It's open source (MIT), runs fully offline, no accounts. Still pre-launch, landing page + waitlist here if you want to see the concept and maybe kill the boredom of doing this by hand too: https://exportrace.vercel.app/
Curious if others hit this same wall, and what backends/hardware you'd actually want covered first.