openplate - MIT licensed calorie/food tracker with photo scan (local LLM support), MIT licensed, no paywalled features
Hey everyone,
so I was kinda surprised to see how expensive the popular calorie/food logging apps still are. Given that the individual API calls to identify foods from an image are really cheap at the moment I decided to build an open source alternative to Yazio and CalAI.
I asked family and friends as to what they would like and what the most important features were to them and came up with a set of features for the first version of a food tracking/logging app.
Feature list:
- id foods from the photo of a plate (multiple at once)
- be able to input a plate as text or voice, as if you're chatting with someone
- cross check the foods with a reliable database of nutritional facts so hallucinations are kept low
- easy to use
- recipe recommendations for a list of ingredients, photo of the fridge
What the self-hoster in me wanted:
- No reliance on the appstores, but a great UX (Progressive Web App)
- Easy to self-host
- Easy to replace upstream inference provider, also the possibility to use a local LLM
- Stay in control of my data
- Be able to invite my family to an instance I manage, so they dont have to signup to openrouter etc.
I had a first demo of this running quite fast and then met a research assistant for an obesity center and learned about their issues with patients and food logging. Obviously the existing solutions solve most of these, but they are paid and institutions can't host them on their own infra. So this led me down the path of splitting up the app into three distinct services.
The client app, can be run as a PWA within your network, you can add your key (BYOK) for any openai compatible API. The data stays in the browser, can be exported or synced through an encryption key with a sync server.
The core app, this app started off as a sync server for client apps, but turned into something like a gateway. You can run this in different modes, one of which is a managed gateway mode, for example to set up an instance for your family. The thought here is you have this running on your own network, configure an API key once and can then invite your friends to use (tailnet or whatever frontdoor youre using), they install the PWA and have an almost-native like app experience.
The inference runtime, if you want to self-host the inference and image recognition, I've setup a service for that too. This uses llama.cpp under the hood and does some pre-procesing for more accurate image identification. Ony my really crappy hardware ryzen 9 and 64gb this took around 180s per image, so you definitely need the hardware for this to make sense.
Once all of these were wired together and I had a better understanding of what the product needs to support, I developed a more polished version and invited family and friends to try this out and report bugs. This was basically the last 4 weeks. I now feel comfortable to open this up to everyone.
There is a project website that explains the stack in more detail and hosts the docs outside of github via https://openplate.de
The repository contains installation instructions and everything else you need via: https://github.com/LowCarbCheck/openplate (includes the compose or quadlet files).
Tech stack:
- React Router, Vite, Tailwind + Shadcn
- PostgreSQL
- Docker/Podman
- llama.cpp for local inference
Disclaimer: There is a hosted/paid version with a free trial that is linked to from the project website, but that's totally optional, no paywalled features and self-hosting should be easy enough (requires an API key or a local LLM though).
Looking forward to feedback on the app itself and how setting it up goes for everyone. Here to help if you're facing issues running this, also happy to accept PRs on GitHub!