Immich ML Proxy
This allows forwarding embedding calculation to one or more external instance (llama.cpp). I started this since Immich ML has been broken since v2.5.2 on GTX 960. So you can use embeddinggemma-2, or you have enough resources, you could run Qwen3-VL-Embedding-8B as well.
You would still need an Immich ML instance to handle OCR/face recognition workload.
Another caveat is, you need to set a model that matches the dimension.
Tested on Immich v3.3.0, new PP-OCR-v6 requests can be forwarded without problems.
One more note, the proxy support multiple llama.cpp backends, I run llama.cpp on both Mac/Steam Deck/a gaming PC during the backfill.
Lastly, this is heavily assisted by AI, but I did test things out that it does work (at least on happy path)
------
I initially run the model directly with sentence transformer, but using llama.cpp allows use of quantized models (gguf format) easily, and it is a very actively maintained project, if new hardware/arch gets released, chances are llama.cpp will get support first among all the engines. If you have a Mac, you can use mlx-embeddings as well.