X-ray like feature
I've been working on a side project for a while to add a X-ray like feature (i.e. when you pause you get information about actors in the current scene. Professionally I have worked intensively with face-recogntion, and this is not such a tough problem to solve, and if you make a gallery of all actors in your media library you can also find some fun (at least in my opinion) things like cameos or in the case of the second screenshot a poster of another actor being recognized.
Right now I have the process split in two:
- Offline scanning of media to get actor epochs (currently private while I make sure I am not accidentally leaking any IP from my employer through retained knowledge)
- A Plugin (JRay) that serves a json (or modifies the html player) with the actors in the scene.
Right now it's really a POC. I wanted to get input from any other interested developers. Some specific things:
- Does the standardization on Actor/Epoch fit well compared to Epoch/Actor?
- Right now it is just who is on screen, should we leave room for more info (e.g. filming location, or some fun fact)
- I view this as really being split, I have a low power n100 as my jellyfin host (keeps the power bills down) and a gaming rig, I can run the face-rec on the gaming rig when it is not in use, I will go bankrupt if I add the GPU to my homelab :)
- maybe more for a lawyer, but sharing the scene info? Would save a lot of energy if this gets more widely used...
Also happy to discuss with any one who wants to colab.
Yes this project has relied heavily on LLMs, no I would not have time to do this all without.