u/Head_Invite3039

Hello, everyone. I‘m going to buy a P40 to run some local large models below 20B to reduce the token cost of my AI programming. My idea is to let it do some simple tasks, and complex tasks call the paid big model. Will this work? How can I implement it on the software so that it can hand over complex tasks to the paid model by itself?The machine translation I use may have stiff sentences, sorry.

Hello everyone, I have developed an app that runs an LLM service on a mobile device and allows other software to call it.

Regarding the API service, I have ensured compatibility between the OpenAI and Ollam interface specifications.

Furthermore, I've integrated Hugging Face's Hub Point, allowing you to directly search, download, and import Hugging Face models within the application.

Link :

https://play.google.com/store/apps/details?id=com.chaterminal.mobilellm