Hiya, just found the tool today and it looks great. But on the android app it seems like the first letter of each new reply, and the first letter after doing a backspace get duplicated. I am using heliboard keyboard on Samsung s25
Does anyone have a fix for this? Tool seems amazing but really hard to use currently.
Looking to experiment with these for LLM
Hey all,
I have built myself a WebUI for configuring and managing llama-server sessions, and want to share the code and concept. Python and a bit of JS. Hack away!
Local only.
https://github.com/m94301/llama-studio
The major use case is running various instances of llama-server on fixed ports to act as infrastructure for home development (and entertainment) frameworks. Read: Fiddling with settings, comparing experimental builds to mainline, and optimizing. Also good for everyday fooling around.
Configs are saved per model in a json, consisting of all launch args and optional paths for custom llama-server. I have a launch arg browser with search using the current llama-server's actual -help output. I hate forgetting a launch arg format and having to open a new terminal to do -help. Spec MTP what? Draft type who?
Launch to choice of GPU, monitor VRAM, load, and temp. And a somewhat rudimentary VRAM calculator to help estimate what fits where when using what quant.
Last, a reasonable mobile interface to run tests and fool with config on phone when in a basement or IT closet. Show and hide logs, start, stop, change config. Less keystrokes on tiny phone keyboards. Sanity +100.
Just a quick note that I got a nice result using am17an's MTP branch of llama.cpp on v100 32GB SXM module using one of those pcie card adapters. Pulled and built in one shot, and llama-server ran without a hitch.
Tested using am17an's MTP GGUF, q8_0 kv cache and 200k cache limit acting as vscode copilot.
29-30 t/s without MTP
54-55t/s with MTP, using 150W power limit on the card.
Falls to 40-45 t/s after choking down 50k tokens, but doing great with tool calls, sub agents, and made some very insightful code reviews and refactors.
Thank you am17an! Can't wait to see this branch mature, this is great stuff.