r/LocalLLM

I would like to share my latest open source local LLM inference tool implemented in C#. It supports models like Gemma4, Qwen3.6 with multi-modal (image, vision, audio), reasoning and function tool. It can run on Windows/MacOS/Linux and fully leverage GPU's capability. The API is completely compatible with OpenAI and Ollama interface.

Really appreciated if you can try it and give me some feedback. If you like it, it will be a big thank you if you can star it. Thank you very much!

Local coding models have a repo-context problem.

When using llama/qwen/mistral/gemma for coding, the hard part is often not the model itself. It is getting the right files/functions into context without dumping too much raw source.

Long context helps, but it does not solve retrieval.

If the model never sees the right file, it still guesses.

I’ve been building SigMap, a zero-dependency CLI that creates a compact repo map for coding workflows.

Instead of sending raw source first, it extracts:

• function signatures
• classes/interfaces
• exports
• import relationships
• ranked file matches per query

The workflow is simple:

repo map first → find likely files → read full source only where needed

Benchmarked across 18 repos / 90 tasks:

• 81.1% hit@5 vs 13.6% random baseline
• ~6× better file retrieval
• 96.9% token reduction in the benchmark setup
• 41.4% fewer prompts per task

No embeddings. No vector DB. No npm dependencies.

This is not meant to replace LSPs, grep, agent search, MCP tools, or full-file reads.

It is meant to give local coding models / agents a cheap first-pass structure map before deeper inspection.

Repo: https://github.com/manojmallick/sigmap

Benchmark suite: https://github.com/manojmallick/sigmap-benchmark-suite

Curious how people here handle repo context with local coding models.

Are you mostly using grep/search, RAG, repo maps, MCP tools, or just relying on longer-context models?

Edit: Good point from the comments — SigMap core is model-agnostic. The docs currently look too focused on proprietary assistants, so I’ll add clearer examples for VSCodium/Open VSX, Continue, Cline/Roo Code, Aider, OpenHands, and local Ollama/llama.cpp workflows.

So I'd like to be able to get the text out of this image. Is that a thing that an LLM can help with? I have OLLAMA installed and running.

I am just not sure what tools to throw at the problem? It'd be great if it was structured in some kind of way, but I'd settle for the text.

Anyone any pointers?

Thanks!

I've been building an open-source agent (Apache-2.0) and wanted to share it here because it's designed to be fully local and self-hostable: it talks to any OpenAI-compatible endpoint, so Ollama / llama.cpp / vLLM / LM Studio all work as the backend. No cloud lock-in, your keys and data stay yours.

The core idea is LLM-Fusion: for the hard steps it can run a panel of models on the same prompt, have a judge model cross-check them (consensus / contradictions / blind spots), and a synthesizer write the final answer. Locally this is fun because you can mix a few small local models and let them cross-check each other. A cost/latency-aware router keeps easy turns on a single model so you're not paying panel latency for everything.

Beyond that it's a full agent: plan -> act -> verify-or-revert (it runs your tests and treats the result as ground truth), layered memory (SQLite + FTS recall, cross-session profile, consolidation), a governance kernel, cron/proactive jobs, MCP client + OpenAPI-to-tool import, and an isolated subagent/crew layer (parallel git worktrees with per-worker verify gates). Runs on a laptop or a $5 VPS via Docker.

Honest status: it's alpha - 463 tests, mypy --strict clean, but no production mileage yet. Local reasoning quality obviously depends on the models you point it at, so I'd genuinely love to hear which local models people find good enough to actually drive an agent loop (reliable tool use + self-correction) - that's the make-or-break for going fully local.

Repo: https://github.com/brcampidelli/chimera-agent

Agents waste time and tokens re-learning every site. On each run they screenshot, snapshot the DOM, and figure out the page from scratch.

I built an open source catalog of reusable browser skills. Skills capture each site's network requests and DOM, making it 30 times faster.

You can upload your own skills or request new sites.

Github repo: https://github.com/browser-memory/bmem

Somewhere you can't quite leave. Choose a way through. https://huggingface.co/spaces/alvations/hallway8

This is a interactive fiction escape room kinda game and the whole game is created by just me and my AI agents through Claude Code. The full game is opened source and anyone looking to replicate any workflow processes used to create the game can do so at https://github.com/alvations/melon-eight

Hi everyone, I've developed an application that runs an LLM service on mobile devices and makes it available as a service node for other applications to access via a local area network.

It supports LiteRT-LM and llmam.cpp, and can run the vast majority of models (provided your hardware configuration allows). Currently, mobile hardware typically supports models up to 3B.

Regarding the API service, I've ensured it's compatible with the OpenAI and Ollam interface specifications.

Furthermore, I've integrated Hugging Face's Hub Point, allowing you to directly search, download, and import Hugging Face models within the application.

Link :

https://play.google.com/store/apps/details?id=com.chaterminal.mobilellm

I am trying to run Qwen 3.5 9B locally but it keeps thinking for very long amounts of time. I am running it on Linux with a 9060xt 16G. Is this normal?

Hey everyone,

I just open-sourced TuneForge.

The goal is simple: let your coding agent manage the full LLM improvement loop without ever leaving the chat window.

You can now tell your agent something like:

“Build me a customer support bot from this FAQ”

…and it can:

• Generate a clean synthetic instruction dataset (with LLM judging for quality)

• Run LoRA supervised fine-tuning on any Hugging Face causal LM

• Do a quick policy-gradient RL step using Ollama as the reward judge

• Merge the adapter, evaluate on a test set, and iterate

Everything runs locally, uses 4-bit quantization so it fits on modest hardware, and uses background jobs (with job_id polling) so long training tasks don’t freeze the MCP connection.

It’s built around the Model Context Protocol (MCP) for seamless integration with Claude Desktop, Cursor, Zed, Continue.dev, etc.

Tech: Python + Transformers + PEFT + bitsandbytes + Ollama + SQLite for job state.

Super early stage (just released), MIT licensed.

Would love feedback or ideas on what to add next. If you’re into agentic fine-tuning workflows, give it a try and let me know how it goes!

One thing has felt odd to me while working with AI agents.

We've gotten pretty good at giving them memory for text.

They can search documentation, index repositories, retrieve past conversations, and even build long-term memory over time.

Videos, though, are still treated as temporary input.

The agent watches a recording, answers a few questions, and when the session ends, that understanding is usually gone. Next session, the same video gets processed all over again.

That feels like an architectural gap rather than a model limitation.

A video isn't fundamentally different from any other source of information. Once you've extracted transcripts, OCR, visual observations, and timestamps, why throw that work away?

I ended up building an open-source project around this idea.

Instead of asking the agent to repeatedly "watch" the same video, it builds a persistent local index the first time. Future questions become retrieval instead of video analysis.

It changed how I think about video in agent workflows.

I'm curious whether others see this as a real missing piece, or if you've already solved it another way.

GitHub: https://github.com/oxbshw/watch-skill