Ask questions across your Markdown notes using a fully local Graph RAG engine. Built for Obsidian vaults, works with any folder of Markdown files. Extracts entity-relation triples from wikilinks & YAML frontmatter, retrieves answers via hybrid search (vector + BM25 + temporal). Multilingual. No cloud. Runs on Ollama.
I would like to share my latest open source local LLM inference tool implemented in C#. It supports models like Gemma4, Qwen3.6 with multi-modal (image, vision, audio), reasoning and function tool. It can run on Windows/MacOS/Linux and fully leverage GPU's capability. The API is completely compatible with OpenAI and Ollama interface.
Really appreciated if you can try it and give me some feedback. If you like it, it will be a big thank you if you can star it. Thank you very much!
Hi everyone, I've developed an application that runs an LLM service on mobile devices and makes it available as a service node for other applications to access via a local area network.
It supports LiteRT-LM and llmam.cpp, and can run the vast majority of models (provided your hardware configuration allows). Currently, mobile hardware typically supports models up to 3B.
Regarding the API service, I've ensured it's compatible with the OpenAI and Ollam interface specifications.
Furthermore, I've integrated Hugging Face's Hub Point, allowing you to directly search, download, and import Hugging Face models within the application.
Link :
https://play.google.com/store/apps/details?id=com.chaterminal.mobilellm
Open Source Palantir
We're building OSIRIS - The Open-Source Palantir Alternative
Just launched at osirisai.live - a free, open-source global intelligence platform:
-Real-Time Tracking:
-10,000+ commercial, military and private aircraft live on a 3D globe
- 2,000+ satellites including ISS
- 1,400+ worldwide CCTV camera feeds
- Earthquakes, wildfires, nuclear facilities and severe weather
Built-In OSINT Tools (no installs needed):
Nmap port scanning from the browser
- DNS record lookup and enumeration
- WHOIS domain intelligence
- SSL/TLS certificate transparency
- BGP routing and ASN lookup
- Threat intelligence and IP reputation
All running on a 3D interactive globe with day/night cycle, 20+ live API feeds, and a SIGINT news aggregator.
Live: https://osirisai.live
GitHub: https://github.com/simplifaisoul/osiris
Free. Open Source. No sign-up required.
Hey r/StrixHalo — I built hal0.dev with the goal of optimizing for exactly this hardware and extracting the best possible performance, functionality, and value from it.
We're finally ready and opening public beta this weekend. Would love to have you kick the tires — I've had limited testers so far and we're ready for more.
The idea. A Strix Halo box is a genuinely special piece of kit — Radeon iGPU, XDNA NPU, and one big unified-memory pool — and hal0's goal is to extract the most performance, value, and functionality possible from it.
Chat, embeddings, rerank, transcription, live speech, image gen — answers on one local /v1/ API.
This is my first real shot at something this ambitious, so the philosophy is deliberately narrow: high impact features, reliable, proven tools, wired up automatically, and integrated deeply across the platform.
One-line install builds and wires up — automatically
Slots: every model runs in a "slot" — one model, one container, with a typed lifecycle and a GPU arbiter that assigns unified-memory to either always-on concurrent LLMs or image gen, one group at a time — so GPU workloads never fight over the pool, yet multiple LLMs stay concurrent and always ready.
Agents & memory: striving for the deepest, most seamless Hermes integration possible — kanban, delegation, and hal0 administration, all out of the box. Memory is a constantly improving shared brain: a fully built-out Hindsight custom-provider system with a primary private bank (seeded per child profile) plus a shared bank with MCP access, so agents like claude-code, pi, and opencode can learn from and teach your agent as the homelab evolves.
Developed on a Ryzen AI Max+ 395 / 128 GB. I run mine in a Proxmox LXC for the exceptional quality-of-life wins — resource sharing/allocation without being captured, plus the reliability. Bare-metal Ubuntu and WSL2 (WIP) paths are in the docs too. It's hardware-agnostic in principle but tuned for Strix Halo first, particularly on Proxmox — NVIDIA/CUDA is being worked toward as a supported runtime device, but don't count on it working just yet.
Open-source, Apache-2.0. Come kick it around and tell me what falls off 🙂
⭐ https://github.com/Hal0ai/hal0 - Give Us A Star!
🗪 https://discord.gg/n2ftGqYr8 - Join Us In The Discord!
💫 https://hal0.dev - Promo, Info, Docs & More
Been building this solo for a while and finally feel good sharing it: Ares, a personal AI assistant that actually remembers you, runs in your terminal (or a desktop app), and keeps everything on your machine instead of shipping your life to some company's server.
The idea was simple: I wanted something like Jarvis — not a chatbot that forgets everything the second you close the tab, but something that builds up real context about me over time and actually does things instead of just talking about them.
What it can do right now:
SKILL.md playbooks it can load on demand instead of cramming everything into one giant promptThe privacy part actually matters to me. Memories, conversations, everything — stored locally in SQLite. No telemetry. No analytics. Where most assistants reach for a convenience API layer to hook up Gmail, this one does the OAuth dance directly so nothing extra sees your data.
It's still very much a work in progress — I'm actively hardening the architecture and building out a proper task system right now — but it's genuinely usable today and I'd love feedback, contributions, or just someone else to yell at me about what's broken.
GitHub: https://github.com/akyourowngames/friday
Happy to answer questions about the architecture, the memory system, or why I made specific choices — building this thing has taught me more than any tutorial ever did.
I built AI-Monitoring: A lightweight, zero-dependency dashboard for Ollama, LiteLLM, llama.cpp & GPUs (No Prometheus/Agents)
I would love to get your feedback, feature requests, or contributions!
I've been toying with the idea of getting a loan to get into hosting here in Canada. The loan isn't so bad, maybe no worse than 10% interest and no need to make principal payments for the first year.
The hard part is figuring out a way to turn that into profit.
At first I was toying with the idea of buying a couple of RTX 6000 Pro Blackwell cards but looking at the competitive price point, I started figuring two of them really isn't *that* competitive. So I started looking at the H100, getting a single card.
I'd probably end up having the ability to spend maybe $50K CAD to set up the system, that'd be like $35K USD.
My expenses would end up probably being about $900-950 a month, and after a year that'd jump to about $2K. OR in USD $630-670 and $1400 respectively.
But I understand utilization shouldn't be expected to be more than 50-60% for new users. Is there *ever* a point at which you can expect 80-100% utilization? And is there *ever* a point at which one could sell a H100 for rent for over $2 USD an hour?
I also see Vast.AI has a financing programme. Anyone have experience with it?
Builders-welcome post with the substance up front (disclosure: I'm the maintainer). OmniRoute is a free, MIT, self-hosted AI gateway — one OpenAI-compatible endpoint over 237 providers — built around two problems: runs dying on a provider 429, and tokens bleeding on tool/log output.
One endpoint, 237 providers — 90+ of them free. You point any tool or agent at a single OpenAI-compatible endpoint (localhost:20128/v1) and it can reach 237 LLM providers without you rewriting anything. 90+ have free tiers and 11 are free forever (no card), which aggregates to ~1.6B documented free tokens/month — and that's honest, pool-deduped math (we count each shared pool once instead of inflating it; the methodology is public in the repo). There's a one-command setup-* for 13+ coding tools (Claude Code, Codex, Cursor, Cline, Roo, Kilo, Gemini CLI…), so switching your existing setup over takes seconds.
Fallback combos — so it never stops mid-task. A "combo" is a ladder of models the router walks automatically: your subscription first, then API keys, then cheap models, then free ones. When a provider returns a 500 or you hit a rate limit, it slides to the next target in milliseconds, mid-request, and your tool never even sees the error. There are 17 routing strategies (priority, weighted, round-robin, cost-optimized, auto/coding:fast…) plus three resilience layers — a per-provider circuit breaker, a per-key cooldown, and a per-model lockout — so one dead key can't take down a whole provider.
Fusion — an ensemble mode for the hard steps. Beyond simple routing, there's a fusion strategy that fans a single prompt out to a panel of different models in parallel and then has a judge model synthesize one best answer (mixture-of-agents, built in). It's cost-aware, so easy turns stay on one fast model and it only fuses when the step is worth it.
A 10-engine compression pipeline — the part most routers don't have. Every request flows through a transparent compression pass you can toggle/stack per combo. Instead of one trick, it stacks the best of the open-source ecosystem: RTK filters command/tool output (git diffs, test logs, builds) at 60–90%, Microsoft's LLMLingua-2 does ML semantic pruning, Caveman handles prose, session-dedup strips repeats across turns. Critically, code, URLs and JSON are preserved byte-perfect, and a default-on inflation guard throws the compressed version away and sends the original if compressing would actually grow the prompt — it never makes things worse. On tool-heavy sessions that's ~89% average input-token reduction (an 8k-token git diff becomes a few hundred). Full credit to every upstream project (RTK, Caveman, LLMLingua-2, Troglodita) is in the README.
Agent-native — the agent can drive the router itself. There's a built-in MCP server (95 tools across 30 audited scopes, over stdio / SSE / streamable-HTTP), plus A2A (v0.3, JSON-RPC 2.0) support. That means an agent can query providers, switch combos, read its own remaining quota and manage memory through the gateway — not just consume tokens through it.
It's 100% local (zero telemetry, AES-256-GCM at rest), MIT-licensed, has a prompt-injection guard on every LLM route, opt-in memory, and runs on npm, Docker, desktop or your phone via Termux.
For context on whether it's worth your time: it's grown to ~9.8K GitHub stars, 1,490+ forks and 280+ contributors in ~4.5 months, with 21,000+ automated tests and 1,830+ issues closed — so it's a battle-tested project, not a brand-new experiment.
npm install -g omniroute
GitHub: https://github.com/diegosouzapw/OmniRoute · Site: https://omniroute.online
Would value a critique of the routing/compression architecture from this crowd.
Hey! Finally releasing code I've put the past 4-5 months of my life into, I had an idea and wanted to fix some things that really irritated me with LLMs. Aimee runs agents that actually remember. Self-hosted, your keys. No subscriptions, no costs, purely open source. First public beta release, but the results have already exceeded my expectations.
- Persistent searchable memory across runs. No starting from zero. Shared across all agents models and users.
- Delegates bounded sub-tasks to multiple model backends in parallel, each with a role and persona. Use local LLMs, subscriptions, or API keys.
- Indexes your codebase, records past decisions, and curates all associated documents so agents have real context and a knowledgebase of past decisions, not just a prompt.
- Exposes OpenAI/Anthropic-compatible APIs, so Claude Code, Codex, or your own orchestrator can drive it. You can also do the inverse, and run any model you have hooked up to aimee as your model for Claude Code, Codex, etc.
- Switch models, TUIs, etc. at anytime, and keep your decisions, knowledge, and other information!
- Works with anything that can use MCP, plugins, web APIs, or ACP.
Built for people tired of stateless one-shot agents. Try it out: https://github.com/RakuenSoftware/aimee
What it does:
•Fuses 30+ free, keyless live feeds into one real-time world-state on a live globe — conflict (GDELT, Ukraine front lines), natural hazards (quakes, storms, wildfires), markets (indices, crypto, Polymarket odds), humanitarian (displacement, disease, food insecurity), and movement (flights, satellites, maritime)
•Fully agent-ready: one local call returns a prose world summary + every event with coordinates + live forecasts. OpenAPI spec, SSE streaming, runtime model-switching
•A local model forecasts across 24h → 1 year, with a council of four personas (Strategist, Economist, Naturalist, Skeptic) surfacing where they agree and split
Stack: Ollama for the local LLM, FastAPI + HTTP/JSON/SSE for the API, built by fusing two existing OSS projects — MiroFish (swarm prediction) and Osiris (live globe). MIT.
A coding agent called Zero launched today and the pitch caught my attention: your model, your machine, your rules.
It's built from scratch in Go — single binary, no Python environment, no dependency hell. Install is one line:
npm install -g u/gitlawb/zero
Repo: https://github.com/Gitlawb/zero
What stood out vs the usual coding agents:
- You pick the model provider, it's not locked to one API
- Runs on your machine, not their cloud
- Go means it's lean — the team behind it (GitLawb) claims 5x faster than their previous harness
It's early ,someone already flagged that only one provider can be active at a time, and the maintainers said to file a feature request. So rough edges exist but they're responsive.
Anyone tried pointing it at a local model through Ollama yet? Curious how it handles smaller coding models vs the big API ones.
Hey,
I got tired of debugging my AI agents with print() statements so I built PeekAI.
It's a lightweight, framework-agnostic observability tool for Python AI agents. Zero config, no cloud, no account needed.
What it does:
Install in 2 lines:
pip install peekai
import peekai peekai.init() # that's it
It's early (v0.1) and open source (MIT). Would love feedback from anyone building agents — especially multi-agent systems.
GitHub: https://github.com/oussamaKH63/peekai PyPI: https://pypi.org/project/peekai
Disclosure up front: I'm the dev, this is my project, and there's a paid version — I'll mention it at the end so it's not a stealth ad. I'm really here for this community's brutal technical feedback, because you'll find the holes faster than anyone.
What it is: a Debian-based OS built around local AI as a first-class citizen instead of a browser tab. Everything runs on your own hardware, fully offline — no cloud, no API keys, no token meter.
Under the hood (no magic — it's open models orchestrated into an OS):
The point isn't a new frontier model — it's that the whole sovereign stack is integrated, offline, and yours, instead of you gluing 8 repos together.
Honest limits: it's beta, and the local models are smaller than frontier cloud (I don't claim Midjourney/GPT parity — the trade is sovereignty + zero per-use cost, not raw quality).
Genuinely want to know: what would you want in a "local-AI-first OS" that nothing does well yet — and where do you think this approach breaks? (Paid founding beta link in a comment to respect the sub; the feedback is why I'm posting.)
Rika is a fully autonomous, sovereign, voice-first personal AI agent built from scratch. Runs entirely locally on Windows, boots with the system, operates as a persistent background intelligence.
What she does:
* Wake word activation, sub-300ms voice-to-response
* Sees the screen via 3-tier targeting (Windows Accessibility, RapidOCR, Pixtral)
* Ghost-types into any application by voice (code, emails, documents)
* Six-layer persistent memory with midnight consolidation
* Multi-agent swarm (concurrent background agents)
* Full Telegram remote control from phone
* Full-duplex interruption (stops mid-syllable when you speak)
* Emotion-reactive UI (interface shifts color based on state)
* Custom neural voice (`rika.pt` tensor, 3-way blend)
* Self-evolving (reads docs, writes her own integration code)
* Real-time data interception (zero-latency feeds injected into context)
The part nobody talks about:
Everyone obsesses over the LLM call. The actual bottleneck is everything around it. I spent more time debugging audio buffer sizes for interruption latency than I did on the entire memory architecture. And the midnight daemon that consolidates her memories? First 3 versions just hallucinated fake memories. Had to build an adversarial staleness detector that cross-checks new summaries against raw interaction logs before writing anything permanent.
GitHub: https://github.com/nssriraam/rika
Ask me anything!
We have just released our Claude Mythos Fine Tune based on synthetic CoT generated from Fable-5 and Mythos-5 session logs.
You can find the model here: https://huggingface.co/empero-ai/Qwythos-9B-Claude-Mythos-5-1M
GGUFs are also available here:
https://huggingface.co/empero-ai/Qwythos-9B-Claude-Mythos-5-1M-GGUF
We also have some sample outputs here for you: https://huggingface.co/empero-ai/Qwythos-9B-Claude-Mythos-5-1M/blob/main/evals/sample_generations.md
We hope you can find some use in it! :)
Been building this for a while and finally put out a first release. Not going to oversell it, just going to describe what it actually does.
The core idea came from being tired of AI tools that give you one confident answer and leave you to figure out if it's right. So I built something where the output you see has already been challenged internally before it reaches you. Not the same model second-guessing itself. A genuinely separate process with a different job, specifically designed to find problems with what was just produced.
There are two sides to the app.
The first is a council mode where you load local AI models and assign them different roles. One role breaks down your task and makes a plan. Another executes against that plan. A third receives both the plan and the result and checks one against the other. For coding tasks it actually runs the code before the reviewer sees it, so problems get caught by execution rather than by a model guessing whether it looks correct. If problems are found it either patches the specific issues or rewrites entirely depending on how bad it is. What you get at the end has been through all of that.
It also has session memory that builds up as you work, a document pipeline that processes files into structured knowledge before you start asking questions, task history, a diff view showing exactly what changed between the original output and any revision, and confidence labels on every result.
The second is a normal chat mode that runs Python, JavaScript, C#, Java and PowerShell inline and shows execution results inside the conversation. Web search with full page content extraction, LaTeX math rendering, a thinking mode, document attachment, and chat branching where you can fork from any point in the conversation.
Both modes run locally on your machine using GGUF models. If you don't want to manage model files there is a cloud mode through OpenRouter using their free models, same full pipeline, no local setup needed.
No account. No signup. No subscription. Open the app and use it.
MIT licensed. GitHub: github.com/YoMosa2009/Axiom
Happy to answer questions about anything.
Been running a full local LLM stack on a GEEKOM A9 Mega for a few weeks. 128GB unified memory, 170mm mini PC, runs models that normally need an A100. The hardware delivers. The AMD software ecosystem around it is still catching up.
Sharing the friction points because I couldn't find anything specific to gfx1151 when I was setting this up.
Specs
- CPU: AMD Ryzen AI Max+ 395, 16C/32T, 5.1GHz boost
- GPU: Radeon 8060S, RDNA 3.5, gfx1151
- RAM: 128GB LPDDR5x unified, 96GB carved to VRAM in BIOS
- OS: Ubuntu 24.04, OEM kernel 6.17
Current stack: Qwen3-235B (107GB), Qwen3-30B, DeepSeek-R1 70B, Qwen3-VL 30B vision, few 27B variants. All tested on one box.
The issues; none of these are hardware faults, all ecosystem/tooling maturity
ROCm 7.2 lies about VRAM on gfx1151
hipMemGetInfo returns ~26GB (system free RAM) instead of the actual 96GB. Model loads hang forever at "fitting params to device memory." Fix is HIP_VISIBLE_DEVICES=-1 in your Ollama service environment to force Vulkan/RADV, which correctly sees 111.5 GiB. gfx1151 is new enough that ROCm just hasn't caught up yet.
MTP is blocked
llama-server's multi-token prediction path uses HIP compute dispatch throws WALKER_ERROR and MAPPING_ERROR in dmesg on gfx1151 then page-faults. No workaround, waiting on ROCm 8.0. Not a hardware limitation, purely a driver gap.
Vulkan caps context efficiency around 32K
Token gen is good ~63 t/s on 30B, ~15 t/s on 235B. But prompt processing on long contexts is slow. ROCm would be 3x faster on prefill for 130K+ context. Since ROCm is broken you feel this on large document ingestion. Again a tooling problem not a silicon one.
Ollama rough edges
ollama pull hf.co/ fails due to a redirect auth bug download GGUFs manually with the hf CLI instead. Split GGUFs (00001-of-00009) can't be registered directly, merge with llama-gguf-split first. Neither is AMD-specific, just things you hit when most community docs assume CUDA.
The most frustrating documentation everything assumes Nvidia, AMD need to up their game here else good hardware with no or limited tooling support will discourage adoption.
-----
Bottom line
The silicon is ahead of its software support. AMD is putting out genuinely competitive hardware for local inference — 128GB unified at this price and form factor is hard to beat. But gfx1151 is new enough that you're in early-adopter territory. ROCm docs mostly cover gfx1100/1101, community guides assume Nvidia, and you'll be reading kernel logs more than you'd like.
If you want plug-and-play today, wait for ROCm 8.0. If you're okay with some manual setup it's worth it.
Happy to answer questions on the Vulkan setup or specific model configs.
----------Update on what all was tried and failed 😄 -----------
Done exhaustive kernel testing trying to get ROCm HIP working for llama.cpp inference. Everything fails with the same page fault:
amdgpu: [gfxhub] page fault (src_id:0 ring:153 vmid:8 pasid:35)
GCVM_L2_PROTECTION_FAULT_STATUS: 0x00800932
PERMISSION_FAULTS: 0x3 ← both read AND write denied
WALKER_ERROR: 0x1
MAPPING_ERROR: 0x1
Tested on every kernel I could find. All fail identically:
- Ubuntu OEM 6.17.0-1025-oem → PERMISSION_FAULTS 0x3
- Ubuntu OEM 6.17.0 + amdgpu-dkms 6.19.4 (AMD 31.30 repo) → PERMISSION_FAULTS 0x3
- Ubuntu mainline 6.18.9 → PERMISSION_FAULTS 0x3
- Ubuntu mainline 7.0.14 → PERMISSION_FAULTS 0x3
- Fedora 42 kernel 6.18.0-rc5 vanilla → WORKS (per kyuz0/amd-strix-halo-toolboxes benchmarks)
Also tried every env var and kernel param people suggest:
- amdgpu.noretry=0 → no effect, XNACK stays NO regardless
- HSA_XNACK=1 → no effect
- amdgpu.vm_fragment_size=9 → no effect on permissions
- GGML_HIP_UMA=OFF (forces regular hipMalloc instead of SVM) → same faults
- amd_iommu=off + amdgpu.gttsize=126976 → GTT confirmed at 124GB, fault unchanged
Key finding: Checked kernel config on both Ubuntu kernels — they have identical flags to Fedora (CONFIG_HSA_AMD_SVM=y, CONFIG_HMM_MIRROR=y, CONFIG_DEVICE_PRIVATE=y, CONFIG_ZONE_DEVICE=y). The fix in Fedora's kernel is not a config difference. It's amdgpu driver code patches — presumably from AMD's drm-next/amdgpu-next branch — that haven't landed in Ubuntu mainline or the 6.18 stable series yet.
The fault PERMISSION_FAULTS: 0x3 means both read AND write are denied. The GPU driver is mapping memory into the GPU's address space but the page table entries are missing the r/W permission bits. gfx1151-specific bug.
Workaround: Ollama with Vulkan/RADV backend (HIP_VISIBLE_DEVICES=-1 disables broken ROCm, forces Vulkan). Running ~17.5 t/s on Qwen3-235B Q3_K_M. Not as fast as HIP but stable.
Can anyone confirm which specific commits fix gfx1151 page table permissions in Fedora's tree? Is this targeting a specific drm-next PR for 6.19? Would help users know whether to patch or just wait.
I have been building taOS, a self hosted operating system where you and AI agents work on projects together, and I wanted to share it and get some honest feedback.
The short version: it is a web desktop OS (windows, dock, files, an app store) that runs on your own hardware, anything from an Orange Pi up to a small cluster. The difference from a normal chat tool is that the agents are first class citizens of the OS. You deploy an agent and it gets its own identity, memory, and tools, and it lives alongside you in the workspace instead of in a throwaway chat tab.
Everything is organised around projects. You spin up a project, drop in agents, and they collaborate with you and with each other on it. There is a shared canvas next to the chat where an agent can show you a mockup, a comparison, or a set of options to pick from, plus a coordination bus so several agents can hand work back and forth without stepping on each other.
A few things I care about:
• Local first. Your data and your agents stay on your hardware. No cloud account required to use it.
• Framework agnostic. The agent harness is swappable, so you are not locked into one agent framework.
• Cluster aware. You can pair extra machines as workers and run agents and models across them.
• A real OS feel, not just a dashboard: themes, multi window, a mobile PWA, and an app store with things like an image studio and a browser.
It is still early and very much a work in progress, built mostly by me, so I would rather hear what is missing than oversell it. If you self host, or you have wanted your local models to actually do work for you instead of just answering questions, I would love to know what you would want from something like this.
Happy to answer anything in the comments.