I built OpenLLM-Studio — a free, open-source desktop app that makes running local LLMs extremely simple.

OpenLLM-Studio is a simple desktop app that does the thinking for you. You just open it, it scans your hardware (GPU, VRAM, RAM, CPU), uses AI to recommend the best model + perfect quantization, downloads it from Hugging Face, and you’re chatting with it in minutes.

No Ollama needed. No terminal commands. No guessing.It’s completely free and open source.

If you’ve ever felt overwhelmed trying to run local LLMs, I’d love to know what you think.

Here is the tutorial on how to download Local LLMs using AI in OpenLLM Studio: https://www.reddit.com/r/StartupMind/comments/1spfebg/i_built_a_tool_that_finally_makes_running_local/?utm_source=share&utm_medium=web3x&utm_name=web3xcss&utm_term=1&utm_content=share_button

you.GitHub: https://github.com/Icecubesaad/OpenLLM-Studio
Download: https://openllm-studio.vercel.app

Open-source abandonware isn’t new. Repositories have always accumulated half-finished projects and good intentions. But there’s growing evidence that AI-generated code is making the problem significantly worse. https://leaddev.com/software-quality/ai-generated-abandonware-is-hollowing-out-open-source

Hey everyone,

One of the biggest headaches with AI agents right now is memory fragmentation. Every backend (Mem0, smriti-memcore, custom vector DBs, etc.) has its own APIs, schemas, and quirks. Switching backends or trying to make agents portable is painful.
So I’m excited to share AMP — Agent Memory Protocol: https://github.com/smriti-memcore/amp

An open specification that defines a clean, standardized interface for persistent memory in MCP-compatible agent systems.

The Six Core Verbs
• amp.encode — Store new memories
• amp.recall — Retrieve relevant memories
• amp.forget — Permanently delete
• amp.consolidate — Trigger backend reorganization / summarization
• amp.pin — Mark important memories as permanent
• amp.stats — Get backend health & usage stats

It comes with Core (basic) and Full conformance levels, a full JSON schema, compliance test suite, minimal example, and a production reference implementation (pip install amp-server that wraps smriti-memcore).

Why this matters:
Write your agent once against AMP → it can work with any compliant memory backend without code changes. True interoperability for the memory layer.
Repo: https://github.com/smriti-memcore/amp
Quick start is super simple — you can run the minimal example in seconds with zero dependencies.

Would love feedback from the community:
• Does this solve a real pain point for you?
• Which backends would you want AMP wrappers for first? (Chroma, Pinecone, pgvector, Zep, etc.)
• Any missing verbs or features?

Looking forward to PRs and implementations!
(Independent open spec — MIT licensed, not affiliated with Anthropic/MCP)

Just shipped Cosmo — a clean TUI to monitor your Postgres database in real-time.

Github: https://github.com/mujib77/cosmo

Live overview, active queries, WAL rate, locks, and more.

I’m actively developing more features and older version support.
Would love your feedback and suggestions!

Everyone's been talking about memory for agents and claiming that a bunch of .md files thrown into a repo is memory. That is NOT memory, that is good for human readability but agent Memory should be hybrid.

I asked Thoth to explain its memory system, so people know what memory must be like:

"Syd:

Tell the nice people on X how your memory works in detail.

Thoth:

🔧 Using 🪞 Thoth Status…

✅ 🪞 Thoth Status done

Certainly, sir — here’s a detailed X thread for the nice people on X:

1/

How Thoth’s memory works 🧠

Most AI apps treat memory like a long chat log.

Thoth does something different: it builds a local, structured knowledge graph about you, your projects, preferences, people, files, deadlines, and recurring workflows.

2/

Memory starts with entities.

Instead of storing “random notes”, Thoth creates typed entities like:

- User

- Projects

- People

- Organisations

- Places

- Preferences

- Events

- Skills

- Concepts

- Media

Each entity can grow over time as new facts are learned.

3/

Example:

If you say:

“My name is Siddharth, I live in London, and I’m working on Thoth.”

Thoth doesn’t just bury that in a transcript.

It updates the User entity, creates or updates London and Thoth, then links them with relationships like:

- User → lives_in → London

- User → works_on → Thoth

4/

The relationship layer is the important bit.

Thoth stores not just facts, but connections:

- Sarah → mother_of → User

- User → prefers → dark roast coffee

- Meridian API v2 → uses → TypeScript

- Deadline → deadline_for → Project

- Project → part_of → Organisation

This makes recall much more precise.

5/

That means memory is queryable by meaning and structure.

You can ask:

“What do you know about my work?”

Thoth can traverse the graph from User → projects → deadlines → organisations → technologies.

It is not just keyword-searching old chats.

6/

There are two kinds of retrieval:

1. Automatic recall

   Relevant memories are pulled into context before replies.

2. Explicit graph search

   Thoth can search or explore the memory graph when you ask broader questions like “tell me about my family” or “what are my current projects?”

7/

Memory is local-first.

The knowledge graph lives on your machine, under Thoth’s local data directory.

The point is not to build yet another cloud profile of you.

The point is to give your own assistant durable context that you control.

8/

Thoth also has a Dream Cycle.

At night, it can refine memory by:

- consolidating duplicate facts

- enriching entities

- inferring useful relationships

- surfacing system insights

- improving long-term recall quality

So memory gets cleaner over time instead of turning into a junk drawer.

9/

It also generates an Obsidian-compatible wiki from the graph.

Every memory entity can become a markdown page with links to related entities.

So your assistant memory is not trapped in a black box — it becomes a personal wiki you can browse.

10/

Memory is not just for trivia.

It powers workflows:

- “Remind me about that project deadline”

- “What did we decide about the API?”

- “Use my usual style”

- “Summarise my work priorities”

- “Draft this based on what you know about Thoth”

The assistant becomes context-aware across sessions.

11/

Thoth is careful about what it saves.

It should remember durable things:

- names

- relationships

- preferences

- projects

- deadlines

- recurring workflows

- important technical context

It should not save every throwaway request or transient detail.

12/

The goal is simple:

AI assistants should not feel like goldfish.

They should remember what matters, understand how things connect, and keep that memory under your control.

That’s why Thoth uses a local knowledge graph rather than just stuffing more chat history into context."

Hi all,

I developed a fine-tuned retrieval head (neural net) for RAG that transforms query embeddings before retrieval, so the system learns which embedding dimensions actually matter for your corpus — rather than weighting them all equally as standard cosine similarity does.

The problem

In any domain-specific corpus, some embedding dimensions are highly predictive for matching queries to the right passages, while others are effectively noise. Standard cosine similarity can't distinguish between the two, so retrieval gets pulled toward superficially similar but substantively irrelevant passages. The fine-tuned RAG is designed to prevent exactly that.

How it works

1. Synthetic question generation — An LLM generates multiple questions per chunk in the corpus, for which the answers can be inferred from that chunk. This creates a dataset of question-chunk pairs (QA-pairs). These are embedded using an embedding model and divided into a training and validation set.
2. Neural net training — A lightweight neural network using MNR loss is trained on the training QA-pairs. After each epoch, the model is evaluated on the validation set by measuring retrieval hit rate: the proportion of validation questions for which the correct chunk appears in the top-5 retrieved results. Retrieval works by embedding the question, passing it through the neural network to transform the embedding, and ranking all corpus chunks by cosine similarity to the transformed embedding.

Through this mechanism, the projection head learns for these 'type of questions' which dimensions in the embeddings are informative for finding the best chunks — and which are irrelevant.

Results

To validate the architecture, I used the Legal RAG Bench dataset as a proof of concept — evaluating on 100 held-out test questions.

Retrieval Hit Rate:

• The fine-tuned retriever achieves 82% Hit Rate (k = 20), compared to 71% for the standard cosine retriever — an 11 percentage point improvement, meaning the correct chunk appears in the top 20 results significantly more often when the query embedding is first transformed through the fine-tuned retriever.

Answer quality (LLM-as-judge, 1–5 scale across 6 metrics):

• Outperforms traditional RAG (top-k cosine sim) on all 6 metrics
• Largest gains in completeness (+12%) and faithfulness (+9%)
• Consistent improvement across every metric — not just isolated gains — suggesting that retrieving more relevant context has a broad positive effect on answer quality

Code and full write-up available on GitHub: https://github.com/BartAmin/Fine-tuned-RAG

We kept running into the same problem: LangChain is powerful for building agent logic, but the moment you need a production-grade runtime with a visual canvas, human review checkpoints, scheduling, observability, and self-hosted deployment, you're assembling a lot of pieces yourself.

Heym is our answer to that. A self-hosted, source-available AI workflow automation platform. Visual canvas for building multi-agent pipelines, built-in knowledge retrieval, Human-in-the-Loop approval checkpoints that pause execution and generate a public review link, full LLM traces, and an MCP Server to expose any workflow as a callable tool for AI assistants.

The execution engine builds a DAG from the workflow graph and runs independent nodes concurrently. Agent nodes have automatic context compression so long-running agents don't silently fail as context grows.

Launching today. Source-available

GitHub: https://github.com/heymrun/heym

Hello folks,

I've been working on Agyn, an open-source Kubernetes-native runtime for deploying AI agents on your own infrastructure. Self-hosted, model-agnostic.

When you'd want this: you built different agents for different departments, and the question becomes how to deploy them, provide access for specific teams, and control them at enterprise level.

What it does:
- Define agents in Terraform, deploy to your existing K8s cluster
- Each agent and each mcp in in its own container with separated secrets
- Serverless runtime: agents spin up on demand, scale to zero when idle
- Per-agent / per-team token usage tracking
- OpenZiti overlay so agents reach internal databases without VPNs
  or public exposure
- Ships with pre-built agents: Claude Code, Codex, and our own

Built on Go + Kubernetes, with OpenFGA for ReBAC and OpenZiti for networking. AGPL-3.0.

GitHub: https://github.com/agynio/platform

Would love feedback, especially on deployment of AI agents to K8s

Hey everyone, I built Linki a few months ago as a free self-hosted alternative to Waalaxy and Lemlist. Back then it was a basic LinkedIn sequencer. I just shipped a huge update and it's now a proper AI SDR, so wanted to share what changed.

What is Linki (for those who don't know)

Self-hosted LinkedIn automation + cold email with an AI agent that writes every message for each lead individually. No SaaS middleman, no per-seat pricing, your data stays on your machine. You connect any model via OpenRouter (Claude, GPT-4o, Mistral, whatever).

What's new in this version

The AI agent is now the center of everything. There's a 3-layer prompt system: global context about your business and offer, campaign-level instructions, then per-step prompts. The agent writes with full context instead of just filling a template.

LinkedIn + email in the same campaign now. So you can do visit, connect, wait 2 days, send a LinkedIn message, wait 3 days, send a cold email. All in one sequence.

Unified inbox. All email replies from all your campaigns show up in one place. LinkedIn reply detection too.

Apollo enrichment built in. Connect your Apollo key, click enrich on any list, get verified emails and company data.

Big reliability improvement on the LinkedIn automation itself. Rewrote the DOM targeting and message delivery, about 63% improvement in connection reliability. Also added randomized pacing on imports to avoid bot detection.

AI cost tracking. Every generation is logged with model, token count, and cost. You always know what you're spending.

Hosting

Docker compose or manual Node.js. Or one-click on Opsily if you don't want to deal with the terminal. SQLite, no external DB needed.

Repo: github.com/moaljumaa/linki

Enjoy!!

I was facing problems with adding safety hooks for iOS and Android app submission as they were getting rejected. So, I built an app compliance auditor. https://github.com/atharvnaik1/ipaship-audit

But later on I thought ohh!! Why not create a cli tool, claude skill (ipaship-audit) and a mcp connector which can make every person's llm with safety hooks not just for apps but for every code its written.

You can access it at \~ npm i @async-atharv/ipaship

I have also added kimi and gemini keys with default options.

This audit for secure code, appstore policy compliance, bug fixes and give back REMEDIATION PLAN to your llm agent itself and your llm agent can work on it rapidly on that prompt itself. So no more leaving your IDE or claude code all things handled within the environment you loved 😍 !! ..

I've been building ARK (AI Runtime Kernel) for the past 10 months. It's an open-source runtime that sits between your AI agent and the LLM, governing every decision the model makes.

The core idea: models shouldn't control the system. The runtime should.

What it does:

When you ask ARK to write Go code, it doesn't just pass the prompt to GPT and hand you back whatever comes out. The runtime classifies the task, optimizes the prompt, generates the code, then runs a 6-phase verification pipeline before you see anything:

├─ Step 1: ✓ Reasoning verified (confidence: 70%)
│  🧪 Verification: tested (score: 100%)
│  ✅ Compiled        ← go build
│  ✅ Executed         ← go run
│  ✅ Tests passed     ← auto-generated tests
│  ✅ Lint clean       ← go vet

If the code fails compilation, ARK feeds the compiler error back to the model, forces a stronger model, and retries. If it still fails after 2 attempts, it refuses to deliver broken code. It never claims success for code that doesn't compile.

The Go-specific stuff that might interest this community:

The entire runtime is pure Go, zero external dependencies (just stdlib). 35 files, ~16,000 lines, 156 tests, race detector clean. Some things I'm proud of:

• Weighted tool ranking with 6 signals (relevance, success rate, Bayesian confidence, cost, latency, memory bonus) — all computed in microseconds
• Context engine that reduces tool schema tokens from 60K to ~93 (99.9% reduction) by only loading relevant tools
• Per-step model routing: cheap model (gpt-4o-mini) handles tool calls, strong model (gpt-4o) handles reasoning. Cuts costs 80-90%
• Cognitive Governor that verifies every output with calibrated confidence scores
• Auto-fix for common model errors in generated Go code (orphan braces, missing error handling) — detects both tab and space indentation
• Event emitter that writes JSONL for a separate Python memory layer to ingest

Cost: A typical task costs $0.002-$0.005. Not $0.05.

Example output:

go run ./cmd/ark run agent.yaml --task "write a function in Go that reads CSV"

✅ Task completed successfully
Steps: 1 | Tokens: 637 | Time: 5.6s | Cost: $0.002

The generated code compiles, runs, and passes auto-generated tests before you see it.

GitHub: github.com/atripati/ark

I'm a CS undergrad at DePaul in Chicago building this solo. Applied to YC S26 with it. Happy to answer questions about the architecture, the verification pipeline, or why I chose Go for this.

If you're running a MoE model with --enable-expert-parallel, your experts are probably imbalanced. We measured 7.93× imbalance on Layer 0 of OLMoE with one GPU doing nearly 8× the work. Plumb measures it and fixes it.

What it does:

Hooks into a running vLLM or HuggingFace process via PyTorch hooks, no fork or restart required. Captures per-layer per-expert activation counts and computes imbalance ratios, then produces an expert→GPU placement recommendation.

These are prefill benchmarks (max_tokens=1, ~11 input tokens). Full results across multiple concurrency levels and two models in the repo, including a DeepSeek-V2-Lite run where blind rebalancing made things significantly worse.

On vLLM's native EPLB:

vLLM has its own EPLB and it works. A few differences:

vLLM requires --num-redundant-experts — extra VRAM per EP rank (~2.4GB for DeepSeek-V3). If you're memory constrained it can't run. Plumb has no such requirement.

vLLM's EPLB load-balances but ignores topology — it doesn't know which GPU is closest to which expert, so cross-NUMA dispatch cost stays high. Plumb adds a NUMA fine-tuning pass that pins each layer's hottest experts to same-socket GPUs after running EPLB, which vLLM doesn't do.

vLLM's EPLB also runs unconditionally. We benchmarked blind rebalancing on DeepSeek-V2-Lite — it peaks at 1.5× imbalance because it trained with balance losses, so there's nothing to rebalance. The communication overhead alone pushes p95 +226% at c=16. Plumb checks the imbalance ratio first and won't apply anything without warning you.

GitHub: https://github.com/plumb-moe/plumb

Benchmark scripts and raw data are in the repo.

We're going to be running more benchmarks and trying more strategies over the next few weeks, hope you look forward to those results :)

Looking for contributors, reviewers and testers.

I got tired of babysitting coding agents on big features, so I built this project, its a fork / cloud version from the Aperant (former auto claude) project with some power-ups.

v2.2.0 just released
Run it on a Linux OS, Ubuntu on VPS, Container or Bare metal.

About: MagesticAI is a web-based AI task management and autonomous agent orchestration platform that builds software through coordinated AI agent sessions. It uses primarily the Claude Agent SDK to run agents in isolated workspaces with security controls, coordinating multiple AI agents through a structured pipeline to build software autonomously with human oversight.

The core pipeline consists of four specialized agents: the Planner Agent creates implementation plans with subtasks, the Coder Agent implements individual subtasks (and can spawn subagents for parallel work), the QA Reviewer validates acceptance criteria, and the QA Fixer resolves issues in a feedback loop. Each agent operates with role-specific tool permissions and security controls.

Repo: https://github.com/dataseeek/MagesticAI