I am sharing a Go project I am building from past few months called knowledgeMesh. Short version is this. It is a P2P inference mesh where buyer sends OpenAI-style chat completion requests to a local API on 127.0.0.1:8080, and seller runs the actual model on their own Ollama somewhere else on network. A small control pane handles auth, matching and accounting. Every piece is self-hostable only.

Why I am building this when Ollama is already there. Ollama is good if your laptop can run the model you want. Mine cannot. I am having one GPU box which can run, but exposing it across LAN with proper auth and quotas was either hand-rolled reverse proxy or some third-party service. I wanted small process on laptop, small process on GPU box, and the local API on laptop should speak OpenAI's protocol itself so existing tools like Open WebUI, LibreChat or the official SDKs can work without any changes.

What is there in the box:

• OpenAI and Anthropic style HTTP APIs on buyer process. Point any OpenAI-compatible client to http://127.0.0.1:8080 and it will work.
• Sellers expose models over libp2p with QUIC. AutoNAT v2, DCUtR hole punching, relay fallback, network-change watcher are all there. Sellers sitting behind home NATs can reach buyers without doing any port forwarding.
• Postgres-backed control API with versioned migrations (golang-migrate), JWT auth, wallets, quotas and a transaction ledger. Migrations run on startup itself.
• Matchmaker picks sellers by skill, duty status and price cap, then ranks by cheapest with reputation as tie-breaker. Settlement is idempotent per request ID so retries are not double-charging.
• Separate relay binary for circuit relay v2 if your sellers cannot accept direct dials.
• Sandbox mode (knowledgeMesh serve) runs the buyer HTTP API with mock inference for UI testing without Postgres.

What is not there yet:

• No web UI for control pane. CLI only.
• No payments integration. Wallet is internal credit only.
• No metrics exporter. Logs only.

Stack is Go 1.24, libp2p (QUIC/TCP), Postgres, Cobra CLI, golang-migrate.

Two-minute try, no Postgres or Ollama needed (this uses a hosted mesh for the demo):

go run ./cmd/buyer register --name "you" --email you@example.com --password 'pw'
go run ./cmd/buyer serve --email you@example.com --password 'pw' &
go run ./cmd/buyer prompt --email you@example.com --password 'pw' \
  --api-url http://127.0.0.1:8080 --prompt 'haiku about distributed systems'

Full local stack walkthrough (Postgres, control, relay, seller, buyer, around 15 minutes) is there in the README.

Repo: github.com/Knowledge-Mesh/knowledgeMesh

Few caveats before you clone. It is early. The CLI will change. Hosted public DB is shared so emails need to be unique. Also the go.mod path is not resolving via go install currently, so you have to clone and build only. On the hosted mesh your prompts will cross someone else's seller, so for anything serious kindly run your own stack.

If anyone here has tried sharing inference with friends across NATs before and hit walls, kindly revert with what broke for you. libp2p side has been the most interesting part to debug and I am suspecting there are still some rough edges I haven't found.