u/pmv143

Worth knowing that it’s a dense model, not MoE, so unified memory setups (DGX Spark, Mac mini, MacBook) will struggle with it . might fit, but it’ll be slow.

Dense models need real GPU horsepower.

Try it out on InferX

By popular request — Qwen 3.8 27B is up on InferX at 50% off.

One thing worth knowing if you’ve been thinking about running it locally: it’s a dense model, not MoE, so unified memory setups (DGX Spark, Mac mini, MacBook) will struggle — it might fit, but it’ll be slow. Dense models want real GPU horsepower, which is exactly what we’re running it on.

Try it: https://inferx.net

Pay $1, get $5 in credits.

Token pricing is 50% off. V4 Flash input is $0.07/1M, output is $0.14/1M.

That $5? It goes forever. Hardly burns at all.
Sub-second cold starts. No idle GPU tax. Test it, iterate, actually build something.

inferx.net

Wanted to share results from a third-party stress test — not our own benchmarks, run by an outside team.

Setup: fixed 100 RPM, ramping target TPM from 0.10M up to 30.00M by increasing prompt size per request. 1,500 total requests.

Results: 1,495/1,500 succeeded (99.67%), zero-error all the way to 25M target TPM, highest observed successful throughput 18.41M TPM. Cache read ratio came in at 99.81%.

A handful of 429s (rate-limit rejections, not crashes) showed up starting at 22M and again at 30M — full stage-by-stage numbers are in the gallery if you want to see exactly where and how.
Happy to answer questions on methodology.

We’ve expanded the InferX lineup with some of the most requested open models:

⭐ DeepSeek V4 Flash
⭐ Mimo v2.5
⭐ Devstral 2 123B

Qwen3 Coder Next FP8
Qwen3.6 27B FP8
Qwen3.6 35B A3B FP8
Ornith 1.0 35B FP8
Gemma 4 31B FP8

To celebrate the launch, DeepSeek V4 Flash and Mimo v2.5 are available at nearly 50% below prevailing market pricing for a limited time.

As always, we’d love your feedback. Which models should we add next?

A few weeks ago I posted about replacing RAG with persistent KV cache. A lot of you resonated. We took it further now.

Here’s what we built on top of that.

You upload a PDF. We automatically convert it into skill experts. each one its own model, its own context, its own reasoning. One snapshot per section.

you can combine those experts into an orchestrator skill. Skills call other skills . your query automatically reaches the right expert. Cross-section queries hit multiple experts and synthesize.

The whole thing is exposed as an MCP server.

For example: take your company knowledge across legal, finance, HR, and product. turn each into a skill expert, combine them into one orchestrator, and query across your entire company knowledge base. Right expert answers every time.

No vector database. No embeddings. No retrieval step. No document size limit. 70-90% cheaper than loading everything into one context window.

Demo here:
https://youtu.be/2SIEk7ZX60w

Been building with Claude skills since SKILL.md launched. Loved the concept — wanted to take it further.

The problems with current structure:

Every skill you add loads into your shared context window. One expensive flagship model handles everything from summarizing a routine email to complex legal analysis. And every skill file sitting on your local machine runs with full system privileges — NVIDIA analyzed 42,447 public skills and found 26.1% contain security vulnerabilities and 5.2% are outright malware.

What we built:

Skill Function — skills that live in the cloud as callable endpoints instead of local files.

Instead of downloading a skill:

POST api.inferx.net/skills/saas-pricing

{
  "input": "B2B SaaS, $50 ACV, PLG motion"
}

→ Expert output returned. Instructions never exposed.

How it works technically:

Each Skill Function runs in its own isolated cloud context with its own dedicated model. A simple classification skill uses a 7B model. A complex reasoning skill uses a 70B model. The agent never forces everything through one expensive flagship model.

The orchestrator pattern — this is where it gets interesting:

[ USER INPUT ]
      │
      ▼
┌─────────────────────┐
│  ORCHESTRATOR SKILL │  ← reads your intent
└──────────┬──────────┘
           │
    ┌──────┼──────┐
    ▼      ▼      ▼
[Skill A] [Skill B] [Skill C]  ← each runs in isolated context
    │      │      │
[Model A] [Model B] [Model C]  ← right model per skill

One call from your agent. The orchestrator decides which sub-skill runs. Each sub-skill executes in its own dedicated context with its own model. Results aggregate back to you.

Works via MCP:

Subscribe to a skill → it auto-appears in your Claude Code, Cursor, or OpenClaw tool list through MCP discovery. No local installation. No environment variables. No manual config.

The benefits:

→ 70-90% lower inference cost — right model per task → Zero local security risk — no file access, no shell, no credentials → Unlimited context — skills call skills, each in isolated context → MCP native — auto-discovers in any MCP-compatible agent → Create your own — import any SKILL.md and run it as a protected endpoint

Try it free: https://inferx.net/skill-function

White paper with full technical architecture: [link]

Happy to answer any technical questions. (please feel free to roast the idea) :)

Go to https://inferx.net to subscribe

We’re opening a very limited beta for Sovereign Endpoints™.

$10/month

Dedicated instance
Tool calling enabled
Long context (260k)
No throttling
No queuing

Your own endpoint, not a shared pool
Built for agents that need to run for hours or days without constantly running into context limits or noisy-neighbor issues.

The goal isn’t to give you access to hundreds of models. The goal is to give you a reliable endpoint that behaves consistently when your agent is doing real work.

We’re keeping this rollout intentionally small while we collect feedback and usage data.
If you’re building agents and want to try it, let us know what you’re working on.

https://inferx.net

Open source model adoption is growing.

According to LangChain, 1 in 3 AI teams now use open models, up from 1 in 5 less than a year ago.

What’s interesting is that Qwen and Gemma have consistently been our top two model families on Inferx(Inferx.net)

When we launched our $10/month plan, we made a bet that developers wanted longer context, tool calling enabled by default, and dedicated instances rather than shared endpoints.

So far, that bet seems to be paying off.

Super exciting to watch open source models getting adapted so fast though . Especially Qwen.

Tired of your coding agent getting throttled mid-task?

We just dropped to $10/month.

Dedicated H100. Your instance. Nobody else’s.

OpenCode + InferX = no interruptions. Ever.

inferx.net

Rent a 16GB GPU and keep it running 24/7:

~$360/month.

Whether you’re using it or not.

Or run the same model footprint on InferX:

• Dedicated instance
• H100 performance
• Pay only when the model runs
• No idle GPU bill
• Longer context windows
• No noisy neighbors

AI infrastructure should charge for usage, not waiting.

Try it free → inferx.net