u/Stock-Pepper4884

So recently I decided that it would be nice to run my agent against some popular benchmarks. And oh my god, the cost to run a single benchmark, such as terminal-bench or swe-bench will cost you thousands of dollars in tokens just for a single run. And obviously you want to run multiple times to get some average result eventually.

Running terminal-bench with opus 4.6 might cost you up to $40k. Just one run.

Wtf is that? Anyone knows some popular benchmarks that will not put you $200k down to get a reasonable output?

Yes, we do.

We have a new harness that is looking exceptionally good.

One example of a win we have, is our agent design consistently hits 95% to 99% cache hit rate.

The project itself looks absolutely crazy in capabilities.

If you happen to be a developer who wants to get into this game, and to have a party for that, let me know.

What the title is saying.

AI agents are a big deal, and I think everyone wants something of their own out of it. The capabilities are pretty much endless in what you can build with it. Current agents are just scratching the surface of what can be achieved.

But the problem is community, how do you bring people in?

How do I make the environment safe?

What stops people from contributing usually?

I see some people complain about some frameworks or harnesses, what stops you from contributing the feature you need right into hermes/opencode/whatever ?