I’ve developed a method for ranking AI users instead of models—two developers face off on a new problem, the code is run (no LLM referee), and the one who makes the fewest moves, acts the fastest, or writes the best prompts wins
Benchmarks rank the model. But on a real team, the variable that actually moves output is the person steering it — and nobody measures that. "I'm good with AI" is an unprovable line on a CV. I wanted a scoreboard for the wielder.
So I built a prototype. The design:
- Two devs, one fresh problem, live. Each drives their own model over MCP — works with anything: local Llama/Qwen/DeepSeek/GLM/Kimi or hosted Claude/GPT.
- The solution is executed against tests — no LLM-as-judge (judges get gamed). Pass/fail is mechanical.
- Winner = fewest human-authorized moves, recorded on a hash-chained, tamper-evident log.
- Per-model leaderboards. A Glicko rating per model so it's skill-vs-skill, not compute-vs-compute. Cross-model duels settle the rest.
- No money, no stakes. The rank is the only reward — a public, verifiable record.
First real 2-player duel ran this week: both attacked "find the N+1 query," the verifier rejected the first answer (missed the batch-load) and accepted the rigorous one. Felt right.