u/kwizzle

A while ago, I posted here getting OSS-120B and GLM-4.6 playing full games of Civilization V. Since then, models have moved pretty far, and we wanted a better understanding about models' capabilities playing the game.

Introducing the controlled version of CivBench on newer models:

The controlled version of CivBench (Chen et al., 2026, extending our COLM 2026 work)

We are currently testing GPT-6.1-Sol, GPT-6-Astra, etc. Feel free to suggest some models (especially interesting open-weight ones) for our next run!

What is Civilization? Civilization V ($7.49 today on Steam promotion) is a turn-based strategy game where you take a civilization through hundreds of turns of expansion, science, diplomacy, war and eventually the space age. That makes it useful for testing something LLM benchmarks often struggle with: decisions whose consequences may not show up until 50 or 100+ turns later.

A LLM strategist playing as Byzantine. Can Theodora rebuilt Rome?

What makes this a controlled experiment? Instead of giving each model unrelated games, we rotate them through the same three fixed starts. Each game has eight civilizations: two using the tested LLM strategist and six using the standard Vox Populi AI. The LLM sets high-level strategy; Civ's existing AI handles low-level execution.

Can I see how the models actually play? Yes. A few examples:

• GLM-5.3: China, Cultural victory
• Opus-5.5: Morocco, Cultural victory
• Qwen-3.8-27B: China, Science victory

Can I play a round now? Yes. If you own the game, Vox Deorum is open source and has an installer. You can play Civilization V yourself against LLM-powered civilizations, watch a full AI-vs-AI game, or even chat with your opponents. You can also have LLMs as your teammates and work together towards a win!

Guess I can't avoid an unequal treaty as a pacifist. At least I can get a bargain?

Can I use local models or my existing subscriptions? Yes. Local OpenAI-compatible servers are supported, and Qwen-3.8-27B can do an excellent job. You can also use your existing Claude or Codex subscriptions. (I use them to run a ton of evaluation games! GPT-6-Luna is basically free to play. About $0.5 in API cost per player per game.)

What else did you learn? Please check out our COLM 2026 paper for methodology and EMNLP 2026 paper for whether models would authorize nuclear strikes on others. I guess Civilization is just a game, don't you think so?

Can we at least have a chat, please?

(Sorry for sending and deleting this repeatedly. Guess I shouldn't use in-flight wifi to send a post with many pictures. I hope they go through! Please let me know if you can't see them.)

I'll start. Thousands of hours in, I only just discovered that if you have an agreement to not settle new cities close to another player and then you ask them to not settle near you, then you are no longer beholden to your initial promise. Works the other way around as well.

Not sure if that's VP specific but it's been years that I only play VP so I don't know what's a vanilla mechanic and a VP mechanic anymore.

I'm half way through the renaissance and I have 50/100 spy points and zero spies. How do I get more? I'm playing 5.4 and haven't played for a while.

I've reinstalled the intel drivers. Had error 43 and ran a script I found on a forum to fix the error. I updated the motherboard firmware to latest. Checked every cable connection and card seating more times than I can remember. Enabled Thunderbolt Pcie tunneling in the BIOS as well as discrete thunderbolt.

I'm using the thunderbolt cable that came with the Razer core x v2. I tried two different GPUs (a 3090 and a 3060) that are both cards that work. I have an 850W PSU (Corsair RM850x) in the eGPU which should provide more than enough power.

I don't know what else I can try. Does anyone have any ideas?

I was wondering if anyone had experience pairing a 4090 with another card to get a bit more VRAM without losing too much speed.

I know all about how this card or that card has x y z tensor cores or memory speed but I want to know what your experiences are. For example I get around 40-45 tok/s on qwen 27B, so if I were to pair it with a 3090, 5070 or 5080 would I be able to get a similar token rate?

I got a code from Eurail in the mail that I could use to sign up for identity protection with Transunion on their website following the data leak.

I entered all my info then after the last step I got an error message saying that the website was having problems and I should try again later. So I tried the next day and same problem. I tried different browsers as well and still the same problem.

I then called and their support told me to just try again later and I had to repetitively and firmly tell the customer support agent that I was not going to try yet again that that he was going to help me sign up. To his credit once he understood that I was not going to hang up he did put in a decent effort to help me. Unfortunately he was not able to get my account working but he did arrange for a manager to call me a few days later.

I did get a phone call a few days later and it took a while but I was finally able to sign up and get identity protection.

So if you're having trouble signing up don't give up, call them and insist they help and if the first line agent can't figure it out insist on a call back from a manager.