Run an open beta for the AI update
Cross-posting from the Discord:
Every time there’s a rocky release somebody suggests an open beta, but I think this would be pointless 99% of the time. If 1.13 had an open beta nothing would be different today. The beta would have the same bugs and people would play 1.12 if they didn’t want to put up with them, just like they can today.
Not to mention the reaction to the fundamental design of the new systems has actually been quite positive, so it’s not like an open beta would’ve led to something completely different on that front. Some QoL and UX feedback would’ve been useful, but there was no risk of needing to start over.
An AI update is different. While there’s a lot of things about the AI that can be tested by a small team with observer mode, like GDP benchmarks, a huge amount of what makes an AI successful comes down to vibes. For an AI update to be received as a success players will need to feel like the AI is performing better, and making decisions that feel logical and realistic. How the AI behaves and how that behaviour is perceived is completely intangible, and I just don’t think you can effectively judge if AI changes are having the desired effects on that front without putting it in front of more players.
Internal testing will let the team know if AI is succeeding at the numbers games, but an open beta would allow the team to better assess whether it’s giving off the right vibes and iterate accordingly.
A release outside of the DLC cycle is the perfect opportunity to run an open beta, and depending on the timing it makes the July holidays less of a looming deadline. Instead of rushing to get an update out before the holidays and being unable to fix it over the break, a beta can be left open until the team returns.