False Advertising by Backgammon Galaxy + Computer Olympiad Challenge
TL;DR: If 1-ply is now “2-ply,” and “rollouts” are still 20 to 26 mP away from the rollout you’re comparing to, and that still somehow ends at “strongest analysis on the market,” then we’ve left science and entered branding. At that point just bring the engines back to the Computer Olympiad, plug them togeather and let Galaxy, GNU, and BGBlitz (and maybe XG and Sage) settle it the only way the universe respects... RNG dice and no marketing department in sight.
I got a chance to skim the Backgammon Galaxy Engine Analysis article, and it seems like there is a bit of false advertising:
1. The article states:"Free Members: Get access to Gnu 1-ply (called 2-ply on Galaxy)."
"Ply" is a standard term in game-tree search, so it already has a clear meaning. Calling GNU 1-ply "2-ply" is misleading because it suggests a deeper search than is actually happening. It also makes comparisons with engines like GNU, XG, and BGBlitz harder, since it breaks the standard terminology and can give the impression of extra strength that isn’t really there.
2. "Galaxy Gnu Rollout" and "Galaxy Gnu Deep Rollout." are not really rollouts
These names suggest they're comparable to full rollout analysis, yet they're still 26.5 mP and 20.2 mP away from the GNU Target Rollout benchmark. The article doesn't explain why a product called a "Rollout" is so different from its own rollout reference...
3. "Increased evaluation speed by more than 7x, enabling significantly deeper and stronger rollouts."
Without knowing what's under the hood, I would be skeptical of snake oil which makes something faster and more powerful, especially in the computing world. A faster engine doesn't automatically produce deeper rollouts. Unless the rollout parameters (trials, search depth, etc.) were actually increased, speed alone doesn't make the analysis stronger. The article never quantifies what became "deeper" (other than maybe their pockets after memberships).
4. "New and improved 4-ply eval."
From the table, the difference is 0.0005 of a difference (36.8 mP for GNU 4 ply and Galaxy GNU 36.3 mP). Small enough to be a statistical error... without doing a T test (assuming normal distribution, or another test) it's a joke to make such a claim.
5. Claiming to be the strongest and deepest analysis of any playform on the marke
Making this claim goes beyond what the methodology they used can support. The evaluation is based on equity prediction error against internal rollout benchmarks, but those benchmarks are not a single ground truth. Combined with a dataset built from user blunders and category sampling, the results are inherently skewed toward difficult, high-disagreement positions rather than a neutral cross-section of play.
More importantly, this still isn’t a direct comparison of engines under identical conditions. It’s an indirect proxy based on internal rollouts and calibration error.
If the goal is a definitive ranking of engine strength, the most convincing method remains head-to-head evaluation, seeing which engine wins after millions or billions of matches. The Computer Olympiad is a good setup, where engines like GNU, BGBlitz, and others were historically compared directly... maybe it's time for GB Galaxy, Sage and new XG to step up to the plate and face off.