Screeps Reinforcement Learning
▲ 21 r/screeps+1 crossposts

Screeps Reinforcement Learning

I haven't been playing the game for awhile, doing other programming things like machine learning instead. I've learned a lot, especially from Kaggle challenges, and decided to try my hand at a machine learning in Screeps.

Code link: https://github.com/CarsonBurke/xxscreeps/tree/main/samples/rl

First, I pretrain the model on my bot, behavioural cloning so it just tries to copy what the bot does using a bunch of data I collected running it. It can perform all kinds of actions but isn't very efficient or directional.

Then I do reinforcement learning on it. I reward it for harvesting and upgrading and expect the rest to be emergent, which sort of happens. It gets really good at harvesting and upgrading, does stationary harvesters and hauling really well, but also learns to stop building eventually. The rollouts are too short that it doesn't end up seeing the benefits of building. Regardless, I'm happy with the results given my budget.

To get more technical, it's a ~1.5M parameter model with a ViT over patches room tiles (terrain, sources, controller) and entity transformer for the entities (creeps, spawns, towers) each getting a head that outputs actions for the entity. Inspired somewhat by AlphaStar, though I figured autoregressive actioning and temporal recursion would be too expensive for marginal benefit, so I skipped them.

Then a separate 1.5M parameter model critic where I take a VAPO style approach of making the critic Monte-Carlo so it can get signal from the whole rollout for itself and also the actor's advantages. It hurts learning speed a bit, but makes it so the model can learn from very far out returns.

xxscreeps allows me to do really fast parallel rollouts. I think I did 12 games at once for 512 steps, each taking under a couple seconds. If I used the normal screeps engine it would have taken days to train. This partly avoids the immense parallelism that Ben had to do in his approach.

There were a lot of compromises to get it training fast on my single RTX 5090. If I had a bigger budget I would have done longer rollouts to allow it to build and explore more game features. It should be able to claim, colonize and expand too. Maybe a project for someone with more resources than me.

Thanks to Ben for the inspiration!

youtube.com
u/MarvinTMB — 2 days ago

I created this a few weeks ago and have been using it periodically for stock research to get more breadth in my watchlist, and to examine my portfolio for blind spots. By no means is it a replacement for manual research, but it's a useful tool.

Example output:

T1 Energy (TE) — Analysis

T1 Energy Inc. (NYSE: TE, formerly FREYR Battery) is a U.S. vertically-integrated solar + battery manufacturer.

Price Action (last month: 3/26 → 4/24)

  • Close 4/24: $5.29 — down ~16% over the month, in a sharp V-shape.
  • 3/31: -22% gap-down on Q4 earnings miss ($6.50 → $4.39 in two sessions).
  • 4/7: Bottomed at $3.93.
  • 4/14: -10% on $125M convert announcement (later upsized to $160M).
  • 4/15–4/24: Rebounded ~35% off the low as the dilution overhang cleared.
  • 6-month context: well off the $9.78 high (1/28) but well above the $2.52 low (11/21). 12-month range roughly: $2.50 → $9.78.

Key Events

Date Event
12/17/25 Started construction on G2_Austin 2.1 GW solar cell fab ($400–425M Phase 1)
12/30/25 First $160M sale of Section 45X tax credits — FEOC compliance restructure with Trina
3/18/26 Secured 50MW Nordic data-center grid allocation (legacy FREYR asset)
3/31/26 Q4 miss: Rev $358.6M (est $368.2M), loss 87¢/sh (cont-ops 70¢ vs 10¢ est)
4/2/26 Insider buying: Chairman Barcelo +333k shares, CFO Calio +303k. Needham PT $10→$8, maintained Buy
4/14–17 Priced $160M of 4% convertible senior notes due 2031, conv. price $6.80 (40% premium); $151.6M for G2_Austin

Sentiment & Evidence

  • Cautiously constructive but bruised. Post-earnings drop, Needham PT cut, and convert announcement are bearish; the rapid rebound, large insider buys two days after the gap-down, and maintained 2026 production guidance (3.1–4.2 GW) are bullish.
  • Volume confirms — 3/31 down-day (26M shares), 4/14 selloff (34M), and 4/15 + 4/22 rallies (25M each) all on heavy turnover suggest active institutional repositioning, not just retail.

Controversy & Risk

  • Liquidity vs capex: $182.5M unrestricted cash at YE25 against $400–425M Phase 1 capex; mgmt explicitly seeking "larger financing solution with significant debt component" beyond the convert. More dilution likely.
  • FEOC / 45X dependency: Business model leans heavily on Section 45X tax-credit monetization. Restructuring with Trina maintains eligibility, but it is policy-sensitive in 2026.
  • Execution: G2_Austin Phase 1 not producing cells until end of 2027 — long runway, large checks to write first.
  • Margin quality: Q4 EPS miss far worse than the revenue miss implies — points to cost/start-up drag at G1_Dallas.

Bull Case

  • Only credible U.S. silicon solar-cell pure-play; G2_Austin Phase 1 alone reportedly exceeds existing U.S. silicon-cell capacity.
  • Insider conviction (chairman + CFO 6-figure buys post-earnings) is a real signal.
  • Vertical integration (Texas cells → Dallas modules) + 45X economics = potential cost moat if executed.
  • Run-rate: $358M Q4 revenue from $2.9M y/y shows the Trina-acquired module business is real and scaling.

Bear Case

  • Cash burn + dilution treadmill; the convert is the first of several financings needed.
  • Solar sector facing Chinese oversupply pricing and U.S. policy risk.
  • Q4 miss suggests cost discipline still poor.
  • Stock has round-tripped from $9.78 to ~$5 — supply from underwater Jan/Feb buyers will cap rallies.

Bottom Line

A high-beta thematic play on U.S. solar reshoring with real revenue but real solvency questions. The 4% convert at $6.80 effectively defines a near-term ceiling around the conversion price; insider buying near $4 effectively defined a floor. The $4 → $5.50 zone is contested territory; a confirmed break above $6 would signal the financing overhang has cleared.

Sources:

u/MarvinTMB — 4 months ago