u/Any-Surprise-9336

I’m the developer behind CIKADA, a KBO baseball analytics and community site. I wanted to share the evaluation results and get feedback from people working on sports probability models.

The site brings together game schedules, matchup win probabilities, starting-pitcher and bullpen information, archived predictions and results, and a fan discussion area.

For the 2025 evaluation, the model used 27 numerical features with logistic regression, trained on 2024 data and tested on a separate 2025 dataset.

The recorded results are:

• 2025 OOS: 618 evaluated games, 368 correct, 59.55% accuracy, Brier: 0.2385.
• 2026 historical replay/reference dataset: 494 evaluated games, 284 correct, 57.49% accuracy, Brier: 0.2455.

The 2026 dataset is a separate retrospective research sample and may contain different model/data sources. It should not be treated as verified live performance or a continuation of the same frozen model’s OOS record.

These are prediction metrics, not betting ROI. I’d welcome feedback on calibration checks and how to structure a fair comparison against timestamped market probabilities.

Disclosure: I built and operate the site.

CIKADA — cikadabaseball.com

I've been building CIKADA, a website that lets readers explore KBO game predictions and compare them with recorded outcomes.

The site includes:

• Matchup-level win probabilities for both teams.
• Game schedules, starting pitcher information, and bullpen information.
• An archive of prediction records and results.
• Historical evaluation information, including hit rate and Brier score.
• A fan discussion area alongside the analytical content.

The purpose is to let readers inspect more than a predicted winner. They can see the probability assigned to a matchup, explore the game information, and review how past predictions performed.

The site presents its 2025 out-of-sample evaluation separately from current predictions. Retrospective reference records also need to be distinguished from predictions recorded before a game; those aren't interchangeable evidence of model performance.

I spent about eight months on this alongside my day job. AI helped with implementation, while much of my own time went into checking game identities, matching results, and resolving inconsistent records.

Disclosure: this is my own project.

https://www.cikadabaseball.com

I'd welcome feedback on the probability displays and archive. Can you find enough information to assess a prediction, and what additional calibration or baseline comparisons would you want to see?

Hi everyone, I wanted to share a KBO website I've spent about eight months building alongside my full-time job.

It's called CIKADA. The idea is to bring game information, predictions, and fan conversations together in one place.

Here's what you can explore:

• Game schedules and matchups by date.
• Each team's predicted win probability for a matchup.
• Starting pitcher and bullpen information to help you look into the game.
• A prediction archive where you can compare recorded probabilities with actual results.
• A fan lounge for team discussions and sharing baseball stories and photos.

For example, you can check today's matchups, look at the probabilities and pitcher information, then visit the archive to see how previous predictions turned out.

The community side matters to me too. I'd like CIKADA to become a place where fans can discuss what they're seeing on the field alongside what the data suggests.

I'm 40 and wanted to learn by building something with AI. AI helped with development, but I spent a lot of the eight months checking numbers and correcting inconsistencies myself.

Disclosure: I built and run the site.

https://www.cikadabaseball.com

If you give it a try, I'd appreciate hearing which parts are useful and what you'd want added as a KBO fan.

I’ve been working solo on a KBO pre-game prediction model for a few months. The current locked 2025 OOS test is:

- 629 KBO regular-season games

- Brier around 0.240–0.243 depending on the locked variant

- hit rate around 57.6%–58.3%

These are not final multi-year numbers, and I’m not claiming edge from this alone.

The next step is to compare the locked model against Pinnacle closing moneyline prices on the same joined games. Before doing that, I’m trying to make sure the comparison is clean:

- only pre-first-pitch odds

- timestamp < scheduled start

- full-game moneyline only

- ties treated as void / excluded from binary Brier

- no changing the model after seeing market data

- paired Brier comparison on the same games

I previously found an old odds sample where many rows were updated after first pitch, which made the market look unrealistically accurate. So I’m being careful with timestamp validation before running the final comparison.

For people who have validated models against closing lines: is paired Brier vs closing price on the same games the right way to frame this? Anything else I should watch out for?

I’ve been working solo on a KBO prediction model for about five months now.

It started as a curiosity project, but it slowly turned into a pretty serious backtest / calibration study. I’m not doing this for a company or a betting group — just a personal quantitative project that got much deeper than I expected.

Right now, the locked internal model track is a fairly simple pre-game model. The current 2025 out-of-sample test looks like this:

- 629 KBO regular-season games

- Variant B: Brier 0.2427, hit rate 57.6%

- Variant D: Brier 0.2401, hit rate 58.3%

These are not final multi-year numbers. They are only from the locked 2025 OOS common-universe test. The reported Brier may move once I run the same locked protocol across 2024–2026 and once the final Pinnacle joined sample is fixed.

The numbers look decent internally, but I’m trying not to fool myself.

A model can look good against results and still have no edge over the market. So the next step is to buy historical Pinnacle KBO moneyline data and compare my locked model probabilities against Pinnacle closing lines on the exact same games.

Honestly, most of the work has not been “adding more features.” The hard part has been trying to make the comparison clean:

- making sure features are pre-game only

- avoiding post-game or post-start information

- handling doubleheaders properly

- handling ties / voids correctly

- comparing model and market on the same joined sample

- not changing the model after seeing the market data

- checking timestamps so that odds really existed before first pitch

That last point turned out to be more important than I expected. I checked an old KBO odds sample from another source, and many of the Pinnacle rows had been updated after first pitch. If I had used them blindly, the market looked absurdly accurate — something like an 88% hit rate. Obviously that was not a real pre-game market signal. It was a timestamp problem.

That made me slow down and lock the comparison rules before looking at the real Pinnacle data.

So the plan now is:

- seal the model variants and inclusion/exclusion rules

- pull Pinnacle full-game KBO moneyline closing data

- filter only prices that existed before first pitch

- exclude ties/voids from the binary Brier comparison

- compare model Brier vs Pinnacle closing Brier on the same games

- call it inconclusive if the joined sample or timing checks fail

The final result might be no edge. It might be inconclusive. Or maybe both locked variants beat the closing line on paired Brier.

I don’t know yet — that’s the point of doing this properly.

Mostly sharing because I think this is the part of sports modeling that people underestimate. Internal OOS results are useful, but at some point the model has to face the closing market.

Has anyone here done similar closing-line validation for a smaller baseball league or a less-covered market?

I’ve been working on a KBO probabilistic forecasting model as a personal project.

Current holdout Brier score is around 0.24.

Still very much a work in progress, but I’m curious if anyone here is interested in KBO prediction models, probability forecasting, or market efficiency.

Happy to share more if anyone’s interested.

I’ve been working on a KBO probabilistic forecasting model as a personal project.

Current holdout Brier score is around 0.24.

Still very much a work in progress, but I’m curious if anyone here is interested in KBO prediction models, probability forecasting, or market efficiency.

Happy to share more if anyone’s interested.!!

I’ve been working on a KBO probabilistic forecasting model as a personal project.

Current holdout Brier score is around 0.24.

Still very much a work in progress, but I’m curious if anyone here is interested in KBO prediction models, probability forecasting, or market efficiency.

Happy to share more if anyone’s interested.

I’ve been working on a KBO probabilistic forecasting model as a personal project.

Current holdout Brier score is around 0.24.

Still very much a work in progress, but I’m curious if anyone here is interested in KBO prediction models, probability forecasting, or market efficiency.

Happy to share more if anyone’s interested.