I couldn't find tick-level data for Polymarket, so I collected it myself. The model still loses money

I wanted to answer what I thought was a fairly simple question:

Does order-book microstructure contain enough information to beat Polymarket's 5-minute crypto markets after transaction costs?

Turns out the harder part wasn't training the model.

It was getting the data.

As far as I could tell, there wasn't any public historical dataset with tick-level quotes, trades and order-book updates for these markets, so at the end of June I started collecting everything myself.

The archive now covers BTC, ETH, SOL and XRP 5-minute markets.

For every trading day it contains best bid/ask quotes, trade prints, spot prices, and (from 20 July onward) full L2 depth. Everything is stored as daily Parquet files with manifests and measured uptime for each day.

Once I finally had enough history, I trained a regime-routed LightGBM ensemble using only information available at the decision timestamp.

The disappointing part is that the model actually behaves reasonably well.

Calibration is good.

Out-of-sample Brier scores are consistently around 0.06–0.10 depending on asset and market regime.

But the strategy still loses money.

Across roughly 4,300 simulated trades it finishes around -$277.

The hit rate is about 47%.

After spread and fees you need something closer to 49% just to break even.

That two-percentage-point gap turned out to be the entire problem.

Looking back, most of the important work wasn't machine learning at all.

I found that my simulator had been using fees that were 3.5× too low.

One validation gate wasn't actually filtering anything because of a bad threshold.

At one point the trading system quietly stopped opening new positions because several expired markets hadn't been marked as resolved yet, while every monitoring dashboard still showed the process as healthy.

Those bugs changed the conclusions far more than another week of feature engineering.

I'm curious whether anyone here has managed to build a strategy on prediction-market microstructure that survives realistic execution costs.

If you have, I'd genuinely like to hear what worked.

And if anyone wants to reproduce the experiment or try a completely different approach, I'm happy to share the dataset. It now contains a little over a month of tick-level history and keeps growing every day.

reddit.com
u/parrot_chirik — 14 days ago
▲ 3 r/CryptoTradingBot+1 crossposts

I couldn't find tick-level data for Polymarket, so I collected it myself. The model still loses money

I wanted to answer what I thought was a fairly simple question:

Does order-book microstructure contain enough information to beat Polymarket's 5-minute crypto markets after transaction costs?

Turns out the harder part wasn't training the model.

It was getting the data.

As far as I could tell, there wasn't any public historical dataset with tick-level quotes, trades and order-book updates for these markets, so at the end of June I started collecting everything myself.

The archive now covers BTC, ETH, SOL and XRP 5-minute markets.

For every trading day it contains best bid/ask quotes, trade prints, spot prices, and (from 20 July onward) full L2 depth. Everything is stored as daily Parquet files with manifests and measured uptime for each day.

Once I finally had enough history, I trained a regime-routed LightGBM ensemble using only information available at the decision timestamp.

The disappointing part is that the model actually behaves reasonably well.

Calibration is good.

Out-of-sample Brier scores are consistently around 0.06–0.10 depending on asset and market regime.

But the strategy still loses money.

Across roughly 4,300 simulated trades it finishes around -$277.

The hit rate is about 47%.

After spread and fees you need something closer to 49% just to break even.

That two-percentage-point gap turned out to be the entire problem.

Looking back, most of the important work wasn't machine learning at all.

I found that my simulator had been using fees that were 3.5× too low.

One validation gate wasn't actually filtering anything because of a bad threshold.

At one point the trading system quietly stopped opening new positions because several expired markets hadn't been marked as resolved yet, while every monitoring dashboard still showed the process as healthy.

Those bugs changed the conclusions far more than another week of feature engineering.

I'm curious whether anyone here has managed to build a strategy on prediction-market microstructure that survives realistic execution costs.

If you have, I'd genuinely like to hear what worked.

And if anyone wants to reproduce the experiment or try a completely different approach, I'm happy to share the dataset. It now contains a little over a month of tick-level history and keeps growing every day.

reddit.com
u/parrot_chirik — 12 days ago
▲ 1 r/algotradingcrypto+1 crossposts

[PAID] Polymarket 5m/15m up-down markets: tick-level quotes, trades and L2 depth (BTC/ETH/SOL/XRP), 58k markets, ~5.7B rows, June-August 2026

Disclosure: this is my own dataset and I sell access to it. Four days are free, linked below, so you can check the schema before deciding anything.

I run a collector against Polymarket's public API for the 5-minute and 15-minute up/down markets on BTC, ETH, SOL and XRP. It has been running since 26 June 2026. What has accumulated so far:

  • 58,600 markets (14,653 per asset — roughly 11k five-minute, 3.7k fifteen-minute)
  • ~5.69 billion rows across six tables
  • One Parquet file per asset, per table, per UTC day, each with a manifest carrying sha256, row count and interval bounds

Tables: book_snaps (best bid/ask), book_depth (ladder levels, ~88 per event), clob_trades and clob_trades_ws (trades with taker side, the second carries tx hash and fee for VPIN), spot_klines, book_meta.

Limitations, because they decide whether this is useful to you:

  • book_snaps is top of book only. There are level and size columns but they are 0 in every row — they exist to keep the schema stable across tables.
  • book_depth starts 20 July 2026. Fifteen days of the archive have it; everything before is quotes, trades and spot candles.
  • The clock changes partway through. Up to 20 July the timestamp is receipt time rounded to the second, so intra-second ordering is lost and cannot be recovered. From 21 July it is exchange time in milliseconds.
  • Per-day uptime runs 95.8% to 100%. Every day carries its own measured figure.

Free sample (original source): 26-29 June, four full days, no payment and no signup — https://huggingface.co/datasets/astrnvk/polymarket-orderbook-data-preview

Paid access covers 30 June onward and is priced per day ($5/day for short ranges down to $3/day past a month), with incomplete days billed proportionally lower — a day missing depth costs about half. Happy to answer schema questions here either way.

u/parrot_chirik — 16 days ago

How do you prep cook the entire meal?

I've recently joined this community and have seen many recipes for meals prepared days or weeks in advance. So, I wondered, how do you eat them? Because, in my opinion, if a dish is refrigerated for several days, it becomes tasteless or has a terrible texture after reheating. I can't imagine freezing an entire dish (unless it's soup or pasta sauce, etc.) and having it taste as good as freshly prepared.

I prefer to do meal prep the way it's done in restaurants (preparing and cutting chicken, beef, pork, you name it) and then cooking it when needed. You wouldn't be able to tell the difference between fresh chicken and chicken that was frozen, as long as it was thawed properly. But it's possible because only part of dish was "prepared" in a correct way separatly from other ingridients comparing to freezing whole meal.

reddit.com
u/parrot_chirik — 2 months ago